-----============= acceptance-small: sanity-hsm ============----- Sat Aug 15 09:00:42 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg243-server: /home/green/git/lustre-release/lustre/tests/except/sanity-hsm.ex - see MDS1_VERSION <= CLIENT_VERSION (34682625 <= 34682625) for LU-18924, go 100 105 excepting tests: === sanity-hsm: start setup 09:01:03 (1786798863) === Starting client oleg243-client.virtnet: -o user_xattr,flock 192.168.202.143@tcp:/lustre /mnt/lustre2 Mount client oleg243-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.202.143@tcp:/lustre /mnt/lustre2 Started clients oleg243-client.virtnet: 192.168.202.143@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg243-client.virtnet: executing check_config_client /mnt/lustre oleg243-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg243-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9d738319d000.idle_timeout=debug osc.lustre-OST0000-osc-ffff9d73988cb800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9d738319d000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9d73988cb800.idle_timeout=debug disable quota as required oleg243-server: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity-hsm: finish setup 09:01:46 (1786798906) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7594] nobody:x:65534: Killing existing copytools on oleg243-client.virtnet with signal TERM pkill: pidfile not valid Try `pkill --help' for more information. Set HSM on and start Set sanity-hsm HSM policy == sanity-hsm test 1A: lfs hsm flags root/non-root access ========================================================== 09:02:13 (1786798933) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_clear] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--exists] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] lfs hsm_set: change hsm flags for '/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm' failed: Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] PASS 1A (8s) == sanity-hsm test 1a: mmap & cat a HSM released file ==== 09:02:21 (1786798941) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.442191 s, 2.4 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Verifying released state: PASS 1a (11s) == sanity-hsm test 1b: Archive, Release and Restore composite file ========================================================== 09:02:33 (1786798953) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835452 s, 12.6 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' release /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm verify released state: pass restore /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass PASS 1b (14s) == sanity-hsm test 1c: Check setting archive-id in lfs hsm_set ========================================================== 09:02:46 (1786798966) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is still 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 33 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 16 PASS 1c (8s) == sanity-hsm test 1d: Archive, Release and Restore DoM file ========================================================== 09:02:54 (1786798974) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300871 s, 34.9 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm release /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm verify released state: pass restore /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm verify restored state: pass PASS 1d (10s) == sanity-hsm test 1e: Archive, Release and Restore SEL file ========================================================== 09:03:04 (1786798984) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927897 s, 11.3 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm release /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm verify released state: pass restore /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass PASS 1e (11s) == sanity-hsm test 1f: DoM file release after restore ==== 09:03:16 (1786798996) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116995 s, 9.0 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm release /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm verify released state: pass restore /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass release again /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 verify released state: pass PASS 1f (13s) == sanity-hsm test 2: Check file dirtyness when doing setattr ========================================================== 09:03:29 (1786799009) 0+0 records in 0+0 records out 0 bytes copied, 0.00363556 s, 0.0 kB/s 0x200000402:0x9:0x0 PASS 2 (9s) == sanity-hsm test 3: Check file dirtyness when opening for write ========================================================== 09:03:38 (1786799018) Succeed in opening file "/mnt/lustre/d3.sanity-hsm/f3.sanity-hsm"(flags=O_WRONLY) 3+0 records in 3+0 records out 3+0 records in 3+0 records out PASS 3 (11s) == sanity-hsm test 4: Useless cancel must not be registered ========================================================== 09:03:49 (1786799029) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205735 s, 5.1 MB/s PASS 4 (8s) == sanity-hsm test 8: Test default archive number ======== 09:03:57 (1786799037) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_8/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 8 (11s) == sanity-hsm test 9A: Use of explicit archive number, with dedicated copytool ========================================================== 09:04:08 (1786799048) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_9A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 3 "/mnt/lustre2"' oleg243-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg243-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=3e4300d8-c80c-4b65-a617-ce9375c889c0 archive_id=3 requests=[current:0 ok:0 errors:0] on mds1 Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 9A (26s) == sanity-hsm test 9a: Multiple remote agents ============ 09:04:34 (1786799074) SKIP: sanity-hsm test_9a Need 3 or more clients, have 1 SKIP 9a (4s) == sanity-hsm test 10a: Archive a file =================== 09:04:38 (1786799078) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 996: warning: command substitution: ignored null byte in input Verifying content Verifying hsm state Verifying archive number is 2 PASS 10a (10s) == sanity-hsm test 10b: Archive of non dirty file must work without doing request ========================================================== 09:04:49 (1786799089) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 10b (9s) == sanity-hsm test 10c: Check forbidden archive ========== 09:04:58 (1786799098) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d10c.sanity-hsm/f10c.sanity-hsm): Operation not permitted PASS 10c (8s) == sanity-hsm test 10d: Archive a file on the default archive id ========================================================== 09:05:07 (1786799107) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 10d (12s) == sanity-hsm test 11a: Import a file ==================== 09:05:18 (1786799118) lhsmtool_posix: 1786799121.059983 [21152]: action=1 src=d11a.sanity-hsm/f11a.sanity-hsm dst=/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786799121.067971 [21152]: importing '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1786799121.124638 [21152]: imported '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//042b/0x200000401:0x2a:0x0'=='/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1786799121.124698 [21152]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying imported size 158=158 Verifying released pattern: raid0,released Verifying new fid 0x200000401:0x2a:0x0 in archive PASS 11a (7s) == sanity-hsm test 11b: Import a deleted file using its FID ========================================================== 09:05:26 (1786799126) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_11b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786799129.302119 [21836]: action=1 src=0x200000402:0x12:0x0 dst=/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786799129.315522 [21836]: Resolving [0x200000402:0x12:0x0] to /tmp/arc1/sanity-hsm.test_11b//0410/0x200000402:0x12:0x0 lhsmtool_posix: 1786799129.315942 [21836]: importing '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//0410/0x200000402:0x12:0x0' lhsmtool_posix: 1786799129.364387 [21836]: imported '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//042d/0x200000401:0x2c:0x0'=='/tmp/arc1/sanity-hsm.test_11b//0410/0x200000402:0x12:0x0' lhsmtool_posix: 1786799129.365287 [21836]: process finished, errs: 0 major, 0 minor, rc=0 (Success) /mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm: OK PASS 11b (8s) == sanity-hsm test 11c: Import a file to a directory with a pool ========================================================== 09:05:34 (1786799134) Creating new pool test_11c oleg243-server: Pool lustre.test_11c created Adding targets to pool oleg243-server: OST lustre-OST0001_UUID added to pool lustre.test_11c lhsmtool_posix: 1786799143.684227 [22524]: action=1 src=d11c.sanity-hsm/f11c.sanity-hsm dst=/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786799143.700981 [22524]: importing '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1786799143.756074 [22524]: imported '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//042e/0x200000401:0x2f:0x0'=='/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1786799143.756138 [22524]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Destroy the created pools: test_11c lustre.test_11c oleg243-server: OST lustre-OST0001_UUID removed from pool lustre.test_11c oleg243-server: Pool lustre.test_11c destroyed PASS 11c (23s) == sanity-hsm test 12a: Restore an imported file explicitly ========================================================== 09:05:57 (1786799157) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786799160.136211 [23244]: action=1 src=d12a.sanity-hsm/f12a.sanity-hsm dst=/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786799160.141659 [23244]: importing '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1786799160.183211 [23244]: imported '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//0433/0x200000401:0x32:0x0'=='/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1786799160.183281 [23244]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying file state: PASS 12a (11s) == sanity-hsm test 12b: Restore an imported file implicitly ========================================================== 09:06:09 (1786799169) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786799171.686896 [23951]: action=1 src=d12b.sanity-hsm/f12b.sanity-hsm dst=/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786799171.692242 [23951]: importing '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1786799171.737518 [23951]: imported '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//0434/0x200000401:0x35:0x0'=='/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1786799171.738521 [23951]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying file state after restore: PASS 12b (10s) == sanity-hsm test 12c: Restore a file with stripe of 2 == 09:06:19 (1786799179) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.208758 s, 25.1 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d12c.sanity-hsm/f12c.sanity-hsm: OK PASS 12c (11s) == sanity-hsm test 12d: Restore of a non archived, non released file must work ========================================================== 09:06:30 (1786799190) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12d (10s) == sanity-hsm test 12e: Check forbidden restore ========== 09:06:40 (1786799200) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 Cannot send HSM request (use of /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm): Operation not permitted PASS 12e (10s) == sanity-hsm test 12f: Restore a released file explicitly ========================================================== 09:06:50 (1786799210) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying file state: PASS 12f (12s) == sanity-hsm test 12g: Restore a released file implicitly ========================================================== 09:07:03 (1786799223) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 12g (12s) == sanity-hsm test 12h: Restore a released file implicitly from a second node ========================================================== 09:07:15 (1786799235) SKIP: sanity-hsm test_12h Need 2 or more clients, have 1 SKIP 12h (3s) == sanity-hsm test 12m: Archive/release/implicit restore ========================================================== 09:07:19 (1786799239) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12m/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 12m (10s) == sanity-hsm test 12n: Import/implicit restore/release == 09:07:29 (1786799249) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12n/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786799251.775076 [28751]: action=1 src=d12n.sanity-hsm/f12n.sanity-hsm dst=/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786799251.783058 [28751]: importing '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1786799251.843705 [28751]: imported '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//0445/0x200000401:0x44:0x0'=='/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1786799251.844583 [28751]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12n (9s) == sanity-hsm test 12o: Layout-swap failure during Restore leaves file released ========================================================== 09:07:38 (1786799258) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12o/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x152 mdt.lustre-MDT0000.hsm.policy=+NRA diff: /mnt/lustre/d12o.sanity-hsm/f12o.sanity-hsm: No data available mdt.lustre-MDT0000.hsm.policy=-NRA fail_loc=0 mdt.lustre-MDT0000.hsm_control=purge Waiting 200s for 'SUCCEED' Updated after 3s: want 'SUCCEED' got 'SUCCEED' PASS 12o (32s) == sanity-hsm test 12p: implicit restore of a file on copytool mount point ========================================================== 09:08:10 (1786799290) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12p/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12p (13s) == sanity-hsm test 12q: file attributes are refreshed after restore ========================================================== 09:08:23 (1786799303) Starting client: oleg243-client.virtnet: -o user_xattr,flock 192.168.202.143@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3304 s, 3.2 MB/s Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 1s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Stopping client oleg243-client.virtnet /mnt/lustre3 (opts:) PASS 12q (21s) == sanity-hsm test 12r: lseek restores released file ===== 09:08:44 (1786799324) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12r/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12r (11s) == sanity-hsm test 12s: race between restore requests ==== 09:08:56 (1786799336) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12s/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x8000018b PASS 12s (18s) == sanity-hsm test 12t: Multiple parallel reads for a HSM imported file ========================================================== 09:09:13 (1786799353) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12t/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786799357.570963 [32815]: action=1 src=0x200000402:0x2c:0x0 dst=/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786799357.577455 [32815]: Resolving [0x200000402:0x2c:0x0] to /tmp/arc1/sanity-hsm.test_12t//042e/0x200000402:0x2c:0x0 lhsmtool_posix: 1786799357.577597 [32815]: importing '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//042e/0x200000402:0x2c:0x0' lhsmtool_posix: 1786799357.619706 [32815]: imported '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//0453/0x200000401:0x52:0x0'=='/tmp/arc1/sanity-hsm.test_12t//042e/0x200000402:0x2c:0x0' lhsmtool_posix: 1786799357.622156 [32815]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12t (10s) == sanity-hsm test 12u: Multiple reads on multiple HSM imported files in parallel ========================================================== 09:09:23 (1786799363) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12u/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786799378.045601 [33682]: action=1 src=0x200000402:0x2e:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0 mount_point=/mnt/lustre2 lhsmtool_posix: 1786799378.087914 [33682]: Resolving [0x200000402:0x2e:0x0] to /tmp/arc1/sanity-hsm.test_12u//042c/0x200000402:0x2e:0x0 lhsmtool_posix: 1786799378.087985 [33682]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//042c/0x200000402:0x2e:0x0' lhsmtool_posix: 1786799378.181221 [33682]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//0455/0x200000401:0x54:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042c/0x200000402:0x2e:0x0' lhsmtool_posix: 1786799378.181281 [33682]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786799378.565924 [33711]: action=1 src=0x200000402:0x2f:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1 mount_point=/mnt/lustre2 lhsmtool_posix: 1786799378.580422 [33711]: Resolving [0x200000402:0x2f:0x0] to /tmp/arc1/sanity-hsm.test_12u//042d/0x200000402:0x2f:0x0 lhsmtool_posix: 1786799378.580535 [33711]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//042d/0x200000402:0x2f:0x0' lhsmtool_posix: 1786799378.703001 [33711]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//0454/0x200000401:0x55:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042d/0x200000402:0x2f:0x0' lhsmtool_posix: 1786799378.703898 [33711]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786799379.126338 [33740]: action=1 src=0x200000402:0x30:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2 mount_point=/mnt/lustre2 lhsmtool_posix: 1786799379.135631 [33740]: Resolving [0x200000402:0x30:0x0] to /tmp/arc1/sanity-hsm.test_12u//0432/0x200000402:0x30:0x0 lhsmtool_posix: 1786799379.140936 [33740]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//0432/0x200000402:0x30:0x0' lhsmtool_posix: 1786799379.187337 [33740]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//0457/0x200000401:0x56:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0432/0x200000402:0x30:0x0' lhsmtool_posix: 1786799379.187410 [33740]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786799379.526469 [33769]: action=1 src=0x200000402:0x31:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3 mount_point=/mnt/lustre2 lhsmtool_posix: 1786799379.537832 [33769]: Resolving [0x200000402:0x31:0x0] to /tmp/arc1/sanity-hsm.test_12u//0433/0x200000402:0x31:0x0 lhsmtool_posix: 1786799379.538470 [33769]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//0433/0x200000402:0x31:0x0' lhsmtool_posix: 1786799379.577171 [33769]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//0456/0x200000401:0x57:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0433/0x200000402:0x31:0x0' lhsmtool_posix: 1786799379.577231 [33769]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786799379.910834 [33799]: action=1 src=0x200000402:0x32:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4 mount_point=/mnt/lustre2 lhsmtool_posix: 1786799379.916624 [33799]: Resolving [0x200000402:0x32:0x0] to /tmp/arc1/sanity-hsm.test_12u//0430/0x200000402:0x32:0x0 lhsmtool_posix: 1786799379.917036 [33799]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0430/0x200000402:0x32:0x0' lhsmtool_posix: 1786799379.951742 [33799]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0459/0x200000401:0x58:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0430/0x200000402:0x32:0x0' lhsmtool_posix: 1786799379.952031 [33799]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786799380.277700 [33828]: action=1 src=0x200000402:0x33:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5 mount_point=/mnt/lustre2 lhsmtool_posix: 1786799380.287297 [33828]: Resolving [0x200000402:0x33:0x0] to /tmp/arc1/sanity-hsm.test_12u//0431/0x200000402:0x33:0x0 lhsmtool_posix: 1786799380.287769 [33828]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0431/0x200000402:0x33:0x0' lhsmtool_posix: 1786799380.338003 [33828]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0458/0x200000401:0x59:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0431/0x200000402:0x33:0x0' lhsmtool_posix: 1786799380.338921 [33828]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786799380.627402 [33857]: action=1 src=0x200000402:0x34:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6 mount_point=/mnt/lustre2 lhsmtool_posix: 1786799380.632176 [33857]: Resolving [0x200000402:0x34:0x0] to /tmp/arc1/sanity-hsm.test_12u//0436/0x200000402:0x34:0x0 lhsmtool_posix: 1786799380.632466 [33857]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0436/0x200000402:0x34:0x0' lhsmtool_posix: 1786799380.674122 [33857]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//045b/0x200000401:0x5a:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0436/0x200000402:0x34:0x0' lhsmtool_posix: 1786799380.675253 [33857]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786799381.080829 [33886]: action=1 src=0x200000402:0x35:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7 mount_point=/mnt/lustre2 lhsmtool_posix: 1786799381.092202 [33886]: Resolving [0x200000402:0x35:0x0] to /tmp/arc1/sanity-hsm.test_12u//0437/0x200000402:0x35:0x0 lhsmtool_posix: 1786799381.092297 [33886]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0437/0x200000402:0x35:0x0' lhsmtool_posix: 1786799381.164268 [33886]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//045a/0x200000401:0x5b:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0437/0x200000402:0x35:0x0' lhsmtool_posix: 1786799381.164330 [33886]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786799381.598902 [33915]: action=1 src=0x200000402:0x36:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8 mount_point=/mnt/lustre2 lhsmtool_posix: 1786799381.607868 [33915]: Resolving [0x200000402:0x36:0x0] to /tmp/arc1/sanity-hsm.test_12u//0434/0x200000402:0x36:0x0 lhsmtool_posix: 1786799381.608067 [33915]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0434/0x200000402:0x36:0x0' lhsmtool_posix: 1786799381.681239 [33915]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//045d/0x200000401:0x5c:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0434/0x200000402:0x36:0x0' lhsmtool_posix: 1786799381.689610 [33915]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1786799381.937357 [33944]: action=1 src=0x200000402:0x37:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9 mount_point=/mnt/lustre2 lhsmtool_posix: 1786799381.944751 [33944]: Resolving [0x200000402:0x37:0x0] to /tmp/arc1/sanity-hsm.test_12u//0435/0x200000402:0x37:0x0 lhsmtool_posix: 1786799381.951804 [33944]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0435/0x200000402:0x37:0x0' lhsmtool_posix: 1786799382.037104 [33944]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//045c/0x200000401:0x5d:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0435/0x200000402:0x37:0x0' lhsmtool_posix: 1786799382.037166 [33944]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12u (29s) == sanity-hsm test 13: Recursively import and restore a directory ========================================================== 09:09:52 (1786799392) lhsmtool_posix: 1786799401.606544 [35709]: action=1 src=subdir dst=/mnt/lustre/d13.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786799401.730843 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1786799401.811873 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//0460/0x200000401:0x61:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1786799401.812875 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1786799401.859416 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//0463/0x200000401:0x62:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1786799401.859689 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1786799401.907715 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//0462/0x200000401:0x63:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1786799401.907925 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1786799401.957510 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//0465/0x200000401:0x64:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1786799401.958611 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1786799402.006216 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//0464/0x200000401:0x65:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1786799402.006418 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1786799402.048051 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//0467/0x200000401:0x66:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1786799402.048270 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1786799402.117278 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//0466/0x200000401:0x67:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1786799402.117486 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1786799402.155111 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//0469/0x200000401:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1786799402.155334 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1786799402.186356 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//0468/0x200000401:0x69:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1786799402.187171 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1786799402.228835 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//046b/0x200000401:0x6a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1786799402.234389 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1786799402.284242 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//046d/0x200000401:0x6c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1786799402.285511 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1786799402.320528 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//046c/0x200000401:0x6d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1786799402.321539 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1786799402.358621 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//046f/0x200000401:0x6e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1786799402.358833 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1786799402.393464 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//046e/0x200000401:0x6f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1786799402.394208 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1786799402.432263 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//0471/0x200000401:0x70:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1786799402.432986 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1786799402.477757 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//0470/0x200000401:0x71:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1786799402.478115 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1786799402.512174 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//0473/0x200000401:0x72:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1786799402.512397 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1786799402.558874 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//0472/0x200000401:0x73:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1786799402.559083 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1786799402.616967 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//0475/0x200000401:0x74:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1786799402.617895 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1786799402.682481 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//0474/0x200000401:0x75:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1786799402.687373 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1786799402.767384 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//0476/0x200000401:0x77:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1786799402.768044 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1786799402.818426 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//0479/0x200000401:0x78:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1786799402.826721 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1786799402.874063 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//0478/0x200000401:0x79:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1786799402.874619 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1786799402.911954 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//047b/0x200000401:0x7a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1786799402.912170 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1786799402.950677 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//047a/0x200000401:0x7b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1786799402.950902 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1786799402.999342 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//047d/0x200000401:0x7c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1786799403.000464 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1786799403.052908 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//047c/0x200000401:0x7d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1786799403.053962 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1786799403.116307 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//047f/0x200000401:0x7e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1786799403.116899 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1786799403.204968 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//047e/0x200000401:0x7f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1786799403.208541 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1786799403.293623 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//0481/0x200000401:0x80:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1786799403.306761 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1786799403.387387 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//0483/0x200000401:0x82:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1786799403.388359 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1786799403.450066 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//0482/0x200000401:0x83:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1786799403.450281 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1786799403.489608 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//0485/0x200000401:0x84:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1786799403.489815 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1786799403.540279 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//0484/0x200000401:0x85:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1786799403.540523 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1786799403.587452 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//0487/0x200000401:0x86:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1786799403.587714 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1786799403.673008 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//0486/0x200000401:0x87:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1786799403.673361 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1786799403.767296 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//0489/0x200000401:0x88:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1786799403.767681 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1786799403.815395 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//0488/0x200000401:0x89:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1786799403.815911 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1786799403.859662 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//048b/0x200000401:0x8a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1786799403.859908 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1786799403.929523 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//048a/0x200000401:0x8b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1786799403.935584 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1786799403.978860 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//048c/0x200000401:0x8d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1786799403.979183 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1786799404.021377 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//048f/0x200000401:0x8e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1786799404.023344 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1786799404.080514 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//048e/0x200000401:0x8f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1786799404.080791 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1786799404.117545 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//0491/0x200000401:0x90:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1786799404.118417 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1786799404.158687 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//0490/0x200000401:0x91:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1786799404.159100 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1786799404.205291 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//0493/0x200000401:0x92:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1786799404.206206 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1786799404.259021 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//0492/0x200000401:0x93:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1786799404.259822 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1786799404.299391 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//0495/0x200000401:0x94:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1786799404.300266 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1786799404.335613 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//0494/0x200000401:0x95:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1786799404.336982 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1786799404.376270 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//0497/0x200000401:0x96:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1786799404.384655 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1786799404.435375 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//0499/0x200000401:0x98:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1786799404.436228 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1786799404.477929 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//0498/0x200000401:0x99:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1786799404.480367 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1786799404.560996 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//049b/0x200000401:0x9a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1786799404.561213 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1786799404.618419 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//049a/0x200000401:0x9b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1786799404.624631 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1786799404.691474 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//049d/0x200000401:0x9c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1786799404.695845 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1786799404.770197 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//049c/0x200000401:0x9d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1786799404.770401 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1786799404.859457 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//049f/0x200000401:0x9e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1786799404.859738 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1786799404.911237 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//049e/0x200000401:0x9f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1786799404.912295 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1786799404.951157 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//04a1/0x200000401:0xa0:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1786799404.951332 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1786799404.989311 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//04a0/0x200000401:0xa1:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1786799404.995881 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1786799405.105523 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//04a2/0x200000401:0xa3:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1786799405.111405 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1786799405.165321 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//04a5/0x200000401:0xa4:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1786799405.165660 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1786799405.241086 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//04a4/0x200000401:0xa5:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1786799405.241315 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1786799405.294234 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//04a7/0x200000401:0xa6:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1786799405.294453 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1786799405.380851 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//04a6/0x200000401:0xa7:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1786799405.381063 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1786799405.445696 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//04a9/0x200000401:0xa8:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1786799405.445926 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1786799405.490188 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//04a8/0x200000401:0xa9:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1786799405.490401 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1786799405.542132 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//04ab/0x200000401:0xaa:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1786799405.542328 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1786799405.583407 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//04aa/0x200000401:0xab:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1786799405.584332 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1786799405.619131 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//04ad/0x200000401:0xac:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1786799405.628927 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1786799405.678706 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//04af/0x200000401:0xae:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1786799405.678918 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1786799405.723828 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//04ae/0x200000401:0xaf:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1786799405.724040 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1786799405.789862 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//04b1/0x200000401:0xb0:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1786799405.790091 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1786799405.855290 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//04b0/0x200000401:0xb1:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1786799405.855510 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1786799405.894500 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//04b3/0x200000401:0xb2:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1786799405.894749 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1786799405.927509 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//04b2/0x200000401:0xb3:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1786799405.928208 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1786799405.963971 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//04b5/0x200000401:0xb4:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1786799405.964248 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1786799405.997715 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//04b4/0x200000401:0xb5:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1786799405.997922 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1786799406.040864 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//04b7/0x200000401:0xb6:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1786799406.041461 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1786799406.076772 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//04b6/0x200000401:0xb7:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1786799406.083325 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1786799406.122223 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//04b8/0x200000401:0xb9:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1786799406.123067 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1786799406.162085 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//04bb/0x200000401:0xba:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1786799406.162299 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1786799406.202080 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//04ba/0x200000401:0xbb:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1786799406.202300 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1786799406.237514 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//04bd/0x200000401:0xbc:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1786799406.237765 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1786799406.270685 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//04bc/0x200000401:0xbd:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1786799406.270890 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1786799406.311802 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//04bf/0x200000401:0xbe:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1786799406.311999 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1786799406.345694 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//04be/0x200000401:0xbf:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1786799406.346736 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1786799406.379344 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//04c1/0x200000401:0xc0:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1786799406.380156 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1786799406.417620 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//04c0/0x200000401:0xc1:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1786799406.417818 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1786799406.453425 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//04c3/0x200000401:0xc2:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1786799406.457850 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1786799406.498771 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//04c5/0x200000401:0xc4:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1786799406.498969 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1786799406.544746 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//04c4/0x200000401:0xc5:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1786799406.554058 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1786799406.627645 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//04c7/0x200000401:0xc6:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1786799406.627966 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1786799406.685066 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//04c6/0x200000401:0xc7:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1786799406.685297 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1786799406.734885 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//04c9/0x200000401:0xc8:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1786799406.735421 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1786799406.769981 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//04c8/0x200000401:0xc9:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1786799406.770192 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1786799406.806515 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//04cb/0x200000401:0xca:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1786799406.806918 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1786799406.844645 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//04ca/0x200000401:0xcb:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1786799406.844907 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1786799406.879314 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//04cd/0x200000401:0xcc:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1786799406.879525 [35709]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1786799406.934121 [35709]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//04cc/0x200000401:0xcd:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1786799406.934714 [35709]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_13/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 13 (40s) == sanity-hsm test 14: Rebind archived file to a new fid ========================================================== 09:10:32 (1786799432) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_14/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.398947 s, 2.6 MB/s 0+0 records in 0+0 records out 0 bytes copied, 0.0227218 s, 0.0 kB/s rebind 0x200000402:0xa7:0x0 to 0x200000402:0xa8:0x0 lhsmtool_posix: 1786799437.079030 [36661]: action=2 src=0x200000402:0xa7:0x0 dst=0x200000402:0xa8:0x0 mount_point=/mnt/lustre2 lhsmtool_posix: 1786799437.099145 [36661]: rebind [0x200000402:0xa7:0x0] to [0x200000402:0xa8:0x0] lhsmtool_posix: 1786799437.099364 [36661]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 14 (12s) == sanity-hsm test 15: Rebind a list of files ============ 09:10:44 (1786799444) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_15/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168404 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138425 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.280791 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105968 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118241 s, 8.9 MB/s pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 0+0 records in 0+0 records out 0 bytes copied, 0.0218303 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00339418 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00751313 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00437368 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.0106392 s, 0.0 kB/s rebind list of files lhsmtool_posix: 1786799450.112993 [37444]: action=2 src=/tmp/tmp.7594 dst=(null) mount_point=/mnt/lustre2 lhsmtool_posix: 1786799450.114754 [37444]: rebind [0x200000402:0xab:0x0] to [0x200000402:0xb0:0x0] lhsmtool_posix: 1786799450.114900 [37444]: rebind [0x200000402:0xac:0x0] to [0x200000402:0xb1:0x0] lhsmtool_posix: 1786799450.114957 [37444]: rebind [0x200000402:0xad:0x0] to [0x200000402:0xb2:0x0] lhsmtool_posix: 1786799450.115015 [37444]: rebind [0x200000402:0xae:0x0] to [0x200000402:0xb3:0x0] lhsmtool_posix: 1786799450.115258 [37444]: rebind [0x200000402:0xaf:0x0] to [0x200000402:0xb4:0x0] lhsmtool_posix: 1786799450.115326 [37444]: 5 lines read from '/tmp/tmp.7594', 5 rebind successful lhsmtool_posix: 1786799450.115344 [37444]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 15 (19s) == sanity-hsm test 16: Test CT bandwith control option === 09:11:03 (1786799463) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_16/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 0.079834 s, 263 MB/s Waiting 200s for 'SUCCEED' Waiting 190s for 'SUCCEED' Updated after 20s: want 'SUCCEED' got 'SUCCEED' PASS 16 (31s) == sanity-hsm test 20: Release is not permitted ========== 09:11:34 (1786799494) 0+0 records in 0+0 records out 0 bytes copied, 0.0143162 s, 0.0 kB/s 0x200000402:0xbc:0x0 Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted PASS 20 (7s) == sanity-hsm test 21: Simple release tests ============== 09:11:41 (1786799501) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_21/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178589 s, 5.9 MB/s 1+0 records in 1+0 records out 512 bytes copied, 0.0775652 s, 6.6 kB/s Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 21 (14s) == sanity-hsm test 22: Could not swap a release file ===== 09:11:55 (1786799515) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_22/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201389 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276459 s, 3.8 MB/s 0x200000402:0xc1:0x0 lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d22.sanity-hsm/test_swap' and '/mnt/lustre/d22.sanity-hsm/test_release': Operation not supported (95) PASS 22 (9s) == sanity-hsm test 23: Release does not change a/mtime (utime) ========================================================== 09:12:04 (1786799524) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_23/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168965 s, 6.2 MB/s PASS 23 (9s) == sanity-hsm test 24a: Archive, release, and restore does not change a/mtime (i/o) ========================================================== 09:12:13 (1786799533) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169708 s, 6.2 MB/s Waiting 200s for 'SUCCEED' Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s 192.168.202.143@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg243-client.virtnet /mnt/lustre (opts:) Starting client: oleg243-client.virtnet: -o user_xattr,flock 192.168.202.143@tcp:/lustre /mnt/lustre PASS 24a (16s) == sanity-hsm test 24b: root can archive, release, and restore user files ========================================================== 09:12:29 (1786799549) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226852 s, 4.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] /mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm: (0x00000009) exists archived, archive_id:1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [md5sum] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] PASS 24b (12s) == sanity-hsm test 24c: check that user,group,other request masks work ========================================================== 09:12:41 (1786799561) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213137 s, 4.9 MB/s 0x200000402:0xcc:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162495 s, 6.5 MB/s 0x200000402:0xcd:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215684 s, 4.9 MB/s 0x200000402:0xce:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] PASS 24c (17s) == sanity-hsm test 24d: check that read-only mounts are respected ========================================================== 09:12:58 (1786799578) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196901 s, 5.3 MB/s 0x200000402:0xd0:0x0 /mnt/lustre/d24d.sanity-hsm/f24d.sanity-hsm lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 105 0x69 0x240000400 Starting client: oleg243-client.virtnet: -o user_xattr,flock 192.168.202.143@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Waiting 200s for 'SUCCEED' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 1.10468 s, 949 kB/s Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Stopping client oleg243-client.virtnet /mnt/lustre3 (opts:) PASS 24d (19s) == sanity-hsm test 24e: tar succeeds on HSM released files ========================================================== 09:13:17 (1786799597) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159454 s, 6.6 MB/s /mnt/lustre/d24e.sanity-hsm/f24e.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 tar: Removing leading `/' from member names PASS 24e (10s) == sanity-hsm test 24f: root can archive, release, and restore tar files ========================================================== 09:13:27 (1786799607) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' f24f.sanity-hsm f24f.sanity-hsm Sum0 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm, sum1 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm PASS 24f (12s) == sanity-hsm test 24g: write by non-owner still sets dirty ========================================================== 09:13:39 (1786799619) RUNAS = 'runas -u 500 -g 500' Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Please listen carefully as our options have changed. running as uid/gid/euid/egid 500/500/500/500, groups: 500 [tee] [/mnt/lustre/d24g.sanity-hsm/f24g.sanity-hsm] To be electrocuted by your telephone, press #. PASS 24g (9s) == sanity-hsm test 25a: Restore lost file (HS_LOST flag) from import ========================================================== 09:13:48 (1786799628) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786799630.897206 [46313]: action=1 src=d25a.sanity-hsm/f25a.sanity-hsm dst=/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786799630.905433 [46313]: importing '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1786799631.001050 [46313]: imported '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//0411/0x200000404:0x15:0x0'=='/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1786799631.001178 [46313]: process finished, errs: 0 major, 0 minor, rc=0 (Success) md5sum: /mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm: No data available PASS 25a (8s) == sanity-hsm test 25b: Restore lost file (HS_LOST flag) after release ========================================================== 09:13:57 (1786799637) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' md5sum: /mnt/lustre/d25b.sanity-hsm/f25b.sanity-hsm: No data available PASS 25b (8s) == sanity-hsm test 26A: Remove the archive of a valid file ========================================================== 09:14:06 (1786799646) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00672454 s, 0.0 kB/s PASS 26A (10s) == sanity-hsm test 26a: Remove Archive On Last Unlink (RAoLU) policy ========================================================== 09:14:16 (1786799656) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 6s: want 'SUCCEED' got 'SUCCEED' PASS 26a (30s) == sanity-hsm test 26b: RAoLU policy when CDT off ======== 09:14:46 (1786799686) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 26b (18s) == sanity-hsm test 26c: RAoLU effective when file closed ========================================================== 09:15:04 (1786799704) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' multiop /mnt/lustre/d26c.sanity-hsm/f26c.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7594 Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 9s: want 'SUCCEED' got 'SUCCEED' PASS 26c (30s) == sanity-hsm test 26d: RAoLU when Client eviction ======= 09:15:34 (1786799734) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199713 s, 5.3 MB/s Waiting 200s for 'SUCCEED' multiop /mnt/lustre/d26d.sanity-hsm/f26d.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7594 Waiting 200s for 'SUCCEED' Updated after 10s: want 'SUCCEED' got 'SUCCEED' PASS 26d (31s) == sanity-hsm test 26e: RAoLU with a non-started coordinator ========================================================== 09:16:05 (1786799765) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248878 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154635 s, 6.8 MB/s Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s Failing mds1 on oleg243-server Stopping /mnt/lustre-mds1 (opts:) on oleg243-server 09:16:38 (1786799798) shut down facet: mds1 facet_host: oleg243-server facet_failover_host: oleg243-server Failover mds1 to oleg243-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg243-server: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Started lustre-MDT0000 09:17:00 (1786799820) targets are mounted 09:17:00 (1786799820) facet_failover done oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Check cookie from RAoLU request (last: 1786799125, remove: 1786799126) mdt.lustre-MDT0000.hsm_control=enabled Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 26e (101s) == sanity-hsm test 27a: Remove the archive of an imported file (Operation not permitted) ========================================================== 09:17:46 (1786799866) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.641991 s, 63.7 MB/s lhsmtool_posix: 1786799870.066883 [53065]: action=1 src=d27a.sanity-hsm/f27a.sanity-hsm dst=/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786799870.070729 [53065]: importing '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1786799870.112009 [53065]: imported '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//0405/0x200000406:0x3:0x0'=='/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1786799870.112378 [53065]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Cannot send HSM request (use of /mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm): Operation not permitted PASS 27a (8s) == sanity-hsm test 27b: Remove the archive of a relased file (Operation not permitted) ========================================================== 09:17:55 (1786799875) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00491939 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Cannot send HSM request (use of /mnt/lustre/d27b.sanity-hsm/f27b.sanity-hsm): Operation not permitted PASS 27b (9s) == sanity-hsm test 28: Concurrent archive/file remove ==== 09:18:04 (1786799884) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_28/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.0037516 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled PASS 28 (12s) == sanity-hsm test 29a: Tests --mntpath and --archive options ========================================================== 09:18:16 (1786799896) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 7 "/mnt/lustre"' Cannot send HSM request (use of /mnt/lustre): Invalid argument hsm: '/qwerty/uyt' is not a valid FID PASS 29a (6s) == sanity-hsm test 29b: Archive/delete/remove by FID from the archive. ========================================================== 09:18:22 (1786799902) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175909 s, 6.0 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 29b (10s) == sanity-hsm test 29c: Archive/delete/remove by FID, using a file list. ========================================================== 09:18:32 (1786799912) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131504 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133934 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190456 s, 5.5 MB/s Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' PASS 29c (15s) == sanity-hsm test 29d: hsm_remove by FID with archive_id 0 for unlinked file cause ========================================================== 09:18:47 (1786799927) SKIP: sanity-hsm test_29d Need 3 or more clients, have 1 SKIP 29d (3s) == sanity-hsm test 30a: Restore at exec (import case) ==== 09:18:50 (1786799930) SKIP: sanity-hsm test_30a Need 2 or more clients, have 1 SKIP 30a (2s) == sanity-hsm test 30b: Restore at exec (release case) === 09:18:53 (1786799933) SKIP: sanity-hsm test_30b Need 2 or more clients, have 1 SKIP 30b (3s) == sanity-hsm test 30c: Update during exec of released file must fail ========================================================== 09:18:56 (1786799936) SKIP: sanity-hsm test_30c Need 2 or more clients, have 1 SKIP 30c (3s) == sanity-hsm test 31a: Import a large file and check size during restore ========================================================== 09:18:59 (1786799939) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.975175 s, 41.9 MB/s lhsmtool_posix: 1786799942.260707 [58148]: action=1 src=d31a.sanity-hsm/f31a.sanity-hsm dst=/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786799942.265455 [58148]: importing '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1786799942.307980 [58148]: imported '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//040b/0x200000406:0xd:0x0'=='/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1786799942.308433 [58148]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' size seen is right: 40894464 == 40894464 done PASS 31a (19s) == sanity-hsm test 31b: Restore a large unaligned file and check size during restore ========================================================== 09:19:18 (1786799958) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.969246 s, 40.2 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' size seen is right: 39000000 == 39000000 done PASS 31b (23s) == sanity-hsm test 31c: Restore a large aligned file and check size during restore ========================================================== 09:19:41 (1786799981) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.15928 s, 35.3 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' size seen is right: 40894464 == 40894464 done PASS 31c (23s) == sanity-hsm test 33: Kill a restore waiting process ==== 09:20:04 (1786800004) 0+0 records in 0+0 records out 0 bytes copied, 0.00420077 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_33/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2893: 60165 Terminated md5sum $f > /dev/null PASS 33 (12s) == sanity-hsm test 34: Remove file during restore ======== 09:20:16 (1786800016) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_34/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00725553 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg243-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Copytool is continued on oleg243-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2925: kill: (60826) - No such process PASS 34 (13s) == sanity-hsm test 35: Overwrite file during restore ===== 09:20:29 (1786800029) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_35/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00483143 s, 0.0 kB/s Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2962: kill: (61504) - No such process PASS 35 (10s) == sanity-hsm test 36: Move file during restore ========== 09:20:39 (1786800039) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_36/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.0113134 s, 0.0 kB/s Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2998: kill: (62171) - No such process PASS 36 (10s) == sanity-hsm test 37: re-archive a dirty file =========== 09:20:49 (1786800049) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_37/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288052 s, 3.6 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101433 s, 10.3 MB/s PASS 37 (74s) == sanity-hsm test 40: Parallel archive requests ========= 09:22:03 (1786800123) oleg243-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1152921504606846976: Cannot allocate memory oleg243-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1152921504606846976': Cannot allocate memory pdsh@oleg243-client: oleg243-server: ssh exited with exit code 12 set max_requests=1152921504606846976 failed with 12 [ 1499.256667] Lustre: lustre-MDT0000: No more memory to set HSM max_requests=1152921504606846976 (max request memory: 461MB, current total 3/975941): rc = -12 Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_40/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge Wait for all requests being enqueued...100 archive requests submitted 100 archive requests submitted 100 archive requests submitted 100 archive requests submitted OK pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 PASS 40 (150s) == sanity-hsm test 50: Archive with large number of pending HSM actions ========================================================== 09:24:34 (1786800274) oleg243-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1000000: Cannot allocate memory oleg243-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1000000': Cannot allocate memory pdsh@oleg243-client: oleg243-server: ssh exited with exit code 12 Filesystem Inodes IUsed IFree IUse% Mounted on 192.168.202.143@tcp:/lustre 236094 990 235104 1% /mnt/lustre - create 1373 (time 1786800290.51 total 10.00 last 137.27) - create 3014 (time 1786800300.53 total 20.02 last 163.73) - create 4596 (time 1786800310.53 total 30.03 last 158.18) - create 6190 (time 1786800320.54 total 40.03 last 159.31) - create 7761 (time 1786800330.54 total 50.03 last 157.06) - create 9338 (time 1786800340.54 total 60.03 last 157.70) total: 10000 create in 64.42 seconds: 155.24 ops/second 10000 - unlinked 0 (time 1786800432 ; total 0 ; last 0) total: 10000 unlinks in 70 seconds: 142.857147 unlinks/second Start Phase files_num: 10000 time: 82 - create 1450 (time 1786800516.80 total 10.00 last 144.99) - create 2825 (time 1786800526.80 total 20.00 last 137.45) - create 4300 (time 1786800536.82 total 30.03 last 147.18) - create 5694 (time 1786800546.83 total 40.03 last 139.35) - create 7054 (time 1786800556.83 total 50.03 last 135.96) - create 8510 (time 1786800566.88 total 60.08 last 144.94) - create 9862 (time 1786800576.88 total 70.09 last 135.11) - create 11218 (time 1786800586.89 total 80.09 last 135.58) - create 12578 (time 1786800596.89 total 90.09 last 135.95) - create 13892 (time 1786800606.90 total 100.10 last 131.30) - create 14838 (time 1786800616.90 total 110.10 last 94.56) - create 16229 (time 1786800626.90 total 120.10 last 139.07) - create 17792 (time 1786800636.91 total 130.11 last 156.19) - create 19146 (time 1786800646.92 total 140.12 last 135.31) total: 20000 create in 146.24 seconds: 136.76 ops/second - unlinked 0 (time 1786800821 ; total 1 ; last 1) - unlinked 10000 (time 1786800895 ; total 75 ; last 74) total: 20000 unlinks in 149 seconds: 134.228195 unlinks/second Middle Phase files_num: 20000 time: 163 - create 1365 (time 1786800984.62 total 10.01 last 136.39) - create 2719 (time 1786800994.62 total 20.01 last 135.33) - create 4141 (time 1786801004.62 total 30.02 last 142.15) - create 5504 (time 1786801014.63 total 40.02 last 136.20) - create 6633 (time 1786801024.64 total 50.04 last 112.76) - create 8225 (time 1786801034.65 total 60.04 last 159.16) - create 9668 (time 1786801044.65 total 70.05 last 144.21) total: 10000 create in 72.18 seconds: 138.54 ops/second 40000 - unlinked 0 (time 1786801144 ; total 0 ; last 0) total: 10000 unlinks in 72 seconds: 138.888885 unlinks/second End Phase files_num: 10000 time: 88 40000 mdt.lustre-MDT0000.hsm_control=purge PASS 50 (957s) == sanity-hsm test 52: Opened for write file on an evicted client should be set dirty ========================================================== 09:40:31 (1786801231) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_52/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196321 s, 5.3 MB/s multiop /mnt/lustre/d52.sanity-hsm/f52.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7594 PASS 52 (15s) == sanity-hsm test 53: Opened for read file on an evicted client should not be set dirty ========================================================== 09:40:46 (1786801246) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_53/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.343515 s, 3.1 MB/s multiop /mnt/lustre/d53.sanity-hsm/f53.sanity-hsm vo_c TMPPIPE=/tmp/multiop_open_wait_pipe.7594 pdsh@oleg243-client: oleg243-client: ssh exited with exit code 5 PASS 53 (16s) == sanity-hsm test 54: Write during an archive cancels it ========================================================== 09:41:02 (1786801262) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.931116 s, 41.9 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_54/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA Waiting 200s for 'FAILED' Waiting 180s for 'FAILED' Updated after 35s: want 'FAILED' got 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA PASS 54 (48s) == sanity-hsm test 55: Truncate during an archive cancels it ========================================================== 09:41:50 (1786801310) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.927465 s, 42.1 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_55/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NRA PASS 55 (13s) == sanity-hsm test 56: Setattr during an archive is ok === 09:42:03 (1786801323) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.977439 s, 39.9 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_56/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Waiting 200s for 'SUCCEED' Waiting 180s for 'SUCCEED' Updated after 36s: want 'SUCCEED' got 'SUCCEED' PASS 56 (48s) == sanity-hsm test 57: Archive a file with dirty cache on another node ========================================================== 09:42:51 (1786801371) SKIP: sanity-hsm test_57 Need 2 or more clients, have 1 SKIP 57 (4s) == sanity-hsm test 58: Truncate a released file will trigger restore ========================================================== 09:42:55 (1786801375) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_58/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' truncate up from 1516 to 3032 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate down from 1516 to 758 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate to 0 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 PASS 58 (16s) == sanity-hsm test 59: Release stripeless file with non-zero size ========================================================== 09:43:11 (1786801391) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_59/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 59 (7s) == sanity-hsm test 60: Changing progress update interval from default ========================================================== 09:43:18 (1786801398) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.246872 s, 42.5 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_60/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 --update-interval 5 "/mnt/lustre2"' Waiting 100s for 'start copy' Expecting a progress update within 20 seconds... 5242880 bytes copied in 2 seconds. Wait for on going archive hsm action to complete Waiting 10s for 'copied' Updated after 5s: want 'copied' got 'copied' PASS 60 (21s) == sanity-hsm test 61: Waiting archive of a removed file should fail ========================================================== 09:43:39 (1786801419) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_61/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled PASS 61 (12s) == sanity-hsm test 70: Copytool logs JSON register/unregister events to FIFO ========================================================== 09:43:52 (1786801432) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_70/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --event-fifo /tmp/sanity-hsm.test_70.7fY8/fifo "/mnt/lustre2"' Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s Register/Unregister events look OK. PASS 70 (11s) == sanity-hsm test 71: Copytool logs JSON archive events to FIFO ========================================================== 09:44:03 (1786801443) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_71/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_71.WT5C/fifo "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=-NRA 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185549 s, 5.7 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Archive events look OK. mdt.lustre-MDT0000.hsm.policy=-NRA PASS 71 (21s) == sanity-hsm test 72: Copytool logs JSON restore events to FIFO ========================================================== 09:44:24 (1786801464) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_72/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_72.I1ky/fifo "/mnt/lustre2"' 16+0 records in 16+0 records out 16000000 bytes (16 MB, 15 MiB) copied, 0.423534 s, 37.8 MB/s lhsmtool_posix: 1786801468.939856 [80835]: action=1 src=d72.sanity-hsm/f72.sanity-hsm dst=/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786801468.946661 [80835]: importing '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1786801469.010950 [80835]: imported '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//0bc8/0x200000bd2:0x1a:0x0'=='/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1786801469.011321 [80835]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Restore events look OK. PASS 72 (17s) == sanity-hsm test 90: Archive/restore a file list ======= 09:44:41 (1786801481) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_90/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge Waiting 200s for '' pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Waiting 200s for '' Changed after 2s: from 'lrh=[type=10680000 len=136 idx=4/10061] fid=[0x200000402:0x2a2:0x0] dfid=[0x200000402:0x2a2:0x0] compound/cookie=0x0/0x6a810248 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10062] fid=[0x200000402:0x2a3:0x0] dfid=[0x200000402:0x2a3:0x0] compound/cookie=0x0/0x6a810249 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10063] fid=[0x200000402:0x2a4:0x0] dfid=[0x200000402:0x2a4:0x0] compound/cookie=0x0/0x6a81024a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10064] fid=[0x200000402:0x2a5:0x0] dfid=[0x200000402:0x2a5:0x0] compound/cookie=0x0/0x6a81024b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10066] fid=[0x200000402:0x2a6:0x0] dfid=[0x200000402:0x2a6:0x0] compound/cookie=0x0/0x6a81024c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10067] fid=[0x200000402:0x2a7:0x0] dfid=[0x200000402:0x2a7:0x0] compound/cookie=0x0/0x6a81024d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10068] fid=[0x200000402:0x2a8:0x0] dfid=[0x200000402:0x2a8:0x0] compound/cookie=0x0/0x6a81024e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10069] fid=[0x200000402:0x2a9:0x0] dfid=[0x200000402:0x2a9:0x0] compound/cookie=0x0/0x6a81024f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10070] fid=[0x200000402:0x2aa:0x0] dfid=[0x200000402:0x2aa:0x0] compound/cookie=0x0/0x6a810250 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10071] fid=[0x200000402:0x2ab:0x0] dfid=[0x200000402:0x2ab:0x0] compound/cookie=0x0/0x6a810251 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10072] fid=[0x200000402:0x2ac:0x0] dfid=[0x200000402:0x2ac:0x0] compound/cookie=0x0/0x6a810252 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10073] fid=[0x200000402:0x2ad:0x0] dfid=[0x200000402:0x2ad:0x0] compound/cookie=0x0/0x6a810253 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10074] fid=[0x200000402:0x2ae:0x0] dfid=[0x200000402:0x2ae:0x0] compound/cookie=0x0/0x6a810254 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10075] fid=[0x200000402:0x2af:0x0] dfid=[0x200000402:0x2af:0x0] compound/cookie=0x0/0x6a810255 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10076] fid=[0x200000402:0x2b0:0x0] dfid=[0x200000402:0x2b0:0x0] compound/cookie=0x0/0x6a810256 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10077] fid=[0x200000402:0x2b1:0x0] dfid=[0x200000402:0x2b1:0x0] compound/cookie=0x0/0x6a810257 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10078] fid=[0x200000402:0x2b2:0x0] dfid=[0x200000402:0x2b2:0x0] compound/cookie=0x0/0x6a810258 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10079] fid=[0x200000402:0x2b3:0x0] dfid=[0x200000402:0x2b3:0x0] compound/cookie=0x0/0x6a810259 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10080] fid=[0x200000402:0x2b4:0x0] dfid=[0x200000402:0x2b4:0x0] compound/cookie=0x0/0x6a81025a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10081] fid=[0x200000402:0x2b5:0x0] dfid=[0x200000402:0x2b5:0x0] compound/cookie=0x0/0x6a81025b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10082] fid=[0x200000402:0x2b6:0x0] dfid=[0x200000402:0x2b6:0x0] compound/cookie=0x0/0x6a81025c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000402:0x2b7:0x0] dfid=[0x200000402:0x2b7:0x0] compound/cookie=0x0/0x6a81025d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000402:0x2b8:0x0] dfid=[0x200000402:0x2b8:0x0] compound/cookie=0x0/0x6a81025e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000402:0x2b9:0x0] dfid=[0x200000402:0x2b9:0x0] compound/cookie=0x0/0x6a81025f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000402:0x2ba:0x0] dfid=[0x200000402:0x2ba:0x0] compound/cookie=0x0/0x6a810260 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000402:0x2bb:0x0] dfid=[0x200000402:0x2bb:0x0] compound/cookie=0x0/0x6a810261 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000402:0x2bc:0x0] dfid=[0x200000402:0x2bc:0x0] compound/cookie=0x0/0x6a810262 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000402:0x2bd:0x0] dfid=[0x200000402:0x2bd:0x0] compound/cookie=0x0/0x6a810263 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000402:0x2be:0x0] dfid=[0x200000402:0x2be:0x0] compound/cookie=0x0/0x6a810264 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000402:0x2bf:0x0] dfid=[0x200000402:0x2bf:0x0] compound/cookie=0x0/0x6a810265 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000402:0x2c0:0x0] dfid=[0x200000402:0x2c0:0x0] compound/cookie=0x0/0x6a810266 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000402:0x2c1:0x0] dfid=[0x200000402:0x2c1:0x0] compound/cookie=0x0/0x6a810267 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000402:0x2c2:0x0] dfid=[0x200000402:0x2c2:0x0] compound/cookie=0x0/0x6a810268 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000402:0x2c3:0x0] dfid=[0x200000402:0x2c3:0x0] compound/cookie=0x0/0x6a810269 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000402:0x2c4:0x0] dfid=[0x200000402:0x2c4:0x0] compound/cookie=0x0/0x6a81026a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000402:0x2c5:0x0] dfid=[0x200000402:0x2c5:0x0] compound/cookie=0x0/0x6a81026b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000402:0x2c6:0x0] dfid=[0x200000402:0x2c6:0x0] compound/cookie=0x0/0x6a81026c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000402:0x2c7:0x0] dfid=[0x200000402:0x2c7:0x0] compound/cookie=0x0/0x6a81026d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000402:0x2c8:0x0] dfid=[0x200000402:0x2c8:0x0] compound/cookie=0x0/0x6a81026e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000402:0x2c9:0x0] dfid=[0x200000402:0x2c9:0x0] compound/cookie=0x0/0x6a81026f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000402:0x2ca:0x0] dfid=[0x200000402:0x2ca:0x0] compound/cookie=0x0/0x6a810270 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000402:0x2cb:0x0] dfid=[0x200000402:0x2cb:0x0] compound/cookie=0x0/0x6a810271 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000402:0x2cc:0x0] dfid=[0x200000402:0x2cc:0x0] compound/cookie=0x0/0x6a810272 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000402:0x2cd:0x0] dfid=[0x200000402:0x2cd:0x0] compound/cookie=0x0/0x6a810273 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000402:0x2ce:0x0] dfid=[0x200000402:0x2ce:0x0] compound/cookie=0x0/0x6a810274 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000402:0x2cf:0x0] dfid=[0x200000402:0x2cf:0x0] compound/cookie=0x0/0x6a810275 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000402:0x2d0:0x0] dfid=[0x200000402:0x2d0:0x0] compound/cookie=0x0/0x6a810276 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000402:0x2d1:0x0] dfid=[0x200000402:0x2d1:0x0] compound/cookie=0x0/0x6a810277 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=4/10072] fid=[0x200000402:0x2ac:0x0] dfid=[0x200000402:0x2ac:0x0] compound/cookie=0x0/0x6a810252 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10073] fid=[0x200000402:0x2ad:0x0] dfid=[0x200000402:0x2ad:0x0] compound/cookie=0x0/0x6a810253 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10074] fid=[0x200000402:0x2ae:0x0] dfid=[0x200000402:0x2ae:0x0] compound/cookie=0x0/0x6a810254 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10075] fid=[0x200000402:0x2af:0x0] dfid=[0x200000402:0x2af:0x0] compound/cookie=0x0/0x6a810255 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10076] fid=[0x200000402:0x2b0:0x0] dfid=[0x200000402:0x2b0:0x0] compound/cookie=0x0/0x6a810256 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10077] fid=[0x200000402:0x2b1:0x0] dfid=[0x200000402:0x2b1:0x0] compound/cookie=0x0/0x6a810257 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10078] fid=[0x200000402:0x2b2:0x0] dfid=[0x200000402:0x2b2:0x0] compound/cookie=0x0/0x6a810258 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10079] fid=[0x200000402:0x2b3:0x0] dfid=[0x200000402:0x2b3:0x0] compound/cookie=0x0/0x6a810259 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10080] fid=[0x200000402:0x2b4:0x0] dfid=[0x200000402:0x2b4:0x0] compound/cookie=0x0/0x6a81025a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10081] fid=[0x200000402:0x2b5:0x0] dfid=[0x200000402:0x2b5:0x0] compound/cookie=0x0/0x6a81025b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10082] fid=[0x200000402:0x2b6:0x0] dfid=[0x200000402:0x2b6:0x0] compound/cookie=0x0/0x6a81025c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000402:0x2b7:0x0] dfid=[0x200000402:0x2b7:0x0] compound/cookie=0x0/0x6a81025d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000402:0x2b8:0x0] dfid=[0x200000402:0x2b8:0x0] compound/cookie=0x0/0x6a81025e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000402:0x2b9:0x0] dfid=[0x200000402:0x2b9:0x0] compound/cookie=0x0/0x6a81025f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000402:0x2ba:0x0] dfid=[0x200000402:0x2ba:0x0] compound/cookie=0x0/0x6a810260 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000402:0x2bb:0x0] dfid=[0x200000402:0x2bb:0x0] compound/cookie=0x0/0x6a810261 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000402:0x2bc:0x0] dfid=[0x200000402:0x2bc:0x0] compound/cookie=0x0/0x6a810262 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000402:0x2bd:0x0] dfid=[0x200000402:0x2bd:0x0] compound/cookie=0x0/0x6a810263 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000402:0x2be:0x0] dfid=[0x200000402:0x2be:0x0] compound/cookie=0x0/0x6a810264 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000402:0x2bf:0x0] dfid=[0x200000402:0x2bf:0x0] compound/cookie=0x0/0x6a810265 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000402:0x2c0:0x0] dfid=[0x200000402:0x2c0:0x0] compound/cookie=0x0/0x6a810266 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000402:0x2c1:0x0] dfid=[0x200000402:0x2c1:0x0] compound/cookie=0x0/0x6a810267 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000402:0x2c2:0x0] dfid=[0x200000402:0x2c2:0x0] compound/cookie=0x0/0x6a810268 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000402:0x2c3:0x0] dfid=[0x200000402:0x2c3:0x0] compound/cookie=0x0/0x6a810269 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000402:0x2c4:0x0] dfid=[0x200000402:0x2c4:0x0] compound/cookie=0x0/0x6a81026a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000402:0x2c5:0x0] dfid=[0x200000402:0x2c5:0x0] compound/cookie=0x0/0x6a81026b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000402:0x2c6:0x0] dfid=[0x200000402:0x2c6:0x0] compound/cookie=0x0/0x6a81026c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000402:0x2c7:0x0] dfid=[0x200000402:0x2c7:0x0] compound/cookie=0x0/0x6a81026d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000402:0x2c8:0x0] dfid=[0x200000402:0x2c8:0x0] compound/cookie=0x0/0x6a81026e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000402:0x2c9:0x0] dfid=[0x200000402:0x2c9:0x0] compound/cookie=0x0/0x6a81026f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000402:0x2ca:0x0] dfid=[0x200000402:0x2ca:0x0] compound/cookie=0x0/0x6a810270 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000402:0x2cb:0x0] dfid=[0x200000402:0x2cb:0x0] compound/cookie=0x0/0x6a810271 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000402:0x2cc:0x0] dfid=[0x200000402:0x2cc:0x0] compound/cookie=0x0/0x6a810272 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000402:0x2cd:0x0] dfid=[0x200000402:0x2cd:0x0] compound/cookie=0x0/0x6a810273 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000402:0x2ce:0x0] dfid=[0x200000402:0x2ce:0x0] compound/cookie=0x0/0x6a810274 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000402:0x2cf:0x0] dfid=[0x200000402:0x2cf:0x0] compound/cookie=0x0/0x6a810275 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000402:0x2d0:0x0] dfid=[0x200000402:0x2d0:0x0] compound/cookie=0x0/0x6a810276 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000402:0x2d1:0x0] dfid=[0x200000402:0x2d1:0x0] compound/cookie=0x0/0x6a810277 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 4s: from 'lrh=[type=10680000 len=136 idx=4/10072] fid=[0x200000402:0x2ac:0x0] dfid=[0x200000402:0x2ac:0x0] compound/cookie=0x0/0x6a810252 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10073] fid=[0x200000402:0x2ad:0x0] dfid=[0x200000402:0x2ad:0x0] compound/cookie=0x0/0x6a810253 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10074] fid=[0x200000402:0x2ae:0x0] dfid=[0x200000402:0x2ae:0x0] compound/cookie=0x0/0x6a810254 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10075] fid=[0x200000402:0x2af:0x0] dfid=[0x200000402:0x2af:0x0] compound/cookie=0x0/0x6a810255 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10076] fid=[0x200000402:0x2b0:0x0] dfid=[0x200000402:0x2b0:0x0] compound/cookie=0x0/0x6a810256 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10077] fid=[0x200000402:0x2b1:0x0] dfid=[0x200000402:0x2b1:0x0] compound/cookie=0x0/0x6a810257 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10078] fid=[0x200000402:0x2b2:0x0] dfid=[0x200000402:0x2b2:0x0] compound/cookie=0x0/0x6a810258 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10079] fid=[0x200000402:0x2b3:0x0] dfid=[0x200000402:0x2b3:0x0] compound/cookie=0x0/0x6a810259 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10080] fid=[0x200000402:0x2b4:0x0] dfid=[0x200000402:0x2b4:0x0] compound/cookie=0x0/0x6a81025a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10081] fid=[0x200000402:0x2b5:0x0] dfid=[0x200000402:0x2b5:0x0] compound/cookie=0x0/0x6a81025b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10082] fid=[0x200000402:0x2b6:0x0] dfid=[0x200000402:0x2b6:0x0] compound/cookie=0x0/0x6a81025c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000402:0x2b7:0x0] dfid=[0x200000402:0x2b7:0x0] compound/cookie=0x0/0x6a81025d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000402:0x2b8:0x0] dfid=[0x200000402:0x2b8:0x0] compound/cookie=0x0/0x6a81025e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000402:0x2b9:0x0] dfid=[0x200000402:0x2b9:0x0] compound/cookie=0x0/0x6a81025f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000402:0x2ba:0x0] dfid=[0x200000402:0x2ba:0x0] compound/cookie=0x0/0x6a810260 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000402:0x2bb:0x0] dfid=[0x200000402:0x2bb:0x0] compound/cookie=0x0/0x6a810261 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000402:0x2bc:0x0] dfid=[0x200000402:0x2bc:0x0] compound/cookie=0x0/0x6a810262 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000402:0x2bd:0x0] dfid=[0x200000402:0x2bd:0x0] compound/cookie=0x0/0x6a810263 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000402:0x2be:0x0] dfid=[0x200000402:0x2be:0x0] compound/cookie=0x0/0x6a810264 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000402:0x2bf:0x0] dfid=[0x200000402:0x2bf:0x0] compound/cookie=0x0/0x6a810265 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000402:0x2c0:0x0] dfid=[0x200000402:0x2c0:0x0] compound/cookie=0x0/0x6a810266 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000402:0x2c1:0x0] dfid=[0x200000402:0x2c1:0x0] compound/cookie=0x0/0x6a810267 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000402:0x2c2:0x0] dfid=[0x200000402:0x2c2:0x0] compound/cookie=0x0/0x6a810268 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000402:0x2c3:0x0] dfid=[0x200000402:0x2c3:0x0] compound/cookie=0x0/0x6a810269 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000402:0x2c4:0x0] dfid=[0x200000402:0x2c4:0x0] compound/cookie=0x0/0x6a81026a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000402:0x2c5:0x0] dfid=[0x200000402:0x2c5:0x0] compound/cookie=0x0/0x6a81026b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000402:0x2c6:0x0] dfid=[0x200000402:0x2c6:0x0] compound/cookie=0x0/0x6a81026c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000402:0x2c7:0x0] dfid=[0x200000402:0x2c7:0x0] compound/cookie=0x0/0x6a81026d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000402:0x2c8:0x0] dfid=[0x200000402:0x2c8:0x0] compound/cookie=0x0/0x6a81026e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000402:0x2c9:0x0] dfid=[0x200000402:0x2c9:0x0] compound/cookie=0x0/0x6a81026f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000402:0x2ca:0x0] dfid=[0x200000402:0x2ca:0x0] compound/cookie=0x0/0x6a810270 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000402:0x2cb:0x0] dfid=[0x200000402:0x2cb:0x0] compound/cookie=0x0/0x6a810271 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000402:0x2cc:0x0] dfid=[0x200000402:0x2cc:0x0] compound/cookie=0x0/0x6a810272 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000402:0x2cd:0x0] dfid=[0x200000402:0x2cd:0x0] compound/cookie=0x0/0x6a810273 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000402:0x2ce:0x0] dfid=[0x200000402:0x2ce:0x0] compound/cookie=0x0/0x6a810274 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000402:0x2cf:0x0] dfid=[0x200000402:0x2cf:0x0] compound/cookie=0x0/0x6a810275 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000402:0x2d0:0x0] dfid=[0x200000402:0x2d0:0x0] compound/cookie=0x0/0x6a810276 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000402:0x2d1:0x0] dfid=[0x200000402:0x2d1:0x0] compound/cookie=0x0/0x6a810277 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000402:0x2b7:0x0] dfid=[0x200000402:0x2b7:0x0] compound/cookie=0x0/0x6a81025d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000402:0x2b8:0x0] dfid=[0x200000402:0x2b8:0x0] compound/cookie=0x0/0x6a81025e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000402:0x2ba:0x0] dfid=[0x200000402:0x2ba:0x0] compound/cookie=0x0/0x6a810260 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000402:0x2bb:0x0] dfid=[0x200000402:0x2bb:0x0] compound/cookie=0x0/0x6a810261 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000402:0x2bc:0x0] dfid=[0x200000402:0x2bc:0x0] compound/cookie=0x0/0x6a810262 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000402:0x2bd:0x0] dfid=[0x200000402:0x2bd:0x0] compound/cookie=0x0/0x6a810263 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000402:0x2be:0x0] dfid=[0x200000402:0x2be:0x0] compound/cookie=0x0/0x6a810264 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000402:0x2bf:0x0] dfid=[0x200000402:0x2bf:0x0] compound/cookie=0x0/0x6a810265 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000402:0x2c0:0x0] dfid=[0x200000402:0x2c0:0x0] compound/cookie=0x0/0x6a810266 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000402:0x2c1:0x0] dfid=[0x200000402:0x2c1:0x0] compound/cookie=0x0/0x6a810267 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000402:0x2c2:0x0] dfid=[0x200000402:0x2c2:0x0] compound/cookie=0x0/0x6a810268 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000402:0x2c3:0x0] dfid=[0x200000402:0x2c3:0x0] compound/cookie=0x0/0x6a810269 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000402:0x2c4:0x0] dfid=[0x200000402:0x2c4:0x0] compound/cookie=0x0/0x6a81026a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000402:0x2c5:0x0] dfid=[0x200000402:0x2c5:0x0] compound/cookie=0x0/0x6a81026b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000402:0x2c6:0x0] dfid=[0x200000402:0x2c6:0x0] compound/cookie=0x0/0x6a81026c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000402:0x2c7:0x0] dfid=[0x200000402:0x2c7:0x0] compound/cookie=0x0/0x6a81026d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000402:0x2c8:0x0] dfid=[0x200000402:0x2c8:0x0] compound/cookie=0x0/0x6a81026e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000402:0x2c9:0x0] dfid=[0x200000402:0x2c9:0x0] compound/cookie=0x0/0x6a81026f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000402:0x2ca:0x0] dfid=[0x200000402:0x2ca:0x0] compound/cookie=0x0/0x6a810270 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000402:0x2cb:0x0] dfid=[0x200000402:0x2cb:0x0] compound/cookie=0x0/0x6a810271 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000402:0x2cc:0x0] dfid=[0x200000402:0x2cc:0x0] compound/cookie=0x0/0x6a810272 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000402:0x2cd:0x0] dfid=[0x200000402:0x2cd:0x0] compound/cookie=0x0/0x6a810273 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000402:0x2ce:0x0] dfid=[0x200000402:0x2ce:0x0] compound/cookie=0x0/0x6a810274 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000402:0x2cf:0x0] dfid=[0x200000402:0x2cf:0x0] compound/cookie=0x0/0x6a810275 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000402:0x2d0:0x0] dfid=[0x200000402:0x2d0:0x0] compound/cookie=0x0/0x6a810276 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000402:0x2d1:0x0] dfid=[0x200000402:0x2d1:0x0] compound/cookie=0x0/0x6a810277 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 6s: from 'lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000402:0x2b7:0x0] dfid=[0x200000402:0x2b7:0x0] compound/cookie=0x0/0x6a81025d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000402:0x2b8:0x0] dfid=[0x200000402:0x2b8:0x0] compound/cookie=0x0/0x6a81025e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000402:0x2ba:0x0] dfid=[0x200000402:0x2ba:0x0] compound/cookie=0x0/0x6a810260 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000402:0x2bb:0x0] dfid=[0x200000402:0x2bb:0x0] compound/cookie=0x0/0x6a810261 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000402:0x2bc:0x0] dfid=[0x200000402:0x2bc:0x0] compound/cookie=0x0/0x6a810262 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000402:0x2bd:0x0] dfid=[0x200000402:0x2bd:0x0] compound/cookie=0x0/0x6a810263 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000402:0x2be:0x0] dfid=[0x200000402:0x2be:0x0] compound/cookie=0x0/0x6a810264 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000402:0x2bf:0x0] dfid=[0x200000402:0x2bf:0x0] compound/cookie=0x0/0x6a810265 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000402:0x2c0:0x0] dfid=[0x200000402:0x2c0:0x0] compound/cookie=0x0/0x6a810266 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000402:0x2c1:0x0] dfid=[0x200000402:0x2c1:0x0] compound/cookie=0x0/0x6a810267 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000402:0x2c2:0x0] dfid=[0x200000402:0x2c2:0x0] compound/cookie=0x0/0x6a810268 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000402:0x2c3:0x0] dfid=[0x200000402:0x2c3:0x0] compound/cookie=0x0/0x6a810269 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000402:0x2c4:0x0] dfid=[0x200000402:0x2c4:0x0] compound/cookie=0x0/0x6a81026a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000402:0x2c5:0x0] dfid=[0x200000402:0x2c5:0x0] compound/cookie=0x0/0x6a81026b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000402:0x2c6:0x0] dfid=[0x200000402:0x2c6:0x0] compound/cookie=0x0/0x6a81026c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000402:0x2c7:0x0] dfid=[0x200000402:0x2c7:0x0] compound/cookie=0x0/0x6a81026d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000402:0x2c8:0x0] dfid=[0x200000402:0x2c8:0x0] compound/cookie=0x0/0x6a81026e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000402:0x2c9:0x0] dfid=[0x200000402:0x2c9:0x0] compound/cookie=0x0/0x6a81026f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000402:0x2ca:0x0] dfid=[0x200000402:0x2ca:0x0] compound/cookie=0x0/0x6a810270 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000402:0x2cb:0x0] dfid=[0x200000402:0x2cb:0x0] compound/cookie=0x0/0x6a810271 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000402:0x2cc:0x0] dfid=[0x200000402:0x2cc:0x0] compound/cookie=0x0/0x6a810272 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000402:0x2cd:0x0] dfid=[0x200000402:0x2cd:0x0] compound/cookie=0x0/0x6a810273 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000402:0x2ce:0x0] dfid=[0x200000402:0x2ce:0x0] compound/cookie=0x0/0x6a810274 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000402:0x2cf:0x0] dfid=[0x200000402:0x2cf:0x0] compound/cookie=0x0/0x6a810275 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000402:0x2d0:0x0] dfid=[0x200000402:0x2d0:0x0] compound/cookie=0x0/0x6a810276 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000402:0x2d1:0x0] dfid=[0x200000402:0x2d1:0x0] compound/cookie=0x0/0x6a810277 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000402:0x2c6:0x0] dfid=[0x200000402:0x2c6:0x0] compound/cookie=0x0/0x6a81026c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000402:0x2c7:0x0] dfid=[0x200000402:0x2c7:0x0] compound/cookie=0x0/0x6a81026d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000402:0x2c8:0x0] dfid=[0x200000402:0x2c8:0x0] compound/cookie=0x0/0x6a81026e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000402:0x2c9:0x0] dfid=[0x200000402:0x2c9:0x0] compound/cookie=0x0/0x6a81026f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000402:0x2ca:0x0] dfid=[0x200000402:0x2ca:0x0] compound/cookie=0x0/0x6a810270 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000402:0x2cb:0x0] dfid=[0x200000402:0x2cb:0x0] compound/cookie=0x0/0x6a810271 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000402:0x2cc:0x0] dfid=[0x200000402:0x2cc:0x0] compound/cookie=0x0/0x6a810272 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000402:0x2cd:0x0] dfid=[0x200000402:0x2cd:0x0] compound/cookie=0x0/0x6a810273 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000402:0x2ce:0x0] dfid=[0x200000402:0x2ce:0x0] compound/cookie=0x0/0x6a810274 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000402:0x2cf:0x0] dfid=[0x200000402:0x2cf:0x0] compound/cookie=0x0/0x6a810275 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000402:0x2d0:0x0] dfid=[0x200000402:0x2d0:0x0] compound/cookie=0x0/0x6a810276 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000402:0x2d1:0x0] dfid=[0x200000402:0x2d1:0x0] compound/cookie=0x0/0x6a810277 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 PASS 90 (92s) == sanity-hsm test 100: Set coordinator /proc tunables === 09:46:13 (1786801573) Testing mdt.lustre-MDT0000.hsm.loop_period Set loop_period to 0 must failed oleg243-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/loop_period=0: Invalid argument oleg243-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/loop_period'='0': Invalid argument pdsh@oleg243-client: oleg243-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.grace_delay Set grace_delay to 0 must failed oleg243-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/grace_delay=0: Invalid argument oleg243-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/grace_delay'='0': Invalid argument pdsh@oleg243-client: oleg243-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.active_request_timeout Set active_request_timeout to 0 must failed oleg243-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/active_request_timeout=0: Invalid argument oleg243-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/active_request_timeout'='0': Invalid argument pdsh@oleg243-client: oleg243-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.max_requests Set max_requests to 0 must failed oleg243-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=0: Invalid argument oleg243-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='0': Invalid argument pdsh@oleg243-client: oleg243-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.default_archive_id Set default_archive_id to 0 must failed oleg243-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/default_archive_id=0: Invalid argument oleg243-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/default_archive_id'='0': Invalid argument pdsh@oleg243-client: oleg243-server: ssh exited with exit code 22 PASS 100 (30s) == sanity-hsm test 102: Verify coordinator control ======= 09:46:43 (1786801603) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled PASS 102 (13s) == sanity-hsm test 103: Purge all requests =============== 09:46:56 (1786801616) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge Current requests pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 PASS 103 (12s) == sanity-hsm test 103a: Purge pending restore requests == 09:47:08 (1786801628) mdt.lustre-MDT0000.hsm.policy=-NBR Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' (0x200000402:0x319:0x0|0x200000402:0x31a:0x0|0x200000402:0x31b:0x0|0x200000402:0x31c:0x0|0x200000402:0x31d:0x0|0x200000402:0x31e:0x0|0x200000402:0x31f:0x0|0x200000402:0x320:0x0).*action=ARCHIVE.*status=SUCCEED Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s cat: '/mnt/lustre/.lustre/fid/0x200000402:0x31c:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x31a:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x31b:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x31d:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x31f:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x31e:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x319:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x320:0x0': No data available mdt.lustre-MDT0000.hsm_control=purge PASS 103a (17s) == sanity-hsm test 104: Copy tool data field ============= 09:47:25 (1786801645) 0+0 records in 0+0 records out 0 bytes copied, 0.0117207 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=purge PASS 104 (10s) == sanity-hsm test 105: Restart of coordinator =========== 09:47:35 (1786801655) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=purge PASS 105 (22s) == sanity-hsm test 106a: Copytool register/unregister ==== 09:47:58 (1786801678) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg243-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg243-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=3e4300d8-c80c-4b65-a617-ce9375c889c0 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 oleg243-client.virtnet: 87090 Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 uuid not found in agent list on mds1 Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg243-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg243-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=3e4300d8-c80c-4b65-a617-ce9375c889c0 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 PASS 106a (32s) == sanity-hsm test 106b: Unregister agent after a copytool crash ========================================================== 09:48:29 (1786801709) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg243-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg243-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=3e4300d8-c80c-4b65-a617-ce9375c889c0 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 oleg243-client.virtnet: 88783 Killing existing copytools on oleg243-client.virtnet with signal KILL copytools stopped in 0s pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 uuid not found in agent list on mds1 Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' found agent uuid=3e4300d8-c80c-4b65-a617-ce9375c889c0 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 PASS 106b (19s) == sanity-hsm test 106c: Unregister copytool on client eviction ========================================================== 09:48:48 (1786801728) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg243-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg243-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=3e4300d8-c80c-4b65-a617-ce9375c889c0 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 oleg243-client.virtnet: 89977 oleg243-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg243-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg243-client.virtnet: executing get_client_uuid /mnt/lustre2 mdc.lustre-MDT0000-mdc-ffff9d738319d000.active=0 mdt.lustre-MDT0000.evict_client=3e4300d8-c80c-4b65-a617-ce9375c889c0 pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 uuid not found in agent list on mds1 mdc.lustre-MDT0000-mdc-ffff9d738319d000.active=1 oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec found agent uuid=3e4300d8-c80c-4b65-a617-ce9375c889c0 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 mdc.lustre-MDT0000-mdc-ffff9d738319d000.active=1 PASS 106c (43s) == sanity-hsm test 107: Copytool re-register after MDS restart ========================================================== 09:49:31 (1786801771) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_107/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Failing mds1 on oleg243-server Stopping /mnt/lustre-mds1 (opts:) on oleg243-server 09:49:39 (1786801779) shut down facet: mds1 facet_host: oleg243-server facet_failover_host: oleg243-server Failover mds1 to oleg243-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg243-server: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Started lustre-MDT0000 09:50:01 (1786801801) targets are mounted 09:50:01 (1786801801) facet_failover done oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 107 (49s) == sanity-hsm test 109: Policy display/change ============ 09:50:21 (1786801821) mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0000.hsm.policy=NRA NBR Next set_param must failed oleg243-server: error: set_param: setting /sys/kernel/debug/lustre/mdt/lustre-MDT0000/hsm/policy=wrong: Invalid argument oleg243-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/policy'='wrong': Invalid argument pdsh@oleg243-client: oleg243-server: ssh exited with exit code 22 Back to default policy mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR PASS 109 (26s) == sanity-hsm test 110a: Non blocking restore policy (import case) ========================================================== 09:50:47 (1786801847) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786801850.115304 [94486]: action=1 src=d110a.sanity-hsm/f110a.sanity-hsm dst=/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786801850.133388 [94486]: importing '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1786801850.237308 [94486]: imported '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//0bba/0x200000bd2:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1786801850.237373 [94486]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NBR md5sum: /mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm: No data available mdt.lustre-MDT0000.hsm.policy=-NBR PASS 110a (15s) == sanity-hsm test 110b: Non blocking restore policy (release case) ========================================================== 09:51:03 (1786801863) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NBR md5sum: /mnt/lustre/d110b.sanity-hsm/f110b.sanity-hsm: No data available mdt.lustre-MDT0000.hsm.policy=-NBR PASS 110b (13s) == sanity-hsm test 111a: No retry policy (import case), restore will error ========================================================== 09:51:17 (1786801877) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786801880.193122 [95827]: action=1 src=d111a.sanity-hsm/f111a.sanity-hsm dst=/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786801880.202540 [95827]: importing '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1786801880.263916 [95827]: imported '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//0bbf/0x200000bd2:0x6d:0x0'=='/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1786801880.272420 [95827]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NRA Remove from backend: 0x200000bd2:0x6d:0x0 = /tmp/arc1/sanity-hsm.test_111a/0bbf/0x200000bd2:0x6d:0x0 mdt.lustre-MDT0000.hsm.policy=-NRA PASS 111a (13s) == sanity-hsm test 111b: No retry policy (release case), restore will error ========================================================== 09:51:30 (1786801890) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA Remove from backend: 0x200000bd3:0x7:0x0 = /tmp/arc1/sanity-hsm.test_111b/0bd4/0x200000bd3:0x7:0x0 mdt.lustre-MDT0000.hsm.policy=-NRA PASS 111b (11s) == sanity-hsm test 112: State of recorded request ======== 09:51:42 (1786801902) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_112/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled /mnt/lustre/d112.sanity-hsm/f112.sanity-hsm: ARCHIVE waiting (from 0 to EOF) mdt.lustre-MDT0000.hsm_control=enabled PASS 112 (12s) == sanity-hsm test 113: wrong stat after restore ========= 09:51:54 (1786801914) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0670193 s, 15.6 MB/s Starting client: oleg243-client.virtnet: -o user_xattr,flock 192.168.202.143@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_113/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Verifying released state: multiop /mnt/lustre/d113.sanity-hsm/f113.sanity-hsm voO_WRONLY:O_APPEND:_w4c TMPPIPE=/tmp/multiop_open_wait_pipe.7594 File: /mnt/lustre2/d113.sanity-hsm/f113.sanity-hsm Size: 1048576 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115238860488715 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-15 09:51:56.000000000 -0400 Modify: 2026-08-15 09:51:56.000000000 -0400 Change: 2026-08-15 09:51:56.000000000 -0400 Birth: 2026-08-15 09:51:56.000000000 -0400 Stopping client oleg243-client.virtnet /mnt/lustre3 (opts:) PASS 113 (17s) == sanity-hsm test 114: Incompatible request does not set other requests as STARTED ========================================================== 09:52:11 (1786801931) 0+0 records in 0+0 records out 0 bytes copied, 0.0050373 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.0270072 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_114/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled PASS 114 (18s) == sanity-hsm test 200: Register/Cancel archive ========== 09:52:30 (1786801950) 0+0 records in 0+0 records out 0 bytes copied, 0.00386973 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_200/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet PASS 200 (15s) == sanity-hsm test 201: Register/Cancel restore ========== 09:52:44 (1786801964) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_201/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.972421 s, 42.1 MB/s lhsmtool_posix: 1786801968.568173 [100056]: action=1 src=d201.sanity-hsm/f201.sanity-hsm dst=/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786801968.576691 [100056]: importing '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1786801968.626722 [100056]: imported '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//0ba5/0x200000bd2:0x77:0x0'=='/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1786801968.626792 [100056]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled PASS 201 (18s) == sanity-hsm test 202: Register/Cancel remove =========== 09:53:02 (1786801982) 0+0 records in 0+0 records out 0 bytes copied, 0.0145716 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_202/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg243-client.virtnet PASS 202 (12s) == sanity-hsm test 220A: Changelog for archive =========== 09:53:14 (1786801994) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' lustre-MDT0000.3 16HSM 13:53:20.324376893 2026.08.15 0x0 t=[0x200000bd3:0x14:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 220A (17s) == sanity-hsm test 220a: Changelog for failed archive ==== 09:53:32 (1786802012) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl2' Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet lustre-MDT0000.6 16HSM 13:53:40.083139066 2026.08.15 0x2 t=[0x200000bd3:0x15:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 220a (21s) == sanity-hsm test 221: Changelog for archive canceled === 09:53:52 (1786802032) 0+0 records in 0+0 records out 0 bytes copied, 0.00810363 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_221/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl3' Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet lustre-MDT0000.8 16HSM 13:54:01.057855734 2026.08.15 0x7d t=[0x200000bd3:0x17:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 221 (18s) == sanity-hsm test 222a: Changelog for explicit restore == 09:54:10 (1786802050) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786802053.125418 [103614]: action=1 src=d222a.sanity-hsm/f222a.sanity-hsm dst=/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786802053.131930 [103614]: importing '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1786802053.197377 [103614]: imported '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//0bac/0x200000bd2:0x7e:0x0'=='/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1786802053.197470 [103614]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl4' lustre-MDT0000.11 16HSM 13:54:16.982285883 2026.08.15 0x80 t=[0x200000bd2:0x7e:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 222a (16s) == sanity-hsm test 222b: Changelog for implicit restore == 09:54:26 (1786802066) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl5' 4ef076ad20e5d4a2f4729d6ed41a3bf9 /mnt/lustre/d222b.sanity-hsm/f222b.sanity-hsm lustre-MDT0000.20 16HSM 13:54:35.159622887 2026.08.15 0x80 t=[0x200000bd3:0x19:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 PASS 222b (21s) == sanity-hsm test 222c: Changelog for failed explicit restore ========================================================== 09:54:47 (1786802087) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1786802090.571821 [105089]: action=1 src=d222c.sanity-hsm/f222c.sanity-hsm dst=/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786802090.594312 [105089]: importing '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1786802090.707896 [105089]: imported '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//0b51/0x200000bd2:0x83:0x0'=='/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1786802090.718076 [105089]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl6' Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet lustre-MDT0000.22 16HSM 13:54:55.850406803 2026.08.15 0x82 t=[0x200000bd2:0x83:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 222c (21s) == sanity-hsm test 222d: Changelog for failed implicit restore ========================================================== 09:55:08 (1786802108) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl7' Remove from backend: 0x200000bd3:0x1b:0x0 = /tmp/arc1/sanity-hsm.test_222d/0bc8/0x200000bd3:0x1b:0x0 md5sum: /mnt/lustre/d222d.sanity-hsm/f222d.sanity-hsm: No data available lustre-MDT0000.29 16HSM 13:55:16.662892451 2026.08.15 0x82 t=[0x200000bd3:0x1b:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x200000bd2:0x81:0x0] lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 222d (19s) == sanity-hsm test 223a: Changelog for restore canceled (import case) ========================================================== 09:55:27 (1786802127) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.84367 s, 22.2 MB/s mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl8' lhsmtool_posix: 1786802135.229664 [106645]: action=1 src=d223a.sanity-hsm/f223a.sanity-hsm dst=/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1786802135.239719 [106645]: importing '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1786802135.287007 [106645]: imported '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//0b5a/0x200000bd2:0x88:0x0'=='/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1786802135.287078 [106645]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Waiting 200s for 'SUCCEED' Waiting 180s for 'SUCCEED' Updated after 28s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.36 16HSM 13:56:04.745143652 2026.08.15 0xfd t=[0x200000bd2:0x88:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000: clear the changelog for cl8 of all records lustre-MDT0000: Deregistered changelog user #8 PASS 223a (49s) == sanity-hsm test 223b: Changelog for restore canceled (release case) ========================================================== 09:56:16 (1786802176) 0+0 records in 0+0 records out 0 bytes copied, 0.0105728 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl9' Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet lustre-MDT0000.43 16HSM 13:56:26.699465268 2026.08.15 0xfd t=[0x200000bd3:0x1f:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x200000bd2:0x7a:0x0] lustre-MDT0000: clear the changelog for cl9 of all records lustre-MDT0000: Deregistered changelog user #9 PASS 223b (21s) == sanity-hsm test 224A: Changelog for remove ============ 09:56:37 (1786802197) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl10' lustre-MDT0000.48 16HSM 13:56:45.426942006 2026.08.15 0x200 t=[0x200000bd3:0x21:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x200000bd2:0x86:0x0] lustre-MDT0000: clear the changelog for cl10 of all records lustre-MDT0000: Deregistered changelog user #10 PASS 224A (20s) == sanity-hsm test 224a: Changelog for failed remove ===== 09:56:57 (1786802217) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl11' Remove from backend: 0x200000bd3:0x22:0x0 = /tmp/arc1/sanity-hsm.test_224a/0bf1/0x200000bd3:0x22:0x0 Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet lustre-MDT0000.53 16HSM 13:57:05.535313437 2026.08.15 0x202 t=[0x200000bd3:0x22:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x200000bd2:0x8c:0x0] lustre-MDT0000: clear the changelog for cl11 of all records lustre-MDT0000: Deregistered changelog user #11 PASS 224a (19s) == sanity-hsm test 225: Changelog for remove canceled ==== 09:57:16 (1786802236) Test disabled PASS 225 (8s) == sanity-hsm test 226: changelog for last rm/mv with exiting archive ========================================================== 09:57:24 (1786802244) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_226/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0x200000bd3:0x25:0x0 mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl12' lustre-MDT0000.54 16HSM 13:57:30.114940305 2026.08.15 0x280 t=[0x200000bd3:0x23:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.55 16HSM 13:57:30.318642583 2026.08.15 0x280 t=[0x200000bd3:0x23:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x200000bd2:0x86:0x0] lustre-MDT0000.56 16HSM 13:57:30.318810963 2026.08.15 0x0 t=[0x200000bd3:0x23:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x200000bd2:0x86:0x0] lustre-MDT0000.57 16HSM 13:57:31.332619807 2026.08.15 0x280 t=[0x200000bd3:0x24:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.58 16HSM 13:57:31.483073790 2026.08.15 0x280 t=[0x200000bd3:0x24:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x200000bd2:0x86:0x0] lustre-MDT0000.59 16HSM 13:57:31.483310623 2026.08.15 0x0 t=[0x200000bd3:0x24:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x200000bd2:0x86:0x0] lustre-MDT0000.60 06UNLNK 13:57:32.337642142 2026.08.15 0x3 t=[0x200000bd3:0x23:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x200000bd2:0x8d:0x0] f226.sanity-hsm-1 lustre-MDT0000.60 06UNLNK 13:57:32.337642142 2026.08.15 0x3 t=[0x200000bd3:0x23:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x200000bd2:0x8d:0x0] f226.sanity-hsm-1 lustre-MDT0000.61 08RENME 13:57:32.875846958 2026.08.15 0x3 t=[0x200000bd3:0x24:0x0] j=mv.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x200000bd2:0x8d:0x0] f226.sanity-hsm-2 s=[0x200000bd3:0x25:0x0] sp=[0x200000bd2:0x8d:0x0] f226.sanity-hsm-3 lustre-MDT0000: clear the changelog for cl12 of all records lustre-MDT0000: Deregistered changelog user #12 PASS 226 (18s) == sanity-hsm test 227: changelog when explicit setting of HSM flags ========================================================== 09:57:43 (1786802263) 0+0 records in 0+0 records out 0 bytes copied, 0.0349969 s, 0.0 kB/s mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl13' lustre-MDT0000: clear the changelog for cl13 of all records lustre-MDT0000: Deregistered changelog user #13 PASS 227 (19s) == sanity-hsm test 228: On released file, return extend to FIEMAP. For [cp,tar] --sparse ========================================================== 09:58:01 (1786802281) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_228/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936586 s, 11.2 MB/s /mnt/lustre/f228.sanity-hsm: 1 extent found tar: Removing leading `/' from member names mnt/lustre/f228.sanity-hsm PASS 228 (14s) == sanity-hsm test 250: Coordinator max request ========== 09:58:15 (1786802295) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_250/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.0159095 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00575421 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00633749 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00537503 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00654921 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00983734 s, 0.0 kB/s Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet PASS 250 (26s) == sanity-hsm test 251: Coordinator request timeout ====== 09:58:41 (1786802321) 0+0 records in 0+0 records out 0 bytes copied, 0.00588771 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_251/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg243-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Waiting 200s for 'CANCELED' Updated after 2s: want 'CANCELED' got 'CANCELED' PASS 251 (28s) == sanity-hsm test 252: Timeout'ed running archive of a removed file should be canceled ========================================================== 09:59:09 (1786802349) 0+0 records in 0+0 records out 0 bytes copied, 0.0074692 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_252/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet PASS 252 (18s) == sanity-hsm test 253: Check for wrong file size after release ========================================================== 09:59:27 (1786802367) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_253/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 10+0 records in 10+0 records out 10000000 bytes (10 MB, 9.5 MiB) copied, 0.255361 s, 39.2 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' fail_loc=0x807 Cannot send HSM request (use of /mnt/lustre/d253.sanity-hsm/f253.sanity-hsm): Invalid argument could not release file PASS 253 (11s) == sanity-hsm test 254a: Request counters are initialized to zero ========================================================== 09:59:39 (1786802379) PASS 254a (9s) == sanity-hsm test 254b: Request counters are correctly incremented and decremented ========================================================== 09:59:48 (1786802388) Will launch 57 requests of each type Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_254b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Checking archive requests Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet Checking restore requests Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet Checking remove requests Copytool is suspended on oleg243-client.virtnet Copytool is continued on oleg243-client.virtnet PASS 254b (48s) == sanity-hsm test 255: Copytool registration wakes the coordinator up ========================================================== 10:00:36 (1786802436) 0+0 records in 0+0 records out 0 bytes copied, 0.00720858 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s 0+0 records in 0+0 records out 0 bytes copied, 0.00971318 s, 0.0 kB/s 0x200000bd3:0xad:0x0 mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 255 (23s) == sanity-hsm test 256: HSM copytool register on DNE with stopped MDT must not panic ========================================================== 10:00:59 (1786802459) SKIP: sanity-hsm test_256 needs DNE (>= 2 MDTs) SKIP 256 (3s) == sanity-hsm test 260a: Restore request have priority over other requests ========================================================== 10:01:02 (1786802462) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252838 s, 4.1 MB/s 0x200000bd3:0xaf:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1662 s, 6.3 MB/s 0x200000bd3:0xb0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129583 s, 8.1 MB/s 0x200000bd3:0xb1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139838 s, 7.5 MB/s 0x200000bd3:0xb2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108001 s, 9.7 MB/s 0x200000bd3:0xb3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156029 s, 6.7 MB/s 0x200000bd3:0xb4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110044 s, 9.5 MB/s 0x200000bd3:0xb5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100384 s, 10.4 MB/s 0x200000bd3:0xb6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11087 s, 9.5 MB/s 0x200000bd3:0xb7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126471 s, 8.3 MB/s 0x200000bd3:0xb8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124295 s, 8.4 MB/s 0x200000bd3:0xb9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121263 s, 8.6 MB/s 0x200000bd3:0xba:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120411 s, 8.7 MB/s 0x200000bd3:0xbb:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132352 s, 7.9 MB/s 0x200000bd3:0xbc:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11292 s, 9.3 MB/s 0x200000bd3:0xbd:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162714 s, 6.4 MB/s 0x200000bd3:0xbe:0x0 Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260a (34s) == sanity-hsm test 260b: Restore request have priority over other requests ========================================================== 10:01:36 (1786802496) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255898 s, 4.1 MB/s 0x200000bd3:0xc1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276407 s, 3.8 MB/s 0x200000bd3:0xc2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137919 s, 7.6 MB/s 0x200000bd3:0xc3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137512 s, 7.6 MB/s 0x200000bd3:0xc4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139761 s, 7.5 MB/s 0x200000bd3:0xc5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206768 s, 5.1 MB/s 0x200000bd3:0xc6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180455 s, 5.8 MB/s 0x200000bd3:0xc7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317104 s, 3.3 MB/s 0x200000bd3:0xc8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199162 s, 5.3 MB/s 0x200000bd3:0xc9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134236 s, 7.8 MB/s 0x200000bd3:0xca:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.345135 s, 3.0 MB/s 0x200000bd3:0xcb:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157379 s, 6.7 MB/s 0x200000bd3:0xcc:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158397 s, 6.6 MB/s 0x200000bd3:0xcd:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.274517 s, 3.8 MB/s 0x200000bd3:0xce:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159333 s, 6.6 MB/s 0x200000bd3:0xcf:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.283335 s, 3.7 MB/s 0x200000bd3:0xd0:0x0 Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260b (38s) == sanity-hsm test 260c: Requests are not reordered on the 'hot' path of the coordinator ========================================================== 10:02:14 (1786802534) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278404 s, 3.8 MB/s 0x200000bd3:0xd3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20648 s, 5.1 MB/s 0x200000bd3:0xd4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255843 s, 4.1 MB/s 0x200000bd3:0xd5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.246252 s, 4.3 MB/s 0x200000bd3:0xd6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194458 s, 5.4 MB/s 0x200000bd3:0xd7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224457 s, 4.7 MB/s 0x200000bd3:0xd8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160231 s, 6.5 MB/s 0x200000bd3:0xd9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201347 s, 5.2 MB/s 0x200000bd3:0xda:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159648 s, 6.6 MB/s 0x200000bd3:0xdb:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306927 s, 3.4 MB/s 0x200000bd3:0xdc:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252088 s, 4.2 MB/s 0x200000bd3:0xdd:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13598 s, 7.7 MB/s 0x200000bd3:0xde:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13871 s, 7.6 MB/s 0x200000bd3:0xdf:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152617 s, 6.9 MB/s 0x200000bd3:0xe0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160592 s, 6.5 MB/s 0x200000bd3:0xe1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141461 s, 7.4 MB/s 0x200000bd3:0xe2:0x0 Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE RESTORE PASS 260c (39s) == sanity-hsm test 261: Report 0 bytes size after HSM release ========================================================== 10:02:53 (1786802573) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_261/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0126526 s, 647 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x00000009) exists archived, archive_id:2 Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x0000000d) released exists archived, archive_id:2 PASS 261 (17s) == sanity-hsm test 262: The client should return 1 block for HSM released files ========================================================== 10:03:10 (1786802590) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_262/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0165454 s, 495 kB/s PASS 262 (9s) == sanity-hsm test 300: On disk coordinator state kept between MDT umount/mount ========================================================== 10:03:19 (1786802599) Stop coordinator and remove coordinator state at mount mdt.lustre-MDT0000.hsm_control=shutdown Failing mds1 on oleg243-server Stopping /mnt/lustre-mds1 (opts:) on oleg243-server 10:03:29 (1786802609) shut down facet: mds1 facet_host: oleg243-server facet_failover_host: oleg243-server Failover mds1 to oleg243-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg243-server: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Started lustre-MDT0000 10:03:51 (1786802631) targets are mounted 10:03:51 (1786802631) facet_failover done oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Set coordinator start at mount, and start coordinator Failing mds1 on oleg243-server Stopping /mnt/lustre-mds1 (opts:) on oleg243-server 10:04:32 (1786802672) shut down facet: mds1 facet_host: oleg243-server facet_failover_host: oleg243-server Failover mds1 to oleg243-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg243-server: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Started lustre-MDT0000 10:05:03 (1786802703) targets are mounted 10:05:03 (1786802703) facet_failover done oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 300 (130s) == sanity-hsm test 301: HSM tunnable are persistent ====== 10:05:30 (1786802730) Failing mds1 on oleg243-server Stopping /mnt/lustre-mds1 (opts:) on oleg243-server 10:05:38 (1786802738) shut down facet: mds1 facet_host: oleg243-server facet_failover_host: oleg243-server Failover mds1 to oleg243-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg243-server: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Started lustre-MDT0000 10:06:09 (1786802769) targets are mounted 10:06:09 (1786802769) facet_failover done oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 301 (54s) == sanity-hsm test 302: HSM tunnable are persistent when CDT is off ========================================================== 10:06:24 (1786802784) mdt.lustre-MDT0000.hsm_control=shutdown Failing mds1 on oleg243-server Stopping /mnt/lustre-mds1 (opts:) on oleg243-server 10:06:34 (1786802794) shut down facet: mds1 facet_host: oleg243-server facet_failover_host: oleg243-server Failover mds1 to oleg243-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg243-server: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Started lustre-MDT0000 10:06:56 (1786802816) targets are mounted 10:06:56 (1786802816) facet_failover done oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 302 (51s) == sanity-hsm test 400: Single request is sent to the right MDT ========================================================== 10:07:16 (1786802836) SKIP: sanity-hsm test_400 needs >= 2 MDTs SKIP 400 (4s) == sanity-hsm test 401: Compound requests split and sent to their respective MDTs ========================================================== 10:07:19 (1786802839) SKIP: sanity-hsm test_401 needs >= 2 MDTs SKIP 401 (4s) == sanity-hsm test 402a: Copytool start fails if all MDTs are inactive ========================================================== 10:07:23 (1786802843) deactivate lustre-MDT0000-mdc-ffff9d73841f0000 on oleg243-client.virtnet deactivate lustre-MDT0000-mdc-ffff9d738319d000 on oleg243-client.virtnet Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Failed to start copytool agt1 on 'oleg243-client.virtnet' pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 uuid not found in agent list on mds1 activate lustre-MDT0000-mdc-ffff9d73841f0000 on oleg243-client.virtnet activate lustre-MDT0000-mdc-ffff9d738319d000 on oleg243-client.virtnet PASS 402a (9s) == sanity-hsm test 402b: CDT must retry request upon slow start of CT ========================================================== 10:07:32 (1786802852) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x14d fail_loc=0 Waiting 200s for 'SUCCEED' Updated after 6s: want 'SUCCEED' got 'SUCCEED' PASS 402b (26s) == sanity-hsm test 403: Copytool starts with inactive MDT and register on reconnect ========================================================== 10:07:58 (1786802878) SKIP: sanity-hsm test_403 needs >= 2 MDTs SKIP 403 (4s) == sanity-hsm test 404: Inactive MDT does not block requests for active MDTs ========================================================== 10:08:02 (1786802882) SKIP: sanity-hsm test_404 needs >= 2 MDTs SKIP 404 (3s) == sanity-hsm test 405: archive and release under striped directory ========================================================== 10:08:05 (1786802885) SKIP: sanity-hsm test_405 needs >= 2 MDTs SKIP 405 (3s) == sanity-hsm test 406: attempting to migrate HSM archived files is safe ========================================================== 10:08:08 (1786802888) SKIP: sanity-hsm test_406 needs >= 2 MDTs SKIP 406 (4s) == sanity-hsm test 407: Check for double RESTORE records in llog ========================================================== 10:08:12 (1786802892) 0+0 records in 0+0 records out 0 bytes copied, 0.00276817 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_407/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' fail_val=5 fail_loc=0x164 Copytool is suspended on oleg243-client.virtnet mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/1] fid=[0x2000032e1:0x2:0x0] dfid=[0x2000032e1:0x2:0x0] compound/cookie=0x0/0x6a80727b action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/2] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x6a80727c action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] Failing mds1 on oleg243-server Stopping /mnt/lustre-mds1 (opts:) on oleg243-server 10:08:25 (1786802905) shut down facet: mds1 facet_host: oleg243-server facet_failover_host: oleg243-server Failover mds1 to oleg243-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 md5sum: /mnt/lustre/d407.sanity-hsm/f407.sanity-hsm: No data available md5sum: /mnt/lustre2/d407.sanity-hsm/f407.sanity-hsm: No data available oleg243-server: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Started lustre-MDT0000 10:08:50 (1786802930) targets are mounted 10:08:50 (1786802930) facet_failover done mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/1] fid=[0x2000032e1:0x2:0x0] dfid=[0x2000032e1:0x2:0x0] compound/cookie=0x0/0x6a80727b action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/2] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x6a80727c action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/3] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x6a80727d action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec fail_loc=0 mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/2] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x6a80727c action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/3] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x6a80727d action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] Copytool is continued on oleg243-client.virtnet pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 PASS 407 (57s) == sanity-hsm test 408: Verify fiemap on release file ==== 10:09:09 (1786802949) Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_408/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.162766 s, 403 kB/s SKIP: sanity-hsm test_408 ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 408 (4s) == sanity-hsm test 409a: Coordinator should not stop when in use ========================================================== 10:09:13 (1786802953) 0+0 records in 0+0 records out 0 bytes copied, 0.00763704 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_val=5 fail_loc=0x164 cat: /mnt/lustre/d409a.sanity-hsm/f409a.sanity-hsm: No data available mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' mdt.lustre-MDT0000.hsm_control=enabled PASS 409a (25s) == sanity-hsm test 409b: getattr released file with CDT stopped after remount ========================================================== 10:09:38 (1786802978) 0+0 records in 0+0 records out 0 bytes copied, 0.00608417 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s Failing mds1 on oleg243-server Stopping /mnt/lustre-mds1 (opts:) on oleg243-server 10:10:07 (1786803007) shut down facet: mds1 facet_host: oleg243-server facet_failover_host: oleg243-server Failover mds1 to oleg243-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg243-server: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Started lustre-MDT0000 10:10:29 (1786803029) targets are mounted 10:10:29 (1786803029) facet_failover done oleg243-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec File: /mnt/lustre/d409b.sanity-hsm/f409b.sanity-hsm Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115406615871496 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-15 10:09:39.000000000 -0400 Modify: 2026-08-15 10:09:39.000000000 -0400 Change: 2026-08-15 10:09:39.000000000 -0400 Birth: 2026-08-15 10:09:39.000000000 -0400 PASS 409b (88s) == sanity-hsm test 410: lfs data_version -s allows release of force-archived file ========================================================== 10:11:06 (1786803066) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154509 s, 6.8 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_410/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d410.sanity-hsm/f410.sanity-hsm): Operation not permitted PASS 410 (8s) == sanity-hsm test 411: hsm_ops rbac role ================ 10:11:14 (1786803074) On MGS 192.168.202.143, active = nodemap.active=1 On MGS 192.168.202.143, test_411.trusted_nodemap = nodemap.test_411.trusted_nodemap=1 Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /mnt/lustre/d411.sanity-hsm/fileA: (0x00000000) Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' Killing existing copytools on oleg243-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' On MGS 192.168.202.143, test_411.rbac = nodemap.test_411.rbac=file_perms,dne_ops,quota_ops,byfid_ops,chlg_ops,fscrypt_admin,server_upcall,ignore_root_prjquota,immutable_flags,local_admin,pool_quota_ops,lqa_quota_ops,projid_set,foreign_ops /mnt/lustre/d411.sanity-hsm/fileB: (0x00000000) Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied lfs hsm_set: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied lfs hsm_clear: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied On MGS 192.168.202.143, active = nodemap.active=0 oleg243-server: error: nodemap 'test_411' does not exist oleg243-server: nodemap_del: Invalid argument pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 On MGS 192.168.202.143, active = nodemap.active=0 PASS 411 (44s) == sanity-hsm test 500: various LLAPI HSM tests ========== 10:11:58 (1786803118) test1 @1786803165: Register/unregister copytool 2000x to check for leaks ...... pass test2 @1786803165: Re/un-register copytool multiple times without error ....... pass llapi_hsm_test: NULL archive numbers llapi_hsm_test: NULL archive numbers llapi_hsm_test: -1 requested when archive id >= 0 is supported test3 @1786803165: Pass bad parameters to llapi_hsm_copytool_register() ....... pass test4 @1786803165: Bad parameters to llapi_hsm_copytool_unregister() .......... pass test5 @1786803165: Test llapi_hsm_copytool_recv() in non-blocking mode ........ pass test6 @1786803165: Test llapi_hsm_copytool_recv() with bogus parameters ....... pass test7 @1786803165: Test event polling (without actual traffic) ................ pass test50 @1786803165: Test llapi_hsm_state_get()/get_fd() ....................... pass test51 @1786803166: Test llapi_hsm_state_set_fd() ............................. pass test52 @1786803166: Test llapi_hsm_current_action() ........................... pass test100 @1786803167: Simple archive creation with no progress reported. ....... pass test101 @1786803171: Simple archive creation with progress every byte. ........ pass test102 @1786803176: Archive creation with progress every byte, backwards ..... pass test103 @1786803177: Archive creation with a single progress report ........... pass test104 @1786803178: Archive creation with two progress reports ............... pass test105 @1786803179: Archive creation with one bogus progress report .......... pass test106 @1786803180: Archive creation with one empty progress report .......... pass test107 @1786803181: Archive creation with one bogus progress report .......... pass test108 @1786803186: Archive creation with same progress report each time ..... pass test109 @1786803187: Archive creation with one report, with large number ...... pass test110 @1786803189: Archive with 10 progress reports, checking progress ...... pass test111 @1786803190: Archive with 10 reports in reverse, checking progress .... pass test112 @1786803191: Archive with 10 reports, duplicated, check progress ...... pass test113 @1786803192: Archive with 9 reports, with 20% overlapping coverage .... pass PASS 500 (80s) == sanity-hsm test 600: Changelog fields 'u=' and 'nid=' ========================================================== 10:13:18 (1786803198) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' mdd.lustre-MDT0000.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d600.sanity-hsm/f600.sanity-hsm] Got NID '192.168.202.43@tcp' lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 600 (13s) == sanity-hsm test 601: OPEN Changelog entry ============= 10:13:31 (1786803211) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl2' mdd.lustre-MDT0000.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000.8 10OPEN 14:13:36.982948892 2026.08.15 0x1 t=[0x2000032e1:0x26:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.202.43@tcp m=r-- lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 601 (14s) == sanity-hsm test 602: Changelog record CLOSE only if open+write or OPEN recorded ========================================================== 10:13:45 (1786803225) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl3' mdd.lustre-MDT0000.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000.14 11CLOSE 14:13:51.441418281 2026.08.15 0x1 t=[0x2000032e1:0x28:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000.15 10OPEN 14:13:51.656962311 2026.08.15 0x242 t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.202.43@tcp m=-w- p=[0x2000032e1:0x27:0x0] lustre-MDT0000.16 13TRUNC 14:13:51.670039684 2026.08.15 0xe t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x2000032e1:0x27:0x0] lustre-MDT0000.17 11CLOSE 14:13:51.713548876 2026.08.15 0x242 t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000.17 11CLOSE 14:13:51.713548876 2026.08.15 0x242 t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp mdd.lustre-MDT0000.changelog_mask=-OPEN lustre-MDT0000: clear the changelog for cl3 of all records f lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000.18 13TRUNC 14:13:54.553242209 2026.08.15 0xe t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp p=[0x2000032e1:0x27:0x0] lustre-MDT0000.19 11CLOSE 14:13:54.587964253 2026.08.15 0x242 t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000.19 11CLOSE 14:13:54.587964253 2026.08.15 0x242 t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 602 (17s) == sanity-hsm test 603: GETXATTR Changelog entry ========= 10:14:02 (1786803242) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl4' mdd.lustre-MDT0000.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl4 of all records getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d603.sanity-hsm/f603.sanity-hsm user.xattr1="value1" lustre-MDT0000.25 23GXATR 14:14:07.179659647 2026.08.15 0x0 t=[0x2000032e1:0x2a:0x0] j=getfattr.0 ef=0x1b u=0:0 nid=192.168.202.43@tcp x=user.xattr1 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 603 (14s) == sanity-hsm test 604: NOPEN Changelog entry ============ 10:14:17 (1786803257) mdd.lustre-MDT0000.changelog_deniednext=20 mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl5' mdd.lustre-MDT0000.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.30 24NOPEN 14:14:25.711063581 2026.08.15 0x1 t=[0x2000032e1:0x2c:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.202.43@tcp m=r-- Got NID '192.168.202.43@tcp' lustre-MDT0000: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.31 24NOPEN 14:14:46.620363257 2026.08.15 0x1 t=[0x2000032e1:0x2c:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.202.43@tcp m=r-- Got NID '192.168.202.43@tcp' lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 mdd.lustre-MDT0000.changelog_deniednext=60 PASS 604 (39s) == sanity-hsm test 605: Test OPEN and CLOSE rate limit in Changelogs ========================================================== 10:14:55 (1786803295) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl6' mdd.lustre-MDT0000.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000.35 10OPEN 14:15:00.687802629 2026.08.15 0x43 t=[0x2000032e1:0x2e:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.202.43@tcp m=rw- p=[0x2000032e1:0x2b:0x0] lustre-MDT0000.35 10OPEN 14:15:00.687802629 2026.08.15 0x43 t=[0x2000032e1:0x2e:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.202.43@tcp m=rw- p=[0x2000032e1:0x2b:0x0] lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000.36 10OPEN 14:15:01.268083364 2026.08.15 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.202.43@tcp m=r-- p=[0x2000032e1:0x27:0x0] lustre-MDT0000.37 11CLOSE 14:15:01.282483935 2026.08.15 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000.36 10OPEN 14:15:01.268083364 2026.08.15 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.202.43@tcp m=r-- p=[0x2000032e1:0x27:0x0] lustre-MDT0000.37 11CLOSE 14:15:01.282483935 2026.08.15 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre/d605.sanity-hsm/f605.sanity-hsm] lustre-MDT0000.38 10OPEN 14:15:02.069254422 2026.08.15 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.202.43@tcp m=r-- lustre-MDT0000.39 11CLOSE 14:15:02.080882355 2026.08.15 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.202.43@tcp lustre-MDT0000.38 10OPEN 14:15:02.069254422 2026.08.15 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.202.43@tcp m=r-- lustre-MDT0000.39 11CLOSE 14:15:02.080882355 2026.08.15 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.202.43@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000.40 11CLOSE 14:15:02.712394262 2026.08.15 0x43 t=[0x2000032e1:0x2e:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000.40 11CLOSE 14:15:02.712394262 2026.08.15 0x43 t=[0x2000032e1:0x2e:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.202.43@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 605 (16s) == sanity-hsm test 606: llog_reader groks changelog fields ========================================================== 10:15:11 (1786803311) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl7' mdd.lustre-MDT0000.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d606.sanity-hsm/f606.sanity-hsm] Stopping /mnt/lustre-mds1 (opts:) on oleg243-server rec #2 type=1064553b len=64 offset 8256 Header size : 8192 llh_size : 64 Time : Sat Aug 15 09:49:55 2026 Number of records: 1 cat_idx: 0 last_idx: 2 Flags: 0x2012 Target uuid : ----------------------- #02 (064)fid=[0x1:0x3b:0x0] path=oi.1/0x1:0x3b:0x0 oleg243-server: llog_reader: The llog is corrupted, 1 errors found Got UID/GID 500:500 Got NID 192.168.202.43@tcp Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg243-server: oleg243-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg243-client: oleg243-server: ssh exited with exit code 1 Started lustre-MDT0000 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 606 (50s) == sanity-hsm test 607a: release file migrated after archive (non-blocking migrate) ========================================================== 10:16:01 (1786803361) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278143 s, 3.8 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 607a (13s) == sanity-hsm test 607b: Migrate should not change the HSM attribute of dirty files ========================================================== 10:16:14 (1786803374) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144676 s, 7.2 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' llite.lustre-ffff9d738319d000.xattr_cache=0 llite.lustre-ffff9d73841f0000.xattr_cache=0 /mnt/lustre/d607b.sanity-hsm/f607b.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 llite.lustre-ffff9d738319d000.xattr_cache=1 llite.lustre-ffff9d73841f0000.xattr_cache=1 PASS 607b (9s) == sanity-hsm test 607c: 'lfs swap_layouts' should set dirty flag on HSM file ========================================================== 10:16:23 (1786803383) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299018 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183768 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129208 s, 8.1 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-1' and '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-3': Operation not permitted (1) PASS 607c (10s) == sanity-hsm test 607d: Migrate should not set HSM dirty flag for an empty file ========================================================== 10:16:33 (1786803393) 0+0 records in 0+0 records out 0 bytes copied, 0.013261 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 607d (14s) == sanity-hsm test 607e: release file migrated after archive (blocking migrate) ========================================================== 10:16:47 (1786803407) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214145 s, 4.9 MB/s Starting copytool 'agt1' on 'oleg243-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' PASS 607e (13s) == sanity-hsm test complete, duration 4576 sec =========== 10:17:00 (1786803420) === sanity-hsm: start cleanup 10:17:02 (1786803422) === Stopping clients: oleg243-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg243-client.virtnet /mnt/lustre2 opts: === sanity-hsm: finish cleanup 10:17:32 (1786803452) ===