-----============= acceptance-small: sanity-hsm ============----- Mon Jan 12 15:51:06 EST 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg358-server: /home/green/git/lustre-release/lustre/tests/except/sanity-hsm.ex - see MDS1_VERSION <= CLIENT_VERSION (34668545 <= 34668545) for LU-18924, go 100 105 excepting tests: === sanity-hsm: start setup 15:51:26 (1768251086) === Starting client oleg358-client.virtnet: -o user_xattr,flock 192.168.203.158@tcp:/lustre /mnt/lustre2 Mount client oleg358-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.203.158@tcp:/lustre /mnt/lustre2 Started clients oleg358-client.virtnet: 192.168.203.158@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg358-client.virtnet: executing check_config_client /mnt/lustre oleg358-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg358-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff995ac65c4800.idle_timeout=debug osc.lustre-OST0000-osc-ffff995ac6623000.idle_timeout=debug osc.lustre-OST0001-osc-ffff995ac65c4800.idle_timeout=debug osc.lustre-OST0001-osc-ffff995ac6623000.idle_timeout=debug disable quota as required oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity-hsm: finish setup 15:52:05 (1768251125) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f8387] nobody:x:65534: Killing existing copytools on oleg358-client.virtnet pkill: pidfile not valid Try `pkill --help' for more information. Set HSM on and start Set sanity-hsm HSM policy == sanity-hsm test 1A: lfs hsm flags root/non-root access ========================================================== 15:52:34 (1768251154) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_clear] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--exists] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] lfs hsm_set: change hsm flags for '/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm' failed: Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] PASS 1A (9s) == sanity-hsm test 1a: mmap & cat a HSM released file ==== 15:52:43 (1768251163) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253552 s, 4.1 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying released state: PASS 1a (12s) == sanity-hsm test 1b: Archive, Release and Restore composite file ========================================================== 15:52:55 (1768251175) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0745701 s, 14.1 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm release /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm verify released state: pass restore /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm verify restored state: pass PASS 1b (11s) == sanity-hsm test 1c: Check setting archive-id in lfs hsm_set ========================================================== 15:53:06 (1768251186) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is still 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 33 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 16 PASS 1c (7s) == sanity-hsm test 1d: Archive, Release and Restore DoM file ========================================================== 15:53:14 (1768251194) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.05367 s, 19.5 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm release /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm verify released state: pass restore /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm verify restored state: pass PASS 1d (9s) == sanity-hsm test 1e: Archive, Release and Restore SEL file ========================================================== 15:53:23 (1768251203) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0976807 s, 10.7 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' release /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm verify released state: pass restore /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass PASS 1e (13s) == sanity-hsm test 1f: DoM file release after restore ==== 15:53:36 (1768251216) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0599271 s, 17.5 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' release /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm verify released state: pass restore /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass release again /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 verify released state: pass PASS 1f (14s) == sanity-hsm test 2: Check file dirtyness when doing setattr ========================================================== 15:53:51 (1768251231) 0+0 records in 0+0 records out 0 bytes copied, 0.0327388 s, 0.0 kB/s 0x200000403:0x8:0x0 PASS 2 (8s) == sanity-hsm test 3: Check file dirtyness when opening for write ========================================================== 15:53:59 (1768251239) Succeed in opening file "/mnt/lustre/d3.sanity-hsm/f3.sanity-hsm"(flags=O_WRONLY) 3+0 records in 3+0 records out 3+0 records in 3+0 records out PASS 3 (10s) == sanity-hsm test 4: Useless cancel must not be registered ========================================================== 15:54:09 (1768251249) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37701 s, 2.8 MB/s PASS 4 (8s) == sanity-hsm test 8: Test default archive number ======== 15:54:17 (1768251257) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_8/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 8 (7s) == sanity-hsm test 9A: Use of explicit archive number, with dedicated copytool ========================================================== 15:54:25 (1768251265) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_9A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 3 "/mnt/lustre2"' oleg358-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg358-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=6eb0ca4f-e1e8-4487-8c1c-58970c977094 archive_id=3 requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=6eb0ca4f-e1e8-4487-8c1c-58970c977094 archive_id=3 requests=[current:0 ok:0 errors:0] on mds2 PASS 9A (23s) == sanity-hsm test 9a: Multiple remote agents ============ 15:54:47 (1768251287) SKIP: sanity-hsm test_9a Need 3 or more clients, have 1 SKIP 9a (3s) == sanity-hsm test 10a: Archive a file =================== 15:54:50 (1768251290) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 938: warning: command substitution: ignored null byte in input Verifying content Verifying hsm state Verifying archive number is 2 PASS 10a (10s) == sanity-hsm test 10b: Archive of non dirty file must work without doing request ========================================================== 15:55:00 (1768251300) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 10b (10s) == sanity-hsm test 10c: Check forbidden archive ========== 15:55:10 (1768251310) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d10c.sanity-hsm/f10c.sanity-hsm): Operation not permitted PASS 10c (6s) == sanity-hsm test 10d: Archive a file on the default archive id ========================================================== 15:55:17 (1768251317) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 10d (10s) == sanity-hsm test 11a: Import a file ==================== 15:55:27 (1768251327) lhsmtool_posix: 1768251330.466506 [22161]: action=1 src=d11a.sanity-hsm/f11a.sanity-hsm dst=/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768251330.479188 [22161]: importing '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1768251330.538376 [22161]: imported '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//0425/0x200000402:0x27:0x0'=='/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1768251330.544201 [22161]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying imported size 158=158 Verifying released pattern: raid0,released Verifying new fid 0x200000402:0x27:0x0 in archive PASS 11a (10s) == sanity-hsm test 11b: Import a deleted file using its FID ========================================================== 15:55:37 (1768251337) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_11b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1768251342.229836 [22854]: action=1 src=0x200000403:0x11:0x0 dst=/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768251342.245461 [22854]: Resolving [0x200000403:0x11:0x0] to /tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0 lhsmtool_posix: 1768251342.246480 [22854]: importing '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0' lhsmtool_posix: 1768251342.307073 [22854]: imported '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//042b/0x200000402:0x29:0x0'=='/tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0' lhsmtool_posix: 1768251342.308440 [22854]: process finished, errs: 0 major, 0 minor, rc=0 (Success) /mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm: OK PASS 11b (11s) == sanity-hsm test 11c: Import a file to a directory with a pool ========================================================== 15:55:48 (1768251348) Creating new pool test_11c oleg358-server: Pool lustre.test_11c created Adding targets to pool oleg358-server: OST lustre-OST0001_UUID added to pool lustre.test_11c lhsmtool_posix: 1768251358.406938 [23570]: action=1 src=d11c.sanity-hsm/f11c.sanity-hsm dst=/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768251358.413014 [23570]: importing '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1768251358.455923 [23570]: imported '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//0404/0x240000402:0x6:0x0'=='/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1768251358.455975 [23570]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Destroy the created pools: test_11c lustre.test_11c oleg358-server: OST lustre-OST0001_UUID removed from pool lustre.test_11c oleg358-server: Pool lustre.test_11c destroyed PASS 11c (26s) == sanity-hsm test 12a: Restore an imported file explicitly ========================================================== 15:56:15 (1768251375) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1768251378.597195 [24311]: action=1 src=d12a.sanity-hsm/f12a.sanity-hsm dst=/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768251378.610507 [24311]: importing '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1768251378.721562 [24311]: imported '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//042e/0x200000402:0x2c:0x0'=='/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1768251378.721626 [24311]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying file state: PASS 12a (11s) == sanity-hsm test 12b: Restore an imported file implicitly ========================================================== 15:56:25 (1768251385) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1768251388.756919 [25017]: action=1 src=d12b.sanity-hsm/f12b.sanity-hsm dst=/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768251388.767923 [25017]: importing '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1768251388.813506 [25017]: imported '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//042d/0x200000402:0x2f:0x0'=='/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1768251388.815425 [25017]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying file state after restore: PASS 12b (10s) == sanity-hsm test 12c: Restore a file with stripe of 2 == 15:56:35 (1768251395) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.21834 s, 24.0 MB/s /mnt/lustre/d12c.sanity-hsm/f12c.sanity-hsm: OK PASS 12c (11s) == sanity-hsm test 12d: Restore of a non archived, non released file must work ========================================================== 15:56:46 (1768251406) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 12d (12s) == sanity-hsm test 12e: Check forbidden restore ========== 15:56:58 (1768251418) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 Cannot send HSM request (use of /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm): Operation not permitted PASS 12e (8s) == sanity-hsm test 12f: Restore a released file explicitly ========================================================== 15:57:06 (1768251426) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Verifying file state: PASS 12f (9s) == sanity-hsm test 12g: Restore a released file implicitly ========================================================== 15:57:15 (1768251435) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12g (9s) == sanity-hsm test 12h: Restore a released file implicitly from a second node ========================================================== 15:57:24 (1768251444) SKIP: sanity-hsm test_12h Need 2 or more clients, have 1 SKIP 12h (4s) == sanity-hsm test 12m: Archive/release/implicit restore ========================================================== 15:57:28 (1768251448) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12m/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12m (9s) == sanity-hsm test 12n: Import/implicit restore/release == 15:57:37 (1768251457) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12n/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1768251460.019056 [29847]: action=1 src=d12n.sanity-hsm/f12n.sanity-hsm dst=/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768251460.027540 [29847]: importing '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1768251460.085767 [29847]: imported '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//040b/0x240000402:0x9:0x0'=='/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1768251460.095414 [29847]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12n (9s) == sanity-hsm test 12o: Layout-swap failure during Restore leaves file released ========================================================== 15:57:46 (1768251466) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12o/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x152 mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA diff: /mnt/lustre/d12o.sanity-hsm/f12o.sanity-hsm: No data available mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA fail_loc=0 mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 12o (33s) == sanity-hsm test 12p: implicit restore of a file on copytool mount point ========================================================== 15:58:19 (1768251499) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12p/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12p (10s) == sanity-hsm test 12q: file attributes are refreshed after restore ========================================================== 15:58:29 (1768251509) Starting client: oleg358-client.virtnet: -o user_xattr,flock 192.168.203.158@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125588 s, 8.3 MB/s Killing existing copytools on oleg358-client.virtnet copytools stopped in 0s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Stopping client oleg358-client.virtnet /mnt/lustre3 (opts:) PASS 12q (18s) == sanity-hsm test 12r: lseek restores released file ===== 15:58:47 (1768251527) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12r/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12r (10s) == sanity-hsm test 12s: race between restore requests ==== 15:58:57 (1768251537) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12s/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x8000018b PASS 12s (16s) == sanity-hsm test 12t: Multiple parallel reads for a HSM imported file ========================================================== 15:59:13 (1768251553) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12t/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' lhsmtool_posix: 1768251558.179027 [34000]: action=1 src=0x200000403:0x2a:0x0 dst=/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768251558.186829 [34000]: Resolving [0x200000403:0x2a:0x0] to /tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0 lhsmtool_posix: 1768251558.187392 [34000]: importing '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0' lhsmtool_posix: 1768251558.212943 [34000]: imported '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//044a/0x200000402:0x48:0x0'=='/tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0' lhsmtool_posix: 1768251558.213541 [34000]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12t (11s) == sanity-hsm test 12u: Multiple reads on multiple HSM imported files in parallel ========================================================== 15:59:24 (1768251564) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12u/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1768251577.578840 [34880]: action=1 src=0x200000403:0x2c:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0 mount_point=/mnt/lustre2 lhsmtool_posix: 1768251577.586441 [34880]: Resolving [0x200000403:0x2c:0x0] to /tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0 lhsmtool_posix: 1768251577.586869 [34880]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0' lhsmtool_posix: 1768251577.621321 [34880]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//0448/0x200000402:0x4a:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0' lhsmtool_posix: 1768251577.621383 [34880]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1768251577.855529 [34910]: action=1 src=0x200000403:0x2d:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1 mount_point=/mnt/lustre2 lhsmtool_posix: 1768251577.861862 [34910]: Resolving [0x200000403:0x2d:0x0] to /tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0 lhsmtool_posix: 1768251577.863092 [34910]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0' lhsmtool_posix: 1768251577.908447 [34910]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//0449/0x200000402:0x4b:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0' lhsmtool_posix: 1768251577.908855 [34910]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1768251578.217473 [34939]: action=1 src=0x200000403:0x2e:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2 mount_point=/mnt/lustre2 lhsmtool_posix: 1768251578.228096 [34939]: Resolving [0x200000403:0x2e:0x0] to /tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0 lhsmtool_posix: 1768251578.228693 [34939]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0' lhsmtool_posix: 1768251578.296084 [34939]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//044e/0x200000402:0x4c:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0' lhsmtool_posix: 1768251578.296206 [34939]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1768251578.622154 [34968]: action=1 src=0x200000403:0x2f:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3 mount_point=/mnt/lustre2 lhsmtool_posix: 1768251578.628631 [34968]: Resolving [0x200000403:0x2f:0x0] to /tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0 lhsmtool_posix: 1768251578.629461 [34968]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0' lhsmtool_posix: 1768251578.665859 [34968]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//044f/0x200000402:0x4d:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0' lhsmtool_posix: 1768251578.665925 [34968]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1768251578.946045 [34997]: action=1 src=0x200000403:0x30:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4 mount_point=/mnt/lustre2 lhsmtool_posix: 1768251578.951631 [34997]: Resolving [0x200000403:0x30:0x0] to /tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0 lhsmtool_posix: 1768251578.952791 [34997]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0' lhsmtool_posix: 1768251578.993853 [34997]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//044c/0x200000402:0x4e:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0' lhsmtool_posix: 1768251578.994795 [34997]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1768251579.275651 [35026]: action=1 src=0x200000403:0x31:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5 mount_point=/mnt/lustre2 lhsmtool_posix: 1768251579.287780 [35026]: Resolving [0x200000403:0x31:0x0] to /tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0 lhsmtool_posix: 1768251579.287910 [35026]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0' lhsmtool_posix: 1768251579.372779 [35026]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//044d/0x200000402:0x4f:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0' lhsmtool_posix: 1768251579.372837 [35026]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1768251579.646975 [35055]: action=1 src=0x200000403:0x32:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6 mount_point=/mnt/lustre2 lhsmtool_posix: 1768251579.661057 [35055]: Resolving [0x200000403:0x32:0x0] to /tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0 lhsmtool_posix: 1768251579.661639 [35055]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0' lhsmtool_posix: 1768251579.699695 [35055]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0452/0x200000402:0x50:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0' lhsmtool_posix: 1768251579.699750 [35055]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1768251580.021822 [35084]: action=1 src=0x200000403:0x33:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7 mount_point=/mnt/lustre2 lhsmtool_posix: 1768251580.026646 [35084]: Resolving [0x200000403:0x33:0x0] to /tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0 lhsmtool_posix: 1768251580.027040 [35084]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0' lhsmtool_posix: 1768251580.061059 [35084]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0453/0x200000402:0x51:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0' lhsmtool_posix: 1768251580.061115 [35084]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1768251580.394751 [35113]: action=1 src=0x200000403:0x34:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8 mount_point=/mnt/lustre2 lhsmtool_posix: 1768251580.402330 [35113]: Resolving [0x200000403:0x34:0x0] to /tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0 lhsmtool_posix: 1768251580.402421 [35113]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0' lhsmtool_posix: 1768251580.443095 [35113]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0450/0x200000402:0x52:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0' lhsmtool_posix: 1768251580.443207 [35113]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1768251580.744726 [35142]: action=1 src=0x200000403:0x35:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9 mount_point=/mnt/lustre2 lhsmtool_posix: 1768251580.750836 [35142]: Resolving [0x200000403:0x35:0x0] to /tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0 lhsmtool_posix: 1768251580.750932 [35142]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0' lhsmtool_posix: 1768251580.792594 [35142]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0451/0x200000402:0x53:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0' lhsmtool_posix: 1768251580.792673 [35142]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12u (25s) == sanity-hsm test 13: Recursively import and restore a directory ========================================================== 15:59:49 (1768251589) lhsmtool_posix: 1768251598.235055 [36897]: action=1 src=subdir dst=/mnt/lustre/d13.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768251598.756785 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1768251598.869707 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//040f/0x240000402:0xd:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1768251598.870120 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1768251598.910250 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//040c/0x240000402:0xe:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1768251598.910865 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1768251598.939915 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//040d/0x240000402:0xf:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1768251598.940110 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1768251598.974118 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//0412/0x240000402:0x10:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1768251598.974498 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1768251599.005536 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//0413/0x240000402:0x11:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1768251599.005743 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1768251599.035412 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//0410/0x240000402:0x12:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1768251599.035614 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1768251599.069256 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//0411/0x240000402:0x13:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1768251599.069437 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1768251599.102684 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//0416/0x240000402:0x14:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1768251599.102876 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1768251599.141413 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//0417/0x240000402:0x15:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1768251599.141604 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1768251599.175615 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//0414/0x240000402:0x16:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1768251599.181490 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1768251599.255827 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//0454/0x200000402:0x56:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1768251599.256042 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1768251599.316370 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//0455/0x200000402:0x57:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1768251599.316564 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1768251599.353712 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//045a/0x200000402:0x58:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1768251599.354864 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1768251599.394911 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//045b/0x200000402:0x59:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1768251599.395939 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1768251599.430367 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//0458/0x200000402:0x5a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1768251599.431247 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1768251599.487270 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//0459/0x200000402:0x5b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1768251599.487443 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1768251599.528508 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//045e/0x200000402:0x5c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1768251599.528692 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1768251599.568061 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//045f/0x200000402:0x5d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1768251599.568300 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1768251599.596356 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//045c/0x200000402:0x5e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1768251599.596792 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1768251599.626013 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//045d/0x200000402:0x5f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1768251599.633680 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1768251599.774566 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//041a/0x240000402:0x18:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1768251599.774772 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1768251599.820955 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//041b/0x240000402:0x19:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1768251599.821821 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1768251599.859391 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//0418/0x240000402:0x1a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1768251599.859581 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1768251599.894451 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//0419/0x240000402:0x1b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1768251599.894937 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1768251599.944828 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//041e/0x240000402:0x1c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1768251599.945513 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1768251599.990008 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//041f/0x240000402:0x1d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1768251599.990238 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1768251600.028458 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//041c/0x240000402:0x1e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1768251600.028921 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1768251600.067830 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//041d/0x240000402:0x1f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1768251600.068653 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1768251600.106345 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//0422/0x240000402:0x20:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1768251600.106543 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1768251600.143474 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//0423/0x240000402:0x21:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1768251600.160860 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1768251600.263544 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//0421/0x240000402:0x23:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1768251600.263740 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1768251600.310762 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//0426/0x240000402:0x24:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1768251600.311821 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1768251600.352515 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//0427/0x240000402:0x25:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1768251600.352686 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1768251600.401608 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//0424/0x240000402:0x26:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1768251600.403902 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1768251600.448082 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//0425/0x240000402:0x27:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1768251600.451293 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1768251600.488699 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//042a/0x240000402:0x28:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1768251600.488886 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1768251600.551905 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//042b/0x240000402:0x29:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1768251600.556655 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1768251600.594238 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//0428/0x240000402:0x2a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1768251600.594903 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1768251600.638081 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//0429/0x240000402:0x2b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1768251600.638672 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1768251600.676006 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//042e/0x240000402:0x2c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1768251600.687514 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1768251600.765906 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//042c/0x240000402:0x2e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1768251600.766101 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1768251600.798975 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//042d/0x240000402:0x2f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1768251600.799201 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1768251600.830380 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//0432/0x240000402:0x30:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1768251600.831012 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1768251600.861682 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//0433/0x240000402:0x31:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1768251600.861872 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1768251600.890394 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//0430/0x240000402:0x32:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1768251600.890582 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1768251600.917596 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//0431/0x240000402:0x33:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1768251600.917756 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1768251600.942440 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//0436/0x240000402:0x34:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1768251600.943092 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1768251600.975086 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//0437/0x240000402:0x35:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1768251600.975490 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1768251601.003112 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//0434/0x240000402:0x36:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1768251601.003574 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1768251601.031583 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//0435/0x240000402:0x37:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1768251601.036168 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1768251601.084508 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//0463/0x200000402:0x61:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1768251601.085325 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1768251601.144532 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//0460/0x200000402:0x62:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1768251601.149077 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1768251601.235331 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//0461/0x200000402:0x63:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1768251601.235537 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1768251601.322601 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//0466/0x200000402:0x64:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1768251601.323693 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1768251601.390534 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//0467/0x200000402:0x65:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1768251601.390744 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1768251601.464373 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//0464/0x200000402:0x66:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1768251601.464558 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1768251601.511628 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//0465/0x200000402:0x67:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1768251601.511826 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1768251601.555440 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//046a/0x200000402:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1768251601.555652 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1768251601.594999 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//046b/0x200000402:0x69:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1768251601.595891 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1768251601.632921 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//0468/0x200000402:0x6a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1768251601.638546 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1768251601.708873 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//046e/0x200000402:0x6c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1768251601.709559 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1768251601.848436 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//046f/0x200000402:0x6d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1768251601.848628 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1768251601.952572 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//046c/0x200000402:0x6e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1768251601.952776 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1768251602.078880 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//046d/0x200000402:0x6f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1768251602.079098 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1768251602.124909 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//0472/0x200000402:0x70:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1768251602.125526 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1768251602.175990 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//0473/0x200000402:0x71:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1768251602.176220 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1768251602.217985 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//0470/0x200000402:0x72:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1768251602.218780 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1768251602.251461 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//0471/0x200000402:0x73:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1768251602.251649 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1768251602.290392 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//0476/0x200000402:0x74:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1768251602.291000 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1768251602.332618 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//0477/0x200000402:0x75:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1768251602.349009 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1768251602.511912 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//043b/0x240000402:0x39:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1768251602.513066 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1768251602.579510 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//0438/0x240000402:0x3a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1768251602.580109 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1768251602.651916 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//0439/0x240000402:0x3b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1768251602.653804 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1768251602.748899 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//043e/0x240000402:0x3c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1768251602.751640 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1768251602.862578 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//043f/0x240000402:0x3d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1768251602.862765 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1768251602.922664 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//043c/0x240000402:0x3e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1768251602.922872 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1768251602.987229 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//043d/0x240000402:0x3f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1768251602.987416 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1768251603.086381 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//0442/0x240000402:0x40:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1768251603.086589 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1768251603.169075 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//0443/0x240000402:0x41:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1768251603.170379 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1768251603.209350 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//0440/0x240000402:0x42:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1768251603.214884 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1768251603.303892 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//0475/0x200000402:0x77:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1768251603.304084 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1768251603.402720 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//047a/0x200000402:0x78:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1768251603.402905 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1768251603.462926 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//047b/0x200000402:0x79:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1768251603.463111 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1768251603.509832 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//0478/0x200000402:0x7a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1768251603.510650 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1768251603.554951 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//0479/0x200000402:0x7b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1768251603.555665 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1768251603.596736 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//047e/0x200000402:0x7c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1768251603.596926 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1768251603.628873 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//047f/0x200000402:0x7d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1768251603.629070 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1768251603.663053 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//047c/0x200000402:0x7e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1768251603.663945 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1768251603.697365 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//047d/0x200000402:0x7f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1768251603.697554 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1768251603.733255 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//0482/0x200000402:0x80:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1768251603.749005 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1768251603.885866 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//0446/0x240000402:0x44:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1768251603.886894 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1768251603.922754 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//0447/0x240000402:0x45:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1768251603.923351 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1768251603.981807 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//0444/0x240000402:0x46:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1768251603.982805 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1768251604.015423 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//0445/0x240000402:0x47:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1768251604.015598 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1768251604.046556 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//044a/0x240000402:0x48:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1768251604.046743 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1768251604.080016 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//044b/0x240000402:0x49:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1768251604.080864 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1768251604.108792 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//0448/0x240000402:0x4a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1768251604.108995 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1768251604.137445 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//0449/0x240000402:0x4b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1768251604.138266 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1768251604.179544 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//044e/0x240000402:0x4c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1768251604.179883 [36897]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1768251604.215909 [36897]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//044f/0x240000402:0x4d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1768251604.215992 [36897]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_13/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 13 (39s) == sanity-hsm test 14: Rebind archived file to a new fid ========================================================== 16:00:28 (1768251628) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_14/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109744 s, 9.6 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 0+0 records in 0+0 records out 0 bytes copied, 0.0066955 s, 0.0 kB/s rebind 0x200000403:0x69:0x0 to 0x200000403:0x6a:0x0 lhsmtool_posix: 1768251634.275782 [37869]: action=2 src=0x200000403:0x69:0x0 dst=0x200000403:0x6a:0x0 mount_point=/mnt/lustre2 lhsmtool_posix: 1768251634.277665 [37869]: rebind [0x200000403:0x69:0x0] to [0x200000403:0x6a:0x0] lhsmtool_posix: 1768251634.279404 [37869]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 14 (12s) == sanity-hsm test 15: Rebind a list of files ============ 16:00:40 (1768251640) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_15/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251344 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150927 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.402941 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198885 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.272641 s, 3.8 MB/s Waiting 300s for '' pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 0+0 records in 0+0 records out 0 bytes copied, 0.00388738 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00364504 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00954109 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.0130039 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00498984 s, 0.0 kB/s rebind list of files lhsmtool_posix: 1768251649.452507 [38663]: action=2 src=/tmp/tmp.8387 dst=(null) mount_point=/mnt/lustre2 lhsmtool_posix: 1768251649.455642 [38663]: rebind [0x200000403:0x6d:0x0] to [0x200000403:0x72:0x0] lhsmtool_posix: 1768251649.455798 [38663]: rebind [0x200000403:0x6e:0x0] to [0x200000403:0x73:0x0] lhsmtool_posix: 1768251649.456537 [38663]: rebind [0x200000403:0x6f:0x0] to [0x200000403:0x74:0x0] lhsmtool_posix: 1768251649.457068 [38663]: rebind [0x200000403:0x70:0x0] to [0x200000403:0x75:0x0] lhsmtool_posix: 1768251649.457157 [38663]: rebind [0x200000403:0x71:0x0] to [0x200000403:0x76:0x0] lhsmtool_posix: 1768251649.457228 [38663]: 5 lines read from '/tmp/tmp.8387', 5 rebind successful lhsmtool_posix: 1768251649.457246 [38663]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 15 (21s) == sanity-hsm test 16: Test CT bandwith control option === 16:01:01 (1768251661) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_16/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 0.0708173 s, 296 MB/s Waiting 200s for 'SUCCEED' Changed after 2s: from 'WAITING' to 'STARTED' Waiting 190s for 'SUCCEED' Updated after 20s: want 'SUCCEED' got 'SUCCEED' PASS 16 (30s) == sanity-hsm test 20: Release is not permitted ========== 16:01:31 (1768251691) 0+0 records in 0+0 records out 0 bytes copied, 0.00298248 s, 0.0 kB/s 0x240000403:0x40:0x0 Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted PASS 20 (7s) == sanity-hsm test 21: Simple release tests ============== 16:01:39 (1768251699) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_21/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165331 s, 6.3 MB/s 1+0 records in 1+0 records out 512 bytes copied, 0.111157 s, 4.6 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 21 (14s) == sanity-hsm test 22: Could not swap a release file ===== 16:01:53 (1768251713) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_22/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317105 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247613 s, 4.2 MB/s 0x200000403:0x81:0x0 lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d22.sanity-hsm/test_swap' and '/mnt/lustre/d22.sanity-hsm/test_release': Operation not supported (95) PASS 22 (8s) == sanity-hsm test 23: Release does not change a/mtime (utime) ========================================================== 16:02:02 (1768251722) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_23/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244187 s, 4.3 MB/s PASS 23 (7s) == sanity-hsm test 24a: Archive, release, and restore does not change a/mtime (i/o) ========================================================== 16:02:09 (1768251729) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223527 s, 4.7 MB/s Killing existing copytools on oleg358-client.virtnet copytools stopped in 0s 192.168.203.158@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg358-client.virtnet /mnt/lustre (opts:) Starting client: oleg358-client.virtnet: -o user_xattr,flock 192.168.203.158@tcp:/lustre /mnt/lustre PASS 24a (14s) == sanity-hsm test 24b: root can archive, release, and restore user files ========================================================== 16:02:23 (1768251743) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218442 s, 4.8 MB/s Waiting 200s for 'SUCCEED' running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] /mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm: (0x00000009) exists archived, archive_id:1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [md5sum] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] PASS 24b (13s) == sanity-hsm test 24c: check that user,group,other request masks work ========================================================== 16:02:36 (1768251756) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204532 s, 5.1 MB/s 0x200000403:0x8a:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213234 s, 4.9 MB/s 0x200000403:0x8b:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223438 s, 4.7 MB/s 0x200000403:0x8c:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] PASS 24c (21s) == sanity-hsm test 24d: check that read-only mounts are respected ========================================================== 16:02:57 (1768251777) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157044 s, 6.7 MB/s 0x200000403:0x8d:0x0 /mnt/lustre/d24d.sanity-hsm/f24d.sanity-hsm lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 75 0x4b 0x2c0000401 Starting client: oleg358-client.virtnet: -o user_xattr,flock 192.168.203.158@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 1.34371 s, 780 kB/s Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Stopping client oleg358-client.virtnet /mnt/lustre3 (opts:) PASS 24d (15s) == sanity-hsm test 24e: tar succeeds on HSM released files ========================================================== 16:03:12 (1768251792) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192551 s, 5.4 MB/s /mnt/lustre/d24e.sanity-hsm/f24e.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 tar: Removing leading `/' from member names PASS 24e (8s) == sanity-hsm test 24f: root can archive, release, and restore tar files ========================================================== 16:03:21 (1768251801) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm f24f.sanity-hsm f24f.sanity-hsm Sum0 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm, sum1 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm PASS 24f (11s) == sanity-hsm test 24g: write by non-owner still sets dirty ========================================================== 16:03:32 (1768251812) RUNAS = 'runas -u 500 -g 500' Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Please listen carefully as our options have changed. running as uid/gid/euid/egid 500/500/500/500, groups: 500 [tee] [/mnt/lustre/d24g.sanity-hsm/f24g.sanity-hsm] To be electrocuted by your telephone, press #. PASS 24g (9s) == sanity-hsm test 25a: Restore lost file (HS_LOST flag) from import ========================================================== 16:03:42 (1768251822) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1768251824.510414 [47691]: action=1 src=d25a.sanity-hsm/f25a.sanity-hsm dst=/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768251824.514700 [47691]: importing '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1768251824.559196 [47691]: imported '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//0400/0x240000404:0x4:0x0'=='/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1768251824.559764 [47691]: process finished, errs: 0 major, 0 minor, rc=0 (Success) md5sum: /mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm: No data available PASS 25a (9s) == sanity-hsm test 25b: Restore lost file (HS_LOST flag) after release ========================================================== 16:03:50 (1768251830) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' md5sum: /mnt/lustre/d25b.sanity-hsm/f25b.sanity-hsm: No data available PASS 25b (8s) == sanity-hsm test 26A: Remove the archive of a valid file ========================================================== 16:03:58 (1768251838) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.0025398 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 26A (11s) == sanity-hsm test 26a: Remove Archive On Last Unlink (RAoLU) policy ========================================================== 16:04:09 (1768251849) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 9s: want 'SUCCEED' got 'SUCCEED' PASS 26a (37s) == sanity-hsm test 26b: RAoLU policy when CDT off ======== 16:04:46 (1768251886) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Killing existing copytools on oleg358-client.virtnet copytools stopped in 0s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 26b (23s) == sanity-hsm test 26c: RAoLU effective when file closed ========================================================== 16:05:09 (1768251909) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' multiop /mnt/lustre/d26c.sanity-hsm/f26c.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8387 Waiting 200s for 'SUCCEED' Updated after 9s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 5s: want 'SUCCEED' got 'SUCCEED' PASS 26c (44s) == sanity-hsm test 26d: RAoLU when Client eviction ======= 16:05:53 (1768251953) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.383536 s, 2.7 MB/s multiop /mnt/lustre/d26d.sanity-hsm/f26d.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8387 Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 26d (32s) == sanity-hsm test 26e: RAoLU with a non-started coordinator ========================================================== 16:06:25 (1768251985) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248869 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153561 s, 6.8 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg358-client.virtnet Copytool is continued on oleg358-client.virtnet copytools stopped in 0s Failing mds1 on oleg358-server Stopping /mnt/lustre-mds1 (opts:) on oleg358-server 16:06:59 (1768252019) shut down facet: mds1 facet_host: oleg358-server facet_failover_host: oleg358-server Failover mds1 to oleg358-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Started lustre-MDT0000 16:07:22 (1768252042) targets are mounted 16:07:22 (1768252042) facet_failover done oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Check cookie from RAoLU request (last: 1768251282, remove: 1768251283) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 26e (106s) == sanity-hsm test 27a: Remove the archive of an imported file (Operation not permitted) ========================================================== 16:08:11 (1768252091) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.05801 s, 38.7 MB/s lhsmtool_posix: 1768252095.833586 [54769]: action=1 src=d27a.sanity-hsm/f27a.sanity-hsm dst=/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768252095.844552 [54769]: importing '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1768252095.909746 [54769]: imported '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//0404/0x200000407:0x3:0x0'=='/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1768252095.913072 [54769]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Cannot send HSM request (use of /mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm): Operation not permitted PASS 27a (10s) == sanity-hsm test 27b: Remove the archive of a relased file (Operation not permitted) ========================================================== 16:08:21 (1768252101) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00683644 s, 0.0 kB/s Cannot send HSM request (use of /mnt/lustre/d27b.sanity-hsm/f27b.sanity-hsm): Operation not permitted PASS 27b (9s) == sanity-hsm test 28: Concurrent archive/file remove ==== 16:08:30 (1768252110) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_28/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00462084 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 28 (17s) == sanity-hsm test 29a: Tests --mntpath and --archive options ========================================================== 16:08:47 (1768252127) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 7 "/mnt/lustre"' Cannot send HSM request (use of /mnt/lustre): Invalid argument hsm: '/qwerty/uyt' is not a valid FID PASS 29a (6s) == sanity-hsm test 29b: Archive/delete/remove by FID from the archive. ========================================================== 16:08:53 (1768252133) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174058 s, 6.0 MB/s Waiting 200s for 'SUCCEED' PASS 29b (10s) == sanity-hsm test 29c: Archive/delete/remove by FID, using a file list. ========================================================== 16:09:03 (1768252143) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154692 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183346 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199313 s, 5.3 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 29c (15s) == sanity-hsm test 29d: hsm_remove by FID with archive_id 0 for unlinked file cause ========================================================== 16:09:19 (1768252159) SKIP: sanity-hsm test_29d Need 3 or more clients, have 1 SKIP 29d (3s) == sanity-hsm test 30a: Restore at exec (import case) ==== 16:09:22 (1768252162) SKIP: sanity-hsm test_30a Need 2 or more clients, have 1 SKIP 30a (4s) == sanity-hsm test 30b: Restore at exec (release case) === 16:09:26 (1768252166) SKIP: sanity-hsm test_30b Need 2 or more clients, have 1 SKIP 30b (3s) == sanity-hsm test 30c: Update during exec of released file must fail ========================================================== 16:09:29 (1768252169) SKIP: sanity-hsm test_30c Need 2 or more clients, have 1 SKIP 30c (2s) == sanity-hsm test 31a: Import a large file and check size during restore ========================================================== 16:09:32 (1768252172) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.2068 s, 33.9 MB/s lhsmtool_posix: 1768252175.395248 [59934]: action=1 src=d31a.sanity-hsm/f31a.sanity-hsm dst=/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768252175.404513 [59934]: importing '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1768252175.458527 [59934]: imported '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//040b/0x200000407:0xc:0x0'=='/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1768252175.459317 [59934]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' size seen is right: 40894464 == 40894464 done PASS 31a (20s) == sanity-hsm test 31b: Restore a large unaligned file and check size during restore ========================================================== 16:09:52 (1768252192) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.876862 s, 44.5 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' size seen is right: 39000000 == 39000000 done PASS 31b (22s) == sanity-hsm test 31c: Restore a large aligned file and check size during restore ========================================================== 16:10:13 (1768252213) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.01685 s, 40.2 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' size seen is right: 40894464 == 40894464 done PASS 31c (22s) == sanity-hsm test 33: Kill a restore waiting process ==== 16:10:36 (1768252236) 0+0 records in 0+0 records out 0 bytes copied, 0.00239217 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_33/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg358-client.virtnet Copytool is continued on oleg358-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2829: 61959 Terminated md5sum $f > /dev/null PASS 33 (11s) == sanity-hsm test 34: Remove file during restore ======== 16:10:46 (1768252246) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_34/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00576093 s, 0.0 kB/s Copytool is suspended on oleg358-client.virtnet Copytool is continued on oleg358-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2861: kill: (62621) - No such process PASS 34 (11s) == sanity-hsm test 35: Overwrite file during restore ===== 16:10:57 (1768252257) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_35/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00362212 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg358-client.virtnet Copytool is continued on oleg358-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2898: kill: (63306) - No such process PASS 35 (12s) == sanity-hsm test 36: Move file during restore ========== 16:11:09 (1768252269) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_36/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00399167 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg358-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Copytool is continued on oleg358-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2934: kill: (63988) - No such process PASS 36 (15s) == sanity-hsm test 37: re-archive a dirty file =========== 16:11:24 (1768252284) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_37/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258073 s, 4.1 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675288 s, 15.5 MB/s PASS 37 (74s) == sanity-hsm test 40: Parallel archive requests ========= 16:12:38 (1768252358) oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1152921504606846976: Cannot allocate memory oleg358-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1152921504606846976': Cannot allocate memory pdsh@oleg358-client: oleg358-server: ssh exited with exit code 12 oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=1152921504606846976: Cannot allocate memory oleg358-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='1152921504606846976': Cannot allocate memory pdsh@oleg358-client: oleg358-server: ssh exited with exit code 12 set max_requests=1152921504606846976 failed with 12 [ 1594.028697] Lustre: lustre-MDT0000: No more memory to set HSM max_requests=1152921504606846976 (max request memory: 461MB, current total 6/975940): rc = -12 [ 1595.008575] Lustre: lustre-MDT0001: No more memory to set HSM max_requests=1152921504606846976 (max request memory: 461MB, current total 975940/975940): rc = -12 Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_40/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Wait for all requests being enqueued...100 archive requests submitted 100 archive requests submitted 100 archive requests submitted 100 archive requests submitted OK pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 PASS 40 (153s) == sanity-hsm test 50: Archive with large number of pending HSM actions ========================================================== 16:15:12 (1768252512) oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1000000: Cannot allocate memory oleg358-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1000000': Cannot allocate memory pdsh@oleg358-client: oleg358-server: ssh exited with exit code 12 oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=1000000: Cannot allocate memory oleg358-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='1000000': Cannot allocate memory pdsh@oleg358-client: oleg358-server: ssh exited with exit code 12 Filesystem Inodes IUsed IFree IUse% Mounted on 192.168.203.158@tcp:/lustre 222273 1336 220937 1% /mnt/lustre - create 1777 (time 1768252529.51 total 10.00 last 177.66) - create 3453 (time 1768252539.51 total 20.00 last 167.57) - create 5165 (time 1768252549.51 total 30.00 last 171.19) - create 7243 (time 1768252559.51 total 40.01 last 207.78) - create 9151 (time 1768252569.52 total 50.01 last 190.72) total: 10000 create in 54.22 seconds: 184.44 ops/second 10000 - unlinked 0 (time 1768252652 ; total 0 ; last 0) total: 10000 unlinks in 56 seconds: 178.571426 unlinks/second Start Phase files_num: 10000 time: 72 - create 1927 (time 1768252722.30 total 10.00 last 192.66) - create 3927 (time 1768252732.30 total 20.00 last 200.00) - create 5805 (time 1768252742.31 total 30.01 last 187.65) - create 7069 (time 1768252752.32 total 40.02 last 126.32) - create 8684 (time 1768252762.32 total 50.02 last 161.45) - create 10000 (time 1768252769.49 total 57.19 last 183.50) - create 11874 (time 1768252779.50 total 67.20 last 187.32) - create 13773 (time 1768252789.50 total 77.20 last 189.89) - create 15437 (time 1768252799.51 total 87.20 last 166.26) - create 17309 (time 1768252809.51 total 97.21 last 187.11) - create 18771 (time 1768252819.51 total 107.21 last 146.18) total: 20000 create in 114.86 seconds: 174.12 ops/second - unlinked 0 (time 1768252988 ; total 0 ; last 0) - unlinked 10000 (time 1768253055 ; total 67 ; last 67) total: 20000 unlinks in 128 seconds: 156.250000 unlinks/second Middle Phase files_num: 20000 time: 157 - create 1760 (time 1768253130.24 total 10.01 last 175.89) - create 3628 (time 1768253140.24 total 20.01 last 186.78) - create 5508 (time 1768253150.24 total 30.01 last 187.97) - create 7140 (time 1768253160.25 total 40.01 last 163.11) - create 8407 (time 1768253170.25 total 50.02 last 126.63) - create 9832 (time 1768253180.26 total 60.02 last 142.45) total: 10000 create in 61.53 seconds: 162.52 ops/second 40000 - unlinked 0 (time 1768253269 ; total 0 ; last 0) total: 10000 unlinks in 58 seconds: 172.413788 unlinks/second End Phase files_num: 10000 time: 80 40000 mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 50 (832s) == sanity-hsm test 52: Opened for write file on an evicted client should be set dirty ========================================================== 16:29:03 (1768253343) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_52/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182106 s, 5.8 MB/s multiop /mnt/lustre/d52.sanity-hsm/f52.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8387 pdsh@oleg358-client: oleg358-client: ssh exited with exit code 5 PASS 52 (14s) == sanity-hsm test 53: Opened for read file on an evicted client should not be set dirty ========================================================== 16:29:17 (1768253357) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_53/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313567 s, 3.3 MB/s multiop /mnt/lustre/d53.sanity-hsm/f53.sanity-hsm vo_c TMPPIPE=/tmp/multiop_open_wait_pipe.8387 pdsh@oleg358-client: oleg358-client: ssh exited with exit code 5 PASS 53 (15s) == sanity-hsm test 54: Write during an archive cancels it ========================================================== 16:29:32 (1768253372) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 1.0287 s, 37.9 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_54/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Waiting 200s for 'FAILED' Waiting 170s for 'FAILED' Updated after 34s: want 'FAILED' got 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 54 (49s) == sanity-hsm test 55: Truncate during an archive cancels it ========================================================== 16:30:21 (1768253421) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 1.01428 s, 38.5 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_55/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 55 (13s) == sanity-hsm test 56: Setattr during an archive is ok === 16:30:34 (1768253434) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 1.09271 s, 35.7 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_56/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Waiting 170s for 'SUCCEED' Updated after 36s: want 'SUCCEED' got 'SUCCEED' PASS 56 (47s) == sanity-hsm test 57: Archive a file with dirty cache on another node ========================================================== 16:31:21 (1768253481) SKIP: sanity-hsm test_57 Need 2 or more clients, have 1 SKIP 57 (3s) == sanity-hsm test 58: Truncate a released file will trigger restore ========================================================== 16:31:24 (1768253484) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_58/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' truncate up from 1516 to 3032 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate down from 1516 to 758 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate to 0 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 PASS 58 (19s) == sanity-hsm test 59: Release stripeless file with non-zero size ========================================================== 16:31:43 (1768253503) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_59/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 59 (8s) == sanity-hsm test 60: Changing progress update interval from default ========================================================== 16:31:51 (1768253511) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.282071 s, 37.2 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_60/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 --update-interval 5 "/mnt/lustre2"' Waiting 100s for 'start copy' Expecting a progress update within 20 seconds... 5242880 bytes copied in 3 seconds. Wait for on going archive hsm action to complete Waiting 10s for 'copied' Updated after 3s: want 'copied' got 'copied' PASS 60 (19s) == sanity-hsm test 61: Waiting archive of a removed file should fail ========================================================== 16:32:10 (1768253530) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_61/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 61 (16s) == sanity-hsm test 70: Copytool logs JSON register/unregister events to FIFO ========================================================== 16:32:26 (1768253546) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_70/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --event-fifo /tmp/sanity-hsm.test_70.4QeK/fifo "/mnt/lustre2"' Killing existing copytools on oleg358-client.virtnet copytools stopped in 0s Register/Unregister events look OK. PASS 70 (10s) == sanity-hsm test 71: Copytool logs JSON archive events to FIFO ========================================================== 16:32:36 (1768253556) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_71/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_71.W2wD/fifo "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172027 s, 6.1 MB/s Waiting 200s for 'SUCCEED' Archive events look OK. mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 71 (22s) == sanity-hsm test 72: Copytool logs JSON restore events to FIFO ========================================================== 16:32:58 (1768253578) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_72/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_72.k2hr/fifo "/mnt/lustre2"' 16+0 records in 16+0 records out 16000000 bytes (16 MB, 15 MiB) copied, 0.347823 s, 46.0 MB/s lhsmtool_posix: 1768253582.860417 [82944]: action=1 src=d72.sanity-hsm/f72.sanity-hsm dst=/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768253582.868951 [82944]: importing '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1768253582.926529 [82944]: imported '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//0bc5/0x200000bd2:0x17:0x0'=='/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1768253582.927413 [82944]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Restore events look OK. PASS 72 (18s) == sanity-hsm test 90: Archive/restore a file list ======= 16:33:16 (1768253596) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_90/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Waiting 200s for '' pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Waiting 200s for '' Changed after 2s: from 'lrh=[type=10680000 len=136 idx=4/9652] fid=[0x200000403:0xc4:0x0] dfid=[0x200000403:0xc4:0x0] compound/cookie=0x0/0x6965fc32 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/9655] fid=[0x200000403:0xc7:0x0] dfid=[0x200000403:0xc7:0x0] compound/cookie=0x0/0x6965fc35 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9656] fid=[0x200000403:0xc8:0x0] dfid=[0x200000403:0xc8:0x0] compound/cookie=0x0/0x6965fc36 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9657] fid=[0x200000403:0xc9:0x0] dfid=[0x200000403:0xc9:0x0] compound/cookie=0x0/0x6965fc37 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9658] fid=[0x200000403:0xca:0x0] dfid=[0x200000403:0xca:0x0] compound/cookie=0x0/0x6965fc38 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9659] fid=[0x200000403:0xcb:0x0] dfid=[0x200000403:0xcb:0x0] compound/cookie=0x0/0x6965fc39 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9660] fid=[0x200000403:0xcc:0x0] dfid=[0x200000403:0xcc:0x0] compound/cookie=0x0/0x6965fc3a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9661] fid=[0x200000403:0xcd:0x0] dfid=[0x200000403:0xcd:0x0] compound/cookie=0x0/0x6965fc3b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9662] fid=[0x200000403:0xce:0x0] dfid=[0x200000403:0xce:0x0] compound/cookie=0x0/0x6965fc3c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9663] fid=[0x200000403:0xcf:0x0] dfid=[0x200000403:0xcf:0x0] compound/cookie=0x0/0x6965fc3d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9664] fid=[0x200000403:0xd0:0x0] dfid=[0x200000403:0xd0:0x0] compound/cookie=0x0/0x6965fc3e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9665] fid=[0x200000403:0xd1:0x0] dfid=[0x200000403:0xd1:0x0] compound/cookie=0x0/0x6965fc3f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9666] fid=[0x200000403:0xd2:0x0] dfid=[0x200000403:0xd2:0x0] compound/cookie=0x0/0x6965fc40 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9667] fid=[0x200000403:0xd3:0x0] dfid=[0x200000403:0xd3:0x0] compound/cookie=0x0/0x6965fc41 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9668] fid=[0x200000403:0xd4:0x0] dfid=[0x200000403:0xd4:0x0] compound/cookie=0x0/0x6965fc42 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9669] fid=[0x200000403:0xd5:0x0] dfid=[0x200000403:0xd5:0x0] compound/cookie=0x0/0x6965fc43 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9670] fid=[0x200000403:0xd6:0x0] dfid=[0x200000403:0xd6:0x0] compound/cookie=0x0/0x6965fc44 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9671] fid=[0x200000403:0xd7:0x0] dfid=[0x200000403:0xd7:0x0] compound/cookie=0x0/0x6965fc45 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9672] fid=[0x200000403:0xd8:0x0] dfid=[0x200000403:0xd8:0x0] compound/cookie=0x0/0x6965fc46 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9673] fid=[0x200000403:0xd9:0x0] dfid=[0x200000403:0xd9:0x0] compound/cookie=0x0/0x6965fc47 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9674] fid=[0x200000403:0xda:0x0] dfid=[0x200000403:0xda:0x0] compound/cookie=0x0/0x6965fc48 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9675] fid=[0x200000403:0xdb:0x0] dfid=[0x200000403:0xdb:0x0] compound/cookie=0x0/0x6965fc49 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9676] fid=[0x200000403:0xdc:0x0] dfid=[0x200000403:0xdc:0x0] compound/cookie=0x0/0x6965fc4a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9677] fid=[0x200000403:0xdd:0x0] dfid=[0x200000403:0xdd:0x0] compound/cookie=0x0/0x6965fc4b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9678] fid=[0x200000403:0xde:0x0] dfid=[0x200000403:0xde:0x0] compound/cookie=0x0/0x6965fc4c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9679] fid=[0x200000403:0xdf:0x0] dfid=[0x200000403:0xdf:0x0] compound/cookie=0x0/0x6965fc4d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9680] fid=[0x200000403:0xe0:0x0] dfid=[0x200000403:0xe0:0x0] compound/cookie=0x0/0x6965fc4e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9681] fid=[0x200000403:0xe1:0x0] dfid=[0x200000403:0xe1:0x0] compound/cookie=0x0/0x6965fc4f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9682] fid=[0x200000403:0xe2:0x0] dfid=[0x200000403:0xe2:0x0] compound/cookie=0x0/0x6965fc50 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9683] fid=[0x200000403:0xe3:0x0] dfid=[0x200000403:0xe3:0x0] compound/cookie=0x0/0x6965fc51 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9684] fid=[0x200000403:0xe4:0x0] dfid=[0x200000403:0xe4:0x0] compound/cookie=0x0/0x6965fc52 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9685] fid=[0x200000403:0xe5:0x0] dfid=[0x200000403:0xe5:0x0] compound/cookie=0x0/0x6965fc53 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9686] fid=[0x200000403:0xe6:0x0] dfid=[0x200000403:0xe6:0x0] compound/cookie=0x0/0x6965fc54 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9687] fid=[0x200000403:0xe7:0x0] dfid=[0x200000403:0xe7:0x0] compound/cookie=0x0/0x6965fc55 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9688] fid=[0x200000403:0xe8:0x0] dfid=[0x200000403:0xe8:0x0] compound/cookie=0x0/0x6965fc56 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9689] fid=[0x200000403:0xe9:0x0] dfid=[0x200000403:0xe9:0x0] compound/cookie=0x0/0x6965fc57 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9690] fid=[0x200000403:0xea:0x0] dfid=[0x200000403:0xea:0x0] compound/cookie=0x0/0x6965fc58 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9691] fid=[0x200000403:0xeb:0x0] dfid=[0x200000403:0xeb:0x0] compound/cookie=0x0/0x6965fc59 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9692] fid=[0x200000403:0xec:0x0] dfid=[0x200000403:0xec:0x0] compound/cookie=0x0/0x6965fc5a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9693] fid=[0x200000403:0xed:0x0] dfid=[0x200000403:0xed:0x0] compound/cookie=0x0/0x6965fc5b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9694] fid=[0x200000403:0xee:0x0] dfid=[0x200000403:0xee:0x0] compound/cookie=0x0/0x6965fc5c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9695] fid=[0x200000403:0xef:0x0] dfid=[0x200000403:0xef:0x0] compound/cookie=0x0/0x6965fc5d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9696] fid=[0x200000403:0xf0:0x0] dfid=[0x200000403:0xf0:0x0] compound/cookie=0x0/0x6965fc5e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9697] fid=[0x200000403:0xf1:0x0] dfid=[0x200000403:0xf1:0x0] compound/cookie=0x0/0x6965fc5f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9698] fid=[0x200000403:0xf2:0x0] dfid=[0x200000403:0xf2:0x0] compound/cookie=0x0/0x6965fc60 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9700] fid=[0x200000403:0xf3:0x0] dfid=[0x200000403:0xf3:0x0] compound/cookie=0x0/0x6965fc61 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9701] fid=[0x200000403:0xf4:0x0] dfid=[0x200000403:0xf4:0x0] compound/cookie=0x0/0x6965fc62 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9702] fid=[0x200000403:0xf5:0x0] dfid=[0x200000403:0xf5:0x0] compound/cookie=0x0/0x6965fc63 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9703] fid=[0x200000403:0xf6:0x0] dfid=[0x200000403:0xf6:0x0] compound/cookie=0x0/0x6965fc64 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=4/9670] fid=[0x200000403:0xd6:0x0] dfid=[0x200000403:0xd6:0x0] compound/cookie=0x0/0x6965fc44 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/9671] fid=[0x200000403:0xd7:0x0] dfid=[0x200000403:0xd7:0x0] compound/cookie=0x0/0x6965fc45 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/9672] fid=[0x200000403:0xd8:0x0] dfid=[0x200000403:0xd8:0x0] compound/cookie=0x0/0x6965fc46 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/9673] fid=[0x200000403:0xd9:0x0] dfid=[0x200000403:0xd9:0x0] compound/cookie=0x0/0x6965fc47 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9674] fid=[0x200000403:0xda:0x0] dfid=[0x200000403:0xda:0x0] compound/cookie=0x0/0x6965fc48 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9675] fid=[0x200000403:0xdb:0x0] dfid=[0x200000403:0xdb:0x0] compound/cookie=0x0/0x6965fc49 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9676] fid=[0x200000403:0xdc:0x0] dfid=[0x200000403:0xdc:0x0] compound/cookie=0x0/0x6965fc4a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9677] fid=[0x200000403:0xdd:0x0] dfid=[0x200000403:0xdd:0x0] compound/cookie=0x0/0x6965fc4b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9678] fid=[0x200000403:0xde:0x0] dfid=[0x200000403:0xde:0x0] compound/cookie=0x0/0x6965fc4c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9679] fid=[0x200000403:0xdf:0x0] dfid=[0x200000403:0xdf:0x0] compound/cookie=0x0/0x6965fc4d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9680] fid=[0x200000403:0xe0:0x0] dfid=[0x200000403:0xe0:0x0] compound/cookie=0x0/0x6965fc4e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9681] fid=[0x200000403:0xe1:0x0] dfid=[0x200000403:0xe1:0x0] compound/cookie=0x0/0x6965fc4f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9682] fid=[0x200000403:0xe2:0x0] dfid=[0x200000403:0xe2:0x0] compound/cookie=0x0/0x6965fc50 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9683] fid=[0x200000403:0xe3:0x0] dfid=[0x200000403:0xe3:0x0] compound/cookie=0x0/0x6965fc51 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9684] fid=[0x200000403:0xe4:0x0] dfid=[0x200000403:0xe4:0x0] compound/cookie=0x0/0x6965fc52 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9685] fid=[0x200000403:0xe5:0x0] dfid=[0x200000403:0xe5:0x0] compound/cookie=0x0/0x6965fc53 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9686] fid=[0x200000403:0xe6:0x0] dfid=[0x200000403:0xe6:0x0] compound/cookie=0x0/0x6965fc54 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9687] fid=[0x200000403:0xe7:0x0] dfid=[0x200000403:0xe7:0x0] compound/cookie=0x0/0x6965fc55 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9688] fid=[0x200000403:0xe8:0x0] dfid=[0x200000403:0xe8:0x0] compound/cookie=0x0/0x6965fc56 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9689] fid=[0x200000403:0xe9:0x0] dfid=[0x200000403:0xe9:0x0] compound/cookie=0x0/0x6965fc57 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9690] fid=[0x200000403:0xea:0x0] dfid=[0x200000403:0xea:0x0] compound/cookie=0x0/0x6965fc58 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9691] fid=[0x200000403:0xeb:0x0] dfid=[0x200000403:0xeb:0x0] compound/cookie=0x0/0x6965fc59 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9692] fid=[0x200000403:0xec:0x0] dfid=[0x200000403:0xec:0x0] compound/cookie=0x0/0x6965fc5a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9693] fid=[0x200000403:0xed:0x0] dfid=[0x200000403:0xed:0x0] compound/cookie=0x0/0x6965fc5b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9694] fid=[0x200000403:0xee:0x0] dfid=[0x200000403:0xee:0x0] compound/cookie=0x0/0x6965fc5c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9695] fid=[0x200000403:0xef:0x0] dfid=[0x200000403:0xef:0x0] compound/cookie=0x0/0x6965fc5d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9696] fid=[0x200000403:0xf0:0x0] dfid=[0x200000403:0xf0:0x0] compound/cookie=0x0/0x6965fc5e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9697] fid=[0x200000403:0xf1:0x0] dfid=[0x200000403:0xf1:0x0] compound/cookie=0x0/0x6965fc5f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9698] fid=[0x200000403:0xf2:0x0] dfid=[0x200000403:0xf2:0x0] compound/cookie=0x0/0x6965fc60 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9700] fid=[0x200000403:0xf3:0x0] dfid=[0x200000403:0xf3:0x0] compound/cookie=0x0/0x6965fc61 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9701] fid=[0x200000403:0xf4:0x0] dfid=[0x200000403:0xf4:0x0] compound/cookie=0x0/0x6965fc62 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9702] fid=[0x200000403:0xf5:0x0] dfid=[0x200000403:0xf5:0x0] compound/cookie=0x0/0x6965fc63 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9703] fid=[0x200000403:0xf6:0x0] dfid=[0x200000403:0xf6:0x0] compound/cookie=0x0/0x6965fc64 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 3s: from 'lrh=[type=10680000 len=136 idx=4/9670] fid=[0x200000403:0xd6:0x0] dfid=[0x200000403:0xd6:0x0] compound/cookie=0x0/0x6965fc44 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/9671] fid=[0x200000403:0xd7:0x0] dfid=[0x200000403:0xd7:0x0] compound/cookie=0x0/0x6965fc45 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/9672] fid=[0x200000403:0xd8:0x0] dfid=[0x200000403:0xd8:0x0] compound/cookie=0x0/0x6965fc46 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/9673] fid=[0x200000403:0xd9:0x0] dfid=[0x200000403:0xd9:0x0] compound/cookie=0x0/0x6965fc47 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9674] fid=[0x200000403:0xda:0x0] dfid=[0x200000403:0xda:0x0] compound/cookie=0x0/0x6965fc48 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9675] fid=[0x200000403:0xdb:0x0] dfid=[0x200000403:0xdb:0x0] compound/cookie=0x0/0x6965fc49 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9676] fid=[0x200000403:0xdc:0x0] dfid=[0x200000403:0xdc:0x0] compound/cookie=0x0/0x6965fc4a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9677] fid=[0x200000403:0xdd:0x0] dfid=[0x200000403:0xdd:0x0] compound/cookie=0x0/0x6965fc4b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9678] fid=[0x200000403:0xde:0x0] dfid=[0x200000403:0xde:0x0] compound/cookie=0x0/0x6965fc4c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9679] fid=[0x200000403:0xdf:0x0] dfid=[0x200000403:0xdf:0x0] compound/cookie=0x0/0x6965fc4d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9680] fid=[0x200000403:0xe0:0x0] dfid=[0x200000403:0xe0:0x0] compound/cookie=0x0/0x6965fc4e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9681] fid=[0x200000403:0xe1:0x0] dfid=[0x200000403:0xe1:0x0] compound/cookie=0x0/0x6965fc4f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9682] fid=[0x200000403:0xe2:0x0] dfid=[0x200000403:0xe2:0x0] compound/cookie=0x0/0x6965fc50 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9683] fid=[0x200000403:0xe3:0x0] dfid=[0x200000403:0xe3:0x0] compound/cookie=0x0/0x6965fc51 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9684] fid=[0x200000403:0xe4:0x0] dfid=[0x200000403:0xe4:0x0] compound/cookie=0x0/0x6965fc52 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9685] fid=[0x200000403:0xe5:0x0] dfid=[0x200000403:0xe5:0x0] compound/cookie=0x0/0x6965fc53 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9686] fid=[0x200000403:0xe6:0x0] dfid=[0x200000403:0xe6:0x0] compound/cookie=0x0/0x6965fc54 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9687] fid=[0x200000403:0xe7:0x0] dfid=[0x200000403:0xe7:0x0] compound/cookie=0x0/0x6965fc55 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9688] fid=[0x200000403:0xe8:0x0] dfid=[0x200000403:0xe8:0x0] compound/cookie=0x0/0x6965fc56 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9689] fid=[0x200000403:0xe9:0x0] dfid=[0x200000403:0xe9:0x0] compound/cookie=0x0/0x6965fc57 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9690] fid=[0x200000403:0xea:0x0] dfid=[0x200000403:0xea:0x0] compound/cookie=0x0/0x6965fc58 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9691] fid=[0x200000403:0xeb:0x0] dfid=[0x200000403:0xeb:0x0] compound/cookie=0x0/0x6965fc59 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9692] fid=[0x200000403:0xec:0x0] dfid=[0x200000403:0xec:0x0] compound/cookie=0x0/0x6965fc5a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9693] fid=[0x200000403:0xed:0x0] dfid=[0x200000403:0xed:0x0] compound/cookie=0x0/0x6965fc5b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9694] fid=[0x200000403:0xee:0x0] dfid=[0x200000403:0xee:0x0] compound/cookie=0x0/0x6965fc5c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9695] fid=[0x200000403:0xef:0x0] dfid=[0x200000403:0xef:0x0] compound/cookie=0x0/0x6965fc5d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9696] fid=[0x200000403:0xf0:0x0] dfid=[0x200000403:0xf0:0x0] compound/cookie=0x0/0x6965fc5e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9697] fid=[0x200000403:0xf1:0x0] dfid=[0x200000403:0xf1:0x0] compound/cookie=0x0/0x6965fc5f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9698] fid=[0x200000403:0xf2:0x0] dfid=[0x200000403:0xf2:0x0] compound/cookie=0x0/0x6965fc60 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9700] fid=[0x200000403:0xf3:0x0] dfid=[0x200000403:0xf3:0x0] compound/cookie=0x0/0x6965fc61 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9701] fid=[0x200000403:0xf4:0x0] dfid=[0x200000403:0xf4:0x0] compound/cookie=0x0/0x6965fc62 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9702] fid=[0x200000403:0xf5:0x0] dfid=[0x200000403:0xf5:0x0] compound/cookie=0x0/0x6965fc63 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9703] fid=[0x200000403:0xf6:0x0] dfid=[0x200000403:0xf6:0x0] compound/cookie=0x0/0x6965fc64 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=4/9688] fid=[0x200000403:0xe8:0x0] dfid=[0x200000403:0xe8:0x0] compound/cookie=0x0/0x6965fc56 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/9689] fid=[0x200000403:0xe9:0x0] dfid=[0x200000403:0xe9:0x0] compound/cookie=0x0/0x6965fc57 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/9690] fid=[0x200000403:0xea:0x0] dfid=[0x200000403:0xea:0x0] compound/cookie=0x0/0x6965fc58 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/9691] fid=[0x200000403:0xeb:0x0] dfid=[0x200000403:0xeb:0x0] compound/cookie=0x0/0x6965fc59 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9692] fid=[0x200000403:0xec:0x0] dfid=[0x200000403:0xec:0x0] compound/cookie=0x0/0x6965fc5a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9693] fid=[0x200000403:0xed:0x0] dfid=[0x200000403:0xed:0x0] compound/cookie=0x0/0x6965fc5b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9694] fid=[0x200000403:0xee:0x0] dfid=[0x200000403:0xee:0x0] compound/cookie=0x0/0x6965fc5c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9695] fid=[0x200000403:0xef:0x0] dfid=[0x200000403:0xef:0x0] compound/cookie=0x0/0x6965fc5d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9696] fid=[0x200000403:0xf0:0x0] dfid=[0x200000403:0xf0:0x0] compound/cookie=0x0/0x6965fc5e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9697] fid=[0x200000403:0xf1:0x0] dfid=[0x200000403:0xf1:0x0] compound/cookie=0x0/0x6965fc5f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9698] fid=[0x200000403:0xf2:0x0] dfid=[0x200000403:0xf2:0x0] compound/cookie=0x0/0x6965fc60 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9700] fid=[0x200000403:0xf3:0x0] dfid=[0x200000403:0xf3:0x0] compound/cookie=0x0/0x6965fc61 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9701] fid=[0x200000403:0xf4:0x0] dfid=[0x200000403:0xf4:0x0] compound/cookie=0x0/0x6965fc62 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9702] fid=[0x200000403:0xf5:0x0] dfid=[0x200000403:0xf5:0x0] compound/cookie=0x0/0x6965fc63 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/9703] fid=[0x200000403:0xf6:0x0] dfid=[0x200000403:0xf6:0x0] compound/cookie=0x0/0x6965fc64 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 PASS 90 (88s) == sanity-hsm test 100: Set coordinator /proc tunables === 16:34:44 (1768253684) Testing mdt.lustre-MDT0000.hsm.loop_period Set loop_period to 0 must failed oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/loop_period=0: Invalid argument oleg358-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/loop_period'='0': Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/loop_period=0: Invalid argument oleg358-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/loop_period'='0': Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.grace_delay Set grace_delay to 0 must failed oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/grace_delay=0: Invalid argument oleg358-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/grace_delay'='0': Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/grace_delay=0: Invalid argument oleg358-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/grace_delay'='0': Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.active_request_timeout Set active_request_timeout to 0 must failed oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/active_request_timeout=0: Invalid argument oleg358-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/active_request_timeout'='0': Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/active_request_timeout=0: Invalid argument oleg358-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/active_request_timeout'='0': Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.max_requests Set max_requests to 0 must failed oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=0: Invalid argument oleg358-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='0': Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=0: Invalid argument oleg358-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='0': Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.default_archive_id Set default_archive_id to 0 must failed oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/default_archive_id=0: Invalid argument oleg358-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/default_archive_id'='0': Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 oleg358-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/default_archive_id=0: Invalid argument oleg358-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/default_archive_id'='0': Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 PASS 100 (44s) == sanity-hsm test 102: Verify coordinator control ======= 16:35:28 (1768253728) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 102 (19s) == sanity-hsm test 103: Purge all requests =============== 16:35:48 (1768253748) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Current requests pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 PASS 103 (13s) == sanity-hsm test 103a: Purge pending restore requests == 16:36:01 (1768253761) mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' (0x200000403:0x12a:0x0|0x200000403:0x12b:0x0|0x200000403:0x12c:0x0|0x200000403:0x12d:0x0|0x200000403:0x12e:0x0|0x200000403:0x12f:0x0|0x200000403:0x130:0x0|0x200000403:0x131:0x0).*action=ARCHIVE.*status=SUCCEED Killing existing copytools on oleg358-client.virtnet copytools stopped in 0s cat: '/mnt/lustre/.lustre/fid/0x200000403:0x12a:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x12e:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x12c:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x12f:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x12b:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x131:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x12d:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x130:0x0': No data available mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 103a (17s) == sanity-hsm test 104: Copy tool data field ============= 16:36:18 (1768253778) 0+0 records in 0+0 records out 0 bytes copied, 0.00493325 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 104 (10s) == sanity-hsm test 105: Restart of coordinator =========== 16:36:28 (1768253788) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 105 (32s) == sanity-hsm test 106: Copytool register/unregister ===== 16:37:01 (1768253821) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg358-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg358-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=6eb0ca4f-e1e8-4487-8c1c-58970c977094 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=6eb0ca4f-e1e8-4487-8c1c-58970c977094 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg358-client.virtnet: 89709 Killing existing copytools on oleg358-client.virtnet copytools stopped in 0s pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 uuid not found in agent list on mds2 Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg358-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg358-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=6eb0ca4f-e1e8-4487-8c1c-58970c977094 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=6eb0ca4f-e1e8-4487-8c1c-58970c977094 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 106 (27s) == sanity-hsm test 107: Copytool re-register after MDS restart ========================================================== 16:37:27 (1768253847) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_107/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Failing mds1 on oleg358-server Stopping /mnt/lustre-mds1 (opts:) on oleg358-server 16:37:34 (1768253854) shut down facet: mds1 facet_host: oleg358-server facet_failover_host: oleg358-server Failover mds1 to oleg358-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Started lustre-MDT0000 16:37:54 (1768253874) targets are mounted 16:37:54 (1768253874) facet_failover done oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 107 (43s) == sanity-hsm test 109: Policy display/change ============ 16:38:10 (1768253890) mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0000.hsm.policy=NRA NBR Next set_param must failed oleg358-server: error: set_param: setting /sys/kernel/debug/lustre/mdt/lustre-MDT0000/hsm/policy=wrong: Invalid argument oleg358-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/policy'='wrong': Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 22 Back to default policy mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 109 (22s) == sanity-hsm test 110a: Non blocking restore policy (import case) ========================================================== 16:38:32 (1768253912) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1768253915.055956 [93556]: action=1 src=d110a.sanity-hsm/f110a.sanity-hsm dst=/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768253915.063209 [93556]: importing '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1768253915.099599 [93556]: imported '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//0bb1/0x200000bd2:0x63:0x0'=='/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1768253915.100261 [93556]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0001.hsm.policy=+NBR md5sum: /mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm: No data available Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 110a (13s) == sanity-hsm test 110b: Non blocking restore policy (release case) ========================================================== 16:38:45 (1768253925) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0001.hsm.policy=+NBR md5sum: /mnt/lustre/d110b.sanity-hsm/f110b.sanity-hsm: No data available Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 110b (14s) == sanity-hsm test 111a: No retry policy (import case), restore will error ========================================================== 16:38:59 (1768253939) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1768253941.273785 [94963]: action=1 src=d111a.sanity-hsm/f111a.sanity-hsm dst=/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768253941.290581 [94963]: importing '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1768253941.369995 [94963]: imported '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//0bba/0x200000bd2:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1768253941.370062 [94963]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Remove from backend: 0x200000bd2:0x68:0x0 = /tmp/arc1/sanity-hsm.test_111a/0bba/0x200000bd2:0x68:0x0 Waiting 200s for 'FAILED' Updated after 2s: want 'FAILED' got 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 111a (11s) == sanity-hsm test 111b: No retry policy (release case), restore will error ========================================================== 16:39:10 (1768253950) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Remove from backend: 0x200000403:0x139:0x0 = /tmp/arc1/sanity-hsm.test_111b/053a/0x200000403:0x139:0x0 mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 111b (11s) == sanity-hsm test 112: State of recorded request ======== 16:39:21 (1768253961) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_112/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled /mnt/lustre/d112.sanity-hsm/f112.sanity-hsm: ARCHIVE waiting (from 0 to EOF) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 112 (14s) == sanity-hsm test 113: wrong stat after restore ========= 16:39:35 (1768253975) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0754417 s, 13.9 MB/s Starting client: oleg358-client.virtnet: -o user_xattr,flock 192.168.203.158@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_113/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Verifying released state: multiop /mnt/lustre/d113.sanity-hsm/f113.sanity-hsm voO_WRONLY:O_APPEND:_w4c TMPPIPE=/tmp/multiop_open_wait_pipe.8387 File: /mnt/lustre2/d113.sanity-hsm/f113.sanity-hsm Size: 1048576 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205306057021 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-12 16:39:36.000000000 -0500 Modify: 2026-01-12 16:39:36.000000000 -0500 Change: 2026-01-12 16:39:36.000000000 -0500 Birth: 2026-01-12 16:39:36.000000000 -0500 Stopping client oleg358-client.virtnet /mnt/lustre3 (opts:) PASS 113 (12s) == sanity-hsm test 114: Incompatible request does not set other requests as STARTED ========================================================== 16:39:47 (1768253987) 0+0 records in 0+0 records out 0 bytes copied, 0.0155829 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00698145 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_114/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 114 (16s) == sanity-hsm test 200: Register/Cancel archive ========== 16:40:03 (1768254003) 0+0 records in 0+0 records out 0 bytes copied, 0.00259546 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_200/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg358-client.virtnet Copytool is continued on oleg358-client.virtnet PASS 200 (9s) == sanity-hsm test 201: Register/Cancel restore ========== 16:40:12 (1768254012) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_201/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.0232 s, 40.0 MB/s lhsmtool_posix: 1768254015.129892 [99355]: action=1 src=d201.sanity-hsm/f201.sanity-hsm dst=/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768254015.134318 [99355]: importing '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1768254015.175983 [99355]: imported '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//0ba3/0x200000bd2:0x71:0x0'=='/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1768254015.176039 [99355]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 201 (14s) == sanity-hsm test 202: Register/Cancel remove =========== 16:40:26 (1768254026) 0+0 records in 0+0 records out 0 bytes copied, 0.00674847 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_202/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Copytool is suspended on oleg358-client.virtnet Waiting 200s for 'CANCELED' Updated after 2s: want 'CANCELED' got 'CANCELED' PASS 202 (11s) == sanity-hsm test 220A: Changelog for archive =========== 16:40:37 (1768254037) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' lustre-MDT0000.3 16HSM 21:40:42.827361945 2026.01.12 0x0 t=[0x200000403:0x145:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 220A (14s) == sanity-hsm test 220a: Changelog for failed archive ==== 16:40:51 (1768254051) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' Copytool is suspended on oleg358-client.virtnet Copytool is continued on oleg358-client.virtnet lustre-MDT0000.6 16HSM 21:40:57.468930848 2026.01.12 0x2 t=[0x200000403:0x146:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 220a (15s) == sanity-hsm test 221: Changelog for archive canceled === 16:41:06 (1768254066) 0+0 records in 0+0 records out 0 bytes copied, 0.00379265 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_221/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl3 cl3' Copytool is suspended on oleg358-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Waiting 200s for 'CANCELED' Copytool is continued on oleg358-client.virtnet lustre-MDT0000.8 16HSM 21:41:16.746194007 2026.01.12 0x7d t=[0x200000403:0x148:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001: Deregistered changelog user #3 lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 221 (20s) == sanity-hsm test 222a: Changelog for explicit restore == 16:41:26 (1768254086) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1768254087.883695 [103224]: action=1 src=d222a.sanity-hsm/f222a.sanity-hsm dst=/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768254087.890929 [103224]: importing '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1768254087.940974 [103224]: imported '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//0baa/0x200000bd2:0x78:0x0'=='/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1768254087.941980 [103224]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl4 cl4' Waiting 200s for 'SUCCEED' lustre-MDT0000.11 16HSM 21:41:32.273217475 2026.01.12 0x80 t=[0x200000bd2:0x78:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0001: Deregistered changelog user #4 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 222a (15s) == sanity-hsm test 222b: Changelog for implicit restore == 16:41:41 (1768254101) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl5 cl5' 4ef076ad20e5d4a2f4729d6ed41a3bf9 /mnt/lustre/d222b.sanity-hsm/f222b.sanity-hsm lustre-MDT0000.20 16HSM 21:41:48.025114954 2026.01.12 0x80 t=[0x200000403:0x14a:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0001: Deregistered changelog user #5 lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 PASS 222b (16s) == sanity-hsm test 222c: Changelog for failed explicit restore ========================================================== 16:41:57 (1768254117) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1768254119.481589 [104860]: action=1 src=d222c.sanity-hsm/f222c.sanity-hsm dst=/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768254119.487911 [104860]: importing '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1768254119.538986 [104860]: imported '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//0baf/0x200000bd2:0x7d:0x0'=='/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1768254119.539463 [104860]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl6 cl6' Copytool is suspended on oleg358-client.virtnet Waiting 200s for 'STARTED' Copytool is continued on oleg358-client.virtnet lustre-MDT0000.22 16HSM 21:42:05.956431436 2026.01.12 0x82 t=[0x200000bd2:0x7d:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001: Deregistered changelog user #6 lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 222c (18s) == sanity-hsm test 222d: Changelog for failed implicit restore ========================================================== 16:42:15 (1768254135) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl7 cl7' Remove from backend: 0x200000403:0x14c:0x0 = /tmp/arc1/sanity-hsm.test_222d/054f/0x200000403:0x14c:0x0 md5sum: /mnt/lustre/d222d.sanity-hsm/f222d.sanity-hsm: No data available lustre-MDT0000.29 16HSM 21:42:21.465887627 2026.01.12 0x82 t=[0x200000403:0x14c:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0001: Deregistered changelog user #7 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 222d (15s) == sanity-hsm test 223a: Changelog for restore canceled (import case) ========================================================== 16:42:30 (1768254150) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.637561 s, 64.1 MB/s mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl8 cl8' lhsmtool_posix: 1768254156.524277 [106610]: action=1 src=d223a.sanity-hsm/f223a.sanity-hsm dst=/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1768254156.527954 [106610]: importing '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1768254156.562826 [106610]: imported '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//0b53/0x200000bd2:0x81:0x0'=='/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1768254156.564199 [106610]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Waiting 200s for 'SUCCEED' Waiting 180s for 'SUCCEED' Updated after 29s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.36 16HSM 21:43:06.709682034 2026.01.12 0xfd t=[0x200000bd2:0x81:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp p=[0x200000bd2:0x7b:0x0] lustre-MDT0001: clear the changelog for cl8 of all records lustre-MDT0001: Deregistered changelog user #8 lustre-MDT0000: clear the changelog for cl8 of all records lustre-MDT0000: Deregistered changelog user #8 PASS 223a (46s) == sanity-hsm test 223b: Changelog for restore canceled (release case) ========================================================== 16:43:16 (1768254196) 0+0 records in 0+0 records out 0 bytes copied, 0.00691081 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl9 cl9' Copytool is suspended on oleg358-client.virtnet Waiting 200s for 'CANCELED' Copytool is continued on oleg358-client.virtnet lustre-MDT0000.43 16HSM 21:43:25.139829711 2026.01.12 0xfd t=[0x200000403:0x14f:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp p=[0x200000bd2:0x74:0x0] lustre-MDT0001: clear the changelog for cl9 of all records lustre-MDT0001: Deregistered changelog user #9 lustre-MDT0000: clear the changelog for cl9 of all records lustre-MDT0000: Deregistered changelog user #9 PASS 223b (18s) == sanity-hsm test 224A: Changelog for remove ============ 16:43:34 (1768254214) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl10 cl10' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.48 16HSM 21:43:40.688361130 2026.01.12 0x200 t=[0x200000403:0x151:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001: clear the changelog for cl10 of all records lustre-MDT0001: Deregistered changelog user #10 lustre-MDT0000: clear the changelog for cl10 of all records lustre-MDT0000: Deregistered changelog user #10 PASS 224A (16s) == sanity-hsm test 224a: Changelog for failed remove ===== 16:43:51 (1768254231) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl11 cl11' Waiting 200s for 'SUCCEED' Remove from backend: 0x200000403:0x152:0x0 = /tmp/arc1/sanity-hsm.test_224a/0551/0x200000403:0x152:0x0 Copytool is suspended on oleg358-client.virtnet Copytool is continued on oleg358-client.virtnet lustre-MDT0000.53 16HSM 21:43:58.622490406 2026.01.12 0x202 t=[0x200000403:0x152:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp p=[0x200000bd2:0x85:0x0] lustre-MDT0001: clear the changelog for cl11 of all records lustre-MDT0001: Deregistered changelog user #11 lustre-MDT0000: clear the changelog for cl11 of all records lustre-MDT0000: Deregistered changelog user #11 PASS 224a (16s) == sanity-hsm test 225: Changelog for remove canceled ==== 16:44:07 (1768254247) Test disabled PASS 225 (4s) == sanity-hsm test 226: changelog for last rm/mv with exiting archive ========================================================== 16:44:11 (1768254251) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_226/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0x200000403:0x155:0x0 mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl12 cl12' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.54 16HSM 21:44:17.084957159 2026.01.12 0x280 t=[0x200000403:0x153:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.55 16HSM 21:44:17.215841450 2026.01.12 0x280 t=[0x200000403:0x153:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp p=[0x200000bd2:0x85:0x0] lustre-MDT0000.56 16HSM 21:44:17.215983678 2026.01.12 0x0 t=[0x200000403:0x153:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp p=[0x200000bd2:0x85:0x0] lustre-MDT0000.57 16HSM 21:44:18.240022063 2026.01.12 0x280 t=[0x200000403:0x154:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.58 16HSM 21:44:18.356756461 2026.01.12 0x280 t=[0x200000403:0x154:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp p=[0x200000bd2:0x85:0x0] lustre-MDT0000.59 16HSM 21:44:18.356919803 2026.01.12 0x0 t=[0x200000403:0x154:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp p=[0x200000bd2:0x85:0x0] lustre-MDT0000.60 06UNLNK 21:44:20.083139552 2026.01.12 0x3 t=[0x200000403:0x153:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp p=[0x200000bd2:0x86:0x0] f226.sanity-hsm-1 lustre-MDT0000.60 06UNLNK 21:44:20.083139552 2026.01.12 0x3 t=[0x200000403:0x153:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp p=[0x200000bd2:0x86:0x0] f226.sanity-hsm-1 lustre-MDT0000.61 08RENME 21:44:20.562180677 2026.01.12 0x3 t=[0x200000403:0x154:0x0] j=mv.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp p=[0x200000bd2:0x86:0x0] f226.sanity-hsm-2 s=[0x200000403:0x155:0x0] sp=[0x200000bd2:0x86:0x0] f226.sanity-hsm-3 lustre-MDT0001: clear the changelog for cl12 of all records lustre-MDT0001: Deregistered changelog user #12 lustre-MDT0000: clear the changelog for cl12 of all records lustre-MDT0000: Deregistered changelog user #12 PASS 226 (17s) == sanity-hsm test 227: changelog when explicit setting of HSM flags ========================================================== 16:44:28 (1768254268) 0+0 records in 0+0 records out 0 bytes copied, 0.00622298 s, 0.0 kB/s mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl13 cl13' lustre-MDT0001: clear the changelog for cl13 of all records lustre-MDT0001: Deregistered changelog user #13 lustre-MDT0000: clear the changelog for cl13 of all records lustre-MDT0000: Deregistered changelog user #13 PASS 227 (14s) == sanity-hsm test 228: On released file, return extend to FIEMAP. For [cp,tar] --sparse ========================================================== 16:44:42 (1768254282) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_228/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0558872 s, 18.8 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/f228.sanity-hsm: 1 extent found tar: Removing leading `/' from member names mnt/lustre/f228.sanity-hsm PASS 228 (10s) == sanity-hsm test 250: Coordinator max request ========== 16:44:52 (1768254292) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_250/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00382463 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00290653 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00373636 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00456858 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 0+0 records in 0+0 records out 0 bytes copied, 0.00241052 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00562693 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg358-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Copytool is continued on oleg358-client.virtnet PASS 250 (23s) == sanity-hsm test 251: Coordinator request timeout ====== 16:45:16 (1768254316) 0+0 records in 0+0 records out 0 bytes copied, 0.00802048 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_251/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg358-client.virtnet Waiting 200s for 'STARTED' PASS 251 (19s) == sanity-hsm test 252: Timeout'ed running archive of a removed file should be canceled ========================================================== 16:45:35 (1768254335) 0+0 records in 0+0 records out 0 bytes copied, 0.0101593 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_252/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg358-client.virtnet Copytool is continued on oleg358-client.virtnet PASS 252 (13s) == sanity-hsm test 253: Check for wrong file size after release ========================================================== 16:45:48 (1768254348) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_253/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 10+0 records in 10+0 records out 10000000 bytes (10 MB, 9.5 MiB) copied, 0.255552 s, 39.1 MB/s fail_loc=0x807 Cannot send HSM request (use of /mnt/lustre/d253.sanity-hsm/f253.sanity-hsm): Invalid argument could not release file PASS 253 (6s) == sanity-hsm test 254a: Request counters are initialized to zero ========================================================== 16:45:54 (1768254354) PASS 254a (6s) == sanity-hsm test 254b: Request counters are correctly incremented and decremented ========================================================== 16:46:00 (1768254360) Will launch 52 requests of each type Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_254b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Checking archive requests Copytool is suspended on oleg358-client.virtnet Copytool is continued on oleg358-client.virtnet Checking restore requests Copytool is suspended on oleg358-client.virtnet Copytool is continued on oleg358-client.virtnet Checking remove requests Copytool is suspended on oleg358-client.virtnet Copytool is continued on oleg358-client.virtnet PASS 254b (29s) == sanity-hsm test 255: Copytool registration wakes the coordinator up ========================================================== 16:46:29 (1768254389) 0+0 records in 0+0 records out 0 bytes copied, 0.00334389 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg358-client.virtnet copytools stopped in 0s 0+0 records in 0+0 records out 0 bytes copied, 0.00324251 s, 0.0 kB/s 0x200000403:0x1d1:0x0 mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 255 (19s) == sanity-hsm test 260a: Restore request have priority over other requests ========================================================== 16:46:48 (1768254408) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113862 s, 9.2 MB/s 0x200000403:0x1d2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124254 s, 8.4 MB/s 0x200000403:0x1d3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133323 s, 7.9 MB/s 0x200000403:0x1d4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102887 s, 10.2 MB/s 0x200000403:0x1d5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163414 s, 6.4 MB/s 0x200000403:0x1d6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150981 s, 6.9 MB/s 0x200000403:0x1d7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105548 s, 9.9 MB/s 0x200000403:0x1d8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134244 s, 7.8 MB/s 0x200000403:0x1d9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164751 s, 6.4 MB/s 0x200000403:0x1da:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115389 s, 9.1 MB/s 0x200000403:0x1db:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150408 s, 7.0 MB/s 0x200000403:0x1dc:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135374 s, 7.7 MB/s 0x200000403:0x1dd:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11928 s, 8.8 MB/s 0x200000403:0x1de:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159313 s, 6.6 MB/s 0x200000403:0x1df:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122118 s, 8.6 MB/s 0x200000403:0x1e0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164912 s, 6.4 MB/s 0x200000403:0x1e1:0x0 Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Killing existing copytools on oleg358-client.virtnet copytools stopped in 0s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' ARCHIVE ARCHIVE RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260a (26s) == sanity-hsm test 260b: Restore request have priority over other requests ========================================================== 16:47:14 (1768254434) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141109 s, 7.4 MB/s 0x200000403:0x1e4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131259 s, 8.0 MB/s 0x200000403:0x1e5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117066 s, 9.0 MB/s 0x200000403:0x1e6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137097 s, 7.6 MB/s 0x200000403:0x1e7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109921 s, 9.5 MB/s 0x200000403:0x1e8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116629 s, 9.0 MB/s 0x200000403:0x1e9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0947042 s, 11.1 MB/s 0x200000403:0x1ea:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155619 s, 6.7 MB/s 0x200000403:0x1eb:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154464 s, 6.8 MB/s 0x200000403:0x1ec:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0727772 s, 14.4 MB/s 0x200000403:0x1ed:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168473 s, 6.2 MB/s 0x200000403:0x1ee:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18019 s, 5.8 MB/s 0x200000403:0x1ef:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122967 s, 8.5 MB/s 0x200000403:0x1f0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104841 s, 10.0 MB/s 0x200000403:0x1f1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142471 s, 7.4 MB/s 0x200000403:0x1f2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153648 s, 6.8 MB/s 0x200000403:0x1f3:0x0 Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Killing existing copytools on oleg358-client.virtnet copytools stopped in 0s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260b (27s) == sanity-hsm test 260c: Requests are not reordered on the 'hot' path of the coordinator ========================================================== 16:47:42 (1768254462) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131778 s, 8.0 MB/s 0x200000403:0x1f5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19134 s, 5.5 MB/s 0x200000403:0x1f6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12852 s, 8.2 MB/s 0x200000403:0x1f7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1415 s, 7.4 MB/s 0x200000403:0x1f8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163072 s, 6.4 MB/s 0x200000403:0x1f9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0873506 s, 12.0 MB/s 0x200000403:0x1fa:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113067 s, 9.3 MB/s 0x200000403:0x1fb:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117506 s, 8.9 MB/s 0x200000403:0x1fc:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18538 s, 5.7 MB/s 0x200000403:0x1fd:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0996851 s, 10.5 MB/s 0x200000403:0x1fe:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130637 s, 8.0 MB/s 0x200000403:0x1ff:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114232 s, 9.2 MB/s 0x200000403:0x200:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11238 s, 9.3 MB/s 0x200000403:0x201:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155929 s, 6.7 MB/s 0x200000403:0x202:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128581 s, 8.2 MB/s 0x200000403:0x203:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104813 s, 10.0 MB/s 0x200000403:0x204:0x0 Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg358-client.virtnet copytools stopped in 1s mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE RESTORE PASS 260c (30s) == sanity-hsm test 261: Report 0 bytes size after HSM release ========================================================== 16:48:12 (1768254492) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_261/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0153888 s, 532 kB/s Waiting 200s for 'SUCCEED' /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x00000009) exists archived, archive_id:2 Waiting 200s for 'SUCCEED' /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x0000000d) released exists archived, archive_id:2 PASS 261 (10s) == sanity-hsm test 262: The client should return 1 block for HSM released files ========================================================== 16:48:22 (1768254502) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_262/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0119492 s, 686 kB/s PASS 262 (6s) == sanity-hsm test 300: On disk coordinator state kept between MDT umount/mount ========================================================== 16:48:28 (1768254508) Stop coordinator and remove coordinator state at mount mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown Failing mds1 on oleg358-server Stopping /mnt/lustre-mds1 (opts:) on oleg358-server 16:48:36 (1768254516) shut down facet: mds1 facet_host: oleg358-server facet_failover_host: oleg358-server Failover mds1 to oleg358-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Started lustre-MDT0000 16:48:53 (1768254533) targets are mounted 16:48:53 (1768254533) facet_failover done oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Set coordinator start at mount, and start coordinator Failing mds1 on oleg358-server Stopping /mnt/lustre-mds1 (opts:) on oleg358-server 16:49:25 (1768254565) shut down facet: mds1 facet_host: oleg358-server facet_failover_host: oleg358-server Failover mds1 to oleg358-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Started lustre-MDT0000 16:49:41 (1768254581) targets are mounted 16:49:41 (1768254581) facet_failover done oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 300 (83s) == sanity-hsm test 301: HSM tunnable are persistent ====== 16:49:51 (1768254591) Failing mds1 on oleg358-server Stopping /mnt/lustre-mds1 (opts:) on oleg358-server 16:49:55 (1768254595) shut down facet: mds1 facet_host: oleg358-server facet_failover_host: oleg358-server Failover mds1 to oleg358-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Started lustre-MDT0000 16:50:11 (1768254611) targets are mounted 16:50:11 (1768254611) facet_failover done oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 301 (31s) == sanity-hsm test 302: HSM tunnable are persistent when CDT is off ========================================================== 16:50:22 (1768254622) mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown Failing mds1 on oleg358-server Stopping /mnt/lustre-mds1 (opts:) on oleg358-server 16:50:29 (1768254629) shut down facet: mds1 facet_host: oleg358-server facet_failover_host: oleg358-server Failover mds1 to oleg358-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Started lustre-MDT0000 16:50:47 (1768254647) targets are mounted 16:50:47 (1768254647) facet_failover done oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds2 on oleg358-server Stopping /mnt/lustre-mds2 (opts:) on oleg358-server 16:50:56 (1768254656) shut down facet: mds2 facet_host: oleg358-server facet_failover_host: oleg358-server Failover mds2 to oleg358-server mount facets: mds2 Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Started lustre-MDT0001 16:51:12 (1768254672) targets are mounted 16:51:12 (1768254672) facet_failover done oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 302 (61s) == sanity-hsm test 400: Single request is sent to the right MDT ========================================================== 16:51:23 (1768254683) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_400/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13874 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107217 s, 9.8 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' archive successful on mdt0 archive successful on mdt1 PASS 400 (6s) == sanity-hsm test 401: Compound requests split and sent to their respective MDTs ========================================================== 16:51:29 (1768254689) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_401/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925363 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0915008 s, 11.5 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' archive successful on mdt0 archive successful on mdt1 PASS 401 (6s) == sanity-hsm test 402a: Copytool start fails if all MDTs are inactive ========================================================== 16:51:35 (1768254695) deactivate lustre-MDT0000-mdc-ffff995ac8841000 on oleg358-client.virtnet deactivate lustre-MDT0001-mdc-ffff995ac8841000 on oleg358-client.virtnet deactivate lustre-MDT0000-mdc-ffff995ac6623000 on oleg358-client.virtnet deactivate lustre-MDT0001-mdc-ffff995ac6623000 on oleg358-client.virtnet Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Failed to start copytool agt1 on 'oleg358-client.virtnet' pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 uuid not found in agent list on mds2 activate lustre-MDT0000-mdc-ffff995ac8841000 on oleg358-client.virtnet activate lustre-MDT0001-mdc-ffff995ac8841000 on oleg358-client.virtnet activate lustre-MDT0000-mdc-ffff995ac6623000 on oleg358-client.virtnet activate lustre-MDT0001-mdc-ffff995ac6623000 on oleg358-client.virtnet PASS 402a (4s) == sanity-hsm test 402b: CDT must retry request upon slow start of CT ========================================================== 16:51:40 (1768254700) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x14d fail_loc=0 Waiting 200s for 'SUCCEED' Updated after 3s: want 'SUCCEED' got 'SUCCEED' PASS 402b (18s) == sanity-hsm test 403: Copytool starts with inactive MDT and register on reconnect ========================================================== 16:51:58 (1768254718) deactivate lustre-MDT0001-mdc-ffff995ac8841000 on oleg358-client.virtnet deactivate lustre-MDT0001-mdc-ffff995ac6623000 on oleg358-client.virtnet Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_403/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg358-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg358-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=6eb0ca4f-e1e8-4487-8c1c-58970c977094 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 uuid not found in agent list on mds2 oleg358-client.virtnet: 132960 activate lustre-MDT0001-mdc-ffff995ac8841000 on oleg358-client.virtnet activate lustre-MDT0001-mdc-ffff995ac6623000 on oleg358-client.virtnet found agent uuid=6eb0ca4f-e1e8-4487-8c1c-58970c977094 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=6eb0ca4f-e1e8-4487-8c1c-58970c977094 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 403 (9s) == sanity-hsm test 404: Inactive MDT does not block requests for active MDTs ========================================================== 16:52:07 (1768254727) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_404/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0913336 s, 11.5 MB/s deactivate lustre-MDT0001-mdc-ffff995ac8841000 on oleg358-client.virtnet deactivate lustre-MDT0001-mdc-ffff995ac6623000 on oleg358-client.virtnet Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' archive successful on mdt0 activate lustre-MDT0001-mdc-ffff995ac8841000 on oleg358-client.virtnet activate lustre-MDT0001-mdc-ffff995ac6623000 on oleg358-client.virtnet PASS 404 (6s) == sanity-hsm test 405: archive and release under striped directory ========================================================== 16:52:13 (1768254733) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_405/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450563 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0423071 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321809 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286995 s, 36.5 MB/s Waiting 200s for 'SUCCEED' archive successful on 0x240000bd2:0x1:0x0 archive successful on 0x240000bd2:0x2:0x0 archive successful on 0x240000bd2:0x3:0x0 archive successful on 0x2000032e3:0x6:0x0 PASS 405 (12s) == sanity-hsm test 406: attempting to migrate HSM archived files is safe ========================================================== 16:52:25 (1768254745) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0856414 s, 12.2 MB/s old fid 0x2000032e3:0x9:0x0 Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_406/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' lfs migrate: /mnt/lustre/d406.sanity-hsm/f406.sanity-hsm migrate failed: Operation not supported (95) lfs migrate: cb_migrate_mdt_fini: error completing migration of /mnt/lustre/d406.sanity-hsm: Directory not empty (39) Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' new fid 0x240000bd3:0x7:0x0 Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' debug=+trace Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 406 (14s) == sanity-hsm test 407: Check for double RESTORE records in llog ========================================================== 16:52:39 (1768254759) 0+0 records in 0+0 records out 0 bytes copied, 0.00300421 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_407/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_val=5 fail_loc=0x164 Copytool is suspended on oleg358-client.virtnet mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/3] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x69656cb6 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/4] fid=[0x2000032e3:0x3:0x0] dfid=[0x2000032e3:0x3:0x0] compound/cookie=0x0/0x69656cb7 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x69656cb8 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x69656cb9 action=RESTORE archive#=3 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x69656cba action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x69656cbb action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x69656cbc action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x69656cbd action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] Failing mds1 on oleg358-server Stopping /mnt/lustre-mds1 (opts:) on oleg358-server 16:52:47 (1768254767) shut down facet: mds1 facet_host: oleg358-server facet_failover_host: oleg358-server Failover mds1 to oleg358-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Started lustre-MDT0000 16:53:03 (1768254783) targets are mounted 16:53:03 (1768254783) facet_failover done oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec fail_loc=0 mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/4] fid=[0x2000032e3:0x3:0x0] dfid=[0x2000032e3:0x3:0x0] compound/cookie=0x0/0x69656cb7 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x69656cb8 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x69656cb9 action=RESTORE archive#=3 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x69656cba action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x69656cbb action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x69656cbc action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x69656cbd action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/11] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x69656cbe action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] Copytool is continued on oleg358-client.virtnet d41d8cd98f00b204e9800998ecf8427e /mnt/lustre/d407.sanity-hsm/f407.sanity-hsm d41d8cd98f00b204e9800998ecf8427e /mnt/lustre2/d407.sanity-hsm/f407.sanity-hsm pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x69656cb8 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x69656cb9 action=RESTORE archive#=3 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x69656cba action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x69656cbb action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x69656cbc action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x69656cbd action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/11] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xd:0x0] compound/cookie=0x0/0x69656cbe action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] PASS 407 (35s) == sanity-hsm test 408: Verify fiemap on release file ==== 16:53:14 (1768254794) Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_408/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.109882 s, 596 kB/s SKIP: sanity-hsm test_408 ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 408 (2s) == sanity-hsm test 409a: Coordinator should not stop when in use ========================================================== 16:53:16 (1768254796) 0+0 records in 0+0 records out 0 bytes copied, 0.00382046 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' fail_val=5 fail_loc=0x164 cat: /mnt/lustre/d409a.sanity-hsm/f409a.sanity-hsm: No data available mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Killing existing copytools on oleg358-client.virtnet copytools stopped in 0s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 409a (20s) == sanity-hsm test 409b: getattr released file with CDT stopped after remount ========================================================== 16:53:36 (1768254816) 0+0 records in 0+0 records out 0 bytes copied, 0.0026661 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg358-client.virtnet copytools stopped in 0s Failing mds1 on oleg358-server Stopping /mnt/lustre-mds1 (opts:) on oleg358-server 16:54:03 (1768254843) shut down facet: mds1 facet_host: oleg358-server facet_failover_host: oleg358-server Failover mds1 to oleg358-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Started lustre-MDT0000 16:54:42 (1768254882) targets are mounted 16:54:42 (1768254882) facet_failover done oleg358-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec File: /mnt/lustre/d409b.sanity-hsm/f409b.sanity-hsm Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115406632648720 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-12 16:53:37.000000000 -0500 Modify: 2026-01-12 16:53:37.000000000 -0500 Change: 2026-01-12 16:53:37.000000000 -0500 Birth: 2026-01-12 16:53:36.000000000 -0500 PASS 409b (113s) == sanity-hsm test 410: lfs data_version -s allows release of force-archived file ========================================================== 16:55:29 (1768254929) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21182 s, 5.0 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_410/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d410.sanity-hsm/f410.sanity-hsm): Operation not permitted PASS 410 (7s) == sanity-hsm test 411: hsm_ops rbac role ================ 16:55:36 (1768254936) On MGS 192.168.203.158, active = nodemap.active=1 On MGS 192.168.203.158, test_411.trusted_nodemap = nodemap.test_411.trusted_nodemap=1 Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /mnt/lustre/d411.sanity-hsm/fileA: (0x00000000) Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg358-client.virtnet copytools stopped in 0s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' On MGS 192.168.203.158, test_411.rbac = nodemap.test_411.rbac=file_perms,dne_ops,quota_ops,byfid_ops,chlg_ops,fscrypt_admin,server_upcall,ignore_root_prjquota,local_admin,pool_quota_ops /mnt/lustre/d411.sanity-hsm/fileB: (0x00000000) Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied lfs hsm_set: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied lfs hsm_clear: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied On MGS 192.168.203.158, active = nodemap.active=0 oleg358-server: error: nodemap 'test_411' does not exist oleg358-server: nodemap_del: Invalid argument pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 On MGS 192.168.203.158, active = nodemap.active=0 PASS 411 (39s) == sanity-hsm test 500: various LLAPI HSM tests ========== 16:56:15 (1768254975) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d500.sanity-hsm test1 @1768255040: Register/unregister copytool 2000x to check for leaks ...... pass test2 @1768255040: Re/un-register copytool multiple times without error ....... pass test3 @1768255040: Pass bad parameters to llapi_hsm_copytool_register() ....... pass test4 @1768255040: Bad parameters to llapi_hsm_copytool_unregister() .......... pass test5 @1768255041: Test llapi_hsm_copytool_recv() in non-blocking mode ........ pass test6 @1768255041: Test llapi_hsm_copytool_recv() with bogus parameters ....... pass test7 @1768255041: Test event polling (without actual traffic) ................ pass test50 @1768255041: Test llapi_hsm_state_get()/get_fd() ....................... pass test51 @1768255042: Test llapi_hsm_state_set_fd() ............................. pass test52 @1768255042: Test llapi_hsm_current_action() ........................... pass test100 @1768255043: Simple archive creation with no progress reported. ....... pass test101 @1768255049: Simple archive creation with progress every byte. ........ pass test102 @1768255054: Archive creation with progress every byte, backwards ..... pass test103 @1768255055: Archive creation with a single progress report ........... pass test104 @1768255056: Archive creation with two progress reports ............... pass test105 @1768255057: Archive creation with one bogus progress report .......... pass test106 @1768255058: Archive creation with one empty progress report .......... pass test107 @1768255060: Archive creation with one bogus progress report .......... pass test108 @1768255065: Archive creation with same progress report each time ..... pass test109 @1768255066: Archive creation with one report, with large number ...... pass test110 @1768255067: Archive with 10 progress reports, checking progress ...... pass test111 @1768255068: Archive with 10 reports in reverse, checking progress .... pass test112 @1768255070: Archive with 10 reports, duplicated, check progress ...... pass test113 @1768255071: Archive with 9 reports, with 20% overlapping coverage .... pass PASS 500 (101s) == sanity-hsm test 600: Changelog fields 'u=' and 'nid=' ========================================================== 16:57:56 (1768255076) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d600.sanity-hsm/f600.sanity-hsm] Got NID '192.168.203.58@tcp' lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 600 (18s) == sanity-hsm test 601: OPEN Changelog entry ============= 16:58:14 (1768255094) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001.4 10OPEN 21:58:21.174141376 2026.01.12 0x1 t=[0x240000bd3:0x1b:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.58@tcp m=r-- p=[0x240000bd3:0x1a:0x0] lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 601 (17s) == sanity-hsm test 602: Changelog record CLOSE only if open+write or OPEN recorded ========================================================== 16:58:31 (1768255111) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl3 cl3' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001.10 11CLOSE 21:58:39.388869808 2026.01.12 0x1 t=[0x240000bd3:0x1d:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001.11 10OPEN 21:58:39.881122174 2026.01.12 0x242 t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.58@tcp m=-w- lustre-MDT0001.12 13TRUNC 21:58:39.893537492 2026.01.12 0xe t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp p=[0x240000bd3:0x1c:0x0] lustre-MDT0001.13 11CLOSE 21:58:39.924611677 2026.01.12 0x242 t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001.13 11CLOSE 21:58:39.924611677 2026.01.12 0x242 t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp mdd.lustre-MDT0000.changelog_mask=-OPEN mdd.lustre-MDT0001.changelog_mask=-OPEN lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records f lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001.14 13TRUNC 21:58:42.543987288 2026.01.12 0xe t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp p=[0x240000bd3:0x1c:0x0] lustre-MDT0001.15 11CLOSE 21:58:42.556303914 2026.01.12 0x242 t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001.15 11CLOSE 21:58:42.556303914 2026.01.12 0x242 t=[0x240000bd3:0x1d:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001: Deregistered changelog user #3 lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 602 (22s) == sanity-hsm test 603: GETXATTR Changelog entry ========= 16:58:53 (1768255133) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl4 cl4' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0001: clear the changelog for cl4 of all records getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d603.sanity-hsm/f603.sanity-hsm user.xattr1="value1" lustre-MDT0001.21 23GXATR 21:59:01.107792155 2026.01.12 0x0 t=[0x240000bd3:0x1f:0x0] j=getfattr.0 ef=0x1b u=0:0 nid=192.168.203.58@tcp x=user.xattr1 lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0001: Deregistered changelog user #4 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 603 (18s) == sanity-hsm test 604: NOPEN Changelog entry ============ 16:59:11 (1768255151) mdd.lustre-MDT0000.changelog_deniednext=20 mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl5 cl5' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.9 24NOPEN 21:59:22.446021770 2026.01.12 0x1 t=[0x2000032e2:0x1c:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.58@tcp m=r-- p=[0x2000032e2:0x1b:0x0] Got NID '192.168.203.58@tcp' lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.10 24NOPEN 21:59:43.869070510 2026.01.12 0x1 t=[0x2000032e2:0x1c:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.58@tcp m=r-- p=[0x240000bd3:0x1a:0x0] Got NID '192.168.203.58@tcp' lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0001: Deregistered changelog user #5 lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 mdd.lustre-MDT0000.changelog_deniednext=60 PASS 604 (46s) == sanity-hsm test 605: Test OPEN and CLOSE rate limit in Changelogs ========================================================== 16:59:57 (1768255197) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl6 cl6' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001.25 10OPEN 22:00:05.881538461 2026.01.12 0x43 t=[0x240000bd3:0x21:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.58@tcp m=rw- p=[0x240000bd3:0x20:0x0] lustre-MDT0001.25 10OPEN 22:00:05.881538461 2026.01.12 0x43 t=[0x240000bd3:0x21:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.58@tcp m=rw- p=[0x240000bd3:0x20:0x0] lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001.26 10OPEN 22:00:07.429436482 2026.01.12 0x1 t=[0x240000bd3:0x21:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.58@tcp m=r-- p=[0x200000bd2:0x86:0x0] lustre-MDT0001.27 11CLOSE 22:00:07.466030756 2026.01.12 0x1 t=[0x240000bd3:0x21:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001.26 10OPEN 22:00:07.429436482 2026.01.12 0x1 t=[0x240000bd3:0x21:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.58@tcp m=r-- p=[0x200000bd2:0x86:0x0] lustre-MDT0001.27 11CLOSE 22:00:07.466030756 2026.01.12 0x1 t=[0x240000bd3:0x21:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre/d605.sanity-hsm/f605.sanity-hsm] lustre-MDT0001.28 10OPEN 22:00:09.005500269 2026.01.12 0x1 t=[0x240000bd3:0x21:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.58@tcp m=r-- p=[0x240000bd3:0x1c:0x0] lustre-MDT0001.29 11CLOSE 22:00:09.017236262 2026.01.12 0x1 t=[0x240000bd3:0x21:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.203.58@tcp lustre-MDT0001.28 10OPEN 22:00:09.005500269 2026.01.12 0x1 t=[0x240000bd3:0x21:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.58@tcp m=r-- p=[0x240000bd3:0x1c:0x0] lustre-MDT0001.29 11CLOSE 22:00:09.017236262 2026.01.12 0x1 t=[0x240000bd3:0x21:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.203.58@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001.30 11CLOSE 22:00:09.708052279 2026.01.12 0x43 t=[0x240000bd3:0x21:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001.30 11CLOSE 22:00:09.708052279 2026.01.12 0x43 t=[0x240000bd3:0x21:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.58@tcp lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001: Deregistered changelog user #6 lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 605 (24s) == sanity-hsm test 606: llog_reader groks changelog fields ========================================================== 17:00:21 (1768255221) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl7 cl7' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d606.sanity-hsm/f606.sanity-hsm] Stopping /mnt/lustre-mds1 (opts:) on oleg358-server rec #2 type=1064553b len=64 offset 8256 Header size : 8192 llh_size : 64 Time : Mon Jan 12 16:37:48 2026 Number of records: 1 cat_idx: 0 last_idx: 2 Flags: 0x2012 Target uuid : ----------------------- #02 (064)fid=[0x1:0x3b:0x0] path=oi.1/0x1:0x3b:0x0 Got UID/GID 500:500 Got NID 192.168.203.58@tcp Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg358-server: oleg358-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg358-client: oleg358-server: ssh exited with exit code 1 Started lustre-MDT0000 lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0001: Deregistered changelog user #7 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 606 (57s) == sanity-hsm test 607a: release a file that was migrated after being archived ========================================================== 17:01:18 (1768255278) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.395857 s, 2.6 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 607a (13s) == sanity-hsm test 607b: Migrate should not change the HSM attribute of dirty files ========================================================== 17:01:31 (1768255291) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.377834 s, 2.8 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' llite.lustre-ffff995ac6623000.xattr_cache=0 llite.lustre-ffff995ac8841000.xattr_cache=0 /mnt/lustre/d607b.sanity-hsm/f607b.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 llite.lustre-ffff995ac6623000.xattr_cache=1 llite.lustre-ffff995ac8841000.xattr_cache=1 PASS 607b (11s) == sanity-hsm test 607c: 'lfs swap_layouts' should set dirty flag on HSM file ========================================================== 17:01:42 (1768255302) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.310605 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.416852 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.394163 s, 2.7 MB/s Starting copytool 'agt1' on 'oleg358-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-1' and '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-3': Operation not permitted (1) PASS 607c (16s) == sanity-hsm test complete, duration 4250 sec =========== 17:01:58 (1768255318) === sanity-hsm: start cleanup 17:02:00 (1768255320) === Stopping clients: oleg358-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg358-client.virtnet /mnt/lustre2 opts: === sanity-hsm: finish cleanup 17:02:32 (1768255352) ===