-----============= acceptance-small: sanity-hsm ============----- Sun Nov 16 18:22:16 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg323-server: /home/green/git/lustre-release/lustre/tests/except/sanity-hsm.ex - see MDS1_VERSION <= CLIENT_VERSION (34618676 <= 34618676) for LU-18924, go 100 105 excepting tests: === sanity-hsm: start setup 18:22:33 (1763335353) === Starting client oleg323-client.virtnet: -o user_xattr,flock 192.168.203.123@tcp:/lustre /mnt/lustre2 Mount client oleg323-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.203.123@tcp:/lustre /mnt/lustre2 Started clients oleg323-client.virtnet: 192.168.203.123@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg323-client.virtnet: executing check_config_client /mnt/lustre oleg323-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg323-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8e74038c2000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8e7405fd7000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8e74038c2000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8e7405fd7000.idle_timeout=debug disable quota as required oleg323-server: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity-hsm: finish setup 18:22:59 (1763335379) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7560] nobody:x:65534: Killing existing copytools on oleg323-client.virtnet pkill: pidfile not valid Try `pkill --help' for more information. Set HSM on and start Set sanity-hsm HSM policy == sanity-hsm test 1A: lfs hsm flags root/non-root access ========================================================== 18:23:24 (1763335404) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_clear] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--exists] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] lfs hsm_set: change hsm flags for '/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm' failed: Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] PASS 1A (8s) == sanity-hsm test 1a: mmap & cat a HSM released file ==== 18:23:32 (1763335412) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.261652 s, 4.0 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Verifying released state: PASS 1a (11s) == sanity-hsm test 1b: Archive, Release and Restore composite file ========================================================== 18:23:43 (1763335423) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768392 s, 13.6 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm release /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm verify released state: pass restore /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass PASS 1b (10s) == sanity-hsm test 1c: Check setting archive-id in lfs hsm_set ========================================================== 18:23:54 (1763335434) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is still 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 33 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 16 PASS 1c (6s) == sanity-hsm test 1d: Archive, Release and Restore DoM file ========================================================== 18:24:00 (1763335440) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0475555 s, 22.0 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm Waiting 200s for 'SUCCEED' release /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm verify released state: pass restore /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm verify restored state: pass PASS 1d (10s) == sanity-hsm test 1e: Archive, Release and Restore SEL file ========================================================== 18:24:10 (1763335450) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0760386 s, 13.8 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm release /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm verify released state: pass restore /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass PASS 1e (11s) == sanity-hsm test 1f: DoM file release after restore ==== 18:24:21 (1763335461) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0772179 s, 13.6 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm release /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm verify released state: pass restore /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass release again /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 verify released state: pass PASS 1f (10s) == sanity-hsm test 2: Check file dirtyness when doing setattr ========================================================== 18:24:31 (1763335471) 0+0 records in 0+0 records out 0 bytes copied, 0.0085392 s, 0.0 kB/s 0x200000402:0x9:0x0 PASS 2 (6s) == sanity-hsm test 3: Check file dirtyness when opening for write ========================================================== 18:24:37 (1763335477) Succeed in opening file "/mnt/lustre/d3.sanity-hsm/f3.sanity-hsm"(flags=O_WRONLY) 3+0 records in 3+0 records out 3+0 records in 3+0 records out PASS 3 (7s) == sanity-hsm test 4: Useless cancel must not be registered ========================================================== 18:24:45 (1763335485) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.263636 s, 4.0 MB/s PASS 4 (7s) == sanity-hsm test 8: Test default archive number ======== 18:24:51 (1763335491) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_8/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 8 (9s) == sanity-hsm test 9A: Use of explicit archive number, with dedicated copytool ========================================================== 18:25:00 (1763335500) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_9A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 3 "/mnt/lustre2"' oleg323-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg323-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=33af46a0-e4f6-44ec-9a4f-1f0796276236 archive_id=3 requests=[current:0 ok:0 errors:0] on mds1 PASS 9A (16s) == sanity-hsm test 9a: Multiple remote agents ============ 18:25:17 (1763335517) SKIP: sanity-hsm test_9a Need 3 or more clients, have 1 SKIP 9a (3s) == sanity-hsm test 10a: Archive a file =================== 18:25:20 (1763335520) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 938: warning: command substitution: ignored null byte in input Verifying content Verifying hsm state Verifying archive number is 2 PASS 10a (7s) == sanity-hsm test 10b: Archive of non dirty file must work without doing request ========================================================== 18:25:27 (1763335527) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 10b (9s) == sanity-hsm test 10c: Check forbidden archive ========== 18:25:36 (1763335536) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d10c.sanity-hsm/f10c.sanity-hsm): Operation not permitted PASS 10c (7s) == sanity-hsm test 10d: Archive a file on the default archive id ========================================================== 18:25:43 (1763335543) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 10d (9s) == sanity-hsm test 11a: Import a file ==================== 18:25:52 (1763335552) lhsmtool_posix: 1763335554.933386 [21034]: action=1 src=d11a.sanity-hsm/f11a.sanity-hsm dst=/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763335554.939314 [21034]: importing '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1763335555.012323 [21034]: imported '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//042b/0x200000401:0x2a:0x0'=='/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1763335555.012396 [21034]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying imported size 158=158 Verifying released pattern: raid0,released Verifying new fid 0x200000401:0x2a:0x0 in archive PASS 11a (7s) == sanity-hsm test 11b: Import a deleted file using its FID ========================================================== 18:26:00 (1763335560) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_11b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1763335563.131696 [21718]: action=1 src=0x200000402:0x12:0x0 dst=/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763335563.150043 [21718]: Resolving [0x200000402:0x12:0x0] to /tmp/arc1/sanity-hsm.test_11b//0410/0x200000402:0x12:0x0 lhsmtool_posix: 1763335563.150160 [21718]: importing '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//0410/0x200000402:0x12:0x0' lhsmtool_posix: 1763335563.207150 [21718]: imported '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//042d/0x200000401:0x2c:0x0'=='/tmp/arc1/sanity-hsm.test_11b//0410/0x200000402:0x12:0x0' lhsmtool_posix: 1763335563.207226 [21718]: process finished, errs: 0 major, 0 minor, rc=0 (Success) /mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm: OK PASS 11b (8s) == sanity-hsm test 11c: Import a file to a directory with a pool ========================================================== 18:26:08 (1763335568) Creating new pool test_11c oleg323-server: Pool lustre.test_11c created Adding targets to pool oleg323-server: OST lustre-OST0001_UUID added to pool lustre.test_11c lhsmtool_posix: 1763335575.499699 [22406]: action=1 src=d11c.sanity-hsm/f11c.sanity-hsm dst=/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763335575.510145 [22406]: importing '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1763335575.561442 [22406]: imported '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//042e/0x200000401:0x2f:0x0'=='/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1763335575.561518 [22406]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Destroy the created pools: test_11c lustre.test_11c oleg323-server: OST lustre-OST0001_UUID removed from pool lustre.test_11c oleg323-server: Pool lustre.test_11c destroyed PASS 11c (19s) == sanity-hsm test 12a: Restore an imported file explicitly ========================================================== 18:26:27 (1763335587) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1763335589.325382 [23125]: action=1 src=d12a.sanity-hsm/f12a.sanity-hsm dst=/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763335589.332866 [23125]: importing '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1763335589.377330 [23125]: imported '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//0433/0x200000401:0x32:0x0'=='/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1763335589.377408 [23125]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying file state: PASS 12a (9s) == sanity-hsm test 12b: Restore an imported file implicitly ========================================================== 18:26:36 (1763335596) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1763335599.103620 [23832]: action=1 src=d12b.sanity-hsm/f12b.sanity-hsm dst=/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763335599.117904 [23832]: importing '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1763335599.196349 [23832]: imported '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//0434/0x200000401:0x35:0x0'=='/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1763335599.196427 [23832]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying file state after restore: PASS 12b (9s) == sanity-hsm test 12c: Restore a file with stripe of 2 == 18:26:45 (1763335605) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.127495 s, 41.1 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d12c.sanity-hsm/f12c.sanity-hsm: OK PASS 12c (9s) == sanity-hsm test 12d: Restore of a non archived, non released file must work ========================================================== 18:26:54 (1763335614) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12d (7s) == sanity-hsm test 12e: Check forbidden restore ========== 18:27:01 (1763335621) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 Cannot send HSM request (use of /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm): Operation not permitted PASS 12e (8s) == sanity-hsm test 12f: Restore a released file explicitly ========================================================== 18:27:09 (1763335629) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying file state: PASS 12f (11s) == sanity-hsm test 12g: Restore a released file implicitly ========================================================== 18:27:20 (1763335640) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12g (9s) == sanity-hsm test 12h: Restore a released file implicitly from a second node ========================================================== 18:27:29 (1763335649) SKIP: sanity-hsm test_12h Need 2 or more clients, have 1 SKIP 12h (3s) == sanity-hsm test 12m: Archive/release/implicit restore ========================================================== 18:27:33 (1763335653) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12m/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 12m (10s) == sanity-hsm test 12n: Import/implicit restore/release == 18:27:42 (1763335662) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12n/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1763335664.974731 [28617]: action=1 src=d12n.sanity-hsm/f12n.sanity-hsm dst=/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763335664.984478 [28617]: importing '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1763335665.027230 [28617]: imported '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//0445/0x200000401:0x44:0x0'=='/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1763335665.027573 [28617]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12n (8s) == sanity-hsm test 12o: Layout-swap failure during Restore leaves file released ========================================================== 18:27:50 (1763335670) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12o/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x152 mdt.lustre-MDT0000.hsm.policy=+NRA diff: /mnt/lustre/d12o.sanity-hsm/f12o.sanity-hsm: No data available mdt.lustre-MDT0000.hsm.policy=-NRA fail_loc=0 mdt.lustre-MDT0000.hsm_control=purge PASS 12o (24s) == sanity-hsm test 12p: implicit restore of a file on copytool mount point ========================================================== 18:28:15 (1763335695) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12p/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12p (9s) == sanity-hsm test 12q: file attributes are refreshed after restore ========================================================== 18:28:25 (1763335705) Starting client: oleg323-client.virtnet: -o user_xattr,flock 192.168.203.123@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231242 s, 4.5 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Stopping client oleg323-client.virtnet /mnt/lustre3 (opts:) PASS 12q (17s) == sanity-hsm test 12r: lseek restores released file ===== 18:28:42 (1763335722) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12r/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' PASS 12r (8s) == sanity-hsm test 12s: race between restore requests ==== 18:28:51 (1763335731) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12s/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x8000018b PASS 12s (13s) == sanity-hsm test 12t: Multiple parallel reads for a HSM imported file ========================================================== 18:29:04 (1763335744) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12t/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1763335747.279362 [32686]: action=1 src=0x200000402:0x2c:0x0 dst=/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763335747.286467 [32686]: Resolving [0x200000402:0x2c:0x0] to /tmp/arc1/sanity-hsm.test_12t//042e/0x200000402:0x2c:0x0 lhsmtool_posix: 1763335747.287827 [32686]: importing '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//042e/0x200000402:0x2c:0x0' lhsmtool_posix: 1763335747.342954 [32686]: imported '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//0453/0x200000401:0x52:0x0'=='/tmp/arc1/sanity-hsm.test_12t//042e/0x200000402:0x2c:0x0' lhsmtool_posix: 1763335747.343028 [32686]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12t (8s) == sanity-hsm test 12u: Multiple reads on multiple HSM imported files in parallel ========================================================== 18:29:12 (1763335752) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12u/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1763335761.870028 [33554]: action=1 src=0x200000402:0x2e:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0 mount_point=/mnt/lustre2 lhsmtool_posix: 1763335761.889459 [33554]: Resolving [0x200000402:0x2e:0x0] to /tmp/arc1/sanity-hsm.test_12u//042c/0x200000402:0x2e:0x0 lhsmtool_posix: 1763335761.889557 [33554]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//042c/0x200000402:0x2e:0x0' lhsmtool_posix: 1763335761.967747 [33554]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//0455/0x200000401:0x54:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042c/0x200000402:0x2e:0x0' lhsmtool_posix: 1763335761.967868 [33554]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1763335762.168977 [33583]: action=1 src=0x200000402:0x2f:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1 mount_point=/mnt/lustre2 lhsmtool_posix: 1763335762.178336 [33583]: Resolving [0x200000402:0x2f:0x0] to /tmp/arc1/sanity-hsm.test_12u//042d/0x200000402:0x2f:0x0 lhsmtool_posix: 1763335762.178449 [33583]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//042d/0x200000402:0x2f:0x0' lhsmtool_posix: 1763335762.213038 [33583]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//0454/0x200000401:0x55:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042d/0x200000402:0x2f:0x0' lhsmtool_posix: 1763335762.213112 [33583]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1763335762.425230 [33612]: action=1 src=0x200000402:0x30:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2 mount_point=/mnt/lustre2 lhsmtool_posix: 1763335762.434550 [33612]: Resolving [0x200000402:0x30:0x0] to /tmp/arc1/sanity-hsm.test_12u//0432/0x200000402:0x30:0x0 lhsmtool_posix: 1763335762.434695 [33612]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//0432/0x200000402:0x30:0x0' lhsmtool_posix: 1763335762.491908 [33612]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//0457/0x200000401:0x56:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0432/0x200000402:0x30:0x0' lhsmtool_posix: 1763335762.491969 [33612]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1763335762.707019 [33641]: action=1 src=0x200000402:0x31:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3 mount_point=/mnt/lustre2 lhsmtool_posix: 1763335762.711558 [33641]: Resolving [0x200000402:0x31:0x0] to /tmp/arc1/sanity-hsm.test_12u//0433/0x200000402:0x31:0x0 lhsmtool_posix: 1763335762.711773 [33641]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//0433/0x200000402:0x31:0x0' lhsmtool_posix: 1763335762.750998 [33641]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//0456/0x200000401:0x57:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0433/0x200000402:0x31:0x0' lhsmtool_posix: 1763335762.751064 [33641]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1763335762.944008 [33670]: action=1 src=0x200000402:0x32:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4 mount_point=/mnt/lustre2 lhsmtool_posix: 1763335762.956404 [33670]: Resolving [0x200000402:0x32:0x0] to /tmp/arc1/sanity-hsm.test_12u//0430/0x200000402:0x32:0x0 lhsmtool_posix: 1763335762.956503 [33670]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0430/0x200000402:0x32:0x0' lhsmtool_posix: 1763335762.988132 [33670]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0459/0x200000401:0x58:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0430/0x200000402:0x32:0x0' lhsmtool_posix: 1763335762.988200 [33670]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1763335763.191272 [33699]: action=1 src=0x200000402:0x33:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5 mount_point=/mnt/lustre2 lhsmtool_posix: 1763335763.203027 [33699]: Resolving [0x200000402:0x33:0x0] to /tmp/arc1/sanity-hsm.test_12u//0431/0x200000402:0x33:0x0 lhsmtool_posix: 1763335763.203123 [33699]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0431/0x200000402:0x33:0x0' lhsmtool_posix: 1763335763.243923 [33699]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0458/0x200000401:0x59:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0431/0x200000402:0x33:0x0' lhsmtool_posix: 1763335763.243984 [33699]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1763335763.463425 [33728]: action=1 src=0x200000402:0x34:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6 mount_point=/mnt/lustre2 lhsmtool_posix: 1763335763.477388 [33728]: Resolving [0x200000402:0x34:0x0] to /tmp/arc1/sanity-hsm.test_12u//0436/0x200000402:0x34:0x0 lhsmtool_posix: 1763335763.477763 [33728]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0436/0x200000402:0x34:0x0' lhsmtool_posix: 1763335763.574003 [33728]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//045b/0x200000401:0x5a:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0436/0x200000402:0x34:0x0' lhsmtool_posix: 1763335763.574072 [33728]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1763335763.845448 [33757]: action=1 src=0x200000402:0x35:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7 mount_point=/mnt/lustre2 lhsmtool_posix: 1763335763.852367 [33757]: Resolving [0x200000402:0x35:0x0] to /tmp/arc1/sanity-hsm.test_12u//0437/0x200000402:0x35:0x0 lhsmtool_posix: 1763335763.852463 [33757]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0437/0x200000402:0x35:0x0' lhsmtool_posix: 1763335763.908264 [33757]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//045a/0x200000401:0x5b:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0437/0x200000402:0x35:0x0' lhsmtool_posix: 1763335763.908335 [33757]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1763335764.096239 [33786]: action=1 src=0x200000402:0x36:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8 mount_point=/mnt/lustre2 lhsmtool_posix: 1763335764.103958 [33786]: Resolving [0x200000402:0x36:0x0] to /tmp/arc1/sanity-hsm.test_12u//0434/0x200000402:0x36:0x0 lhsmtool_posix: 1763335764.104045 [33786]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0434/0x200000402:0x36:0x0' lhsmtool_posix: 1763335764.142222 [33786]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//045d/0x200000401:0x5c:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0434/0x200000402:0x36:0x0' lhsmtool_posix: 1763335764.142390 [33786]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1763335764.396479 [33815]: action=1 src=0x200000402:0x37:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9 mount_point=/mnt/lustre2 lhsmtool_posix: 1763335764.407877 [33815]: Resolving [0x200000402:0x37:0x0] to /tmp/arc1/sanity-hsm.test_12u//0435/0x200000402:0x37:0x0 lhsmtool_posix: 1763335764.407990 [33815]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0435/0x200000402:0x37:0x0' lhsmtool_posix: 1763335764.453851 [33815]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//045c/0x200000401:0x5d:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0435/0x200000402:0x37:0x0' lhsmtool_posix: 1763335764.453917 [33815]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12u (18s) == sanity-hsm test 13: Recursively import and restore a directory ========================================================== 18:29:30 (1763335770) lhsmtool_posix: 1763335777.024413 [35554]: action=1 src=subdir dst=/mnt/lustre/d13.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763335777.097225 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1763335777.171553 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//0460/0x200000401:0x61:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1763335777.171841 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1763335777.243755 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//0463/0x200000401:0x62:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1763335777.248457 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1763335777.326769 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//0462/0x200000401:0x63:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1763335777.327012 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1763335777.390717 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//0465/0x200000401:0x64:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1763335777.390951 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1763335777.423751 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//0464/0x200000401:0x65:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1763335777.423977 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1763335777.464295 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//0467/0x200000401:0x66:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1763335777.466200 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1763335777.504629 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//0466/0x200000401:0x67:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1763335777.505144 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1763335777.547357 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//0469/0x200000401:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1763335777.547896 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1763335777.604956 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//0468/0x200000401:0x69:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1763335777.606817 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1763335777.646935 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//046b/0x200000401:0x6a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1763335777.651748 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1763335777.699966 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//046d/0x200000401:0x6c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1763335777.700197 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1763335777.753330 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//046c/0x200000401:0x6d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1763335777.753579 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1763335777.805251 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//046f/0x200000401:0x6e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1763335777.805495 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1763335777.873034 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//046e/0x200000401:0x6f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1763335777.873782 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1763335777.915640 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//0471/0x200000401:0x70:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1763335777.916947 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1763335777.972100 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//0470/0x200000401:0x71:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1763335777.973023 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1763335778.036465 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//0473/0x200000401:0x72:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1763335778.036754 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1763335778.101553 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//0472/0x200000401:0x73:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1763335778.101819 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1763335778.161883 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//0475/0x200000401:0x74:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1763335778.163460 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1763335778.198953 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//0474/0x200000401:0x75:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1763335778.205568 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1763335778.271508 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//0476/0x200000401:0x77:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1763335778.273430 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1763335778.312054 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//0479/0x200000401:0x78:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1763335778.312307 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1763335778.367128 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//0478/0x200000401:0x79:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1763335778.367520 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1763335778.425018 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//047b/0x200000401:0x7a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1763335778.425253 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1763335778.500525 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//047a/0x200000401:0x7b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1763335778.500818 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1763335778.563629 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//047d/0x200000401:0x7c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1763335778.563991 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1763335778.607925 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//047c/0x200000401:0x7d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1763335778.608191 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1763335778.645459 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//047f/0x200000401:0x7e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1763335778.645740 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1763335778.683048 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//047e/0x200000401:0x7f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1763335778.683564 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1763335778.715867 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//0481/0x200000401:0x80:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1763335778.723182 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1763335778.764364 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//0483/0x200000401:0x82:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1763335778.765961 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1763335778.808286 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//0482/0x200000401:0x83:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1763335778.808718 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1763335778.849940 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//0485/0x200000401:0x84:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1763335778.850201 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1763335778.901290 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//0484/0x200000401:0x85:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1763335778.901525 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1763335778.945418 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//0487/0x200000401:0x86:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1763335778.946518 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1763335779.000599 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//0486/0x200000401:0x87:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1763335779.004322 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1763335779.059641 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//0489/0x200000401:0x88:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1763335779.060619 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1763335779.130440 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//0488/0x200000401:0x89:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1763335779.131383 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1763335779.217260 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//048b/0x200000401:0x8a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1763335779.217503 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1763335779.290543 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//048a/0x200000401:0x8b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1763335779.296022 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1763335779.364221 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//048c/0x200000401:0x8d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1763335779.365902 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1763335779.431308 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//048f/0x200000401:0x8e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1763335779.431548 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1763335779.496658 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//048e/0x200000401:0x8f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1763335779.497367 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1763335779.543007 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//0491/0x200000401:0x90:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1763335779.543978 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1763335779.584469 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//0490/0x200000401:0x91:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1763335779.585012 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1763335779.637763 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//0493/0x200000401:0x92:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1763335779.637969 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1763335779.689563 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//0492/0x200000401:0x93:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1763335779.690468 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1763335779.734632 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//0495/0x200000401:0x94:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1763335779.735405 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1763335779.776210 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//0494/0x200000401:0x95:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1763335779.776441 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1763335779.824764 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//0497/0x200000401:0x96:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1763335779.828911 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1763335779.885517 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//0499/0x200000401:0x98:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1763335779.885812 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1763335779.929413 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//0498/0x200000401:0x99:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1763335779.929970 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1763335779.978610 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//049b/0x200000401:0x9a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1763335779.978900 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1763335780.036819 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//049a/0x200000401:0x9b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1763335780.037062 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1763335780.085832 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//049d/0x200000401:0x9c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1763335780.086055 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1763335780.142578 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//049c/0x200000401:0x9d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1763335780.143617 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1763335780.186304 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//049f/0x200000401:0x9e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1763335780.186546 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1763335780.251060 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//049e/0x200000401:0x9f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1763335780.251311 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1763335780.319526 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//04a1/0x200000401:0xa0:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1763335780.326161 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1763335780.402246 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//04a0/0x200000401:0xa1:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1763335780.411593 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1763335780.454388 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//04a2/0x200000401:0xa3:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1763335780.455643 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1763335780.489280 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//04a5/0x200000401:0xa4:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1763335780.490338 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1763335780.522940 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//04a4/0x200000401:0xa5:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1763335780.523221 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1763335780.558304 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//04a7/0x200000401:0xa6:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1763335780.558520 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1763335780.594811 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//04a6/0x200000401:0xa7:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1763335780.595045 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1763335780.663985 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//04a9/0x200000401:0xa8:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1763335780.664368 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1763335780.701341 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//04a8/0x200000401:0xa9:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1763335780.701564 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1763335780.731953 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//04ab/0x200000401:0xaa:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1763335780.732865 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1763335780.772621 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//04aa/0x200000401:0xab:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1763335780.773635 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1763335780.836019 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//04ad/0x200000401:0xac:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1763335780.846096 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1763335780.923590 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//04af/0x200000401:0xae:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1763335780.925536 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1763335780.969564 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//04ae/0x200000401:0xaf:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1763335780.970477 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1763335781.007973 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//04b1/0x200000401:0xb0:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1763335781.008201 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1763335781.047434 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//04b0/0x200000401:0xb1:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1763335781.047648 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1763335781.087877 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//04b3/0x200000401:0xb2:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1763335781.088537 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1763335781.137593 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//04b2/0x200000401:0xb3:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1763335781.138816 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1763335781.178755 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//04b5/0x200000401:0xb4:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1763335781.178978 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1763335781.243898 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//04b4/0x200000401:0xb5:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1763335781.244488 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1763335781.297182 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//04b7/0x200000401:0xb6:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1763335781.299443 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1763335781.351918 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//04b6/0x200000401:0xb7:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1763335781.355588 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1763335781.394297 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//04b8/0x200000401:0xb9:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1763335781.396391 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1763335781.454331 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//04bb/0x200000401:0xba:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1763335781.454584 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1763335781.495023 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//04ba/0x200000401:0xbb:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1763335781.495329 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1763335781.534617 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//04bd/0x200000401:0xbc:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1763335781.535423 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1763335781.593237 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//04bc/0x200000401:0xbd:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1763335781.593468 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1763335781.632964 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//04bf/0x200000401:0xbe:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1763335781.633213 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1763335781.678199 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//04be/0x200000401:0xbf:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1763335781.678406 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1763335781.717481 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//04c1/0x200000401:0xc0:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1763335781.717758 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1763335781.764476 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//04c0/0x200000401:0xc1:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1763335781.765516 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1763335781.808719 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//04c3/0x200000401:0xc2:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1763335781.817908 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1763335781.873023 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//04c5/0x200000401:0xc4:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1763335781.875691 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1763335781.971599 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//04c4/0x200000401:0xc5:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1763335781.971869 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1763335782.058299 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//04c7/0x200000401:0xc6:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1763335782.059461 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1763335782.126314 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//04c6/0x200000401:0xc7:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1763335782.126543 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1763335782.208112 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//04c9/0x200000401:0xc8:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1763335782.208386 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1763335782.263464 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//04c8/0x200000401:0xc9:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1763335782.263746 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1763335782.330473 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//04cb/0x200000401:0xca:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1763335782.330838 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1763335782.383468 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//04ca/0x200000401:0xcb:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1763335782.384190 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1763335782.449453 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//04cd/0x200000401:0xcc:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1763335782.449841 [35554]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1763335782.506862 [35554]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//04cc/0x200000401:0xcd:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1763335782.506968 [35554]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_13/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 13 (30s) == sanity-hsm test 14: Rebind archived file to a new fid ========================================================== 18:30:01 (1763335801) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_14/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152396 s, 6.9 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 0+0 records in 0+0 records out 0 bytes copied, 0.0102914 s, 0.0 kB/s rebind 0x200000402:0xa7:0x0 to 0x200000402:0xa8:0x0 lhsmtool_posix: 1763335805.348021 [36508]: action=2 src=0x200000402:0xa7:0x0 dst=0x200000402:0xa8:0x0 mount_point=/mnt/lustre2 lhsmtool_posix: 1763335805.348991 [36508]: rebind [0x200000402:0xa7:0x0] to [0x200000402:0xa8:0x0] lhsmtool_posix: 1763335805.349142 [36508]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 14 (9s) == sanity-hsm test 15: Rebind a list of files ============ 18:30:10 (1763335810) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_15/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12782 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176047 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160896 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167896 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118019 s, 8.9 MB/s pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 0+0 records in 0+0 records out 0 bytes copied, 0.00566775 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00952188 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00676888 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00397848 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00528153 s, 0.0 kB/s rebind list of files lhsmtool_posix: 1763335816.351416 [37283]: action=2 src=/tmp/tmp.7560 dst=(null) mount_point=/mnt/lustre2 lhsmtool_posix: 1763335816.353192 [37283]: rebind [0x200000402:0xab:0x0] to [0x200000402:0xb0:0x0] lhsmtool_posix: 1763335816.353303 [37283]: rebind [0x200000402:0xac:0x0] to [0x200000402:0xb1:0x0] lhsmtool_posix: 1763335816.353346 [37283]: rebind [0x200000402:0xad:0x0] to [0x200000402:0xb2:0x0] lhsmtool_posix: 1763335816.353385 [37283]: rebind [0x200000402:0xae:0x0] to [0x200000402:0xb3:0x0] lhsmtool_posix: 1763335816.353425 [37283]: rebind [0x200000402:0xaf:0x0] to [0x200000402:0xb4:0x0] lhsmtool_posix: 1763335816.353474 [37283]: 5 lines read from '/tmp/tmp.7560', 5 rebind successful lhsmtool_posix: 1763335816.353490 [37283]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 15 (16s) == sanity-hsm test 16: Test CT bandwith control option === 18:30:26 (1763335826) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_16/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 0.0848375 s, 247 MB/s Waiting 200s for 'SUCCEED' Changed after 1s: from 'WAITING' to 'STARTED' Waiting 190s for 'SUCCEED' Updated after 21s: want 'SUCCEED' got 'SUCCEED' PASS 16 (27s) == sanity-hsm test 20: Release is not permitted ========== 18:30:53 (1763335853) 0+0 records in 0+0 records out 0 bytes copied, 0.00402869 s, 0.0 kB/s 0x200000402:0xbc:0x0 Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted PASS 20 (5s) == sanity-hsm test 21: Simple release tests ============== 18:30:58 (1763335858) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_21/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138597 s, 7.6 MB/s 1+0 records in 1+0 records out 512 bytes copied, 0.0676757 s, 7.6 kB/s Waiting 200s for 'SUCCEED' PASS 21 (8s) == sanity-hsm test 22: Could not swap a release file ===== 18:31:06 (1763335866) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_22/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146183 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119545 s, 8.8 MB/s 0x200000402:0xc1:0x0 lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d22.sanity-hsm/test_swap' and '/mnt/lustre/d22.sanity-hsm/test_release': Operation not supported (95) PASS 22 (5s) == sanity-hsm test 23: Release does not change a/mtime (utime) ========================================================== 18:31:11 (1763335871) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_23/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154364 s, 6.8 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 23 (8s) == sanity-hsm test 24a: Archive, release, and restore does not change a/mtime (i/o) ========================================================== 18:31:19 (1763335879) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164295 s, 6.4 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg323-client.virtnet copytools stopped in 1s 192.168.203.123@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg323-client.virtnet /mnt/lustre (opts:) Starting client: oleg323-client.virtnet: -o user_xattr,flock 192.168.203.123@tcp:/lustre /mnt/lustre PASS 24a (14s) == sanity-hsm test 24b: root can archive, release, and restore user files ========================================================== 18:31:33 (1763335893) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190986 s, 5.5 MB/s Waiting 200s for 'SUCCEED' running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] /mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm: (0x00000009) exists archived, archive_id:1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [md5sum] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] PASS 24b (9s) == sanity-hsm test 24c: check that user,group,other request masks work ========================================================== 18:31:42 (1763335902) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151743 s, 6.9 MB/s 0x200000402:0xcc:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144505 s, 7.3 MB/s 0x200000402:0xcd:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146217 s, 7.2 MB/s 0x200000402:0xce:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] PASS 24c (13s) == sanity-hsm test 24d: check that read-only mounts are respected ========================================================== 18:31:56 (1763335916) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16785 s, 6.2 MB/s 0x200000402:0xd0:0x0 /mnt/lustre/d24d.sanity-hsm/f24d.sanity-hsm lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 106 0x6a 0x280000400 Starting client: oleg323-client.virtnet: -o user_xattr,flock 192.168.203.123@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.872003 s, 1.2 MB/s Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Stopping client oleg323-client.virtnet /mnt/lustre3 (opts:) PASS 24d (13s) == sanity-hsm test 24e: tar succeeds on HSM released files ========================================================== 18:32:09 (1763335929) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137649 s, 7.6 MB/s /mnt/lustre/d24e.sanity-hsm/f24e.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 tar: Removing leading `/' from member names PASS 24e (9s) == sanity-hsm test 24f: root can archive, release, and restore tar files ========================================================== 18:32:18 (1763335938) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm f24f.sanity-hsm f24f.sanity-hsm Sum0 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm, sum1 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm PASS 24f (9s) == sanity-hsm test 24g: write by non-owner still sets dirty ========================================================== 18:32:27 (1763335947) RUNAS = 'runas -u 500 -g 500' Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Please listen carefully as our options have changed. running as uid/gid/euid/egid 500/500/500/500, groups: 500 [tee] [/mnt/lustre/d24g.sanity-hsm/f24g.sanity-hsm] To be electrocuted by your telephone, press #. PASS 24g (9s) == sanity-hsm test 25a: Restore lost file (HS_LOST flag) from import ========================================================== 18:32:36 (1763335956) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1763335958.352561 [46181]: action=1 src=d25a.sanity-hsm/f25a.sanity-hsm dst=/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763335958.367152 [46181]: importing '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1763335958.427851 [46181]: imported '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//0411/0x200000404:0x15:0x0'=='/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1763335958.427920 [46181]: process finished, errs: 0 major, 0 minor, rc=0 (Success) md5sum: /mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm: No data available PASS 25a (6s) == sanity-hsm test 25b: Restore lost file (HS_LOST flag) after release ========================================================== 18:32:42 (1763335962) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' md5sum: /mnt/lustre/d25b.sanity-hsm/f25b.sanity-hsm: No data available PASS 25b (7s) == sanity-hsm test 26A: Remove the archive of a valid file ========================================================== 18:32:49 (1763335969) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00702484 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 26A (8s) == sanity-hsm test 26a: Remove Archive On Last Unlink (RAoLU) policy ========================================================== 18:32:57 (1763335977) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 8s: want 'SUCCEED' got 'SUCCEED' PASS 26a (27s) == sanity-hsm test 26b: RAoLU policy when CDT off ======== 18:33:24 (1763336004) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 26b (16s) == sanity-hsm test 26c: RAoLU effective when file closed ========================================================== 18:33:40 (1763336020) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' multiop /mnt/lustre/d26c.sanity-hsm/f26c.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7560 Waiting 200s for 'SUCCEED' Updated after 7s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 6s: want 'SUCCEED' got 'SUCCEED' PASS 26c (28s) == sanity-hsm test 26d: RAoLU when Client eviction ======= 18:34:08 (1763336048) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192285 s, 5.5 MB/s multiop /mnt/lustre/d26d.sanity-hsm/f26d.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7560 Waiting 200s for 'SUCCEED' Updated after 9s: want 'SUCCEED' got 'SUCCEED' PASS 26d (22s) == sanity-hsm test 26e: RAoLU with a non-started coordinator ========================================================== 18:34:30 (1763336070) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1289 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11958 s, 8.8 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s Failing mds1 on oleg323-server Stopping /mnt/lustre-mds1 (opts:) on oleg323-server 18:35:00 (1763336100) shut down facet: mds1 facet_host: oleg323-server facet_failover_host: oleg323-server Failover mds1 to oleg323-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg323-server: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Started lustre-MDT0000 18:35:18 (1763336118) targets are mounted 18:35:18 (1763336118) facet_failover done oleg323-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Check cookie from RAoLU request (last: 1763335598, remove: 1763335599) mdt.lustre-MDT0000.hsm_control=enabled Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 26e (82s) == sanity-hsm test 27a: Remove the archive of an imported file (Operation not permitted) ========================================================== 18:35:53 (1763336153) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.748604 s, 54.6 MB/s lhsmtool_posix: 1763336155.667149 [52974]: action=1 src=d27a.sanity-hsm/f27a.sanity-hsm dst=/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763336155.679147 [52974]: importing '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1763336155.728560 [52974]: imported '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//0405/0x200000406:0x3:0x0'=='/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1763336155.728767 [52974]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Cannot send HSM request (use of /mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm): Operation not permitted PASS 27a (6s) == sanity-hsm test 27b: Remove the archive of a relased file (Operation not permitted) ========================================================== 18:35:59 (1763336159) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00415694 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Cannot send HSM request (use of /mnt/lustre/d27b.sanity-hsm/f27b.sanity-hsm): Operation not permitted PASS 27b (6s) == sanity-hsm test 28: Concurrent archive/file remove ==== 18:36:05 (1763336165) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_28/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.0043531 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled PASS 28 (8s) == sanity-hsm test 29a: Tests --mntpath and --archive options ========================================================== 18:36:13 (1763336173) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 7 "/mnt/lustre"' Cannot send HSM request (use of /mnt/lustre): Invalid argument hsm: '/qwerty/uyt' is not a valid FID PASS 29a (5s) == sanity-hsm test 29b: Archive/delete/remove by FID from the archive. ========================================================== 18:36:18 (1763336178) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191724 s, 5.5 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 29b (8s) == sanity-hsm test 29c: Archive/delete/remove by FID, using a file list. ========================================================== 18:36:26 (1763336186) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167886 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123164 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154031 s, 6.8 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 29c (10s) == sanity-hsm test 29d: hsm_remove by FID with archive_id 0 for unlinked file cause ========================================================== 18:36:37 (1763336197) SKIP: sanity-hsm test_29d Need 3 or more clients, have 1 SKIP 29d (2s) == sanity-hsm test 30a: Restore at exec (import case) ==== 18:36:39 (1763336199) SKIP: sanity-hsm test_30a Need 2 or more clients, have 1 SKIP 30a (2s) == sanity-hsm test 30b: Restore at exec (release case) === 18:36:41 (1763336201) SKIP: sanity-hsm test_30b Need 2 or more clients, have 1 SKIP 30b (3s) == sanity-hsm test 30c: Update during exec of released file must fail ========================================================== 18:36:44 (1763336204) SKIP: sanity-hsm test_30c Need 2 or more clients, have 1 SKIP 30c (2s) == sanity-hsm test 31a: Import a large file and check size during restore ========================================================== 18:36:46 (1763336206) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.872474 s, 46.9 MB/s lhsmtool_posix: 1763336209.281454 [58038]: action=1 src=d31a.sanity-hsm/f31a.sanity-hsm dst=/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763336209.291863 [58038]: importing '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1763336209.332106 [58038]: imported '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//040b/0x200000406:0xd:0x0'=='/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1763336209.332174 [58038]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' size seen is right: 40894464 == 40894464 done PASS 31a (18s) == sanity-hsm test 31b: Restore a large unaligned file and check size during restore ========================================================== 18:37:04 (1763336224) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.642212 s, 60.7 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' size seen is right: 39000000 == 39000000 done PASS 31b (20s) == sanity-hsm test 31c: Restore a large aligned file and check size during restore ========================================================== 18:37:24 (1763336244) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.601183 s, 68.0 MB/s Waiting 200s for 'SUCCEED' size seen is right: 40894464 == 40894464 done PASS 31c (19s) == sanity-hsm test 33: Kill a restore waiting process ==== 18:37:43 (1763336263) 0+0 records in 0+0 records out 0 bytes copied, 0.00349767 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_33/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg323-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Copytool is continued on oleg323-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2829: 60036 Terminated md5sum $f > /dev/null PASS 33 (8s) == sanity-hsm test 34: Remove file during restore ======== 18:37:51 (1763336271) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_34/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00619041 s, 0.0 kB/s Copytool is suspended on oleg323-client.virtnet Copytool is continued on oleg323-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2861: kill: (60697) - No such process PASS 34 (7s) == sanity-hsm test 35: Overwrite file during restore ===== 18:37:58 (1763336278) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_35/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00591616 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Copytool is suspended on oleg323-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Copytool is continued on oleg323-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2898: kill: (61373) - No such process PASS 35 (11s) == sanity-hsm test 36: Move file during restore ========== 18:38:09 (1763336289) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_36/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00599827 s, 0.0 kB/s Copytool is suspended on oleg323-client.virtnet Copytool is continued on oleg323-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2934: kill: (62047) - No such process PASS 36 (8s) == sanity-hsm test 37: re-archive a dirty file =========== 18:38:17 (1763336297) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_37/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224011 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342651 s, 30.6 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 37 (72s) == sanity-hsm test 40: Parallel archive requests ========= 18:39:29 (1763336369) oleg323-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1152921504606846976: Cannot allocate memory oleg323-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1152921504606846976': Cannot allocate memory pdsh@oleg323-client: oleg323-server: ssh exited with exit code 12 set max_requests=1152921504606846976 failed with 12 [ 1258.233341] Lustre: lustre-MDT0000: No more memory to set HSM max_requests=1152921504606846976 (max request memory: 461MB, current total 3/975941): rc = -12 Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_40/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge Wait for all requests being enqueued...100 archive requests submitted 100 archive requests submitted 100 archive requests submitted 100 archive requests submitted OK pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 PASS 40 (127s) == sanity-hsm test 50: Archive with large number of pending HSM actions ========================================================== 18:41:36 (1763336496) oleg323-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1000000: Cannot allocate memory oleg323-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1000000': Cannot allocate memory pdsh@oleg323-client: oleg323-server: ssh exited with exit code 12 Filesystem Inodes IUsed IFree IUse% Mounted on 192.168.203.123@tcp:/lustre 231274 1002 230272 1% /mnt/lustre - create 2041 (time 1763336509.89 total 10.00 last 204.07) - create 3836 (time 1763336519.89 total 20.00 last 179.47) - create 5502 (time 1763336529.89 total 30.00 last 166.59) - create 7429 (time 1763336539.89 total 40.01 last 192.66) - create 9517 (time 1763336549.89 total 50.01 last 208.74) total: 10000 create in 52.97 seconds: 188.79 ops/second 10000 - unlinked 0 (time 1763336620 ; total 0 ; last 0) total: 10000 unlinks in 61 seconds: 163.934433 unlinks/second Start Phase files_num: 10000 time: 63 - create 1682 (time 1763336695.06 total 10.00 last 168.16) - create 3601 (time 1763336705.08 total 20.02 last 191.50) - create 5473 (time 1763336715.09 total 30.02 last 187.17) - create 7266 (time 1763336725.09 total 40.03 last 179.19) - create 9058 (time 1763336735.10 total 50.04 last 179.12) - create 10000 (time 1763336740.95 total 55.89 last 160.88) - create 11659 (time 1763336750.95 total 65.89 last 165.87) - create 13264 (time 1763336760.96 total 75.90 last 160.45) - create 14891 (time 1763336770.96 total 85.90 last 162.60) - create 16477 (time 1763336780.97 total 95.90 last 158.57) - create 18081 (time 1763336790.97 total 105.91 last 160.33) - create 19720 (time 1763336800.98 total 115.91 last 163.81) total: 20000 create in 117.70 seconds: 169.93 ops/second - unlinked 0 (time 1763336939 ; total 0 ; last 0) - unlinked 10000 (time 1763337010 ; total 71 ; last 71) total: 20000 unlinks in 141 seconds: 141.843979 unlinks/second Middle Phase files_num: 20000 time: 132 - create 1444 (time 1763337094.18 total 10.02 last 144.17) - create 2973 (time 1763337104.18 total 20.02 last 152.86) - create 4495 (time 1763337114.19 total 30.03 last 152.03) - create 5990 (time 1763337124.19 total 40.03 last 149.50) - create 7624 (time 1763337134.19 total 50.03 last 163.39) - create 9197 (time 1763337144.20 total 60.03 last 157.26) total: 10000 create in 65.17 seconds: 153.45 ops/second 40000 - unlinked 0 (time 1763337221 ; total 0 ; last 0) total: 10000 unlinks in 62 seconds: 161.290329 unlinks/second End Phase files_num: 10000 time: 67 40000 mdt.lustre-MDT0000.hsm_control=purge PASS 50 (798s) == sanity-hsm test 52: Opened for write file on an evicted client should be set dirty ========================================================== 18:54:54 (1763337294) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_52/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204583 s, 5.1 MB/s multiop /mnt/lustre/d52.sanity-hsm/f52.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7560 pdsh@oleg323-client: oleg323-client: ssh exited with exit code 5 PASS 52 (10s) == sanity-hsm test 53: Opened for read file on an evicted client should not be set dirty ========================================================== 18:55:04 (1763337304) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_53/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197031 s, 5.3 MB/s multiop /mnt/lustre/d53.sanity-hsm/f53.sanity-hsm vo_c TMPPIPE=/tmp/multiop_open_wait_pipe.7560 PASS 53 (11s) == sanity-hsm test 54: Write during an archive cancels it ========================================================== 18:55:15 (1763337315) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.761786 s, 51.2 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_54/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA Waiting 200s for 'FAILED' Waiting 170s for 'FAILED' Updated after 36s: want 'FAILED' got 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA PASS 54 (47s) == sanity-hsm test 55: Truncate during an archive cancels it ========================================================== 18:56:02 (1763337362) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.82345 s, 47.4 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_55/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NRA PASS 55 (10s) == sanity-hsm test 56: Setattr during an archive is ok === 18:56:12 (1763337372) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.957077 s, 40.7 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_56/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Waiting 190s for 'SUCCEED' Updated after 36s: want 'SUCCEED' got 'SUCCEED' PASS 56 (43s) == sanity-hsm test 57: Archive a file with dirty cache on another node ========================================================== 18:56:56 (1763337416) SKIP: sanity-hsm test_57 Need 2 or more clients, have 1 SKIP 57 (2s) == sanity-hsm test 58: Truncate a released file will trigger restore ========================================================== 18:56:58 (1763337418) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_58/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' truncate up from 1516 to 3032 Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate down from 1516 to 758 Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate to 0 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 PASS 58 (19s) == sanity-hsm test 59: Release stripeless file with non-zero size ========================================================== 18:57:17 (1763337437) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_59/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 59 (7s) == sanity-hsm test 60: Changing progress update interval from default ========================================================== 18:57:24 (1763337444) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.220685 s, 47.5 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_60/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 --update-interval 5 "/mnt/lustre2"' Waiting 100s for 'start copy' Expecting a progress update within 20 seconds... 5242880 bytes copied in 3 seconds. Wait for on going archive hsm action to complete Waiting 10s for 'copied' Updated after 4s: want 'copied' got 'copied' PASS 60 (19s) == sanity-hsm test 61: Waiting archive of a removed file should fail ========================================================== 18:57:43 (1763337463) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_61/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled PASS 61 (9s) == sanity-hsm test 70: Copytool logs JSON register/unregister events to FIFO ========================================================== 18:57:52 (1763337472) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_70/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --event-fifo /tmp/sanity-hsm.test_70.JTVF/fifo "/mnt/lustre2"' Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s Register/Unregister events look OK. PASS 70 (9s) == sanity-hsm test 71: Copytool logs JSON archive events to FIFO ========================================================== 18:58:01 (1763337481) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_71/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_71.r0Je/fifo "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=-NRA 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228363 s, 4.6 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Archive events look OK. mdt.lustre-MDT0000.hsm.policy=-NRA PASS 71 (16s) == sanity-hsm test 72: Copytool logs JSON restore events to FIFO ========================================================== 18:58:17 (1763337497) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_72/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_72.jOEn/fifo "/mnt/lustre2"' 16+0 records in 16+0 records out 16000000 bytes (16 MB, 15 MiB) copied, 0.354162 s, 45.2 MB/s lhsmtool_posix: 1763337500.983637 [80557]: action=1 src=d72.sanity-hsm/f72.sanity-hsm dst=/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763337500.993051 [80557]: importing '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1763337501.046517 [80557]: imported '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//0bc8/0x200000bd2:0x1a:0x0'=='/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1763337501.046873 [80557]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Restore events look OK. PASS 72 (13s) == sanity-hsm test 90: Archive/restore a file list ======= 18:58:30 (1763337510) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_90/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge Waiting 200s for '' pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Waiting 200s for '' Changed after 2s: from 'lrh=[type=10680000 len=136 idx=4/10058] fid=[0x200000402:0x29f:0x0] dfid=[0x200000402:0x29f:0x0] compound/cookie=0x0/0x691afbde action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10059] fid=[0x200000402:0x2a0:0x0] dfid=[0x200000402:0x2a0:0x0] compound/cookie=0x0/0x691afbdf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10060] fid=[0x200000402:0x2a1:0x0] dfid=[0x200000402:0x2a1:0x0] compound/cookie=0x0/0x691afbe0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10061] fid=[0x200000402:0x2a2:0x0] dfid=[0x200000402:0x2a2:0x0] compound/cookie=0x0/0x691afbe1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10062] fid=[0x200000402:0x2a3:0x0] dfid=[0x200000402:0x2a3:0x0] compound/cookie=0x0/0x691afbe2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10063] fid=[0x200000402:0x2a4:0x0] dfid=[0x200000402:0x2a4:0x0] compound/cookie=0x0/0x691afbe3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10064] fid=[0x200000402:0x2a5:0x0] dfid=[0x200000402:0x2a5:0x0] compound/cookie=0x0/0x691afbe4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10066] fid=[0x200000402:0x2a6:0x0] dfid=[0x200000402:0x2a6:0x0] compound/cookie=0x0/0x691afbe5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10067] fid=[0x200000402:0x2a7:0x0] dfid=[0x200000402:0x2a7:0x0] compound/cookie=0x0/0x691afbe6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10068] fid=[0x200000402:0x2a8:0x0] dfid=[0x200000402:0x2a8:0x0] compound/cookie=0x0/0x691afbe7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10069] fid=[0x200000402:0x2a9:0x0] dfid=[0x200000402:0x2a9:0x0] compound/cookie=0x0/0x691afbe8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10070] fid=[0x200000402:0x2aa:0x0] dfid=[0x200000402:0x2aa:0x0] compound/cookie=0x0/0x691afbe9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10071] fid=[0x200000402:0x2ab:0x0] dfid=[0x200000402:0x2ab:0x0] compound/cookie=0x0/0x691afbea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10072] fid=[0x200000402:0x2ac:0x0] dfid=[0x200000402:0x2ac:0x0] compound/cookie=0x0/0x691afbeb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10073] fid=[0x200000402:0x2ad:0x0] dfid=[0x200000402:0x2ad:0x0] compound/cookie=0x0/0x691afbec action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10074] fid=[0x200000402:0x2ae:0x0] dfid=[0x200000402:0x2ae:0x0] compound/cookie=0x0/0x691afbed action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10075] fid=[0x200000402:0x2af:0x0] dfid=[0x200000402:0x2af:0x0] compound/cookie=0x0/0x691afbee action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10076] fid=[0x200000402:0x2b0:0x0] dfid=[0x200000402:0x2b0:0x0] compound/cookie=0x0/0x691afbef action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10077] fid=[0x200000402:0x2b1:0x0] dfid=[0x200000402:0x2b1:0x0] compound/cookie=0x0/0x691afbf0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10078] fid=[0x200000402:0x2b2:0x0] dfid=[0x200000402:0x2b2:0x0] compound/cookie=0x0/0x691afbf1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10079] fid=[0x200000402:0x2b3:0x0] dfid=[0x200000402:0x2b3:0x0] compound/cookie=0x0/0x691afbf2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10080] fid=[0x200000402:0x2b4:0x0] dfid=[0x200000402:0x2b4:0x0] compound/cookie=0x0/0x691afbf3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10081] fid=[0x200000402:0x2b5:0x0] dfid=[0x200000402:0x2b5:0x0] compound/cookie=0x0/0x691afbf4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10082] fid=[0x200000402:0x2b6:0x0] dfid=[0x200000402:0x2b6:0x0] compound/cookie=0x0/0x691afbf5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000402:0x2b7:0x0] dfid=[0x200000402:0x2b7:0x0] compound/cookie=0x0/0x691afbf6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000402:0x2b8:0x0] dfid=[0x200000402:0x2b8:0x0] compound/cookie=0x0/0x691afbf7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000402:0x2b9:0x0] dfid=[0x200000402:0x2b9:0x0] compound/cookie=0x0/0x691afbf8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000402:0x2ba:0x0] dfid=[0x200000402:0x2ba:0x0] compound/cookie=0x0/0x691afbf9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000402:0x2bb:0x0] dfid=[0x200000402:0x2bb:0x0] compound/cookie=0x0/0x691afbfa action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000402:0x2bc:0x0] dfid=[0x200000402:0x2bc:0x0] compound/cookie=0x0/0x691afbfb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000402:0x2bd:0x0] dfid=[0x200000402:0x2bd:0x0] compound/cookie=0x0/0x691afbfc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000402:0x2be:0x0] dfid=[0x200000402:0x2be:0x0] compound/cookie=0x0/0x691afbfd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000402:0x2bf:0x0] dfid=[0x200000402:0x2bf:0x0] compound/cookie=0x0/0x691afbfe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000402:0x2c0:0x0] dfid=[0x200000402:0x2c0:0x0] compound/cookie=0x0/0x691afbff action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000402:0x2c1:0x0] dfid=[0x200000402:0x2c1:0x0] compound/cookie=0x0/0x691afc00 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000402:0x2c2:0x0] dfid=[0x200000402:0x2c2:0x0] compound/cookie=0x0/0x691afc01 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000402:0x2c3:0x0] dfid=[0x200000402:0x2c3:0x0] compound/cookie=0x0/0x691afc02 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000402:0x2c4:0x0] dfid=[0x200000402:0x2c4:0x0] compound/cookie=0x0/0x691afc03 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000402:0x2c5:0x0] dfid=[0x200000402:0x2c5:0x0] compound/cookie=0x0/0x691afc04 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000402:0x2c6:0x0] dfid=[0x200000402:0x2c6:0x0] compound/cookie=0x0/0x691afc05 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000402:0x2c7:0x0] dfid=[0x200000402:0x2c7:0x0] compound/cookie=0x0/0x691afc06 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000402:0x2c8:0x0] dfid=[0x200000402:0x2c8:0x0] compound/cookie=0x0/0x691afc07 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000402:0x2c9:0x0] dfid=[0x200000402:0x2c9:0x0] compound/cookie=0x0/0x691afc08 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000402:0x2ca:0x0] dfid=[0x200000402:0x2ca:0x0] compound/cookie=0x0/0x691afc09 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000402:0x2cb:0x0] dfid=[0x200000402:0x2cb:0x0] compound/cookie=0x0/0x691afc0a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000402:0x2cc:0x0] dfid=[0x200000402:0x2cc:0x0] compound/cookie=0x0/0x691afc0b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000402:0x2cd:0x0] dfid=[0x200000402:0x2cd:0x0] compound/cookie=0x0/0x691afc0c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000402:0x2ce:0x0] dfid=[0x200000402:0x2ce:0x0] compound/cookie=0x0/0x691afc0d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000402:0x2cf:0x0] dfid=[0x200000402:0x2cf:0x0] compound/cookie=0x0/0x691afc0e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000402:0x2d0:0x0] dfid=[0x200000402:0x2d0:0x0] compound/cookie=0x0/0x691afc0f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000402:0x2d1:0x0] dfid=[0x200000402:0x2d1:0x0] compound/cookie=0x0/0x691afc10 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=4/10071] fid=[0x200000402:0x2ab:0x0] dfid=[0x200000402:0x2ab:0x0] compound/cookie=0x0/0x691afbea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10072] fid=[0x200000402:0x2ac:0x0] dfid=[0x200000402:0x2ac:0x0] compound/cookie=0x0/0x691afbeb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10073] fid=[0x200000402:0x2ad:0x0] dfid=[0x200000402:0x2ad:0x0] compound/cookie=0x0/0x691afbec action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10074] fid=[0x200000402:0x2ae:0x0] dfid=[0x200000402:0x2ae:0x0] compound/cookie=0x0/0x691afbed action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10075] fid=[0x200000402:0x2af:0x0] dfid=[0x200000402:0x2af:0x0] compound/cookie=0x0/0x691afbee action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10076] fid=[0x200000402:0x2b0:0x0] dfid=[0x200000402:0x2b0:0x0] compound/cookie=0x0/0x691afbef action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10077] fid=[0x200000402:0x2b1:0x0] dfid=[0x200000402:0x2b1:0x0] compound/cookie=0x0/0x691afbf0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10078] fid=[0x200000402:0x2b2:0x0] dfid=[0x200000402:0x2b2:0x0] compound/cookie=0x0/0x691afbf1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10079] fid=[0x200000402:0x2b3:0x0] dfid=[0x200000402:0x2b3:0x0] compound/cookie=0x0/0x691afbf2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10080] fid=[0x200000402:0x2b4:0x0] dfid=[0x200000402:0x2b4:0x0] compound/cookie=0x0/0x691afbf3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10081] fid=[0x200000402:0x2b5:0x0] dfid=[0x200000402:0x2b5:0x0] compound/cookie=0x0/0x691afbf4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10082] fid=[0x200000402:0x2b6:0x0] dfid=[0x200000402:0x2b6:0x0] compound/cookie=0x0/0x691afbf5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000402:0x2b7:0x0] dfid=[0x200000402:0x2b7:0x0] compound/cookie=0x0/0x691afbf6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000402:0x2b8:0x0] dfid=[0x200000402:0x2b8:0x0] compound/cookie=0x0/0x691afbf7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000402:0x2b9:0x0] dfid=[0x200000402:0x2b9:0x0] compound/cookie=0x0/0x691afbf8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000402:0x2ba:0x0] dfid=[0x200000402:0x2ba:0x0] compound/cookie=0x0/0x691afbf9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000402:0x2bb:0x0] dfid=[0x200000402:0x2bb:0x0] compound/cookie=0x0/0x691afbfa action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000402:0x2bc:0x0] dfid=[0x200000402:0x2bc:0x0] compound/cookie=0x0/0x691afbfb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000402:0x2bd:0x0] dfid=[0x200000402:0x2bd:0x0] compound/cookie=0x0/0x691afbfc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000402:0x2be:0x0] dfid=[0x200000402:0x2be:0x0] compound/cookie=0x0/0x691afbfd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000402:0x2bf:0x0] dfid=[0x200000402:0x2bf:0x0] compound/cookie=0x0/0x691afbfe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000402:0x2c0:0x0] dfid=[0x200000402:0x2c0:0x0] compound/cookie=0x0/0x691afbff action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000402:0x2c1:0x0] dfid=[0x200000402:0x2c1:0x0] compound/cookie=0x0/0x691afc00 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000402:0x2c2:0x0] dfid=[0x200000402:0x2c2:0x0] compound/cookie=0x0/0x691afc01 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000402:0x2c3:0x0] dfid=[0x200000402:0x2c3:0x0] compound/cookie=0x0/0x691afc02 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000402:0x2c4:0x0] dfid=[0x200000402:0x2c4:0x0] compound/cookie=0x0/0x691afc03 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000402:0x2c5:0x0] dfid=[0x200000402:0x2c5:0x0] compound/cookie=0x0/0x691afc04 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000402:0x2c6:0x0] dfid=[0x200000402:0x2c6:0x0] compound/cookie=0x0/0x691afc05 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000402:0x2c7:0x0] dfid=[0x200000402:0x2c7:0x0] compound/cookie=0x0/0x691afc06 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000402:0x2c8:0x0] dfid=[0x200000402:0x2c8:0x0] compound/cookie=0x0/0x691afc07 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000402:0x2c9:0x0] dfid=[0x200000402:0x2c9:0x0] compound/cookie=0x0/0x691afc08 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000402:0x2ca:0x0] dfid=[0x200000402:0x2ca:0x0] compound/cookie=0x0/0x691afc09 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000402:0x2cb:0x0] dfid=[0x200000402:0x2cb:0x0] compound/cookie=0x0/0x691afc0a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000402:0x2cc:0x0] dfid=[0x200000402:0x2cc:0x0] compound/cookie=0x0/0x691afc0b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000402:0x2cd:0x0] dfid=[0x200000402:0x2cd:0x0] compound/cookie=0x0/0x691afc0c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000402:0x2ce:0x0] dfid=[0x200000402:0x2ce:0x0] compound/cookie=0x0/0x691afc0d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000402:0x2cf:0x0] dfid=[0x200000402:0x2cf:0x0] compound/cookie=0x0/0x691afc0e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000402:0x2d0:0x0] dfid=[0x200000402:0x2d0:0x0] compound/cookie=0x0/0x691afc0f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000402:0x2d1:0x0] dfid=[0x200000402:0x2d1:0x0] compound/cookie=0x0/0x691afc10 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 3s: from 'lrh=[type=10680000 len=136 idx=4/10071] fid=[0x200000402:0x2ab:0x0] dfid=[0x200000402:0x2ab:0x0] compound/cookie=0x0/0x691afbea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10072] fid=[0x200000402:0x2ac:0x0] dfid=[0x200000402:0x2ac:0x0] compound/cookie=0x0/0x691afbeb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10073] fid=[0x200000402:0x2ad:0x0] dfid=[0x200000402:0x2ad:0x0] compound/cookie=0x0/0x691afbec action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10074] fid=[0x200000402:0x2ae:0x0] dfid=[0x200000402:0x2ae:0x0] compound/cookie=0x0/0x691afbed action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10075] fid=[0x200000402:0x2af:0x0] dfid=[0x200000402:0x2af:0x0] compound/cookie=0x0/0x691afbee action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10076] fid=[0x200000402:0x2b0:0x0] dfid=[0x200000402:0x2b0:0x0] compound/cookie=0x0/0x691afbef action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10077] fid=[0x200000402:0x2b1:0x0] dfid=[0x200000402:0x2b1:0x0] compound/cookie=0x0/0x691afbf0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10078] fid=[0x200000402:0x2b2:0x0] dfid=[0x200000402:0x2b2:0x0] compound/cookie=0x0/0x691afbf1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10079] fid=[0x200000402:0x2b3:0x0] dfid=[0x200000402:0x2b3:0x0] compound/cookie=0x0/0x691afbf2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10080] fid=[0x200000402:0x2b4:0x0] dfid=[0x200000402:0x2b4:0x0] compound/cookie=0x0/0x691afbf3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10081] fid=[0x200000402:0x2b5:0x0] dfid=[0x200000402:0x2b5:0x0] compound/cookie=0x0/0x691afbf4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10082] fid=[0x200000402:0x2b6:0x0] dfid=[0x200000402:0x2b6:0x0] compound/cookie=0x0/0x691afbf5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000402:0x2b7:0x0] dfid=[0x200000402:0x2b7:0x0] compound/cookie=0x0/0x691afbf6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000402:0x2b8:0x0] dfid=[0x200000402:0x2b8:0x0] compound/cookie=0x0/0x691afbf7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000402:0x2b9:0x0] dfid=[0x200000402:0x2b9:0x0] compound/cookie=0x0/0x691afbf8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000402:0x2ba:0x0] dfid=[0x200000402:0x2ba:0x0] compound/cookie=0x0/0x691afbf9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000402:0x2bb:0x0] dfid=[0x200000402:0x2bb:0x0] compound/cookie=0x0/0x691afbfa action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000402:0x2bc:0x0] dfid=[0x200000402:0x2bc:0x0] compound/cookie=0x0/0x691afbfb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000402:0x2bd:0x0] dfid=[0x200000402:0x2bd:0x0] compound/cookie=0x0/0x691afbfc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000402:0x2be:0x0] dfid=[0x200000402:0x2be:0x0] compound/cookie=0x0/0x691afbfd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000402:0x2bf:0x0] dfid=[0x200000402:0x2bf:0x0] compound/cookie=0x0/0x691afbfe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000402:0x2c0:0x0] dfid=[0x200000402:0x2c0:0x0] compound/cookie=0x0/0x691afbff action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000402:0x2c1:0x0] dfid=[0x200000402:0x2c1:0x0] compound/cookie=0x0/0x691afc00 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000402:0x2c2:0x0] dfid=[0x200000402:0x2c2:0x0] compound/cookie=0x0/0x691afc01 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000402:0x2c3:0x0] dfid=[0x200000402:0x2c3:0x0] compound/cookie=0x0/0x691afc02 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000402:0x2c4:0x0] dfid=[0x200000402:0x2c4:0x0] compound/cookie=0x0/0x691afc03 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000402:0x2c5:0x0] dfid=[0x200000402:0x2c5:0x0] compound/cookie=0x0/0x691afc04 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000402:0x2c6:0x0] dfid=[0x200000402:0x2c6:0x0] compound/cookie=0x0/0x691afc05 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000402:0x2c7:0x0] dfid=[0x200000402:0x2c7:0x0] compound/cookie=0x0/0x691afc06 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000402:0x2c8:0x0] dfid=[0x200000402:0x2c8:0x0] compound/cookie=0x0/0x691afc07 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000402:0x2c9:0x0] dfid=[0x200000402:0x2c9:0x0] compound/cookie=0x0/0x691afc08 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000402:0x2ca:0x0] dfid=[0x200000402:0x2ca:0x0] compound/cookie=0x0/0x691afc09 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000402:0x2cb:0x0] dfid=[0x200000402:0x2cb:0x0] compound/cookie=0x0/0x691afc0a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000402:0x2cc:0x0] dfid=[0x200000402:0x2cc:0x0] compound/cookie=0x0/0x691afc0b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000402:0x2cd:0x0] dfid=[0x200000402:0x2cd:0x0] compound/cookie=0x0/0x691afc0c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000402:0x2ce:0x0] dfid=[0x200000402:0x2ce:0x0] compound/cookie=0x0/0x691afc0d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000402:0x2cf:0x0] dfid=[0x200000402:0x2cf:0x0] compound/cookie=0x0/0x691afc0e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000402:0x2d0:0x0] dfid=[0x200000402:0x2d0:0x0] compound/cookie=0x0/0x691afc0f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000402:0x2d1:0x0] dfid=[0x200000402:0x2d1:0x0] compound/cookie=0x0/0x691afc10 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000402:0x2b7:0x0] dfid=[0x200000402:0x2b7:0x0] compound/cookie=0x0/0x691afbf6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000402:0x2b8:0x0] dfid=[0x200000402:0x2b8:0x0] compound/cookie=0x0/0x691afbf7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000402:0x2b9:0x0] dfid=[0x200000402:0x2b9:0x0] compound/cookie=0x0/0x691afbf8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000402:0x2ba:0x0] dfid=[0x200000402:0x2ba:0x0] compound/cookie=0x0/0x691afbf9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000402:0x2bb:0x0] dfid=[0x200000402:0x2bb:0x0] compound/cookie=0x0/0x691afbfa action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000402:0x2bc:0x0] dfid=[0x200000402:0x2bc:0x0] compound/cookie=0x0/0x691afbfb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000402:0x2bd:0x0] dfid=[0x200000402:0x2bd:0x0] compound/cookie=0x0/0x691afbfc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000402:0x2be:0x0] dfid=[0x200000402:0x2be:0x0] compound/cookie=0x0/0x691afbfd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000402:0x2bf:0x0] dfid=[0x200000402:0x2bf:0x0] compound/cookie=0x0/0x691afbfe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000402:0x2c0:0x0] dfid=[0x200000402:0x2c0:0x0] compound/cookie=0x0/0x691afbff action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000402:0x2c1:0x0] dfid=[0x200000402:0x2c1:0x0] compound/cookie=0x0/0x691afc00 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000402:0x2c2:0x0] dfid=[0x200000402:0x2c2:0x0] compound/cookie=0x0/0x691afc01 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000402:0x2c3:0x0] dfid=[0x200000402:0x2c3:0x0] compound/cookie=0x0/0x691afc02 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000402:0x2c4:0x0] dfid=[0x200000402:0x2c4:0x0] compound/cookie=0x0/0x691afc03 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000402:0x2c5:0x0] dfid=[0x200000402:0x2c5:0x0] compound/cookie=0x0/0x691afc04 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000402:0x2c6:0x0] dfid=[0x200000402:0x2c6:0x0] compound/cookie=0x0/0x691afc05 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000402:0x2c7:0x0] dfid=[0x200000402:0x2c7:0x0] compound/cookie=0x0/0x691afc06 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000402:0x2c8:0x0] dfid=[0x200000402:0x2c8:0x0] compound/cookie=0x0/0x691afc07 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000402:0x2c9:0x0] dfid=[0x200000402:0x2c9:0x0] compound/cookie=0x0/0x691afc08 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000402:0x2ca:0x0] dfid=[0x200000402:0x2ca:0x0] compound/cookie=0x0/0x691afc09 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000402:0x2cb:0x0] dfid=[0x200000402:0x2cb:0x0] compound/cookie=0x0/0x691afc0a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000402:0x2cc:0x0] dfid=[0x200000402:0x2cc:0x0] compound/cookie=0x0/0x691afc0b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000402:0x2cd:0x0] dfid=[0x200000402:0x2cd:0x0] compound/cookie=0x0/0x691afc0c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000402:0x2ce:0x0] dfid=[0x200000402:0x2ce:0x0] compound/cookie=0x0/0x691afc0d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000402:0x2cf:0x0] dfid=[0x200000402:0x2cf:0x0] compound/cookie=0x0/0x691afc0e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000402:0x2d0:0x0] dfid=[0x200000402:0x2d0:0x0] compound/cookie=0x0/0x691afc0f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000402:0x2d1:0x0] dfid=[0x200000402:0x2d1:0x0] compound/cookie=0x0/0x691afc10 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 5s: from 'lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000402:0x2b7:0x0] dfid=[0x200000402:0x2b7:0x0] compound/cookie=0x0/0x691afbf6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000402:0x2b8:0x0] dfid=[0x200000402:0x2b8:0x0] compound/cookie=0x0/0x691afbf7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000402:0x2b9:0x0] dfid=[0x200000402:0x2b9:0x0] compound/cookie=0x0/0x691afbf8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000402:0x2ba:0x0] dfid=[0x200000402:0x2ba:0x0] compound/cookie=0x0/0x691afbf9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000402:0x2bb:0x0] dfid=[0x200000402:0x2bb:0x0] compound/cookie=0x0/0x691afbfa action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000402:0x2bc:0x0] dfid=[0x200000402:0x2bc:0x0] compound/cookie=0x0/0x691afbfb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000402:0x2bd:0x0] dfid=[0x200000402:0x2bd:0x0] compound/cookie=0x0/0x691afbfc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000402:0x2be:0x0] dfid=[0x200000402:0x2be:0x0] compound/cookie=0x0/0x691afbfd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000402:0x2bf:0x0] dfid=[0x200000402:0x2bf:0x0] compound/cookie=0x0/0x691afbfe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000402:0x2c0:0x0] dfid=[0x200000402:0x2c0:0x0] compound/cookie=0x0/0x691afbff action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000402:0x2c1:0x0] dfid=[0x200000402:0x2c1:0x0] compound/cookie=0x0/0x691afc00 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000402:0x2c2:0x0] dfid=[0x200000402:0x2c2:0x0] compound/cookie=0x0/0x691afc01 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000402:0x2c3:0x0] dfid=[0x200000402:0x2c3:0x0] compound/cookie=0x0/0x691afc02 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000402:0x2c4:0x0] dfid=[0x200000402:0x2c4:0x0] compound/cookie=0x0/0x691afc03 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000402:0x2c5:0x0] dfid=[0x200000402:0x2c5:0x0] compound/cookie=0x0/0x691afc04 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000402:0x2c6:0x0] dfid=[0x200000402:0x2c6:0x0] compound/cookie=0x0/0x691afc05 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000402:0x2c7:0x0] dfid=[0x200000402:0x2c7:0x0] compound/cookie=0x0/0x691afc06 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000402:0x2c8:0x0] dfid=[0x200000402:0x2c8:0x0] compound/cookie=0x0/0x691afc07 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000402:0x2c9:0x0] dfid=[0x200000402:0x2c9:0x0] compound/cookie=0x0/0x691afc08 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000402:0x2ca:0x0] dfid=[0x200000402:0x2ca:0x0] compound/cookie=0x0/0x691afc09 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000402:0x2cb:0x0] dfid=[0x200000402:0x2cb:0x0] compound/cookie=0x0/0x691afc0a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000402:0x2cc:0x0] dfid=[0x200000402:0x2cc:0x0] compound/cookie=0x0/0x691afc0b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000402:0x2cd:0x0] dfid=[0x200000402:0x2cd:0x0] compound/cookie=0x0/0x691afc0c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000402:0x2ce:0x0] dfid=[0x200000402:0x2ce:0x0] compound/cookie=0x0/0x691afc0d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000402:0x2cf:0x0] dfid=[0x200000402:0x2cf:0x0] compound/cookie=0x0/0x691afc0e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000402:0x2d0:0x0] dfid=[0x200000402:0x2d0:0x0] compound/cookie=0x0/0x691afc0f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000402:0x2d1:0x0] dfid=[0x200000402:0x2d1:0x0] compound/cookie=0x0/0x691afc10 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000402:0x2c6:0x0] dfid=[0x200000402:0x2c6:0x0] compound/cookie=0x0/0x691afc05 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000402:0x2c7:0x0] dfid=[0x200000402:0x2c7:0x0] compound/cookie=0x0/0x691afc06 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000402:0x2c8:0x0] dfid=[0x200000402:0x2c8:0x0] compound/cookie=0x0/0x691afc07 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000402:0x2c9:0x0] dfid=[0x200000402:0x2c9:0x0] compound/cookie=0x0/0x691afc08 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000402:0x2ca:0x0] dfid=[0x200000402:0x2ca:0x0] compound/cookie=0x0/0x691afc09 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000402:0x2cb:0x0] dfid=[0x200000402:0x2cb:0x0] compound/cookie=0x0/0x691afc0a action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000402:0x2cc:0x0] dfid=[0x200000402:0x2cc:0x0] compound/cookie=0x0/0x691afc0b action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000402:0x2cd:0x0] dfid=[0x200000402:0x2cd:0x0] compound/cookie=0x0/0x691afc0c action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000402:0x2ce:0x0] dfid=[0x200000402:0x2ce:0x0] compound/cookie=0x0/0x691afc0d action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000402:0x2cf:0x0] dfid=[0x200000402:0x2cf:0x0] compound/cookie=0x0/0x691afc0e action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000402:0x2d0:0x0] dfid=[0x200000402:0x2d0:0x0] compound/cookie=0x0/0x691afc0f action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000402:0x2d1:0x0] dfid=[0x200000402:0x2d1:0x0] compound/cookie=0x0/0x691afc10 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 PASS 90 (87s) == sanity-hsm test 100: Set coordinator /proc tunables === 18:59:57 (1763337597) Testing mdt.lustre-MDT0000.hsm.loop_period Set loop_period to 0 must failed oleg323-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/loop_period=0: Invalid argument oleg323-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/loop_period'='0': Invalid argument pdsh@oleg323-client: oleg323-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.grace_delay Set grace_delay to 0 must failed oleg323-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/grace_delay=0: Invalid argument oleg323-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/grace_delay'='0': Invalid argument pdsh@oleg323-client: oleg323-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.active_request_timeout Set active_request_timeout to 0 must failed oleg323-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/active_request_timeout=0: Invalid argument oleg323-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/active_request_timeout'='0': Invalid argument pdsh@oleg323-client: oleg323-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.max_requests Set max_requests to 0 must failed oleg323-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=0: Invalid argument oleg323-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='0': Invalid argument pdsh@oleg323-client: oleg323-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.default_archive_id Set default_archive_id to 0 must failed oleg323-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/default_archive_id=0: Invalid argument oleg323-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/default_archive_id'='0': Invalid argument pdsh@oleg323-client: oleg323-server: ssh exited with exit code 22 PASS 100 (24s) == sanity-hsm test 102: Verify coordinator control ======= 19:00:21 (1763337621) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled PASS 102 (9s) == sanity-hsm test 103: Purge all requests =============== 19:00:31 (1763337631) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge Current requests pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 PASS 103 (10s) == sanity-hsm test 103a: Purge pending restore requests == 19:00:40 (1763337640) mdt.lustre-MDT0000.hsm.policy=-NBR Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' (0x200000402:0x319:0x0|0x200000402:0x31a:0x0|0x200000402:0x31b:0x0|0x200000402:0x31c:0x0|0x200000402:0x31d:0x0|0x200000402:0x31e:0x0|0x200000402:0x31f:0x0|0x200000402:0x320:0x0).*action=ARCHIVE.*status=SUCCEED Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s cat: '/mnt/lustre/.lustre/fid/0x200000402:0x31a:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x31b:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x319:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x31d:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x31c:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x31f:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x31e:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000402:0x320:0x0': No data available mdt.lustre-MDT0000.hsm_control=purge PASS 103a (14s) == sanity-hsm test 104: Copy tool data field ============= 19:00:54 (1763337654) 0+0 records in 0+0 records out 0 bytes copied, 0.00644361 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=purge PASS 104 (9s) == sanity-hsm test 105: Restart of coordinator =========== 19:01:03 (1763337663) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=purge PASS 105 (20s) == sanity-hsm test 106: Copytool register/unregister ===== 19:01:23 (1763337683) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg323-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg323-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=33af46a0-e4f6-44ec-9a4f-1f0796276236 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 oleg323-client.virtnet: 86848 Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 uuid not found in agent list on mds1 Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg323-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg323-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=33af46a0-e4f6-44ec-9a4f-1f0796276236 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 PASS 106 (20s) == sanity-hsm test 107: Copytool re-register after MDS restart ========================================================== 19:01:43 (1763337703) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_107/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Failing mds1 on oleg323-server Stopping /mnt/lustre-mds1 (opts:) on oleg323-server 19:01:52 (1763337712) shut down facet: mds1 facet_host: oleg323-server facet_failover_host: oleg323-server Failover mds1 to oleg323-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg323-server: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Started lustre-MDT0000 19:02:12 (1763337732) targets are mounted 19:02:12 (1763337732) facet_failover done oleg323-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 107 (43s) == sanity-hsm test 109: Policy display/change ============ 19:02:26 (1763337746) mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0000.hsm.policy=NRA NBR Next set_param must failed oleg323-server: error: set_param: setting /sys/kernel/debug/lustre/mdt/lustre-MDT0000/hsm/policy=wrong: Invalid argument oleg323-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/policy'='wrong': Invalid argument pdsh@oleg323-client: oleg323-server: ssh exited with exit code 22 Back to default policy mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR PASS 109 (17s) == sanity-hsm test 110a: Non blocking restore policy (import case) ========================================================== 19:02:44 (1763337764) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1763337766.051172 [90624]: action=1 src=d110a.sanity-hsm/f110a.sanity-hsm dst=/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763337766.064338 [90624]: importing '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1763337766.121942 [90624]: imported '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//0bba/0x200000bd2:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1763337766.122000 [90624]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NBR md5sum: /mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm: No data available Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm.policy=-NBR PASS 110a (11s) == sanity-hsm test 110b: Non blocking restore policy (release case) ========================================================== 19:02:54 (1763337774) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm.policy=+NBR md5sum: /mnt/lustre/d110b.sanity-hsm/f110b.sanity-hsm: No data available mdt.lustre-MDT0000.hsm.policy=-NBR PASS 110b (11s) == sanity-hsm test 111a: No retry policy (import case), restore will error ========================================================== 19:03:06 (1763337786) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1763337788.059233 [91981]: action=1 src=d111a.sanity-hsm/f111a.sanity-hsm dst=/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763337788.065163 [91981]: importing '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1763337788.096235 [91981]: imported '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//0bbf/0x200000bd2:0x6d:0x0'=='/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1763337788.096522 [91981]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NRA Remove from backend: 0x200000bd2:0x6d:0x0 = /tmp/arc1/sanity-hsm.test_111a/0bbf/0x200000bd2:0x6d:0x0 Waiting 200s for 'FAILED' Updated after 2s: want 'FAILED' got 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA PASS 111a (10s) == sanity-hsm test 111b: No retry policy (release case), restore will error ========================================================== 19:03:16 (1763337796) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA Remove from backend: 0x200000402:0x329:0x0 = /tmp/arc1/sanity-hsm.test_111b/072b/0x200000402:0x329:0x0 mdt.lustre-MDT0000.hsm.policy=-NRA PASS 111b (9s) == sanity-hsm test 112: State of recorded request ======== 19:03:25 (1763337805) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_112/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled /mnt/lustre/d112.sanity-hsm/f112.sanity-hsm: ARCHIVE waiting (from 0 to EOF) mdt.lustre-MDT0000.hsm_control=enabled PASS 112 (10s) == sanity-hsm test 113: wrong stat after restore ========= 19:03:35 (1763337815) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0602649 s, 17.4 MB/s Starting client: oleg323-client.virtnet: -o user_xattr,flock 192.168.203.123@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_113/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying released state: multiop /mnt/lustre/d113.sanity-hsm/f113.sanity-hsm voO_WRONLY:O_APPEND:_w4c TMPPIPE=/tmp/multiop_open_wait_pipe.7560 File: /mnt/lustre2/d113.sanity-hsm/f113.sanity-hsm Size: 1048576 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205289280301 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-11-16 19:03:37.000000000 -0500 Modify: 2025-11-16 19:03:37.000000000 -0500 Change: 2025-11-16 19:03:37.000000000 -0500 Birth: 2025-11-16 19:03:36.000000000 -0500 Stopping client oleg323-client.virtnet /mnt/lustre3 (opts:) PASS 113 (13s) == sanity-hsm test 114: Incompatible request does not set other requests as STARTED ========================================================== 19:03:48 (1763337828) 0+0 records in 0+0 records out 0 bytes copied, 0.00285828 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00468629 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_114/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled PASS 114 (12s) == sanity-hsm test 200: Register/Cancel archive ========== 19:04:00 (1763337840) 0+0 records in 0+0 records out 0 bytes copied, 0.00524173 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_200/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg323-client.virtnet Copytool is continued on oleg323-client.virtnet PASS 200 (9s) == sanity-hsm test 201: Register/Cancel restore ========== 19:04:09 (1763337849) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_201/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.23852 s, 33.0 MB/s lhsmtool_posix: 1763337852.731119 [96222]: action=1 src=d201.sanity-hsm/f201.sanity-hsm dst=/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763337852.739094 [96222]: importing '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1763337852.792468 [96222]: imported '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//0ba5/0x200000bd2:0x77:0x0'=='/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1763337852.793864 [96222]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled PASS 201 (13s) == sanity-hsm test 202: Register/Cancel remove =========== 19:04:22 (1763337862) 0+0 records in 0+0 records out 0 bytes copied, 0.0102846 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_202/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg323-client.virtnet Waiting 200s for 'CANCELED' PASS 202 (10s) == sanity-hsm test 220A: Changelog for archive =========== 19:04:32 (1763337872) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.3 16HSM 00:04:37.569467067 2025.11.17 0x0 t=[0x200000402:0x336:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 220A (14s) == sanity-hsm test 220a: Changelog for failed archive ==== 19:04:46 (1763337886) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl2' Copytool is suspended on oleg323-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Copytool is continued on oleg323-client.virtnet lustre-MDT0000.6 16HSM 00:04:52.726271902 2025.11.17 0x2 t=[0x200000402:0x337:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp p=[0x200000bd2:0x7a:0x0] lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 220a (14s) == sanity-hsm test 221: Changelog for archive canceled === 19:05:00 (1763337900) 0+0 records in 0+0 records out 0 bytes copied, 0.00519075 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_221/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl3' Copytool is suspended on oleg323-client.virtnet Copytool is continued on oleg323-client.virtnet lustre-MDT0000.8 16HSM 00:05:06.587173190 2025.11.17 0x7d t=[0x200000402:0x339:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 221 (15s) == sanity-hsm test 222a: Changelog for explicit restore == 19:05:15 (1763337915) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1763337917.309900 [99802]: action=1 src=d222a.sanity-hsm/f222a.sanity-hsm dst=/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763337917.318319 [99802]: importing '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1763337917.410107 [99802]: imported '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//0bac/0x200000bd2:0x7e:0x0'=='/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1763337917.410170 [99802]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl4' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.11 16HSM 00:05:20.302863064 2025.11.17 0x80 t=[0x200000bd2:0x7e:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp p=[0x200000bd2:0x7a:0x0] lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 222a (16s) == sanity-hsm test 222b: Changelog for implicit restore == 19:05:31 (1763337931) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl5' 4ef076ad20e5d4a2f4729d6ed41a3bf9 /mnt/lustre/d222b.sanity-hsm/f222b.sanity-hsm lustre-MDT0000.20 16HSM 00:05:37.111211193 2025.11.17 0x80 t=[0x200000402:0x33b:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 PASS 222b (15s) == sanity-hsm test 222c: Changelog for failed explicit restore ========================================================== 19:05:46 (1763337946) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1763337948.668841 [101286]: action=1 src=d222c.sanity-hsm/f222c.sanity-hsm dst=/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763337948.673984 [101286]: importing '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1763337948.728722 [101286]: imported '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//0b51/0x200000bd2:0x83:0x0'=='/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1763337948.728775 [101286]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl6' Copytool is suspended on oleg323-client.virtnet Copytool is continued on oleg323-client.virtnet lustre-MDT0000.22 16HSM 00:05:51.754770522 2025.11.17 0x82 t=[0x200000bd2:0x83:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 222c (13s) == sanity-hsm test 222d: Changelog for failed implicit restore ========================================================== 19:06:00 (1763337960) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl7' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Remove from backend: 0x200000402:0x33d:0x0 = /tmp/arc1/sanity-hsm.test_222d/073f/0x200000402:0x33d:0x0 md5sum: /mnt/lustre/d222d.sanity-hsm/f222d.sanity-hsm: No data available lustre-MDT0000.29 16HSM 00:06:07.424774090 2025.11.17 0x82 t=[0x200000402:0x33d:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 222d (16s) == sanity-hsm test 223a: Changelog for restore canceled (import case) ========================================================== 19:06:15 (1763337975) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.870817 s, 47.0 MB/s mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl8' lhsmtool_posix: 1763337980.067476 [102848]: action=1 src=d223a.sanity-hsm/f223a.sanity-hsm dst=/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1763337980.078163 [102848]: importing '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1763337980.122610 [102848]: imported '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//0b5a/0x200000bd2:0x88:0x0'=='/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1763337980.122660 [102848]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Waiting 200s for 'STARTED' Waiting 200s for 'SUCCEED' Waiting 180s for 'SUCCEED' Updated after 28s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.36 16HSM 00:06:50.820839667 2025.11.17 0xfd t=[0x200000bd2:0x88:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl8 of all records lustre-MDT0000: Deregistered changelog user #8 PASS 223a (45s) == sanity-hsm test 223b: Changelog for restore canceled (release case) ========================================================== 19:07:00 (1763338020) 0+0 records in 0+0 records out 0 bytes copied, 0.00297817 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl9' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg323-client.virtnet Copytool is continued on oleg323-client.virtnet lustre-MDT0000.43 16HSM 00:07:09.065249338 2025.11.17 0xfd t=[0x200000402:0x341:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl9 of all records lustre-MDT0000: Deregistered changelog user #9 PASS 223b (16s) == sanity-hsm test 224A: Changelog for remove ============ 19:07:16 (1763338036) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl10' lustre-MDT0000.48 16HSM 00:07:21.523478190 2025.11.17 0x200 t=[0x200000402:0x343:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl10 of all records lustre-MDT0000: Deregistered changelog user #10 PASS 224A (12s) == sanity-hsm test 224a: Changelog for failed remove ===== 19:07:29 (1763338049) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl11' Remove from backend: 0x200000402:0x344:0x0 = /tmp/arc1/sanity-hsm.test_224a/0746/0x200000402:0x344:0x0 Copytool is suspended on oleg323-client.virtnet Copytool is continued on oleg323-client.virtnet lustre-MDT0000.53 16HSM 00:07:35.467291187 2025.11.17 0x202 t=[0x200000402:0x344:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl11 of all records lustre-MDT0000: Deregistered changelog user #11 PASS 224a (15s) == sanity-hsm test 225: Changelog for remove canceled ==== 19:07:44 (1763338064) Test disabled PASS 225 (6s) == sanity-hsm test 226: changelog for last rm/mv with exiting archive ========================================================== 19:07:50 (1763338070) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_226/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0x200000402:0x347:0x0 mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl12' lustre-MDT0000.54 16HSM 00:07:54.960816571 2025.11.17 0x280 t=[0x200000402:0x345:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.55 16HSM 00:07:55.080468382 2025.11.17 0x280 t=[0x200000402:0x345:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp p=[0x200000bd2:0x8c:0x0] lustre-MDT0000.56 16HSM 00:07:55.080610561 2025.11.17 0x0 t=[0x200000402:0x345:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp p=[0x200000bd2:0x8c:0x0] lustre-MDT0000.57 16HSM 00:07:56.112914738 2025.11.17 0x280 t=[0x200000402:0x346:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.58 16HSM 00:07:56.241590741 2025.11.17 0x280 t=[0x200000402:0x346:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp p=[0x200000bd2:0x8c:0x0] lustre-MDT0000.59 16HSM 00:07:56.241703399 2025.11.17 0x0 t=[0x200000402:0x346:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp p=[0x200000bd2:0x8c:0x0] lustre-MDT0000.60 06UNLNK 00:07:56.722486070 2025.11.17 0x3 t=[0x200000402:0x345:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp p=[0x200000bd2:0x8d:0x0] f226.sanity-hsm-1 lustre-MDT0000.60 06UNLNK 00:07:56.722486070 2025.11.17 0x3 t=[0x200000402:0x345:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp p=[0x200000bd2:0x8d:0x0] f226.sanity-hsm-1 lustre-MDT0000.61 08RENME 00:07:57.079503172 2025.11.17 0x3 t=[0x200000402:0x346:0x0] j=mv.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp p=[0x200000bd2:0x8d:0x0] f226.sanity-hsm-2 s=[0x200000402:0x347:0x0] sp=[0x200000bd2:0x8d:0x0] f226.sanity-hsm-3 lustre-MDT0000: clear the changelog for cl12 of all records lustre-MDT0000: Deregistered changelog user #12 PASS 226 (14s) == sanity-hsm test 227: changelog when explicit setting of HSM flags ========================================================== 19:08:04 (1763338084) 0+0 records in 0+0 records out 0 bytes copied, 0.00547357 s, 0.0 kB/s mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl13' lustre-MDT0000: clear the changelog for cl13 of all records lustre-MDT0000: Deregistered changelog user #13 PASS 227 (13s) == sanity-hsm test 228: On released file, return extend to FIEMAP. For [cp,tar] --sparse ========================================================== 19:08:17 (1763338097) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_228/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0787764 s, 13.3 MB/s /mnt/lustre/f228.sanity-hsm: 1 extent found tar: Removing leading `/' from member names mnt/lustre/f228.sanity-hsm PASS 228 (11s) == sanity-hsm test 250: Coordinator max request ========== 19:08:28 (1763338108) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_250/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.0193023 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00236814 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00354382 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00567159 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 0+0 records in 0+0 records out 0 bytes copied, 0.00526191 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00616367 s, 0.0 kB/s Copytool is suspended on oleg323-client.virtnet Waiting 200s for 'STARTED' Copytool is continued on oleg323-client.virtnet PASS 250 (22s) == sanity-hsm test 251: Coordinator request timeout ====== 19:08:50 (1763338130) 0+0 records in 0+0 records out 0 bytes copied, 0.0138593 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_251/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg323-client.virtnet Waiting 200s for 'CANCELED' PASS 251 (17s) == sanity-hsm test 252: Timeout'ed running archive of a removed file should be canceled ========================================================== 19:09:07 (1763338147) 0+0 records in 0+0 records out 0 bytes copied, 0.0109644 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_252/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg323-client.virtnet Waiting 200s for 'CANCELED' Updated after 2s: want 'CANCELED' got 'CANCELED' Copytool is continued on oleg323-client.virtnet PASS 252 (15s) == sanity-hsm test 253: Check for wrong file size after release ========================================================== 19:09:22 (1763338162) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_253/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 10+0 records in 10+0 records out 10000000 bytes (10 MB, 9.5 MiB) copied, 0.214317 s, 46.7 MB/s Waiting 200s for 'SUCCEED' fail_loc=0x807 Cannot send HSM request (use of /mnt/lustre/d253.sanity-hsm/f253.sanity-hsm): Invalid argument could not release file PASS 253 (8s) == sanity-hsm test 254a: Request counters are initialized to zero ========================================================== 19:09:31 (1763338171) PASS 254a (8s) == sanity-hsm test 254b: Request counters are correctly incremented and decremented ========================================================== 19:09:39 (1763338179) Will launch 42 requests of each type Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_254b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Checking archive requests Copytool is suspended on oleg323-client.virtnet Copytool is continued on oleg323-client.virtnet Checking restore requests Copytool is suspended on oleg323-client.virtnet Copytool is continued on oleg323-client.virtnet Checking remove requests Copytool is suspended on oleg323-client.virtnet Copytool is continued on oleg323-client.virtnet PASS 254b (35s) == sanity-hsm test 255: Copytool registration wakes the coordinator up ========================================================== 19:10:13 (1763338213) 0+0 records in 0+0 records out 0 bytes copied, 0.015401 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s 0+0 records in 0+0 records out 0 bytes copied, 0.0131215 s, 0.0 kB/s 0x200000402:0x3b0:0x0 mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 255 (23s) == sanity-hsm test 260a: Restore request have priority over other requests ========================================================== 19:10:36 (1763338236) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154765 s, 6.8 MB/s 0x200000402:0x3b2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208362 s, 5.0 MB/s 0x200000402:0x3b3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.249874 s, 4.2 MB/s 0x200000402:0x3b4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186717 s, 5.6 MB/s 0x200000402:0x3b5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126631 s, 8.3 MB/s 0x200000402:0x3b6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159023 s, 6.6 MB/s 0x200000402:0x3b7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192821 s, 5.4 MB/s 0x200000402:0x3b8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22296 s, 4.7 MB/s 0x200000402:0x3b9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230768 s, 4.5 MB/s 0x200000402:0x3ba:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148273 s, 7.1 MB/s 0x200000402:0x3bb:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175804 s, 6.0 MB/s 0x200000402:0x3bc:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183278 s, 5.7 MB/s 0x200000402:0x3bd:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254483 s, 4.1 MB/s 0x200000402:0x3be:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204494 s, 5.1 MB/s 0x200000402:0x3bf:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206004 s, 5.1 MB/s 0x200000402:0x3c0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230982 s, 4.5 MB/s 0x200000402:0x3c1:0x0 Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260a (32s) == sanity-hsm test 260b: Restore request have priority over other requests ========================================================== 19:11:08 (1763338268) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124236 s, 8.4 MB/s 0x200000402:0x3c4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154987 s, 6.8 MB/s 0x200000402:0x3c5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133716 s, 7.8 MB/s 0x200000402:0x3c6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138603 s, 7.6 MB/s 0x200000402:0x3c7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117338 s, 8.9 MB/s 0x200000402:0x3c8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159555 s, 6.6 MB/s 0x200000402:0x3c9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137207 s, 7.6 MB/s 0x200000402:0x3ca:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174616 s, 6.0 MB/s 0x200000402:0x3cb:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138159 s, 7.6 MB/s 0x200000402:0x3cc:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131599 s, 8.0 MB/s 0x200000402:0x3cd:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184253 s, 5.7 MB/s 0x200000402:0x3ce:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209952 s, 5.0 MB/s 0x200000402:0x3cf:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158956 s, 6.6 MB/s 0x200000402:0x3d0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247475 s, 4.2 MB/s 0x200000402:0x3d1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183579 s, 5.7 MB/s 0x200000402:0x3d2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207423 s, 5.1 MB/s 0x200000402:0x3d3:0x0 Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260b (32s) == sanity-hsm test 260c: Requests are not reordered on the 'hot' path of the coordinator ========================================================== 19:11:40 (1763338300) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209557 s, 5.0 MB/s 0x200000402:0x3d6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196606 s, 5.3 MB/s 0x200000402:0x3d7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122903 s, 8.5 MB/s 0x200000402:0x3d8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183614 s, 5.7 MB/s 0x200000402:0x3d9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170652 s, 6.1 MB/s 0x200000402:0x3da:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207772 s, 5.0 MB/s 0x200000402:0x3db:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129435 s, 8.1 MB/s 0x200000402:0x3dc:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166318 s, 6.3 MB/s 0x200000402:0x3dd:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225308 s, 4.7 MB/s 0x200000402:0x3de:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148799 s, 7.0 MB/s 0x200000402:0x3df:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160335 s, 6.5 MB/s 0x200000402:0x3e0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185364 s, 5.7 MB/s 0x200000402:0x3e1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183603 s, 5.7 MB/s 0x200000402:0x3e2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23413 s, 4.5 MB/s 0x200000402:0x3e3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141846 s, 7.4 MB/s 0x200000402:0x3e4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137047 s, 7.7 MB/s 0x200000402:0x3e5:0x0 Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE RESTORE PASS 260c (35s) == sanity-hsm test 261: Report 0 bytes size after HSM release ========================================================== 19:12:15 (1763338335) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_261/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0162313 s, 505 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x00000009) exists archived, archive_id:2 /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x0000000d) released exists archived, archive_id:2 PASS 261 (12s) == sanity-hsm test 262: The client should return 1 block for HSM released files ========================================================== 19:12:27 (1763338347) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_262/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0148094 s, 553 kB/s Waiting 200s for 'SUCCEED' PASS 262 (9s) == sanity-hsm test 300: On disk coordinator state kept between MDT umount/mount ========================================================== 19:12:37 (1763338357) Stop coordinator and remove coordinator state at mount mdt.lustre-MDT0000.hsm_control=shutdown Failing mds1 on oleg323-server Stopping /mnt/lustre-mds1 (opts:) on oleg323-server 19:12:45 (1763338365) shut down facet: mds1 facet_host: oleg323-server facet_failover_host: oleg323-server Failover mds1 to oleg323-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg323-server: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Started lustre-MDT0000 19:13:05 (1763338385) targets are mounted 19:13:05 (1763338385) facet_failover done oleg323-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Set coordinator start at mount, and start coordinator Failing mds1 on oleg323-server Stopping /mnt/lustre-mds1 (opts:) on oleg323-server 19:13:40 (1763338420) shut down facet: mds1 facet_host: oleg323-server facet_failover_host: oleg323-server Failover mds1 to oleg323-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg323-server: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Started lustre-MDT0000 19:14:01 (1763338441) targets are mounted 19:14:01 (1763338441) facet_failover done oleg323-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 300 (97s) == sanity-hsm test 301: HSM tunnable are persistent ====== 19:14:14 (1763338454) Failing mds1 on oleg323-server Stopping /mnt/lustre-mds1 (opts:) on oleg323-server 19:14:20 (1763338460) shut down facet: mds1 facet_host: oleg323-server facet_failover_host: oleg323-server Failover mds1 to oleg323-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg323-server: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Started lustre-MDT0000 19:14:41 (1763338481) targets are mounted 19:14:41 (1763338481) facet_failover done oleg323-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 301 (41s) == sanity-hsm test 302: HSM tunnable are persistent when CDT is off ========================================================== 19:14:56 (1763338496) mdt.lustre-MDT0000.hsm_control=shutdown Failing mds1 on oleg323-server Stopping /mnt/lustre-mds1 (opts:) on oleg323-server 19:15:04 (1763338504) shut down facet: mds1 facet_host: oleg323-server facet_failover_host: oleg323-server Failover mds1 to oleg323-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg323-server: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Started lustre-MDT0000 19:15:35 (1763338535) targets are mounted 19:15:35 (1763338535) facet_failover done oleg323-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 302 (55s) == sanity-hsm test 400: Single request is sent to the right MDT ========================================================== 19:15:51 (1763338551) SKIP: sanity-hsm test_400 needs >= 2 MDTs SKIP 400 (3s) == sanity-hsm test 401: Compound requests split and sent to their respective MDTs ========================================================== 19:15:54 (1763338554) SKIP: sanity-hsm test_401 needs >= 2 MDTs SKIP 401 (3s) == sanity-hsm test 402a: Copytool start fails if all MDTs are inactive ========================================================== 19:15:57 (1763338557) deactivate lustre-MDT0000-mdc-ffff8e7418516000 on oleg323-client.virtnet deactivate lustre-MDT0000-mdc-ffff8e7405fd7000 on oleg323-client.virtnet Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Failed to start copytool agt1 on 'oleg323-client.virtnet' pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 uuid not found in agent list on mds1 activate lustre-MDT0000-mdc-ffff8e7418516000 on oleg323-client.virtnet activate lustre-MDT0000-mdc-ffff8e7405fd7000 on oleg323-client.virtnet PASS 402a (6s) == sanity-hsm test 402b: CDT must retry request upon slow start of CT ========================================================== 19:16:03 (1763338563) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x14d fail_loc=0 Waiting 200s for 'SUCCEED' Updated after 6s: want 'SUCCEED' got 'SUCCEED' PASS 402b (23s) == sanity-hsm test 403: Copytool starts with inactive MDT and register on reconnect ========================================================== 19:16:26 (1763338586) SKIP: sanity-hsm test_403 needs >= 2 MDTs SKIP 403 (3s) == sanity-hsm test 404: Inactive MDT does not block requests for active MDTs ========================================================== 19:16:29 (1763338589) SKIP: sanity-hsm test_404 needs >= 2 MDTs SKIP 404 (3s) == sanity-hsm test 405: archive and release under striped directory ========================================================== 19:16:32 (1763338592) SKIP: sanity-hsm test_405 needs >= 2 MDTs SKIP 405 (2s) == sanity-hsm test 406: attempting to migrate HSM archived files is safe ========================================================== 19:16:34 (1763338594) SKIP: sanity-hsm test_406 needs >= 2 MDTs SKIP 406 (3s) == sanity-hsm test 407: Check for double RESTORE records in llog ========================================================== 19:16:37 (1763338597) 0+0 records in 0+0 records out 0 bytes copied, 0.00609986 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_407/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_val=5 fail_loc=0x164 Copytool is suspended on oleg323-client.virtnet mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/1] fid=[0x2000032e1:0x2:0x0] dfid=[0x2000032e1:0x2:0x0] compound/cookie=0x0/0x691a6923 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/2] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x691a6924 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] Failing mds1 on oleg323-server Stopping /mnt/lustre-mds1 (opts:) on oleg323-server 19:16:48 (1763338608) shut down facet: mds1 facet_host: oleg323-server facet_failover_host: oleg323-server Failover mds1 to oleg323-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg323-server: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Started lustre-MDT0000 19:17:09 (1763338629) targets are mounted 19:17:09 (1763338629) facet_failover done oleg323-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/1] fid=[0x2000032e1:0x2:0x0] dfid=[0x2000032e1:0x2:0x0] compound/cookie=0x0/0x691a6923 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/2] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x691a6924 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/3] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x691a6925 action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec fail_loc=0 mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/1] fid=[0x2000032e1:0x2:0x0] dfid=[0x2000032e1:0x2:0x0] compound/cookie=0x0/0x691a6923 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/2] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x691a6924 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/3] fid=[0x2000032e2:0x2:0x0] dfid=[0x2000032e2:0x2:0x0] compound/cookie=0x0/0x691a6925 action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] Copytool is continued on oleg323-client.virtnet d41d8cd98f00b204e9800998ecf8427e /mnt/lustre/d407.sanity-hsm/f407.sanity-hsm d41d8cd98f00b204e9800998ecf8427e /mnt/lustre2/d407.sanity-hsm/f407.sanity-hsm pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 PASS 407 (47s) == sanity-hsm test 408: Verify fiemap on release file ==== 19:17:24 (1763338644) Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_408/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.207533 s, 316 kB/s SKIP: sanity-hsm test_408 ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 408 (3s) == sanity-hsm test 409a: Coordinator should not stop when in use ========================================================== 19:17:28 (1763338648) 0+0 records in 0+0 records out 0 bytes copied, 0.0058837 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' fail_val=5 fail_loc=0x164 cat: /mnt/lustre/d409a.sanity-hsm/f409a.sanity-hsm: No data available mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm_control=enabled PASS 409a (23s) == sanity-hsm test 409b: getattr released file with CDT stopped after remount ========================================================== 19:17:51 (1763338671) 0+0 records in 0+0 records out 0 bytes copied, 0.00813261 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s Failing mds1 on oleg323-server Stopping /mnt/lustre-mds1 (opts:) on oleg323-server 19:18:19 (1763338699) shut down facet: mds1 facet_host: oleg323-server facet_failover_host: oleg323-server Failover mds1 to oleg323-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg323-server: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Started lustre-MDT0000 19:18:40 (1763338720) targets are mounted 19:18:40 (1763338720) facet_failover done oleg323-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec File: /mnt/lustre/d409b.sanity-hsm/f409b.sanity-hsm Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115406615871496 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-11-16 19:17:53.000000000 -0500 Modify: 2025-11-16 19:17:53.000000000 -0500 Change: 2025-11-16 19:17:53.000000000 -0500 Birth: 2025-11-16 19:17:52.000000000 -0500 PASS 409b (84s) == sanity-hsm test 410: lfs data_version -s allows release of force-archived file ========================================================== 19:19:15 (1763338755) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174247 s, 6.0 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_410/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d410.sanity-hsm/f410.sanity-hsm): Operation not permitted PASS 410 (6s) == sanity-hsm test 411: hsm_ops rbac role ================ 19:19:22 (1763338762) On MGS 192.168.203.123, active = nodemap.active=1 waiting 10 secs for sync On MGS 192.168.203.123, test_411.trusted_nodemap = nodemap.test_411.trusted_nodemap=1 waiting 10 secs for sync Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /mnt/lustre/d411.sanity-hsm/fileA: (0x00000000) Waiting 200s for 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg323-client.virtnet copytools stopped in 0s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' On MGS 192.168.203.123, test_411.rbac = nodemap.test_411.rbac=file_perms,dne_ops,quota_ops,byfid_ops,chlg_ops,fscrypt_admin,server_upcall,ignore_root_prjquota,local_admin,pool_quota_ops waiting 10 secs for sync /mnt/lustre/d411.sanity-hsm/fileB: (0x00000000) Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied lfs hsm_set: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied lfs hsm_clear: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied On MGS 192.168.203.123, active = nodemap.active=0 waiting 10 secs for sync oleg323-server: error: nodemap 'test_411' does not exist oleg323-server: nodemap_del: Invalid argument pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 On MGS 192.168.203.123, active = nodemap.active=0 waiting 10 secs for sync PASS 411 (89s) == sanity-hsm test 500: various LLAPI HSM tests ========== 19:20:50 (1763338850) test1 @1763338891: Register/unregister copytool 2000x to check for leaks ...... pass test2 @1763338891: Re/un-register copytool multiple times without error ....... pass test3 @1763338891: Pass bad parameters to llapi_hsm_copytool_register() ....... pass test4 @1763338891: Bad parameters to llapi_hsm_copytool_unregister() .......... pass test5 @1763338891: Test llapi_hsm_copytool_recv() in non-blocking mode ........ pass test6 @1763338891: Test llapi_hsm_copytool_recv() with bogus parameters ....... pass test7 @1763338891: Test event polling (without actual traffic) ................ pass test50 @1763338891: Test llapi_hsm_state_get()/get_fd() ....................... pass test51 @1763338892: Test llapi_hsm_state_set_fd() ............................. pass test52 @1763338892: Test llapi_hsm_current_action() ........................... pass test100 @1763338892: Simple archive creation with no progress reported. ....... pass test101 @1763338896: Simple archive creation with progress every byte. ........ pass test102 @1763338901: Archive creation with progress every byte, backwards ..... pass test103 @1763338903: Archive creation with a single progress report ........... pass test104 @1763338904: Archive creation with two progress reports ............... pass test105 @1763338905: Archive creation with one bogus progress report .......... pass test106 @1763338906: Archive creation with one empty progress report .......... pass test107 @1763338907: Archive creation with one bogus progress report .......... pass test108 @1763338912: Archive creation with same progress report each time ..... pass test109 @1763338913: Archive creation with one report, with large number ...... pass test110 @1763338914: Archive with 10 progress reports, checking progress ...... pass test111 @1763338915: Archive with 10 reports in reverse, checking progress .... pass test112 @1763338917: Archive with 10 reports, duplicated, check progress ...... pass test113 @1763338918: Archive with 9 reports, with 20% overlapping coverage .... pass PASS 500 (73s) == sanity-hsm test 600: Changelog fields 'u=' and 'nid=' ========================================================== 19:22:03 (1763338923) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl1' mdd.lustre-MDT0000.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d600.sanity-hsm/f600.sanity-hsm] Got NID '192.168.203.23@tcp' lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 600 (11s) == sanity-hsm test 601: OPEN Changelog entry ============= 19:22:14 (1763338934) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl2' mdd.lustre-MDT0000.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000.8 10OPEN 00:22:18.860966232 2025.11.17 0x1 t=[0x2000032e1:0x26:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.23@tcp m=r-- lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 601 (11s) == sanity-hsm test 602: Changelog record CLOSE only if open+write or OPEN recorded ========================================================== 19:22:25 (1763338945) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl3' mdd.lustre-MDT0000.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000.14 11CLOSE 00:22:29.526810441 2025.11.17 0x1 t=[0x2000032e1:0x28:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000.15 10OPEN 00:22:29.721655758 2025.11.17 0x242 t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.23@tcp m=-w- lustre-MDT0000.16 13TRUNC 00:22:29.726599704 2025.11.17 0xe t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp p=[0x2000032e1:0x27:0x0] lustre-MDT0000.17 11CLOSE 00:22:29.744722151 2025.11.17 0x242 t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000.17 11CLOSE 00:22:29.744722151 2025.11.17 0x242 t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp mdd.lustre-MDT0000.changelog_mask=-OPEN lustre-MDT0000: clear the changelog for cl3 of all records f lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000.18 13TRUNC 00:22:31.278682405 2025.11.17 0xe t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp p=[0x2000032e1:0x27:0x0] lustre-MDT0000.19 11CLOSE 00:22:31.297533129 2025.11.17 0x242 t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000.19 11CLOSE 00:22:31.297533129 2025.11.17 0x242 t=[0x2000032e1:0x28:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 602 (12s) == sanity-hsm test 603: GETXATTR Changelog entry ========= 19:22:37 (1763338957) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl4' mdd.lustre-MDT0000.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl4 of all records getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d603.sanity-hsm/f603.sanity-hsm user.xattr1="value1" lustre-MDT0000.25 23GXATR 00:22:41.315775063 2025.11.17 0x0 t=[0x2000032e1:0x2a:0x0] j=getfattr.0 ef=0x1b u=0:0 nid=192.168.203.23@tcp x=user.xattr1 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 603 (9s) == sanity-hsm test 604: NOPEN Changelog entry ============ 19:22:46 (1763338966) mdd.lustre-MDT0000.changelog_deniednext=20 mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl5' mdd.lustre-MDT0000.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.30 24NOPEN 00:22:52.112816602 2025.11.17 0x1 t=[0x2000032e1:0x2c:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.23@tcp m=r-- Got NID '192.168.203.23@tcp' lustre-MDT0000: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.31 24NOPEN 00:23:12.813551946 2025.11.17 0x1 t=[0x2000032e1:0x2c:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.23@tcp m=r-- p=[0x2000032e1:0x27:0x0] Got NID '192.168.203.23@tcp' lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 mdd.lustre-MDT0000.changelog_deniednext=60 PASS 604 (34s) == sanity-hsm test 605: Test OPEN and CLOSE rate limit in Changelogs ========================================================== 19:23:20 (1763339000) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl6' mdd.lustre-MDT0000.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000.35 10OPEN 00:23:24.758873227 2025.11.17 0x43 t=[0x2000032e1:0x2e:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.23@tcp m=rw- p=[0x2000032e1:0x2d:0x0] lustre-MDT0000.35 10OPEN 00:23:24.758873227 2025.11.17 0x43 t=[0x2000032e1:0x2e:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.23@tcp m=rw- p=[0x2000032e1:0x2d:0x0] lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000.36 10OPEN 00:23:25.552608458 2025.11.17 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.23@tcp m=r-- p=[0x2000032e1:0x2b:0x0] lustre-MDT0000.37 11CLOSE 00:23:25.580669716 2025.11.17 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000.36 10OPEN 00:23:25.552608458 2025.11.17 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.23@tcp m=r-- p=[0x2000032e1:0x2b:0x0] lustre-MDT0000.37 11CLOSE 00:23:25.580669716 2025.11.17 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre/d605.sanity-hsm/f605.sanity-hsm] lustre-MDT0000.38 10OPEN 00:23:26.762026472 2025.11.17 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.23@tcp m=r-- lustre-MDT0000.39 11CLOSE 00:23:26.770070632 2025.11.17 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.203.23@tcp lustre-MDT0000.38 10OPEN 00:23:26.762026472 2025.11.17 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.23@tcp m=r-- lustre-MDT0000.39 11CLOSE 00:23:26.770070632 2025.11.17 0x1 t=[0x2000032e1:0x2e:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000.40 11CLOSE 00:23:27.298318573 2025.11.17 0x43 t=[0x2000032e1:0x2e:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000.40 11CLOSE 00:23:27.298318573 2025.11.17 0x43 t=[0x2000032e1:0x2e:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.23@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 605 (14s) == sanity-hsm test 606: llog_reader groks changelog fields ========================================================== 19:23:34 (1763339014) mdd.lustre-MDT0000.changelog_mask=+hsm Registered 1 changelog users: 'cl7' mdd.lustre-MDT0000.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d606.sanity-hsm/f606.sanity-hsm] Stopping /mnt/lustre-mds1 (opts:) on oleg323-server rec #2 type=1064553b len=64 offset 8256 Header size : 8192 llh_size : 64 Time : Sun Nov 16 19:02:07 2025 Number of records: 1 cat_idx: 0 last_idx: 2 Flags: 0x2012 Target uuid : ----------------------- #02 (064)fid=[0x1:0x3b:0x0] path=oi.1/0x1:0x3b:0x0 oleg323-server: llog_reader: The llog is corrupted, 1 errors found Got UID/GID 500:500 Got NID 192.168.203.23@tcp Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg323-server: oleg323-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg323-client: oleg323-server: ssh exited with exit code 1 Started lustre-MDT0000 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 606 (35s) == sanity-hsm test 607a: release a file that was migrated after being archived ========================================================== 19:24:09 (1763339049) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162381 s, 6.5 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 607a (8s) == sanity-hsm test 607b: Migrate should not change the HSM attribute of dirty files ========================================================== 19:24:17 (1763339057) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148671 s, 7.1 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' llite.lustre-ffff8e7405fd7000.xattr_cache=0 llite.lustre-ffff8e7418516000.xattr_cache=0 /mnt/lustre/d607b.sanity-hsm/f607b.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 llite.lustre-ffff8e7405fd7000.xattr_cache=1 llite.lustre-ffff8e7418516000.xattr_cache=1 PASS 607b (10s) == sanity-hsm test 607c: 'lfs swap_layouts' should set dirty flag on HSM file ========================================================== 19:24:27 (1763339067) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187085 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196818 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230436 s, 4.6 MB/s Starting copytool 'agt1' on 'oleg323-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-1' and '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-3': Operation not permitted (1) PASS 607c (12s) == sanity-hsm test complete, duration 3742 sec =========== 19:24:39 (1763339079) === sanity-hsm: start cleanup 19:24:40 (1763339080) === Stopping clients: oleg323-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg323-client.virtnet /mnt/lustre2 opts: === sanity-hsm: finish cleanup 19:25:05 (1763339105) ===