-----============= acceptance-small: sanity-hsm ============----- Wed Sep 16 21:13:24 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg348-server: /home/green/git/lustre-release/lustre/tests/except/sanity-hsm.ex - see MDS1_VERSION <= CLIENT_VERSION (34683431 <= 34683431) for LU-18924, go 100 105 excepting tests: === sanity-hsm: start setup 21:13:41 (1789607621) === Starting client oleg348-client.virtnet: -o user_xattr,flock 192.168.203.148@tcp:/lustre /mnt/lustre2 Mount client oleg348-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.203.148@tcp:/lustre /mnt/lustre2 Started clients oleg348-client.virtnet: 192.168.203.148@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) oleg348-client.virtnet: executing check_config_client /mnt/lustre oleg348-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg348-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9d38c88ce800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9d38d8b9f000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9d38c88ce800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9d38d8b9f000.idle_timeout=debug disable quota as required oleg348-server: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity-hsm: finish setup 21:14:21 (1789607661) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f8459] nobody:x:65534: Killing existing copytools on oleg348-client.virtnet with signal TERM pkill: pidfile not valid Try `pkill --help' for more information. Set HSM on and start Set sanity-hsm HSM policy == sanity-hsm test 1A: lfs hsm flags root/non-root access ========================================================== 21:14:50 (1789607690) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_clear] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--exists] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] lfs hsm_set: change hsm flags for '/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm' failed: Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] PASS 1A (7s) == sanity-hsm test 1a: mmap & cat a HSM released file ==== 21:14:57 (1789607697) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.316733 s, 3.3 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Verifying released state: PASS 1a (11s) == sanity-hsm test 1b: Archive, Release and Restore composite file ========================================================== 21:15:08 (1789607708) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0680874 s, 15.4 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' release /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm verify released state: pass restore /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass PASS 1b (13s) == sanity-hsm test 1c: Check setting archive-id in lfs hsm_set ========================================================== 21:15:22 (1789607722) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is still 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 33 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 16 PASS 1c (8s) == sanity-hsm test 1d: Archive, Release and Restore DoM file ========================================================== 21:15:30 (1789607730) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0679644 s, 15.4 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' release /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm verify released state: pass restore /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass PASS 1d (15s) == sanity-hsm test 1e: Archive, Release and Restore SEL file ========================================================== 21:15:44 (1789607744) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597131 s, 17.6 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm release /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm verify released state: pass restore /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass PASS 1e (14s) == sanity-hsm test 1f: DoM file release after restore ==== 21:15:58 (1789607758) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071415 s, 14.7 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm release /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm verify released state: pass restore /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm verify restored state: pass release again /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 verify released state: pass PASS 1f (9s) == sanity-hsm test 2: Check file dirtyness when doing setattr ========================================================== 21:16:08 (1789607768) 0+0 records in 0+0 records out 0 bytes copied, 0.00454495 s, 0.0 kB/s 0x200000403:0x8:0x0 PASS 2 (7s) == sanity-hsm test 3: Check file dirtyness when opening for write ========================================================== 21:16:15 (1789607775) Succeed in opening file "/mnt/lustre/d3.sanity-hsm/f3.sanity-hsm"(flags=O_WRONLY) 3+0 records in 3+0 records out 3+0 records in 3+0 records out PASS 3 (9s) == sanity-hsm test 4: Useless cancel must not be registered ========================================================== 21:16:24 (1789607784) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239123 s, 4.4 MB/s PASS 4 (7s) == sanity-hsm test 8: Test default archive number ======== 21:16:31 (1789607791) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_8/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 8 (7s) == sanity-hsm test 9A: Use of explicit archive number, with dedicated copytool ========================================================== 21:16:38 (1789607798) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_9A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 3 "/mnt/lustre2"' oleg348-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg348-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=3 requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=3 requests=[current:0 ok:0 errors:0] on mds2 PASS 9A (30s) == sanity-hsm test 9a: Multiple remote agents ============ 21:17:08 (1789607828) SKIP: sanity-hsm test_9a Need 3 or more clients, have 1 SKIP 9a (4s) == sanity-hsm test 10a: Archive a file =================== 21:17:13 (1789607833) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 996: warning: command substitution: ignored null byte in input Verifying content Verifying hsm state Verifying archive number is 2 PASS 10a (9s) == sanity-hsm test 10b: Archive of non dirty file must work without doing request ========================================================== 21:17:23 (1789607843) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 10b (10s) == sanity-hsm test 10c: Check forbidden archive ========== 21:17:33 (1789607853) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d10c.sanity-hsm/f10c.sanity-hsm): Operation not permitted PASS 10c (7s) == sanity-hsm test 10d: Archive a file on the default archive id ========================================================== 21:17:40 (1789607860) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 10d (11s) == sanity-hsm test 11a: Import a file ==================== 21:17:51 (1789607871) lhsmtool_posix: 1789607873.380291 [22297]: action=1 src=d11a.sanity-hsm/f11a.sanity-hsm dst=/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789607873.387432 [22297]: importing '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1789607873.468470 [22297]: imported '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//0424/0x200000402:0x26:0x0'=='/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1789607873.479254 [22297]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying imported size 158=158 Verifying released pattern: raid0,released Verifying new fid 0x200000402:0x26:0x0 in archive PASS 11a (8s) == sanity-hsm test 11b: Import a deleted file using its FID ========================================================== 21:17:59 (1789607879) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_11b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lhsmtool_posix: 1789607885.333097 [22997]: action=1 src=0x200000403:0x11:0x0 dst=/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789607885.345214 [22997]: Resolving [0x200000403:0x11:0x0] to /tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0 lhsmtool_posix: 1789607885.345295 [22997]: importing '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0' lhsmtool_posix: 1789607885.388342 [22997]: imported '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//042a/0x200000402:0x28:0x0'=='/tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0' lhsmtool_posix: 1789607885.388403 [22997]: process finished, errs: 0 major, 0 minor, rc=0 (Success) /mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm: OK PASS 11b (12s) == sanity-hsm test 11c: Import a file to a directory with a pool ========================================================== 21:18:11 (1789607891) Creating new pool test_11c oleg348-server: Pool lustre.test_11c created Adding targets to pool oleg348-server: OST lustre-OST0001_UUID added to pool lustre.test_11c lhsmtool_posix: 1789607902.477955 [23713]: action=1 src=d11c.sanity-hsm/f11c.sanity-hsm dst=/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789607902.486705 [23713]: importing '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1789607902.559621 [23713]: imported '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//0404/0x240000402:0x6:0x0'=='/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1789607902.568299 [23713]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Destroy the created pools: test_11c lustre.test_11c oleg348-server: OST lustre-OST0001_UUID removed from pool lustre.test_11c oleg348-server: Pool lustre.test_11c destroyed PASS 11c (27s) == sanity-hsm test 12a: Restore an imported file explicitly ========================================================== 21:18:38 (1789607918) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1789607920.637132 [24452]: action=1 src=d12a.sanity-hsm/f12a.sanity-hsm dst=/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789607920.648431 [24452]: importing '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1789607920.708834 [24452]: imported '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//0429/0x200000402:0x2b:0x0'=='/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1789607920.708893 [24452]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying file state: PASS 12a (11s) == sanity-hsm test 12b: Restore an imported file implicitly ========================================================== 21:18:49 (1789607929) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1789607931.964475 [25166]: action=1 src=d12b.sanity-hsm/f12b.sanity-hsm dst=/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789607931.968895 [25166]: importing '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1789607932.072871 [25166]: imported '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//042c/0x200000402:0x2e:0x0'=='/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1789607932.072916 [25166]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying file state after restore: PASS 12b (10s) == sanity-hsm test 12c: Restore a file with stripe of 2 == 21:18:59 (1789607939) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.211846 s, 24.7 MB/s /mnt/lustre/d12c.sanity-hsm/f12c.sanity-hsm: OK PASS 12c (12s) == sanity-hsm test 12d: Restore of a non archived, non released file must work ========================================================== 21:19:11 (1789607951) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12d (11s) == sanity-hsm test 12e: Check forbidden restore ========== 21:19:22 (1789607962) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 Cannot send HSM request (use of /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm): Operation not permitted PASS 12e (9s) == sanity-hsm test 12f: Restore a released file explicitly ========================================================== 21:19:31 (1789607971) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Verifying file state: PASS 12f (11s) == sanity-hsm test 12g: Restore a released file implicitly ========================================================== 21:19:42 (1789607982) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12g (11s) == sanity-hsm test 12h: Restore a released file implicitly from a second node ========================================================== 21:19:53 (1789607993) SKIP: sanity-hsm test_12h Need 2 or more clients, have 1 SKIP 12h (4s) == sanity-hsm test 12m: Archive/release/implicit restore ========================================================== 21:19:57 (1789607997) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12m/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 12m (10s) == sanity-hsm test 12n: Import/implicit restore/release == 21:20:07 (1789608007) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12n/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1789608010.170631 [30001]: action=1 src=d12n.sanity-hsm/f12n.sanity-hsm dst=/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789608010.179511 [30001]: importing '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1789608010.284229 [30001]: imported '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//0408/0x240000402:0xa:0x0'=='/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1789608010.284312 [30001]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12n (9s) == sanity-hsm test 12o: Layout-swap failure during Restore leaves file released ========================================================== 21:20:16 (1789608016) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12o/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x152 mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA diff: /mnt/lustre/d12o.sanity-hsm/f12o.sanity-hsm: No data available mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA fail_loc=0 mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 12o (35s) == sanity-hsm test 12p: implicit restore of a file on copytool mount point ========================================================== 21:20:52 (1789608052) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12p/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12p (12s) == sanity-hsm test 12q: file attributes are refreshed after restore ========================================================== 21:21:04 (1789608064) Starting client: oleg348-client.virtnet: -o user_xattr,flock 192.168.203.148@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176071 s, 6.0 MB/s Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Stopping client oleg348-client.virtnet /mnt/lustre3 (opts:) PASS 12q (19s) == sanity-hsm test 12r: lseek restores released file ===== 21:21:23 (1789608083) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12r/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 12r (11s) == sanity-hsm test 12s: race between restore requests ==== 21:21:34 (1789608094) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12s/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x8000018b PASS 12s (16s) == sanity-hsm test 12t: Multiple parallel reads for a HSM imported file ========================================================== 21:21:50 (1789608110) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12t/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1789608113.440037 [34144]: action=1 src=0x200000403:0x2a:0x0 dst=/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789608113.452656 [34144]: Resolving [0x200000403:0x2a:0x0] to /tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0 lhsmtool_posix: 1789608113.456680 [34144]: importing '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0' lhsmtool_posix: 1789608113.517099 [34144]: imported '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//0445/0x200000402:0x47:0x0'=='/tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0' lhsmtool_posix: 1789608113.517175 [34144]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12t (9s) == sanity-hsm test 12u: Multiple reads on multiple HSM imported files in parallel ========================================================== 21:21:59 (1789608119) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12u/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1789608134.497693 [35017]: action=1 src=0x200000403:0x2c:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0 mount_point=/mnt/lustre2 lhsmtool_posix: 1789608134.502717 [35017]: Resolving [0x200000403:0x2c:0x0] to /tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0 lhsmtool_posix: 1789608134.503243 [35017]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0' lhsmtool_posix: 1789608134.540603 [35017]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//044b/0x200000402:0x49:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0' lhsmtool_posix: 1789608134.541220 [35017]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1789608134.941292 [35046]: action=1 src=0x200000403:0x2d:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1 mount_point=/mnt/lustre2 lhsmtool_posix: 1789608134.958528 [35046]: Resolving [0x200000403:0x2d:0x0] to /tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0 lhsmtool_posix: 1789608134.958851 [35046]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0' lhsmtool_posix: 1789608135.026653 [35046]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//0448/0x200000402:0x4a:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0' lhsmtool_posix: 1789608135.026738 [35046]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1789608135.377378 [35075]: action=1 src=0x200000403:0x2e:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2 mount_point=/mnt/lustre2 lhsmtool_posix: 1789608135.387373 [35075]: Resolving [0x200000403:0x2e:0x0] to /tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0 lhsmtool_posix: 1789608135.387454 [35075]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0' lhsmtool_posix: 1789608135.439315 [35075]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//0449/0x200000402:0x4b:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0' lhsmtool_posix: 1789608135.440106 [35075]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1789608135.755515 [35105]: action=1 src=0x200000403:0x2f:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3 mount_point=/mnt/lustre2 lhsmtool_posix: 1789608135.766052 [35105]: Resolving [0x200000403:0x2f:0x0] to /tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0 lhsmtool_posix: 1789608135.767242 [35105]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0' lhsmtool_posix: 1789608135.806511 [35105]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//044e/0x200000402:0x4c:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0' lhsmtool_posix: 1789608135.806584 [35105]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1789608136.188845 [35134]: action=1 src=0x200000403:0x30:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4 mount_point=/mnt/lustre2 lhsmtool_posix: 1789608136.194668 [35134]: Resolving [0x200000403:0x30:0x0] to /tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0 lhsmtool_posix: 1789608136.194844 [35134]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0' lhsmtool_posix: 1789608136.232534 [35134]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//044f/0x200000402:0x4d:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0' lhsmtool_posix: 1789608136.233279 [35134]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1789608136.548151 [35163]: action=1 src=0x200000403:0x31:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5 mount_point=/mnt/lustre2 lhsmtool_posix: 1789608136.554358 [35163]: Resolving [0x200000403:0x31:0x0] to /tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0 lhsmtool_posix: 1789608136.558737 [35163]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0' lhsmtool_posix: 1789608136.606643 [35163]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//044c/0x200000402:0x4e:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0' lhsmtool_posix: 1789608136.610704 [35163]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1789608136.903288 [35192]: action=1 src=0x200000403:0x32:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6 mount_point=/mnt/lustre2 lhsmtool_posix: 1789608136.909897 [35192]: Resolving [0x200000403:0x32:0x0] to /tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0 lhsmtool_posix: 1789608136.909985 [35192]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0' lhsmtool_posix: 1789608136.969953 [35192]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//044d/0x200000402:0x4f:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0' lhsmtool_posix: 1789608136.970001 [35192]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1789608137.249568 [35221]: action=1 src=0x200000403:0x33:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7 mount_point=/mnt/lustre2 lhsmtool_posix: 1789608137.259095 [35221]: Resolving [0x200000403:0x33:0x0] to /tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0 lhsmtool_posix: 1789608137.259173 [35221]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0' lhsmtool_posix: 1789608137.310739 [35221]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0452/0x200000402:0x50:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0' lhsmtool_posix: 1789608137.310848 [35221]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1789608137.544061 [35250]: action=1 src=0x200000403:0x34:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8 mount_point=/mnt/lustre2 lhsmtool_posix: 1789608137.553499 [35250]: Resolving [0x200000403:0x34:0x0] to /tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0 lhsmtool_posix: 1789608137.553591 [35250]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0' lhsmtool_posix: 1789608137.584099 [35250]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0453/0x200000402:0x51:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0' lhsmtool_posix: 1789608137.584161 [35250]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1789608137.844302 [35279]: action=1 src=0x200000403:0x35:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9 mount_point=/mnt/lustre2 lhsmtool_posix: 1789608137.851543 [35279]: Resolving [0x200000403:0x35:0x0] to /tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0 lhsmtool_posix: 1789608137.851629 [35279]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0' lhsmtool_posix: 1789608137.903907 [35279]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0450/0x200000402:0x52:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0' lhsmtool_posix: 1789608137.903965 [35279]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12u (27s) == sanity-hsm test 13: Recursively import and restore a directory ========================================================== 21:22:26 (1789608146) lhsmtool_posix: 1789608155.918313 [37031]: action=1 src=subdir dst=/mnt/lustre/d13.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789608156.036246 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1789608156.167414 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//040f/0x240000402:0xd:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1789608156.169072 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1789608156.263591 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//040c/0x240000402:0xe:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1789608156.264750 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1789608156.375562 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//040d/0x240000402:0xf:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1789608156.382944 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1789608156.453182 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//0412/0x240000402:0x10:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1789608156.453419 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1789608156.496547 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//0413/0x240000402:0x11:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1789608156.496878 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1789608156.549097 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//0410/0x240000402:0x12:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1789608156.549348 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1789608156.591331 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//0411/0x240000402:0x13:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1789608156.592359 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1789608156.628962 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//0416/0x240000402:0x14:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1789608156.629212 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1789608156.680873 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//0417/0x240000402:0x15:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1789608156.681344 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1789608156.725322 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//0414/0x240000402:0x16:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1789608156.743660 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1789608156.904160 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//041a/0x240000402:0x18:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1789608156.904400 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1789608156.999756 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//041b/0x240000402:0x19:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1789608157.000797 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1789608157.073227 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//0418/0x240000402:0x1a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1789608157.074187 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1789608157.130477 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//0419/0x240000402:0x1b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1789608157.130725 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1789608157.183067 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//041e/0x240000402:0x1c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1789608157.195806 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1789608157.242168 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//041f/0x240000402:0x1d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1789608157.242425 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1789608157.294021 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//041c/0x240000402:0x1e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1789608157.294261 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1789608157.346034 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//041d/0x240000402:0x1f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1789608157.346283 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1789608157.394102 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//0422/0x240000402:0x20:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1789608157.396647 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1789608157.460146 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//0423/0x240000402:0x21:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1789608157.480103 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1789608157.552089 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//0454/0x200000402:0x56:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1789608157.558960 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1789608157.623143 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//0455/0x200000402:0x57:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1789608157.623391 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1789608157.688103 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//045a/0x200000402:0x58:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1789608157.689952 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1789608157.748953 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//045b/0x200000402:0x59:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1789608157.749222 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1789608157.813081 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//0458/0x200000402:0x5a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1789608157.815277 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1789608157.889373 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//0459/0x200000402:0x5b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1789608157.889650 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1789608157.945453 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//045e/0x200000402:0x5c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1789608157.946969 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1789608158.040516 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//045f/0x200000402:0x5d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1789608158.042265 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1789608158.140101 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//045c/0x200000402:0x5e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1789608158.140371 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1789608158.196182 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//045d/0x200000402:0x5f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1789608158.226213 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1789608158.344547 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//0421/0x240000402:0x23:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1789608158.345218 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1789608158.387207 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//0426/0x240000402:0x24:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1789608158.391124 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1789608158.437302 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//0427/0x240000402:0x25:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1789608158.437555 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1789608158.500629 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//0424/0x240000402:0x26:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1789608158.501130 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1789608158.542143 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//0425/0x240000402:0x27:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1789608158.543127 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1789608158.575207 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//042a/0x240000402:0x28:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1789608158.575469 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1789608158.622068 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//042b/0x240000402:0x29:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1789608158.622316 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1789608158.678230 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//0428/0x240000402:0x2a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1789608158.678474 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1789608158.717393 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//0429/0x240000402:0x2b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1789608158.718016 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1789608158.803681 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//042e/0x240000402:0x2c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1789608158.825987 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1789608158.897249 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//0463/0x200000402:0x61:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1789608158.897506 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1789608158.975529 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//0460/0x200000402:0x62:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1789608158.975870 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1789608159.029979 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//0461/0x200000402:0x63:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1789608159.030240 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1789608159.077086 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//0466/0x200000402:0x64:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1789608159.077346 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1789608159.152664 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//0467/0x200000402:0x65:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1789608159.156682 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1789608159.220727 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//0464/0x200000402:0x66:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1789608159.221390 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1789608159.275236 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//0465/0x200000402:0x67:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1789608159.276071 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1789608159.336001 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//046a/0x200000402:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1789608159.336412 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1789608159.423373 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//046b/0x200000402:0x69:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1789608159.424446 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1789608159.476194 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//0468/0x200000402:0x6a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1789608159.492217 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1789608159.635341 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//042c/0x240000402:0x2e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1789608159.636738 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1789608159.674959 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//042d/0x240000402:0x2f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1789608159.681477 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1789608159.743357 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//0432/0x240000402:0x30:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1789608159.744300 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1789608159.796189 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//0433/0x240000402:0x31:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1789608159.796550 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1789608159.834292 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//0430/0x240000402:0x32:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1789608159.835403 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1789608159.901190 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//0431/0x240000402:0x33:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1789608159.901402 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1789608159.937235 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//0436/0x240000402:0x34:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1789608159.938226 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1789608159.987695 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//0437/0x240000402:0x35:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1789608159.988162 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1789608160.031472 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//0434/0x240000402:0x36:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1789608160.034159 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1789608160.092598 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//0435/0x240000402:0x37:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1789608160.108001 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1789608160.255396 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//043b/0x240000402:0x39:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1789608160.257318 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1789608160.312928 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//0438/0x240000402:0x3a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1789608160.313162 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1789608160.349484 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//0439/0x240000402:0x3b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1789608160.351209 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1789608160.389388 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//043e/0x240000402:0x3c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1789608160.390356 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1789608160.425389 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//043f/0x240000402:0x3d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1789608160.426294 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1789608160.460700 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//043c/0x240000402:0x3e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1789608160.461479 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1789608160.502128 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//043d/0x240000402:0x3f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1789608160.509223 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1789608160.568948 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//0442/0x240000402:0x40:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1789608160.569331 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1789608160.633368 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//0443/0x240000402:0x41:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1789608160.637401 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1789608160.694130 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//0440/0x240000402:0x42:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1789608160.719205 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1789608160.840073 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//0446/0x240000402:0x44:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1789608160.840328 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1789608160.885202 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//0447/0x240000402:0x45:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1789608160.885541 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1789608160.936056 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//0444/0x240000402:0x46:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1789608160.936742 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1789608161.006660 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//0445/0x240000402:0x47:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1789608161.007246 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1789608161.077327 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//044a/0x240000402:0x48:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1789608161.077926 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1789608161.131309 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//044b/0x240000402:0x49:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1789608161.132441 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1789608161.186633 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//0448/0x240000402:0x4a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1789608161.186941 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1789608161.221212 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//0449/0x240000402:0x4b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1789608161.221898 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1789608161.277281 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//044e/0x240000402:0x4c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1789608161.277540 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1789608161.334835 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//044f/0x240000402:0x4d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1789608161.347711 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1789608161.448700 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//044d/0x240000402:0x4f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1789608161.449011 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1789608161.488165 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//0452/0x240000402:0x50:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1789608161.488943 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1789608161.563404 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//0453/0x240000402:0x51:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1789608161.563658 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1789608161.642100 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//0450/0x240000402:0x52:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1789608161.642738 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1789608161.734938 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//0451/0x240000402:0x53:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1789608161.736096 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1789608161.783145 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//0456/0x240000402:0x54:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1789608161.783639 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1789608161.807586 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//0457/0x240000402:0x55:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1789608161.808093 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1789608161.859300 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//0454/0x240000402:0x56:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1789608161.859923 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1789608161.915148 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//0455/0x240000402:0x57:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1789608161.915537 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1789608161.954362 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//045a/0x240000402:0x58:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1789608161.960013 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1789608161.991970 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//046e/0x200000402:0x6c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1789608161.992200 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1789608162.040621 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//046f/0x200000402:0x6d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1789608162.041199 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1789608162.082336 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//046c/0x200000402:0x6e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1789608162.082603 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1789608162.152372 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//046d/0x200000402:0x6f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1789608162.153470 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1789608162.234972 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//0472/0x200000402:0x70:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1789608162.235994 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1789608162.277374 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//0473/0x200000402:0x71:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1789608162.278339 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1789608162.320283 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//0470/0x200000402:0x72:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1789608162.321245 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1789608162.367302 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//0471/0x200000402:0x73:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1789608162.368404 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1789608162.419216 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//0476/0x200000402:0x74:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1789608162.419436 [37031]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1789608162.464317 [37031]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//0477/0x200000402:0x75:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1789608162.464979 [37031]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_13/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 13 (48s) == sanity-hsm test 14: Rebind archived file to a new fid ========================================================== 21:23:14 (1789608194) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_14/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.42257 s, 2.5 MB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00818794 s, 0.0 kB/s rebind 0x200000403:0x5f:0x0 to 0x200000403:0x60:0x0 lhsmtool_posix: 1789608200.716193 [37998]: action=2 src=0x200000403:0x5f:0x0 dst=0x200000403:0x60:0x0 mount_point=/mnt/lustre2 lhsmtool_posix: 1789608200.717217 [37998]: rebind [0x200000403:0x5f:0x0] to [0x200000403:0x60:0x0] lhsmtool_posix: 1789608200.717358 [37998]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 14 (14s) == sanity-hsm test 15: Rebind a list of files ============ 21:23:28 (1789608208) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_15/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.440648 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319511 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204043 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145699 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318407 s, 3.3 MB/s pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 0+0 records in 0+0 records out 0 bytes copied, 0.0169813 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00810837 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.012282 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.0155713 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00630167 s, 0.0 kB/s rebind list of files lhsmtool_posix: 1789608216.878265 [38785]: action=2 src=/tmp/tmp.8459 dst=(null) mount_point=/mnt/lustre2 lhsmtool_posix: 1789608216.879503 [38785]: rebind [0x200000403:0x62:0x0] to [0x200000403:0x67:0x0] lhsmtool_posix: 1789608216.879634 [38785]: rebind [0x200000403:0x63:0x0] to [0x200000403:0x68:0x0] lhsmtool_posix: 1789608216.879685 [38785]: rebind [0x200000403:0x64:0x0] to [0x200000403:0x69:0x0] lhsmtool_posix: 1789608216.879738 [38785]: rebind [0x200000403:0x65:0x0] to [0x200000403:0x6a:0x0] lhsmtool_posix: 1789608216.879822 [38785]: rebind [0x200000403:0x66:0x0] to [0x200000403:0x6b:0x0] lhsmtool_posix: 1789608216.880093 [38785]: 5 lines read from '/tmp/tmp.8459', 5 rebind successful lhsmtool_posix: 1789608216.880110 [38785]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 15 (22s) == sanity-hsm test 16: Test CT bandwith control option === 21:23:51 (1789608231) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_16/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 0.105936 s, 198 MB/s Waiting 200s for 'SUCCEED' Waiting 190s for 'SUCCEED' Updated after 21s: want 'SUCCEED' got 'SUCCEED' PASS 16 (31s) == sanity-hsm test 20: Release is not permitted ========== 21:24:21 (1789608261) 0+0 records in 0+0 records out 0 bytes copied, 0.00790255 s, 0.0 kB/s 0x240000403:0x4b:0x0 Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted PASS 20 (9s) == sanity-hsm test 21: Simple release tests ============== 21:24:30 (1789608270) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_21/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202941 s, 5.2 MB/s 1+0 records in 1+0 records out 512 bytes copied, 0.102564 s, 5.0 kB/s PASS 21 (13s) == sanity-hsm test 22: Could not swap a release file ===== 21:24:43 (1789608283) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_22/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.30828 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234214 s, 4.5 MB/s 0x200000403:0x75:0x0 lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d22.sanity-hsm/test_swap' and '/mnt/lustre/d22.sanity-hsm/test_release': Operation not supported (95) PASS 22 (10s) == sanity-hsm test 23: Release does not change a/mtime (utime) ========================================================== 21:24:53 (1789608293) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_23/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.301854 s, 3.5 MB/s PASS 23 (9s) == sanity-hsm test 24a: Archive, release, and restore does not change a/mtime (i/o) ========================================================== 21:25:02 (1789608302) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.312018 s, 3.4 MB/s Waiting 200s for 'SUCCEED' Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 0s 192.168.203.148@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg348-client.virtnet /mnt/lustre (opts:) Starting client: oleg348-client.virtnet: -o user_xattr,flock 192.168.203.148@tcp:/lustre /mnt/lustre PASS 24a (18s) == sanity-hsm test 24b: root can archive, release, and restore user files ========================================================== 21:25:20 (1789608320) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172124 s, 6.1 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] /mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm: (0x00000009) exists archived, archive_id:1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [md5sum] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] PASS 24b (17s) == sanity-hsm test 24c: check that user,group,other request masks work ========================================================== 21:25:38 (1789608338) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.41448 s, 2.5 MB/s 0x200000403:0x7d:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231792 s, 4.5 MB/s 0x200000403:0x7e:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177607 s, 5.9 MB/s 0x200000403:0x7f:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] PASS 24c (29s) == sanity-hsm test 24d: check that read-only mounts are respected ========================================================== 21:26:06 (1789608366) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237623 s, 4.4 MB/s 0x200000403:0x80:0x0 /mnt/lustre/d24d.sanity-hsm/f24d.sanity-hsm lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 70 0x46 0x2c0000401 Starting client: oleg348-client.virtnet: -o user_xattr,flock 192.168.203.148@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 1.29856 s, 807 kB/s Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Stopping client oleg348-client.virtnet /mnt/lustre3 (opts:) PASS 24d (17s) == sanity-hsm test 24e: tar succeeds on HSM released files ========================================================== 21:26:23 (1789608383) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201319 s, 5.2 MB/s /mnt/lustre/d24e.sanity-hsm/f24e.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 tar: Removing leading `/' from member names PASS 24e (10s) == sanity-hsm test 24f: root can archive, release, and restore tar files ========================================================== 21:26:33 (1789608393) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm f24f.sanity-hsm f24f.sanity-hsm Sum0 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm, sum1 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm PASS 24f (10s) == sanity-hsm test 24g: write by non-owner still sets dirty ========================================================== 21:26:43 (1789608403) RUNAS = 'runas -u 500 -g 500' Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Please listen carefully as our options have changed. running as uid/gid/euid/egid 500/500/500/500, groups: 500 [tee] [/mnt/lustre/d24g.sanity-hsm/f24g.sanity-hsm] To be electrocuted by your telephone, press #. PASS 24g (9s) == sanity-hsm test 25a: Restore lost file (HS_LOST flag) from import ========================================================== 21:26:52 (1789608412) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1789608415.046743 [47784]: action=1 src=d25a.sanity-hsm/f25a.sanity-hsm dst=/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789608415.052508 [47784]: importing '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1789608415.088028 [47784]: imported '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//0407/0x240000404:0x3:0x0'=='/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1789608415.088098 [47784]: process finished, errs: 0 major, 0 minor, rc=0 (Success) md5sum: /mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm: No data available PASS 25a (8s) == sanity-hsm test 25b: Restore lost file (HS_LOST flag) after release ========================================================== 21:27:00 (1789608420) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' md5sum: /mnt/lustre/d25b.sanity-hsm/f25b.sanity-hsm: No data available PASS 25b (8s) == sanity-hsm test 26A: Remove the archive of a valid file ========================================================== 21:27:09 (1789608429) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00734821 s, 0.0 kB/s PASS 26A (10s) == sanity-hsm test 26a: Remove Archive On Last Unlink (RAoLU) policy ========================================================== 21:27:20 (1789608440) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Changed after 6s: from 'WAITING' to 'STARTED' Updated after 8s: want 'SUCCEED' got 'SUCCEED' PASS 26a (36s) == sanity-hsm test 26b: RAoLU policy when CDT off ======== 21:27:56 (1789608476) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 26b (25s) == sanity-hsm test 26c: RAoLU effective when file closed ========================================================== 21:28:21 (1789608501) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' multiop /mnt/lustre/d26c.sanity-hsm/f26c.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8459 Waiting 200s for 'SUCCEED' Updated after 6s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 8s: want 'SUCCEED' got 'SUCCEED' PASS 26c (39s) == sanity-hsm test 26d: RAoLU when Client eviction ======= 21:29:00 (1789608540) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220548 s, 4.8 MB/s multiop /mnt/lustre/d26d.sanity-hsm/f26d.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8459 Waiting 200s for 'SUCCEED' Updated after 4s: want 'SUCCEED' got 'SUCCEED' PASS 26d (32s) == sanity-hsm test 26e: RAoLU with a non-started coordinator ========================================================== 21:29:32 (1789608572) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232618 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.286389 s, 3.7 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 0s Failing mds1 on oleg348-server Stopping /mnt/lustre-mds1 (opts:) on oleg348-server 21:30:07 (1789608607) shut down facet: mds1 facet_host: oleg348-server facet_failover_host: oleg348-server Failover mds1 to oleg348-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg348-server: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 Started lustre-MDT0000 21:30:38 (1789608638) targets are mounted 21:30:38 (1789608638) facet_failover done oleg348-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Check cookie from RAoLU request (last: 1789607809, remove: 1789607810) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 26e (123s) == sanity-hsm test 27a: Remove the archive of an imported file (Operation not permitted) ========================================================== 21:31:35 (1789608695) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.36602 s, 29.9 MB/s lhsmtool_posix: 1789608699.853015 [54872]: action=1 src=d27a.sanity-hsm/f27a.sanity-hsm dst=/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789608699.858763 [54872]: importing '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1789608700.066320 [54872]: imported '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//0401/0x240000404:0x5:0x0'=='/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1789608700.066657 [54872]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Cannot send HSM request (use of /mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm): Operation not permitted PASS 27a (10s) == sanity-hsm test 27b: Remove the archive of a relased file (Operation not permitted) ========================================================== 21:31:45 (1789608705) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00496335 s, 0.0 kB/s Cannot send HSM request (use of /mnt/lustre/d27b.sanity-hsm/f27b.sanity-hsm): Operation not permitted PASS 27b (9s) == sanity-hsm test 28: Concurrent archive/file remove ==== 21:31:55 (1789608715) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_28/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00601102 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 28 (18s) == sanity-hsm test 29a: Tests --mntpath and --archive options ========================================================== 21:32:13 (1789608733) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 7 "/mnt/lustre"' Cannot send HSM request (use of /mnt/lustre): Invalid argument hsm: '/qwerty/uyt' is not a valid FID PASS 29a (7s) == sanity-hsm test 29b: Archive/delete/remove by FID from the archive. ========================================================== 21:32:21 (1789608741) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324346 s, 3.2 MB/s PASS 29b (11s) == sanity-hsm test 29c: Archive/delete/remove by FID, using a file list. ========================================================== 21:32:31 (1789608751) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223644 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201991 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176439 s, 5.9 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 29c (17s) == sanity-hsm test 29d: hsm_remove by FID with archive_id 0 for unlinked file cause ========================================================== 21:32:48 (1789608768) SKIP: sanity-hsm test_29d Need 3 or more clients, have 1 SKIP 29d (4s) == sanity-hsm test 30a: Restore at exec (import case) ==== 21:32:52 (1789608772) SKIP: sanity-hsm test_30a Need 2 or more clients, have 1 SKIP 30a (3s) == sanity-hsm test 30b: Restore at exec (release case) === 21:32:55 (1789608775) SKIP: sanity-hsm test_30b Need 2 or more clients, have 1 SKIP 30b (3s) == sanity-hsm test 30c: Update during exec of released file must fail ========================================================== 21:32:59 (1789608779) SKIP: sanity-hsm test_30c Need 2 or more clients, have 1 SKIP 30c (3s) == sanity-hsm test 31a: Import a large file and check size during restore ========================================================== 21:33:02 (1789608782) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.17568 s, 34.8 MB/s lhsmtool_posix: 1789608786.472556 [60031]: action=1 src=d31a.sanity-hsm/f31a.sanity-hsm dst=/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789608786.482667 [60031]: importing '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1789608786.569382 [60031]: imported '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//040c/0x200000407:0xb:0x0'=='/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1789608786.573431 [60031]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' size seen is right: 40894464 == 40894464 done PASS 31a (20s) == sanity-hsm test 31b: Restore a large unaligned file and check size during restore ========================================================== 21:33:22 (1789608802) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.847395 s, 46.0 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' size seen is right: 39000000 == 39000000 done PASS 31b (21s) == sanity-hsm test 31c: Restore a large aligned file and check size during restore ========================================================== 21:33:43 (1789608823) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.07827 s, 37.9 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' size seen is right: 40894464 == 40894464 done PASS 31c (23s) == sanity-hsm test 33: Kill a restore waiting process ==== 21:34:07 (1789608847) 0+0 records in 0+0 records out 0 bytes copied, 0.0118456 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_33/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2893: 62062 Terminated md5sum $f > /dev/null PASS 33 (12s) == sanity-hsm test 34: Remove file during restore ======== 21:34:19 (1789608859) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_34/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00375555 s, 0.0 kB/s Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2925: kill: (62723) - No such process PASS 34 (12s) == sanity-hsm test 35: Overwrite file during restore ===== 21:34:32 (1789608872) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_35/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00431351 s, 0.0 kB/s Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2962: kill: (63400) - No such process PASS 35 (11s) == sanity-hsm test 36: Move file during restore ========== 21:34:42 (1789608882) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_36/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00461622 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2998: kill: (64084) - No such process PASS 36 (13s) == sanity-hsm test 37: re-archive a dirty file =========== 21:34:55 (1789608895) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_37/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221178 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571128 s, 18.4 MB/s Waiting 200s for 'SUCCEED' Updated after 5s: want 'SUCCEED' got 'SUCCEED' PASS 37 (78s) == sanity-hsm test 40: Parallel archive requests ========= 21:36:13 (1789608973) oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1152921504606846976: Cannot allocate memory oleg348-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1152921504606846976': Cannot allocate memory pdsh@oleg348-client: oleg348-server: ssh exited with exit code 12 oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=1152921504606846976: Cannot allocate memory oleg348-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='1152921504606846976': Cannot allocate memory pdsh@oleg348-client: oleg348-server: ssh exited with exit code 12 set max_requests=1152921504606846976 failed with 12 [ 1651.977865] Lustre: lustre-MDT0000: No more memory to set HSM max_requests=1152921504606846976 (max request memory: 461MB, current total 6/975940): rc = -12 [ 1653.233673] Lustre: lustre-MDT0001: No more memory to set HSM max_requests=1152921504606846976 (max request memory: 461MB, current total 975940/975940): rc = -12 Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_40/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Wait for all requests being enqueued...100 archive requests submitted 100 archive requests submitted 100 archive requests submitted 100 archive requests submitted OK pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 PASS 40 (171s) == sanity-hsm test 50: Archive with large number of pending HSM actions ========================================================== 21:39:04 (1789609144) oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1000000: Cannot allocate memory oleg348-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1000000': Cannot allocate memory pdsh@oleg348-client: oleg348-server: ssh exited with exit code 12 oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=1000000: Cannot allocate memory oleg348-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='1000000': Cannot allocate memory pdsh@oleg348-client: oleg348-server: ssh exited with exit code 12 Filesystem Inodes IUsed IFree IUse% Mounted on 192.168.203.148@tcp:/lustre 236427 1323 235104 1% /mnt/lustre - create 1299 (time 1789609161.25 total 10.01 last 129.82) - create 2546 (time 1789609171.25 total 20.01 last 124.66) - create 3784 (time 1789609181.25 total 30.01 last 123.78) - create 5178 (time 1789609191.26 total 40.01 last 139.38) - create 6491 (time 1789609201.26 total 50.01 last 131.29) - create 7814 (time 1789609211.26 total 60.02 last 132.25) - create 9176 (time 1789609221.26 total 70.02 last 136.17) total: 10000 create in 76.12 seconds: 131.38 ops/second 10000 - unlinked 0 (time 1789609320 ; total 0 ; last 0) total: 10000 unlinks in 82 seconds: 121.951218 unlinks/second Start Phase files_num: 10000 time: 87 - create 1328 (time 1789609417.43 total 10.00 last 132.75) - create 2742 (time 1789609427.43 total 20.01 last 141.38) - create 3985 (time 1789609437.43 total 30.01 last 124.26) - create 5292 (time 1789609447.44 total 40.02 last 130.56) - create 6499 (time 1789609457.47 total 50.05 last 120.38) - create 7779 (time 1789609467.48 total 60.05 last 127.90) - create 9154 (time 1789609477.48 total 70.06 last 137.46) - create 10000 (time 1789609484.69 total 77.27 last 117.31) - create 11315 (time 1789609494.70 total 87.27 last 131.46) - create 12552 (time 1789609504.70 total 97.27 last 123.68) - create 13805 (time 1789609514.70 total 107.28 last 125.26) - create 15033 (time 1789609524.70 total 117.28 last 122.76) - create 16277 (time 1789609534.71 total 127.29 last 124.32) - create 17621 (time 1789609544.72 total 137.29 last 134.32) - create 18769 (time 1789609554.72 total 147.30 last 114.73) - create 19894 (time 1789609564.73 total 157.31 last 112.42) total: 20000 create in 158.36 seconds: 126.29 ops/second - unlinked 0 (time 1789609746 ; total 0 ; last 0) - unlinked 10000 (time 1789609830 ; total 84 ; last 84) total: 20000 unlinks in 170 seconds: 117.647057 unlinks/second Middle Phase files_num: 20000 time: 176 - create 1482 (time 1789609931.88 total 10.00 last 148.19) - create 2802 (time 1789609941.88 total 20.00 last 131.97) - create 4177 (time 1789609951.89 total 30.01 last 137.42) - create 5584 (time 1789609961.89 total 40.01 last 140.64) - create 6784 (time 1789609971.90 total 50.02 last 119.89) - create 7917 (time 1789609981.91 total 60.02 last 113.26) - create 9202 (time 1789609991.91 total 70.03 last 128.44) total: 10000 create in 76.33 seconds: 131.01 ops/second 40000 - unlinked 0 (time 1789610100 ; total 1 ; last 1) total: 10000 unlinks in 91 seconds: 109.890106 unlinks/second End Phase files_num: 10000 time: 94 40000 mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 50 (1063s) == sanity-hsm test 52: Opened for write file on an evicted client should be set dirty ========================================================== 21:56:47 (1789610207) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_52/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230916 s, 4.5 MB/s multiop /mnt/lustre/d52.sanity-hsm/f52.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8459 PASS 52 (15s) == sanity-hsm test 53: Opened for read file on an evicted client should not be set dirty ========================================================== 21:57:02 (1789610222) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_53/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231759 s, 4.5 MB/s multiop /mnt/lustre/d53.sanity-hsm/f53.sanity-hsm vo_c TMPPIPE=/tmp/multiop_open_wait_pipe.8459 PASS 53 (15s) == sanity-hsm test 54: Write during an archive cancels it ========================================================== 21:57:18 (1789610238) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 1.09536 s, 35.6 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_54/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Waiting 200s for 'FAILED' Updated after 33s: want 'FAILED' got 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 54 (48s) == sanity-hsm test 55: Truncate during an archive cancels it ========================================================== 21:58:07 (1789610287) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 1.17548 s, 33.2 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_55/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 55 (16s) == sanity-hsm test 56: Setattr during an archive is ok === 21:58:22 (1789610302) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 1.24055 s, 31.4 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_56/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Waiting 190s for 'SUCCEED' Waiting 180s for 'SUCCEED' Updated after 36s: want 'SUCCEED' got 'SUCCEED' PASS 56 (49s) == sanity-hsm test 57: Archive a file with dirty cache on another node ========================================================== 21:59:12 (1789610352) SKIP: sanity-hsm test_57 Need 2 or more clients, have 1 SKIP 57 (3s) == sanity-hsm test 58: Truncate a released file will trigger restore ========================================================== 21:59:15 (1789610355) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_58/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' truncate up from 1516 to 3032 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate down from 1516 to 758 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate to 0 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 PASS 58 (19s) == sanity-hsm test 59: Release stripeless file with non-zero size ========================================================== 21:59:34 (1789610374) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_59/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 59 (9s) == sanity-hsm test 60: Changing progress update interval from default ========================================================== 21:59:43 (1789610383) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.236674 s, 44.3 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_60/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 --update-interval 5 "/mnt/lustre2"' Waiting 100s for 'start copy' Expecting a progress update within 20 seconds... 5242880 bytes copied in 3 seconds. Wait for on going archive hsm action to complete Waiting 10s for 'copied' Updated after 3s: want 'copied' got 'copied' PASS 60 (21s) == sanity-hsm test 61: Waiting archive of a removed file should fail ========================================================== 22:00:04 (1789610404) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_61/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 61 (17s) == sanity-hsm test 70: Copytool logs JSON register/unregister events to FIFO ========================================================== 22:00:22 (1789610422) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_70/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --event-fifo /tmp/sanity-hsm.test_70.ZU2r/fifo "/mnt/lustre2"' Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 0s Register/Unregister events look OK. PASS 70 (11s) == sanity-hsm test 71: Copytool logs JSON archive events to FIFO ========================================================== 22:00:34 (1789610434) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_71/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_71.Aisf/fifo "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189339 s, 5.5 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Archive events look OK. mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 71 (24s) == sanity-hsm test 72: Copytool logs JSON restore events to FIFO ========================================================== 22:00:58 (1789610458) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_72/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_72.ntf0/fifo "/mnt/lustre2"' 16+0 records in 16+0 records out 16000000 bytes (16 MB, 15 MiB) copied, 0.786158 s, 20.4 MB/s lhsmtool_posix: 1789610462.766342 [83026]: action=1 src=d72.sanity-hsm/f72.sanity-hsm dst=/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789610462.784810 [83026]: importing '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1789610462.861041 [83026]: imported '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//0bc4/0x200000bd2:0x16:0x0'=='/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1789610462.861100 [83026]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Restore events look OK. PASS 72 (20s) == sanity-hsm test 90: Archive/restore a file list ======= 22:01:17 (1789610477) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_90/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Waiting 200s for '' pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 Waiting 200s for '' Changed after 3s: from 'lrh=[type=10680000 len=136 idx=4/10058] fid=[0x200000403:0x248:0x0] dfid=[0x200000403:0x248:0x0] compound/cookie=0x0/0x6aabddb1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10059] fid=[0x200000403:0x249:0x0] dfid=[0x200000403:0x249:0x0] compound/cookie=0x0/0x6aabddb2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10060] fid=[0x200000403:0x24a:0x0] dfid=[0x200000403:0x24a:0x0] compound/cookie=0x0/0x6aabddb3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10061] fid=[0x200000403:0x24b:0x0] dfid=[0x200000403:0x24b:0x0] compound/cookie=0x0/0x6aabddb4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10062] fid=[0x200000403:0x24c:0x0] dfid=[0x200000403:0x24c:0x0] compound/cookie=0x0/0x6aabddb5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10063] fid=[0x200000403:0x24d:0x0] dfid=[0x200000403:0x24d:0x0] compound/cookie=0x0/0x6aabddb6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10064] fid=[0x200000403:0x24e:0x0] dfid=[0x200000403:0x24e:0x0] compound/cookie=0x0/0x6aabddb7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10066] fid=[0x200000403:0x24f:0x0] dfid=[0x200000403:0x24f:0x0] compound/cookie=0x0/0x6aabddb8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10067] fid=[0x200000403:0x250:0x0] dfid=[0x200000403:0x250:0x0] compound/cookie=0x0/0x6aabddb9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10068] fid=[0x200000403:0x251:0x0] dfid=[0x200000403:0x251:0x0] compound/cookie=0x0/0x6aabddba action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10069] fid=[0x200000403:0x252:0x0] dfid=[0x200000403:0x252:0x0] compound/cookie=0x0/0x6aabddbb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10070] fid=[0x200000403:0x253:0x0] dfid=[0x200000403:0x253:0x0] compound/cookie=0x0/0x6aabddbc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10071] fid=[0x200000403:0x254:0x0] dfid=[0x200000403:0x254:0x0] compound/cookie=0x0/0x6aabddbd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10072] fid=[0x200000403:0x255:0x0] dfid=[0x200000403:0x255:0x0] compound/cookie=0x0/0x6aabddbe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10073] fid=[0x200000403:0x256:0x0] dfid=[0x200000403:0x256:0x0] compound/cookie=0x0/0x6aabddbf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10074] fid=[0x200000403:0x257:0x0] dfid=[0x200000403:0x257:0x0] compound/cookie=0x0/0x6aabddc0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10075] fid=[0x200000403:0x258:0x0] dfid=[0x200000403:0x258:0x0] compound/cookie=0x0/0x6aabddc1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10076] fid=[0x200000403:0x259:0x0] dfid=[0x200000403:0x259:0x0] compound/cookie=0x0/0x6aabddc2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10077] fid=[0x200000403:0x25a:0x0] dfid=[0x200000403:0x25a:0x0] compound/cookie=0x0/0x6aabddc3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10078] fid=[0x200000403:0x25b:0x0] dfid=[0x200000403:0x25b:0x0] compound/cookie=0x0/0x6aabddc4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10079] fid=[0x200000403:0x25c:0x0] dfid=[0x200000403:0x25c:0x0] compound/cookie=0x0/0x6aabddc5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10080] fid=[0x200000403:0x25d:0x0] dfid=[0x200000403:0x25d:0x0] compound/cookie=0x0/0x6aabddc6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10081] fid=[0x200000403:0x25e:0x0] dfid=[0x200000403:0x25e:0x0] compound/cookie=0x0/0x6aabddc7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10082] fid=[0x200000403:0x25f:0x0] dfid=[0x200000403:0x25f:0x0] compound/cookie=0x0/0x6aabddc8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000403:0x260:0x0] dfid=[0x200000403:0x260:0x0] compound/cookie=0x0/0x6aabddc9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000403:0x261:0x0] dfid=[0x200000403:0x261:0x0] compound/cookie=0x0/0x6aabddca action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000403:0x262:0x0] dfid=[0x200000403:0x262:0x0] compound/cookie=0x0/0x6aabddcb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000403:0x263:0x0] dfid=[0x200000403:0x263:0x0] compound/cookie=0x0/0x6aabddcc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000403:0x264:0x0] dfid=[0x200000403:0x264:0x0] compound/cookie=0x0/0x6aabddcd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000403:0x265:0x0] dfid=[0x200000403:0x265:0x0] compound/cookie=0x0/0x6aabddce action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000403:0x266:0x0] dfid=[0x200000403:0x266:0x0] compound/cookie=0x0/0x6aabddcf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000403:0x267:0x0] dfid=[0x200000403:0x267:0x0] compound/cookie=0x0/0x6aabddd0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000403:0x268:0x0] dfid=[0x200000403:0x268:0x0] compound/cookie=0x0/0x6aabddd1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000403:0x269:0x0] dfid=[0x200000403:0x269:0x0] compound/cookie=0x0/0x6aabddd2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000403:0x26a:0x0] dfid=[0x200000403:0x26a:0x0] compound/cookie=0x0/0x6aabddd3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000403:0x26b:0x0] dfid=[0x200000403:0x26b:0x0] compound/cookie=0x0/0x6aabddd4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000403:0x26c:0x0] dfid=[0x200000403:0x26c:0x0] compound/cookie=0x0/0x6aabddd5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000403:0x26d:0x0] dfid=[0x200000403:0x26d:0x0] compound/cookie=0x0/0x6aabddd6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000403:0x26e:0x0] dfid=[0x200000403:0x26e:0x0] compound/cookie=0x0/0x6aabddd7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000403:0x26f:0x0] dfid=[0x200000403:0x26f:0x0] compound/cookie=0x0/0x6aabddd8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000403:0x270:0x0] dfid=[0x200000403:0x270:0x0] compound/cookie=0x0/0x6aabddd9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000403:0x271:0x0] dfid=[0x200000403:0x271:0x0] compound/cookie=0x0/0x6aabddda action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000403:0x272:0x0] dfid=[0x200000403:0x272:0x0] compound/cookie=0x0/0x6aabdddb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000403:0x273:0x0] dfid=[0x200000403:0x273:0x0] compound/cookie=0x0/0x6aabdddc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000403:0x274:0x0] dfid=[0x200000403:0x274:0x0] compound/cookie=0x0/0x6aabdddd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6aabddde action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6aabdddf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6aabdde0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6aabdde1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6aabdde2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6aabdde3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=4/10072] fid=[0x200000403:0x255:0x0] dfid=[0x200000403:0x255:0x0] compound/cookie=0x0/0x6aabddbe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10073] fid=[0x200000403:0x256:0x0] dfid=[0x200000403:0x256:0x0] compound/cookie=0x0/0x6aabddbf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10074] fid=[0x200000403:0x257:0x0] dfid=[0x200000403:0x257:0x0] compound/cookie=0x0/0x6aabddc0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10075] fid=[0x200000403:0x258:0x0] dfid=[0x200000403:0x258:0x0] compound/cookie=0x0/0x6aabddc1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10076] fid=[0x200000403:0x259:0x0] dfid=[0x200000403:0x259:0x0] compound/cookie=0x0/0x6aabddc2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10077] fid=[0x200000403:0x25a:0x0] dfid=[0x200000403:0x25a:0x0] compound/cookie=0x0/0x6aabddc3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10078] fid=[0x200000403:0x25b:0x0] dfid=[0x200000403:0x25b:0x0] compound/cookie=0x0/0x6aabddc4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10079] fid=[0x200000403:0x25c:0x0] dfid=[0x200000403:0x25c:0x0] compound/cookie=0x0/0x6aabddc5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10080] fid=[0x200000403:0x25d:0x0] dfid=[0x200000403:0x25d:0x0] compound/cookie=0x0/0x6aabddc6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10081] fid=[0x200000403:0x25e:0x0] dfid=[0x200000403:0x25e:0x0] compound/cookie=0x0/0x6aabddc7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10082] fid=[0x200000403:0x25f:0x0] dfid=[0x200000403:0x25f:0x0] compound/cookie=0x0/0x6aabddc8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000403:0x260:0x0] dfid=[0x200000403:0x260:0x0] compound/cookie=0x0/0x6aabddc9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000403:0x261:0x0] dfid=[0x200000403:0x261:0x0] compound/cookie=0x0/0x6aabddca action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000403:0x262:0x0] dfid=[0x200000403:0x262:0x0] compound/cookie=0x0/0x6aabddcb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000403:0x263:0x0] dfid=[0x200000403:0x263:0x0] compound/cookie=0x0/0x6aabddcc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000403:0x264:0x0] dfid=[0x200000403:0x264:0x0] compound/cookie=0x0/0x6aabddcd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000403:0x265:0x0] dfid=[0x200000403:0x265:0x0] compound/cookie=0x0/0x6aabddce action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000403:0x266:0x0] dfid=[0x200000403:0x266:0x0] compound/cookie=0x0/0x6aabddcf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000403:0x267:0x0] dfid=[0x200000403:0x267:0x0] compound/cookie=0x0/0x6aabddd0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000403:0x268:0x0] dfid=[0x200000403:0x268:0x0] compound/cookie=0x0/0x6aabddd1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000403:0x269:0x0] dfid=[0x200000403:0x269:0x0] compound/cookie=0x0/0x6aabddd2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000403:0x26a:0x0] dfid=[0x200000403:0x26a:0x0] compound/cookie=0x0/0x6aabddd3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000403:0x26b:0x0] dfid=[0x200000403:0x26b:0x0] compound/cookie=0x0/0x6aabddd4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000403:0x26c:0x0] dfid=[0x200000403:0x26c:0x0] compound/cookie=0x0/0x6aabddd5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000403:0x26d:0x0] dfid=[0x200000403:0x26d:0x0] compound/cookie=0x0/0x6aabddd6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000403:0x26e:0x0] dfid=[0x200000403:0x26e:0x0] compound/cookie=0x0/0x6aabddd7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000403:0x26f:0x0] dfid=[0x200000403:0x26f:0x0] compound/cookie=0x0/0x6aabddd8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000403:0x270:0x0] dfid=[0x200000403:0x270:0x0] compound/cookie=0x0/0x6aabddd9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000403:0x271:0x0] dfid=[0x200000403:0x271:0x0] compound/cookie=0x0/0x6aabddda action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000403:0x272:0x0] dfid=[0x200000403:0x272:0x0] compound/cookie=0x0/0x6aabdddb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000403:0x273:0x0] dfid=[0x200000403:0x273:0x0] compound/cookie=0x0/0x6aabdddc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000403:0x274:0x0] dfid=[0x200000403:0x274:0x0] compound/cookie=0x0/0x6aabdddd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6aabddde action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6aabdddf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6aabdde0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6aabdde1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6aabdde2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6aabdde3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 5s: from 'lrh=[type=10680000 len=136 idx=4/10072] fid=[0x200000403:0x255:0x0] dfid=[0x200000403:0x255:0x0] compound/cookie=0x0/0x6aabddbe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10073] fid=[0x200000403:0x256:0x0] dfid=[0x200000403:0x256:0x0] compound/cookie=0x0/0x6aabddbf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10074] fid=[0x200000403:0x257:0x0] dfid=[0x200000403:0x257:0x0] compound/cookie=0x0/0x6aabddc0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10075] fid=[0x200000403:0x258:0x0] dfid=[0x200000403:0x258:0x0] compound/cookie=0x0/0x6aabddc1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10076] fid=[0x200000403:0x259:0x0] dfid=[0x200000403:0x259:0x0] compound/cookie=0x0/0x6aabddc2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10077] fid=[0x200000403:0x25a:0x0] dfid=[0x200000403:0x25a:0x0] compound/cookie=0x0/0x6aabddc3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10078] fid=[0x200000403:0x25b:0x0] dfid=[0x200000403:0x25b:0x0] compound/cookie=0x0/0x6aabddc4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10079] fid=[0x200000403:0x25c:0x0] dfid=[0x200000403:0x25c:0x0] compound/cookie=0x0/0x6aabddc5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10080] fid=[0x200000403:0x25d:0x0] dfid=[0x200000403:0x25d:0x0] compound/cookie=0x0/0x6aabddc6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10081] fid=[0x200000403:0x25e:0x0] dfid=[0x200000403:0x25e:0x0] compound/cookie=0x0/0x6aabddc7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10082] fid=[0x200000403:0x25f:0x0] dfid=[0x200000403:0x25f:0x0] compound/cookie=0x0/0x6aabddc8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000403:0x260:0x0] dfid=[0x200000403:0x260:0x0] compound/cookie=0x0/0x6aabddc9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000403:0x261:0x0] dfid=[0x200000403:0x261:0x0] compound/cookie=0x0/0x6aabddca action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000403:0x262:0x0] dfid=[0x200000403:0x262:0x0] compound/cookie=0x0/0x6aabddcb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000403:0x263:0x0] dfid=[0x200000403:0x263:0x0] compound/cookie=0x0/0x6aabddcc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000403:0x264:0x0] dfid=[0x200000403:0x264:0x0] compound/cookie=0x0/0x6aabddcd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000403:0x265:0x0] dfid=[0x200000403:0x265:0x0] compound/cookie=0x0/0x6aabddce action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000403:0x266:0x0] dfid=[0x200000403:0x266:0x0] compound/cookie=0x0/0x6aabddcf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000403:0x267:0x0] dfid=[0x200000403:0x267:0x0] compound/cookie=0x0/0x6aabddd0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000403:0x268:0x0] dfid=[0x200000403:0x268:0x0] compound/cookie=0x0/0x6aabddd1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000403:0x269:0x0] dfid=[0x200000403:0x269:0x0] compound/cookie=0x0/0x6aabddd2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000403:0x26a:0x0] dfid=[0x200000403:0x26a:0x0] compound/cookie=0x0/0x6aabddd3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000403:0x26b:0x0] dfid=[0x200000403:0x26b:0x0] compound/cookie=0x0/0x6aabddd4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000403:0x26c:0x0] dfid=[0x200000403:0x26c:0x0] compound/cookie=0x0/0x6aabddd5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000403:0x26d:0x0] dfid=[0x200000403:0x26d:0x0] compound/cookie=0x0/0x6aabddd6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000403:0x26e:0x0] dfid=[0x200000403:0x26e:0x0] compound/cookie=0x0/0x6aabddd7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000403:0x26f:0x0] dfid=[0x200000403:0x26f:0x0] compound/cookie=0x0/0x6aabddd8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000403:0x270:0x0] dfid=[0x200000403:0x270:0x0] compound/cookie=0x0/0x6aabddd9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000403:0x271:0x0] dfid=[0x200000403:0x271:0x0] compound/cookie=0x0/0x6aabddda action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000403:0x272:0x0] dfid=[0x200000403:0x272:0x0] compound/cookie=0x0/0x6aabdddb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000403:0x273:0x0] dfid=[0x200000403:0x273:0x0] compound/cookie=0x0/0x6aabdddc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000403:0x274:0x0] dfid=[0x200000403:0x274:0x0] compound/cookie=0x0/0x6aabdddd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6aabddde action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6aabdddf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6aabdde0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6aabdde1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6aabdde2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6aabdde3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000403:0x260:0x0] dfid=[0x200000403:0x260:0x0] compound/cookie=0x0/0x6aabddc9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000403:0x261:0x0] dfid=[0x200000403:0x261:0x0] compound/cookie=0x0/0x6aabddca action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000403:0x262:0x0] dfid=[0x200000403:0x262:0x0] compound/cookie=0x0/0x6aabddcb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000403:0x263:0x0] dfid=[0x200000403:0x263:0x0] compound/cookie=0x0/0x6aabddcc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000403:0x264:0x0] dfid=[0x200000403:0x264:0x0] compound/cookie=0x0/0x6aabddcd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000403:0x265:0x0] dfid=[0x200000403:0x265:0x0] compound/cookie=0x0/0x6aabddce action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000403:0x266:0x0] dfid=[0x200000403:0x266:0x0] compound/cookie=0x0/0x6aabddcf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000403:0x267:0x0] dfid=[0x200000403:0x267:0x0] compound/cookie=0x0/0x6aabddd0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000403:0x268:0x0] dfid=[0x200000403:0x268:0x0] compound/cookie=0x0/0x6aabddd1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000403:0x269:0x0] dfid=[0x200000403:0x269:0x0] compound/cookie=0x0/0x6aabddd2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000403:0x26a:0x0] dfid=[0x200000403:0x26a:0x0] compound/cookie=0x0/0x6aabddd3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000403:0x26b:0x0] dfid=[0x200000403:0x26b:0x0] compound/cookie=0x0/0x6aabddd4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000403:0x26c:0x0] dfid=[0x200000403:0x26c:0x0] compound/cookie=0x0/0x6aabddd5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000403:0x26d:0x0] dfid=[0x200000403:0x26d:0x0] compound/cookie=0x0/0x6aabddd6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000403:0x26e:0x0] dfid=[0x200000403:0x26e:0x0] compound/cookie=0x0/0x6aabddd7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000403:0x26f:0x0] dfid=[0x200000403:0x26f:0x0] compound/cookie=0x0/0x6aabddd8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000403:0x270:0x0] dfid=[0x200000403:0x270:0x0] compound/cookie=0x0/0x6aabddd9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000403:0x271:0x0] dfid=[0x200000403:0x271:0x0] compound/cookie=0x0/0x6aabddda action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000403:0x272:0x0] dfid=[0x200000403:0x272:0x0] compound/cookie=0x0/0x6aabdddb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000403:0x273:0x0] dfid=[0x200000403:0x273:0x0] compound/cookie=0x0/0x6aabdddc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000403:0x274:0x0] dfid=[0x200000403:0x274:0x0] compound/cookie=0x0/0x6aabdddd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6aabddde action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6aabdddf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6aabdde0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6aabdde1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6aabdde2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6aabdde3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 7s: from 'lrh=[type=10680000 len=136 idx=4/10083] fid=[0x200000403:0x260:0x0] dfid=[0x200000403:0x260:0x0] compound/cookie=0x0/0x6aabddc9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10084] fid=[0x200000403:0x261:0x0] dfid=[0x200000403:0x261:0x0] compound/cookie=0x0/0x6aabddca action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10085] fid=[0x200000403:0x262:0x0] dfid=[0x200000403:0x262:0x0] compound/cookie=0x0/0x6aabddcb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10086] fid=[0x200000403:0x263:0x0] dfid=[0x200000403:0x263:0x0] compound/cookie=0x0/0x6aabddcc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10087] fid=[0x200000403:0x264:0x0] dfid=[0x200000403:0x264:0x0] compound/cookie=0x0/0x6aabddcd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10088] fid=[0x200000403:0x265:0x0] dfid=[0x200000403:0x265:0x0] compound/cookie=0x0/0x6aabddce action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10089] fid=[0x200000403:0x266:0x0] dfid=[0x200000403:0x266:0x0] compound/cookie=0x0/0x6aabddcf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10090] fid=[0x200000403:0x267:0x0] dfid=[0x200000403:0x267:0x0] compound/cookie=0x0/0x6aabddd0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10091] fid=[0x200000403:0x268:0x0] dfid=[0x200000403:0x268:0x0] compound/cookie=0x0/0x6aabddd1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000403:0x269:0x0] dfid=[0x200000403:0x269:0x0] compound/cookie=0x0/0x6aabddd2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000403:0x26a:0x0] dfid=[0x200000403:0x26a:0x0] compound/cookie=0x0/0x6aabddd3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000403:0x26b:0x0] dfid=[0x200000403:0x26b:0x0] compound/cookie=0x0/0x6aabddd4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000403:0x26c:0x0] dfid=[0x200000403:0x26c:0x0] compound/cookie=0x0/0x6aabddd5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000403:0x26d:0x0] dfid=[0x200000403:0x26d:0x0] compound/cookie=0x0/0x6aabddd6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000403:0x26e:0x0] dfid=[0x200000403:0x26e:0x0] compound/cookie=0x0/0x6aabddd7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000403:0x26f:0x0] dfid=[0x200000403:0x26f:0x0] compound/cookie=0x0/0x6aabddd8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000403:0x270:0x0] dfid=[0x200000403:0x270:0x0] compound/cookie=0x0/0x6aabddd9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000403:0x271:0x0] dfid=[0x200000403:0x271:0x0] compound/cookie=0x0/0x6aabddda action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000403:0x272:0x0] dfid=[0x200000403:0x272:0x0] compound/cookie=0x0/0x6aabdddb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000403:0x273:0x0] dfid=[0x200000403:0x273:0x0] compound/cookie=0x0/0x6aabdddc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000403:0x274:0x0] dfid=[0x200000403:0x274:0x0] compound/cookie=0x0/0x6aabdddd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6aabddde action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6aabdddf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6aabdde0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6aabdde1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6aabdde2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6aabdde3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000403:0x269:0x0] dfid=[0x200000403:0x269:0x0] compound/cookie=0x0/0x6aabddd2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000403:0x26a:0x0] dfid=[0x200000403:0x26a:0x0] compound/cookie=0x0/0x6aabddd3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000403:0x26b:0x0] dfid=[0x200000403:0x26b:0x0] compound/cookie=0x0/0x6aabddd4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000403:0x26c:0x0] dfid=[0x200000403:0x26c:0x0] compound/cookie=0x0/0x6aabddd5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000403:0x26d:0x0] dfid=[0x200000403:0x26d:0x0] compound/cookie=0x0/0x6aabddd6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000403:0x26e:0x0] dfid=[0x200000403:0x26e:0x0] compound/cookie=0x0/0x6aabddd7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000403:0x26f:0x0] dfid=[0x200000403:0x26f:0x0] compound/cookie=0x0/0x6aabddd8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000403:0x270:0x0] dfid=[0x200000403:0x270:0x0] compound/cookie=0x0/0x6aabddd9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000403:0x271:0x0] dfid=[0x200000403:0x271:0x0] compound/cookie=0x0/0x6aabddda action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000403:0x272:0x0] dfid=[0x200000403:0x272:0x0] compound/cookie=0x0/0x6aabdddb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000403:0x273:0x0] dfid=[0x200000403:0x273:0x0] compound/cookie=0x0/0x6aabdddc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000403:0x274:0x0] dfid=[0x200000403:0x274:0x0] compound/cookie=0x0/0x6aabdddd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6aabddde action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6aabdddf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6aabdde0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6aabdde1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6aabdde2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6aabdde3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 9s: from 'lrh=[type=10680000 len=136 idx=4/10092] fid=[0x200000403:0x269:0x0] dfid=[0x200000403:0x269:0x0] compound/cookie=0x0/0x6aabddd2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10093] fid=[0x200000403:0x26a:0x0] dfid=[0x200000403:0x26a:0x0] compound/cookie=0x0/0x6aabddd3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10094] fid=[0x200000403:0x26b:0x0] dfid=[0x200000403:0x26b:0x0] compound/cookie=0x0/0x6aabddd4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10095] fid=[0x200000403:0x26c:0x0] dfid=[0x200000403:0x26c:0x0] compound/cookie=0x0/0x6aabddd5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10096] fid=[0x200000403:0x26d:0x0] dfid=[0x200000403:0x26d:0x0] compound/cookie=0x0/0x6aabddd6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10097] fid=[0x200000403:0x26e:0x0] dfid=[0x200000403:0x26e:0x0] compound/cookie=0x0/0x6aabddd7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10098] fid=[0x200000403:0x26f:0x0] dfid=[0x200000403:0x26f:0x0] compound/cookie=0x0/0x6aabddd8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10099] fid=[0x200000403:0x270:0x0] dfid=[0x200000403:0x270:0x0] compound/cookie=0x0/0x6aabddd9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10100] fid=[0x200000403:0x271:0x0] dfid=[0x200000403:0x271:0x0] compound/cookie=0x0/0x6aabddda action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10101] fid=[0x200000403:0x272:0x0] dfid=[0x200000403:0x272:0x0] compound/cookie=0x0/0x6aabdddb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10102] fid=[0x200000403:0x273:0x0] dfid=[0x200000403:0x273:0x0] compound/cookie=0x0/0x6aabdddc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10103] fid=[0x200000403:0x274:0x0] dfid=[0x200000403:0x274:0x0] compound/cookie=0x0/0x6aabdddd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6aabddde action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6aabdddf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6aabdde0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6aabdde1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6aabdde2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6aabdde3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=4/10104] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6aabddde action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10105] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6aabdddf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10106] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6aabdde0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=4/10107] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6aabdde1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10108] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6aabdde2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=4/10109] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6aabdde3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 PASS 90 (102s) == sanity-hsm test 100: Set coordinator /proc tunables === 22:02:59 (1789610579) Testing mdt.lustre-MDT0000.hsm.loop_period Set loop_period to 0 must failed oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/loop_period=0: Invalid argument oleg348-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/loop_period'='0': Invalid argument pdsh@oleg348-client: oleg348-server: ssh exited with exit code 22 oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/loop_period=0: Invalid argument oleg348-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/loop_period'='0': Invalid argument pdsh@oleg348-client: oleg348-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.grace_delay Set grace_delay to 0 must failed oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/grace_delay=0: Invalid argument oleg348-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/grace_delay'='0': Invalid argument pdsh@oleg348-client: oleg348-server: ssh exited with exit code 22 oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/grace_delay=0: Invalid argument oleg348-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/grace_delay'='0': Invalid argument pdsh@oleg348-client: oleg348-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.active_request_timeout Set active_request_timeout to 0 must failed oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/active_request_timeout=0: Invalid argument oleg348-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/active_request_timeout'='0': Invalid argument pdsh@oleg348-client: oleg348-server: ssh exited with exit code 22 oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/active_request_timeout=0: Invalid argument oleg348-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/active_request_timeout'='0': Invalid argument pdsh@oleg348-client: oleg348-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.max_requests Set max_requests to 0 must failed oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=0: Invalid argument oleg348-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='0': Invalid argument pdsh@oleg348-client: oleg348-server: ssh exited with exit code 22 oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=0: Invalid argument oleg348-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='0': Invalid argument pdsh@oleg348-client: oleg348-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.default_archive_id Set default_archive_id to 0 must failed oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/default_archive_id=0: Invalid argument oleg348-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/default_archive_id'='0': Invalid argument pdsh@oleg348-client: oleg348-server: ssh exited with exit code 22 oleg348-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/default_archive_id=0: Invalid argument oleg348-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/default_archive_id'='0': Invalid argument pdsh@oleg348-client: oleg348-server: ssh exited with exit code 22 PASS 100 (54s) == sanity-hsm test 102: Verify coordinator control ======= 22:03:53 (1789610633) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 102 (22s) == sanity-hsm test 103: Purge all requests =============== 22:04:15 (1789610655) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Current requests pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 PASS 103 (16s) == sanity-hsm test 103a: Purge pending restore requests == 22:04:31 (1789610671) mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' (0x200000403:0x2c2:0x0|0x200000403:0x2c3:0x0|0x200000403:0x2c4:0x0|0x200000403:0x2c5:0x0|0x200000403:0x2c6:0x0|0x200000403:0x2c7:0x0|0x200000403:0x2c8:0x0|0x200000403:0x2c9:0x0).*action=ARCHIVE.*status=SUCCEED Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 0s cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2c6:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2c2:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2c4:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2c5:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2c7:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2c3:0x0'cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2c8:0x0': No data available : No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2c9:0x0': No data available mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 103a (18s) == sanity-hsm test 104: Copy tool data field ============= 22:04:49 (1789610689) 0+0 records in 0+0 records out 0 bytes copied, 0.00886303 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 104 (11s) == sanity-hsm test 105: Restart of coordinator =========== 22:05:00 (1789610700) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 105 (37s) == sanity-hsm test 106a: Copytool register/unregister ==== 22:05:37 (1789610737) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg348-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg348-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg348-client.virtnet: 89803 Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 0s pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 uuid not found in agent list on mds2 Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg348-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg348-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 106a (41s) == sanity-hsm test 106b: Unregister agent after a copytool crash ========================================================== 22:06:19 (1789610779) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg348-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg348-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg348-client.virtnet: 91540 Killing existing copytools on oleg348-client.virtnet with signal KILL copytools stopped in 0s pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 uuid not found in agent list on mds2 Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 106b (26s) == sanity-hsm test 106c: Unregister copytool on client eviction ========================================================== 22:06:45 (1789610805) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg348-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg348-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg348-client.virtnet: 92773 oleg348-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg348-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg348-client.virtnet: executing get_client_uuid /mnt/lustre2 mdc.lustre-MDT0000-mdc-ffff9d38d8b9f000.active=0 mdc.lustre-MDT0001-mdc-ffff9d38d8b9f000.active=0 mdt.lustre-MDT0000.evict_client=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 mdt.lustre-MDT0001.evict_client=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 uuid not found in agent list on mds2 mdc.lustre-MDT0000-mdc-ffff9d38d8b9f000.active=1 mdc.lustre-MDT0001-mdc-ffff9d38d8b9f000.active=1 oleg348-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 mdc.lustre-MDT0000-mdc-ffff9d38d8b9f000.active=1 mdc.lustre-MDT0001-mdc-ffff9d38d8b9f000.active=1 PASS 106c (52s) == sanity-hsm test 107: Copytool re-register after MDS restart ========================================================== 22:07:37 (1789610857) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_107/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Failing mds1 on oleg348-server Stopping /mnt/lustre-mds1 (opts:) on oleg348-server 22:07:46 (1789610866) shut down facet: mds1 facet_host: oleg348-server facet_failover_host: oleg348-server Failover mds1 to oleg348-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg348-server: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 Started lustre-MDT0000 22:08:18 (1789610898) targets are mounted 22:08:18 (1789610898) facet_failover done oleg348-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 107 (60s) == sanity-hsm test 109: Policy display/change ============ 22:08:37 (1789610917) mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0000.hsm.policy=NRA NBR Next set_param must failed oleg348-server: error: set_param: setting /sys/kernel/debug/lustre/mdt/lustre-MDT0000/hsm/policy=wrong: Invalid argument oleg348-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/policy'='wrong': Invalid argument pdsh@oleg348-client: oleg348-server: ssh exited with exit code 22 Back to default policy mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 109 (25s) == sanity-hsm test 110a: Non blocking restore policy (import case) ========================================================== 22:09:02 (1789610942) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1789610945.204080 [97387]: action=1 src=d110a.sanity-hsm/f110a.sanity-hsm dst=/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789610945.226567 [97387]: importing '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1789610945.278940 [97387]: imported '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//0bb6/0x200000bd2:0x64:0x0'=='/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1789610945.279004 [97387]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0001.hsm.policy=+NBR md5sum: /mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm: No data available Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 110a (16s) == sanity-hsm test 110b: Non blocking restore policy (release case) ========================================================== 22:09:19 (1789610959) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0001.hsm.policy=+NBR md5sum: /mnt/lustre/d110b.sanity-hsm/f110b.sanity-hsm: No data available mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 110b (12s) == sanity-hsm test 111a: No retry policy (import case), restore will error ========================================================== 22:09:31 (1789610971) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1789610973.999575 [98787]: action=1 src=d111a.sanity-hsm/f111a.sanity-hsm dst=/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789610974.007895 [98787]: importing '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1789610974.087273 [98787]: imported '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//0bba/0x200000bd2:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1789610974.087332 [98787]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Remove from backend: 0x200000bd2:0x68:0x0 = /tmp/arc1/sanity-hsm.test_111a/0bba/0x200000bd2:0x68:0x0 mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 111a (14s) == sanity-hsm test 111b: No retry policy (release case), restore will error ========================================================== 22:09:45 (1789610985) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Remove from backend: 0x200000bd3:0x7:0x0 = /tmp/arc1/sanity-hsm.test_111b/0bd4/0x200000bd3:0x7:0x0 mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 111b (14s) == sanity-hsm test 112: State of recorded request ======== 22:09:59 (1789610999) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_112/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled /mnt/lustre/d112.sanity-hsm/f112.sanity-hsm: ARCHIVE waiting (from 0 to EOF) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 112 (15s) == sanity-hsm test 113: wrong stat after restore ========= 22:10:15 (1789611015) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999088 s, 10.5 MB/s Starting client: oleg348-client.virtnet: -o user_xattr,flock 192.168.203.148@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_113/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Verifying released state: multiop /mnt/lustre/d113.sanity-hsm/f113.sanity-hsm voO_WRONLY:O_APPEND:_w4c TMPPIPE=/tmp/multiop_open_wait_pipe.8459 File: /mnt/lustre2/d113.sanity-hsm/f113.sanity-hsm Size: 1048576 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115238860488714 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-09-16 22:10:16.000000000 -0400 Modify: 2026-09-16 22:10:17.000000000 -0400 Change: 2026-09-16 22:10:17.000000000 -0400 Birth: 2026-09-16 22:10:16.000000000 -0400 Stopping client oleg348-client.virtnet /mnt/lustre3 (opts:) PASS 113 (17s) == sanity-hsm test 114: Incompatible request does not set other requests as STARTED ========================================================== 22:10:33 (1789611033) 0+0 records in 0+0 records out 0 bytes copied, 0.00263936 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00371803 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_114/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 114 (24s) == sanity-hsm test 200: Register/Cancel archive ========== 22:10:56 (1789611056) 0+0 records in 0+0 records out 0 bytes copied, 0.0057641 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_200/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet PASS 200 (12s) == sanity-hsm test 201: Register/Cancel restore ========== 22:11:08 (1789611068) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_201/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.24793 s, 32.8 MB/s lhsmtool_posix: 1789611072.243111 [103169]: action=1 src=d201.sanity-hsm/f201.sanity-hsm dst=/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789611072.252351 [103169]: importing '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1789611072.304729 [103169]: imported '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//0ba3/0x200000bd2:0x71:0x0'=='/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1789611072.304822 [103169]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 201 (21s) == sanity-hsm test 202: Register/Cancel remove =========== 22:11:29 (1789611089) 0+0 records in 0+0 records out 0 bytes copied, 0.0155396 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_202/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg348-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' PASS 202 (14s) == sanity-hsm test 220A: Changelog for archive =========== 22:11:43 (1789611103) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' lustre-MDT0000.3 16HSM 02:11:52.567702903 2026.09.17 0x0 t=[0x200000bd3:0x13:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 220A (22s) == sanity-hsm test 220a: Changelog for failed archive ==== 22:12:05 (1789611125) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet lustre-MDT0000.6 16HSM 02:12:15.181187169 2026.09.17 0x2 t=[0x200000bd3:0x14:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 220a (25s) == sanity-hsm test 221: Changelog for archive canceled === 22:12:30 (1789611150) 0+0 records in 0+0 records out 0 bytes copied, 0.00448598 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_221/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl3 cl3' Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet lustre-MDT0000.8 16HSM 02:12:40.929704928 2026.09.17 0x7d t=[0x200000bd3:0x16:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001: Deregistered changelog user #3 lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 221 (23s) == sanity-hsm test 222a: Changelog for explicit restore == 22:12:54 (1789611174) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1789611176.724443 [107019]: action=1 src=d222a.sanity-hsm/f222a.sanity-hsm dst=/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789611176.732091 [107019]: importing '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1789611176.778510 [107019]: imported '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//0ba5/0x200000bd2:0x77:0x0'=='/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1789611176.778568 [107019]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl4 cl4' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.11 16HSM 02:13:03.911069207 2026.09.17 0x80 t=[0x200000bd2:0x77:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0001: Deregistered changelog user #4 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 222a (23s) == sanity-hsm test 222b: Changelog for implicit restore == 22:13:18 (1789611198) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl5 cl5' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 4ef076ad20e5d4a2f4729d6ed41a3bf9 /mnt/lustre/d222b.sanity-hsm/f222b.sanity-hsm lustre-MDT0000.20 16HSM 02:13:30.118777676 2026.09.17 0x80 t=[0x200000bd3:0x18:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0001: Deregistered changelog user #5 lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 PASS 222b (25s) == sanity-hsm test 222c: Changelog for failed explicit restore ========================================================== 22:13:42 (1789611222) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1789611225.477238 [108666]: action=1 src=d222c.sanity-hsm/f222c.sanity-hsm dst=/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789611225.488635 [108666]: importing '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1789611225.546869 [108666]: imported '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//0bae/0x200000bd2:0x7c:0x0'=='/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1789611225.546936 [108666]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl6 cl6' Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet lustre-MDT0000.22 16HSM 02:13:53.119799539 2026.09.17 0x82 t=[0x200000bd2:0x7c:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001: Deregistered changelog user #6 lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 222c (24s) == sanity-hsm test 222d: Changelog for failed implicit restore ========================================================== 22:14:07 (1789611247) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl7 cl7' Remove from backend: 0x200000bd3:0x1a:0x0 = /tmp/arc1/sanity-hsm.test_222d/0bc9/0x200000bd3:0x1a:0x0 md5sum: /mnt/lustre/d222d.sanity-hsm/f222d.sanity-hsm: No data available lustre-MDT0000.29 16HSM 02:14:18.005235331 2026.09.17 0x82 t=[0x200000bd3:0x1a:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0001: Deregistered changelog user #7 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 222d (23s) == sanity-hsm test 223a: Changelog for restore canceled (import case) ========================================================== 22:14:30 (1789611270) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.37894 s, 29.7 MB/s mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl8 cl8' lhsmtool_posix: 1789611280.990430 [110410]: action=1 src=d223a.sanity-hsm/f223a.sanity-hsm dst=/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1789611281.003663 [110410]: importing '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1789611281.095657 [110410]: imported '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//0b52/0x200000bd2:0x80:0x0'=='/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1789611281.095730 [110410]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Waiting 200s for 'SUCCEED' Waiting 190s for 'SUCCEED' Updated after 28s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.36 16HSM 02:15:10.927138987 2026.09.17 0xfd t=[0x200000bd2:0x80:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0001: clear the changelog for cl8 of all records lustre-MDT0001: Deregistered changelog user #8 lustre-MDT0000: clear the changelog for cl8 of all records lustre-MDT0000: Deregistered changelog user #8 PASS 223a (53s) == sanity-hsm test 223b: Changelog for restore canceled (release case) ========================================================== 22:15:23 (1789611323) 0+0 records in 0+0 records out 0 bytes copied, 0.0301468 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl9 cl9' Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet lustre-MDT0000.43 16HSM 02:15:35.096107721 2026.09.17 0xfd t=[0x200000bd3:0x1e:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0001: clear the changelog for cl9 of all records lustre-MDT0001: Deregistered changelog user #9 lustre-MDT0000: clear the changelog for cl9 of all records lustre-MDT0000: Deregistered changelog user #9 PASS 223b (24s) == sanity-hsm test 224A: Changelog for remove ============ 22:15:48 (1789611348) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl10 cl10' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.48 16HSM 02:15:58.882219865 2026.09.17 0x200 t=[0x200000bd3:0x20:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp p=[0x200000bd2:0x7f:0x0] lustre-MDT0001: clear the changelog for cl10 of all records lustre-MDT0001: Deregistered changelog user #10 lustre-MDT0000: clear the changelog for cl10 of all records lustre-MDT0000: Deregistered changelog user #10 PASS 224A (23s) == sanity-hsm test 224a: Changelog for failed remove ===== 22:16:12 (1789611372) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl11 cl11' Remove from backend: 0x200000bd3:0x21:0x0 = /tmp/arc1/sanity-hsm.test_224a/0bf2/0x200000bd3:0x21:0x0 Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet lustre-MDT0000.53 16HSM 02:16:22.695118262 2026.09.17 0x202 t=[0x200000bd3:0x21:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0001: clear the changelog for cl11 of all records lustre-MDT0001: Deregistered changelog user #11 lustre-MDT0000: clear the changelog for cl11 of all records lustre-MDT0000: Deregistered changelog user #11 PASS 224a (24s) == sanity-hsm test 225: Changelog for remove canceled ==== 22:16:35 (1789611395) Test disabled PASS 225 (7s) == sanity-hsm test 226: changelog for last rm/mv with exiting archive ========================================================== 22:16:42 (1789611402) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_226/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0x200000bd3:0x24:0x0 mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl12 cl12' lustre-MDT0000.54 16HSM 02:16:52.379718669 2026.09.17 0x280 t=[0x200000bd3:0x22:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.55 16HSM 02:16:52.527097652 2026.09.17 0x280 t=[0x200000bd3:0x22:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0000.56 16HSM 02:16:52.527722616 2026.09.17 0x0 t=[0x200000bd3:0x22:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0000.57 16HSM 02:16:53.594221369 2026.09.17 0x280 t=[0x200000bd3:0x23:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.58 16HSM 02:16:53.761104912 2026.09.17 0x280 t=[0x200000bd3:0x23:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0000.59 16HSM 02:16:53.761627291 2026.09.17 0x0 t=[0x200000bd3:0x23:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0000.60 06UNLNK 02:16:54.909700788 2026.09.17 0x3 t=[0x200000bd3:0x22:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp p=[0x200000bd2:0x85:0x0] f226.sanity-hsm-1 lustre-MDT0000.60 06UNLNK 02:16:54.909700788 2026.09.17 0x3 t=[0x200000bd3:0x22:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp p=[0x200000bd2:0x85:0x0] f226.sanity-hsm-1 lustre-MDT0000.61 08RENME 02:16:55.727761140 2026.09.17 0x3 t=[0x200000bd3:0x23:0x0] j=mv.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp p=[0x200000bd2:0x85:0x0] f226.sanity-hsm-2 s=[0x200000bd3:0x24:0x0] sp=[0x200000bd2:0x85:0x0] f226.sanity-hsm-3 lustre-MDT0001: clear the changelog for cl12 of all records lustre-MDT0001: Deregistered changelog user #12 lustre-MDT0000: clear the changelog for cl12 of all records lustre-MDT0000: Deregistered changelog user #12 PASS 226 (27s) == sanity-hsm test 227: changelog when explicit setting of HSM flags ========================================================== 22:17:10 (1789611430) 0+0 records in 0+0 records out 0 bytes copied, 0.00743311 s, 0.0 kB/s mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl13 cl13' lustre-MDT0001: clear the changelog for cl13 of all records lustre-MDT0001: Deregistered changelog user #13 lustre-MDT0000: clear the changelog for cl13 of all records lustre-MDT0000: Deregistered changelog user #13 PASS 227 (24s) == sanity-hsm test 228: On released file, return extend to FIEMAP. For [cp,tar] --sparse ========================================================== 22:17:34 (1789611454) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_228/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0905798 s, 11.6 MB/s /mnt/lustre/f228.sanity-hsm: 1 extent found tar: Removing leading `/' from member names mnt/lustre/f228.sanity-hsm PASS 228 (12s) == sanity-hsm test 250: Coordinator max request ========== 22:17:46 (1789611466) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_250/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.0119056 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.0067546 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.008425 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00382122 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00404752 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00642732 s, 0.0 kB/s Copytool is suspended on oleg348-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Copytool is continued on oleg348-client.virtnet PASS 250 (32s) == sanity-hsm test 251: Coordinator request timeout ====== 22:18:18 (1789611498) 0+0 records in 0+0 records out 0 bytes copied, 0.0056691 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_251/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg348-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Waiting 200s for 'CANCELED' Updated after 2s: want 'CANCELED' got 'CANCELED' PASS 251 (29s) == sanity-hsm test 252: Timeout'ed running archive of a removed file should be canceled ========================================================== 22:18:47 (1789611527) 0+0 records in 0+0 records out 0 bytes copied, 0.0135894 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_252/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet PASS 252 (20s) == sanity-hsm test 253: Check for wrong file size after release ========================================================== 22:19:07 (1789611547) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_253/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 10+0 records in 10+0 records out 10000000 bytes (10 MB, 9.5 MiB) copied, 0.309158 s, 32.3 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' fail_loc=0x807 Cannot send HSM request (use of /mnt/lustre/d253.sanity-hsm/f253.sanity-hsm): Invalid argument could not release file PASS 253 (11s) == sanity-hsm test 254a: Request counters are initialized to zero ========================================================== 22:19:18 (1789611558) PASS 254a (10s) == sanity-hsm test 254b: Request counters are correctly incremented and decremented ========================================================== 22:19:28 (1789611568) Will launch 36 requests of each type Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_254b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Checking archive requests Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet Checking restore requests Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet Waiting 90s for '0' Updated after 2s: want '0' got '0' Checking remove requests Copytool is suspended on oleg348-client.virtnet Copytool is continued on oleg348-client.virtnet PASS 254b (36s) == sanity-hsm test 255: Copytool registration wakes the coordinator up ========================================================== 22:20:04 (1789611604) 0+0 records in 0+0 records out 0 bytes copied, 0.0055846 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 0s 0+0 records in 0+0 records out 0 bytes copied, 0.00831162 s, 0.0 kB/s 0x200000bd3:0x7f:0x0 mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 255 (31s) == sanity-hsm test 256: HSM copytool register on DNE with stopped MDT must not panic ========================================================== 22:20:35 (1789611635) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=enabled PASS 256 (32s) == sanity-hsm test 260a: Restore request have priority over other requests ========================================================== 22:21:07 (1789611667) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.548185 s, 1.9 MB/s 0x200000bd3:0x81:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317741 s, 3.3 MB/s 0x200000bd3:0x82:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264568 s, 4.0 MB/s 0x200000bd3:0x83:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.31156 s, 3.4 MB/s 0x200000bd3:0x84:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253942 s, 4.1 MB/s 0x200000bd3:0x85:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.341615 s, 3.1 MB/s 0x200000bd3:0x86:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238919 s, 4.4 MB/s 0x200000bd3:0x87:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219085 s, 4.8 MB/s 0x200000bd3:0x88:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.420428 s, 2.5 MB/s 0x200000bd3:0x89:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163668 s, 6.4 MB/s 0x200000bd3:0x8a:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221451 s, 4.7 MB/s 0x200000bd3:0x8b:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277991 s, 3.8 MB/s 0x200000bd3:0x8c:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264523 s, 4.0 MB/s 0x200000bd3:0x8d:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.30349 s, 3.5 MB/s 0x200000bd3:0x8e:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273059 s, 3.8 MB/s 0x200000bd3:0x8f:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.375756 s, 2.8 MB/s 0x200000bd3:0x90:0x0 Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 1s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' ARCHIVE ARCHIVE RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260a (48s) == sanity-hsm test 260b: Restore request have priority over other requests ========================================================== 22:21:55 (1789611715) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240569 s, 4.4 MB/s 0x200000bd3:0x92:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.272551 s, 3.8 MB/s 0x200000bd3:0x93:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250782 s, 4.2 MB/s 0x200000bd3:0x94:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.316277 s, 3.3 MB/s 0x200000bd3:0x95:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250792 s, 4.2 MB/s 0x200000bd3:0x96:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221916 s, 4.7 MB/s 0x200000bd3:0x97:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.263979 s, 4.0 MB/s 0x200000bd3:0x98:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235819 s, 4.4 MB/s 0x200000bd3:0x99:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177161 s, 5.9 MB/s 0x200000bd3:0x9a:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155734 s, 6.7 MB/s 0x200000bd3:0x9b:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247793 s, 4.2 MB/s 0x200000bd3:0x9c:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165304 s, 6.3 MB/s 0x200000bd3:0x9d:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241078 s, 4.3 MB/s 0x200000bd3:0x9e:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21854 s, 4.8 MB/s 0x200000bd3:0x9f:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228895 s, 4.6 MB/s 0x200000bd3:0xa0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250054 s, 4.2 MB/s 0x200000bd3:0xa1:0x0 Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260b (44s) == sanity-hsm test 260c: Requests are not reordered on the 'hot' path of the coordinator ========================================================== 22:22:39 (1789611759) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133549 s, 7.9 MB/s 0x200000bd3:0xa4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187816 s, 5.6 MB/s 0x200000bd3:0xa5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18381 s, 5.7 MB/s 0x200000bd3:0xa6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237701 s, 4.4 MB/s 0x200000bd3:0xa7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27316 s, 3.8 MB/s 0x200000bd3:0xa8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244343 s, 4.3 MB/s 0x200000bd3:0xa9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198464 s, 5.3 MB/s 0x200000bd3:0xaa:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264014 s, 4.0 MB/s 0x200000bd3:0xab:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.308037 s, 3.4 MB/s 0x200000bd3:0xac:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245285 s, 4.3 MB/s 0x200000bd3:0xad:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306513 s, 3.4 MB/s 0x200000bd3:0xae:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218647 s, 4.8 MB/s 0x200000bd3:0xaf:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198656 s, 5.3 MB/s 0x200000bd3:0xb0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317927 s, 3.3 MB/s 0x200000bd3:0xb1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228676 s, 4.6 MB/s 0x200000bd3:0xb2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205684 s, 5.1 MB/s 0x200000bd3:0xb3:0x0 Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 0s mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE RESTORE PASS 260c (52s) == sanity-hsm test 261: Report 0 bytes size after HSM release ========================================================== 22:23:31 (1789611811) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_261/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0182741 s, 448 kB/s /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x00000009) exists archived, archive_id:2 /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x0000000d) released exists archived, archive_id:2 PASS 261 (12s) == sanity-hsm test 262: The client should return 1 block for HSM released files ========================================================== 22:23:43 (1789611823) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_262/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0215579 s, 380 kB/s Waiting 200s for 'SUCCEED' PASS 262 (13s) == sanity-hsm test 300: On disk coordinator state kept between MDT umount/mount ========================================================== 22:23:56 (1789611836) Stop coordinator and remove coordinator state at mount mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown Failing mds1 on oleg348-server Stopping /mnt/lustre-mds1 (opts:) on oleg348-server 22:24:10 (1789611850) shut down facet: mds1 facet_host: oleg348-server facet_failover_host: oleg348-server Failover mds1 to oleg348-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg348-server: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 Started lustre-MDT0000 22:24:32 (1789611872) targets are mounted 22:24:32 (1789611872) facet_failover done oleg348-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Set coordinator start at mount, and start coordinator Failing mds1 on oleg348-server Stopping /mnt/lustre-mds1 (opts:) on oleg348-server 22:25:13 (1789611913) shut down facet: mds1 facet_host: oleg348-server facet_failover_host: oleg348-server Failover mds1 to oleg348-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg348-server: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 Started lustre-MDT0000 22:25:49 (1789611949) targets are mounted 22:25:49 (1789611949) facet_failover done oleg348-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 300 (134s) == sanity-hsm test 301: HSM tunnable are persistent ====== 22:26:11 (1789611971) Failing mds1 on oleg348-server Stopping /mnt/lustre-mds1 (opts:) on oleg348-server 22:26:21 (1789611981) shut down facet: mds1 facet_host: oleg348-server facet_failover_host: oleg348-server Failover mds1 to oleg348-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg348-server: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 Started lustre-MDT0000 22:26:57 (1789612017) targets are mounted 22:26:58 (1789612018) facet_failover done oleg348-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 301 (66s) == sanity-hsm test 302: HSM tunnable are persistent when CDT is off ========================================================== 22:27:17 (1789612037) mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown Failing mds1 on oleg348-server Stopping /mnt/lustre-mds1 (opts:) on oleg348-server 22:27:29 (1789612049) shut down facet: mds1 facet_host: oleg348-server facet_failover_host: oleg348-server Failover mds1 to oleg348-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg348-server: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 Started lustre-MDT0000 22:27:52 (1789612072) targets are mounted 22:27:52 (1789612072) facet_failover done oleg348-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds2 on oleg348-server Stopping /mnt/lustre-mds2 (opts:) on oleg348-server 22:28:06 (1789612086) shut down facet: mds2 facet_host: oleg348-server facet_failover_host: oleg348-server Failover mds2 to oleg348-server mount facets: mds2 Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg348-server: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 Started lustre-MDT0001 22:28:30 (1789612110) targets are mounted 22:28:30 (1789612110) facet_failover done oleg348-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 302 (92s) == sanity-hsm test 400: Single request is sent to the right MDT ========================================================== 22:28:50 (1789612130) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_400/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217062 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195491 s, 5.4 MB/s archive successful on mdt0 archive successful on mdt1 PASS 400 (12s) == sanity-hsm test 401: Compound requests split and sent to their respective MDTs ========================================================== 22:29:01 (1789612141) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_401/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18187 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227969 s, 4.6 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' archive successful on mdt0 archive successful on mdt1 PASS 401 (12s) == sanity-hsm test 402a: Copytool start fails if all MDTs are inactive ========================================================== 22:29:13 (1789612153) deactivate lustre-MDT0000-mdc-ffff9d38c6aac800 on oleg348-client.virtnet deactivate lustre-MDT0001-mdc-ffff9d38c6aac800 on oleg348-client.virtnet deactivate lustre-MDT0000-mdc-ffff9d38d8b9f000 on oleg348-client.virtnet deactivate lustre-MDT0001-mdc-ffff9d38d8b9f000 on oleg348-client.virtnet Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Failed to start copytool agt1 on 'oleg348-client.virtnet' pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 uuid not found in agent list on mds2 activate lustre-MDT0000-mdc-ffff9d38c6aac800 on oleg348-client.virtnet activate lustre-MDT0001-mdc-ffff9d38c6aac800 on oleg348-client.virtnet activate lustre-MDT0000-mdc-ffff9d38d8b9f000 on oleg348-client.virtnet activate lustre-MDT0001-mdc-ffff9d38d8b9f000 on oleg348-client.virtnet PASS 402a (10s) == sanity-hsm test 402b: CDT must retry request upon slow start of CT ========================================================== 22:29:23 (1789612163) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x14d fail_loc=0 Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 402b (22s) == sanity-hsm test 403: Copytool starts with inactive MDT and register on reconnect ========================================================== 22:29:45 (1789612185) deactivate lustre-MDT0001-mdc-ffff9d38c6aac800 on oleg348-client.virtnet deactivate lustre-MDT0001-mdc-ffff9d38d8b9f000 on oleg348-client.virtnet Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_403/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg348-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg348-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 uuid not found in agent list on mds2 oleg348-client.virtnet: 137154 activate lustre-MDT0001-mdc-ffff9d38c6aac800 on oleg348-client.virtnet activate lustre-MDT0001-mdc-ffff9d38d8b9f000 on oleg348-client.virtnet found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=33016cd4-ce36-4e7a-b7c0-5d485d1fec44 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 403 (23s) == sanity-hsm test 404: Inactive MDT does not block requests for active MDTs ========================================================== 22:30:09 (1789612209) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_404/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.334049 s, 3.1 MB/s deactivate lustre-MDT0001-mdc-ffff9d38c6aac800 on oleg348-client.virtnet deactivate lustre-MDT0001-mdc-ffff9d38d8b9f000 on oleg348-client.virtnet archive successful on mdt0 activate lustre-MDT0001-mdc-ffff9d38c6aac800 on oleg348-client.virtnet activate lustre-MDT0001-mdc-ffff9d38d8b9f000 on oleg348-client.virtnet PASS 404 (11s) == sanity-hsm test 405: archive and release under striped directory ========================================================== 22:30:20 (1789612220) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_405/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113144 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104242 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0774058 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854882 s, 12.3 MB/s archive successful on 0x240000bd2:0x1:0x0 archive successful on 0x240000bd2:0x2:0x0 archive successful on 0x240000bd2:0x3:0x0 archive successful on 0x2000032e3:0x6:0x0 PASS 405 (20s) == sanity-hsm test 406: attempting to migrate HSM archived files is safe ========================================================== 22:30:40 (1789612240) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34874 s, 3.0 MB/s old fid 0x2000032e3:0x9:0x0 Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_406/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lfs migrate: /mnt/lustre/d406.sanity-hsm/f406.sanity-hsm migrate failed: Operation not supported (95) lfs migrate: cb_migrate_mdt_fini: error completing migration of /mnt/lustre/d406.sanity-hsm: Directory not empty (39) new fid 0x240000bd3:0x7:0x0 debug=+trace Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 406 (17s) == sanity-hsm test 407: Check for double RESTORE records in llog ========================================================== 22:30:57 (1789612257) 0+0 records in 0+0 records out 0 bytes copied, 0.0064997 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_407/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_val=5 fail_loc=0x164 Copytool is suspended on oleg348-client.virtnet mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/4] fid=[0x2000032e3:0x3:0x0] dfid=[0x2000032e3:0x3:0x0] compound/cookie=0x0/0x6aab5027 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x6aab5028 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x6aab5029 action=RESTORE archive#=3 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6aab502a action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6aab502b action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6aab502c action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/10] fid=[0x2000032e3:0xb:0x0] dfid=[0x2000032e3:0xb:0x0] compound/cookie=0x0/0x6aab502d action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] Failing mds1 on oleg348-server Stopping /mnt/lustre-mds1 (opts:) on oleg348-server 22:31:09 (1789612269) shut down facet: mds1 facet_host: oleg348-server facet_failover_host: oleg348-server Failover mds1 to oleg348-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/4] fid=[0x2000032e3:0x3:0x0] dfid=[0x2000032e3:0x3:0x0] compound/cookie=0x0/0x6aab5027 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x6aab5028 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x6aab5029 action=RESTORE archive#=3 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6aab502a action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6aab502b action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6aab502c action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/10] fid=[0x2000032e3:0xb:0x0] dfid=[0x2000032e3:0xb:0x0] compound/cookie=0x0/0x6aab502d action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/11] fid=[0x2000032e3:0xb:0x0] dfid=[0x2000032e3:0xb:0x0] compound/cookie=0x0/0x6aab502e action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] oleg348-server: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 Started lustre-MDT0000 22:31:42 (1789612302) targets are mounted 22:31:42 (1789612302) facet_failover done oleg348-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec fail_loc=0 mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=6/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6aab502c action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/10] fid=[0x2000032e3:0xb:0x0] dfid=[0x2000032e3:0xb:0x0] compound/cookie=0x0/0x6aab502d action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=6/11] fid=[0x2000032e3:0xb:0x0] dfid=[0x2000032e3:0xb:0x0] compound/cookie=0x0/0x6aab502e action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] Copytool is continued on oleg348-client.virtnet d41d8cd98f00b204e9800998ecf8427e /mnt/lustre/d407.sanity-hsm/f407.sanity-hsm d41d8cd98f00b204e9800998ecf8427e /mnt/lustre2/d407.sanity-hsm/f407.sanity-hsm pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 PASS 407 (62s) == sanity-hsm test 408: Verify fiemap on release file ==== 22:32:00 (1789612320) Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_408/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.106827 s, 613 kB/s SKIP: sanity-hsm test_408 ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 408 (3s) == sanity-hsm test 409a: Coordinator should not stop when in use ========================================================== 22:32:04 (1789612324) 0+0 records in 0+0 records out 0 bytes copied, 0.00986694 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_val=5 fail_loc=0x164 cat: /mnt/lustre/d409a.sanity-hsm/f409a.sanity-hsm: No data available mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 1s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 409a (31s) == sanity-hsm test 409b: getattr released file with CDT stopped after remount ========================================================== 22:32:35 (1789612355) 0+0 records in 0+0 records out 0 bytes copied, 0.00415381 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 1s Failing mds1 on oleg348-server Stopping /mnt/lustre-mds1 (opts:) on oleg348-server 22:33:09 (1789612389) shut down facet: mds1 facet_host: oleg348-server facet_failover_host: oleg348-server Failover mds1 to oleg348-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg348-server: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 Started lustre-MDT0000 22:33:41 (1789612421) targets are mounted 22:33:41 (1789612421) facet_failover done oleg348-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec File: /mnt/lustre/d409b.sanity-hsm/f409b.sanity-hsm Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115406632648720 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-09-16 22:32:37.000000000 -0400 Modify: 2026-09-16 22:32:37.000000000 -0400 Change: 2026-09-16 22:32:37.000000000 -0400 Birth: 2026-09-16 22:32:37.000000000 -0400 PASS 409b (107s) == sanity-hsm test 410: lfs data_version -s allows release of force-archived file ========================================================== 22:34:22 (1789612462) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.399172 s, 2.6 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_410/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d410.sanity-hsm/f410.sanity-hsm): Operation not permitted PASS 410 (9s) == sanity-hsm test 411: hsm_ops rbac role ================ 22:34:32 (1789612472) On MGS 192.168.203.148, active = nodemap.active=1 On MGS 192.168.203.148, test_411.trusted_nodemap = nodemap.test_411.trusted_nodemap=1 Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /mnt/lustre/d411.sanity-hsm/fileA: (0x00000000) Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg348-client.virtnet with signal TERM copytools stopped in 1s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' On MGS 192.168.203.148, test_411.rbac = nodemap.test_411.rbac=file_perms,dne_ops,quota_ops,byfid_ops,chlg_ops,fscrypt_admin,server_upcall,ignore_root_prjquota,immutable_flags,local_admin,pool_quota_ops,lqa_quota_ops,projid_set,foreign_ops /mnt/lustre/d411.sanity-hsm/fileB: (0x00000000) Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied lfs hsm_set: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied lfs hsm_clear: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied On MGS 192.168.203.148, active = nodemap.active=0 oleg348-server: error: nodemap 'test_411' does not exist oleg348-server: nodemap_del: Invalid argument pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 On MGS 192.168.203.148, active = nodemap.active=0 PASS 411 (46s) == sanity-hsm test 500: various LLAPI HSM tests ========== 22:35:17 (1789612517) striped dir -i0 -c2 -H crush /mnt/lustre/d500.sanity-hsm test1 @1789612608: Register/unregister copytool 2000x to check for leaks ...... pass test2 @1789612608: Re/un-register copytool multiple times without error ....... pass llapi_hsm_test: NULL archive numbers llapi_hsm_test: NULL archive numbers llapi_hsm_test: -1 requested when archive id >= 0 is supported test3 @1789612608: Pass bad parameters to llapi_hsm_copytool_register() ....... pass test4 @1789612608: Bad parameters to llapi_hsm_copytool_unregister() .......... pass test5 @1789612608: Test llapi_hsm_copytool_recv() in non-blocking mode ........ pass test6 @1789612608: Test llapi_hsm_copytool_recv() with bogus parameters ....... pass test7 @1789612609: Test event polling (without actual traffic) ................ pass test50 @1789612609: Test llapi_hsm_state_get()/get_fd() ....................... pass test51 @1789612610: Test llapi_hsm_state_set_fd() ............................. pass test52 @1789612610: Test llapi_hsm_current_action() ........................... pass test100 @1789612611: Simple archive creation with no progress reported. ....... pass test101 @1789612618: Simple archive creation with progress every byte. ........ pass test102 @1789612625: Archive creation with progress every byte, backwards ..... pass test103 @1789612626: Archive creation with a single progress report ........... pass test104 @1789612628: Archive creation with two progress reports ............... pass test105 @1789612629: Archive creation with one bogus progress report .......... pass test106 @1789612630: Archive creation with one empty progress report .......... pass test107 @1789612631: Archive creation with one bogus progress report .......... pass test108 @1789612637: Archive creation with same progress report each time ..... pass test109 @1789612639: Archive creation with one report, with large number ...... pass test110 @1789612640: Archive with 10 progress reports, checking progress ...... pass test111 @1789612641: Archive with 10 reports in reverse, checking progress .... pass test112 @1789612642: Archive with 10 reports, duplicated, check progress ...... pass test113 @1789612644: Archive with 9 reports, with 20% overlapping coverage .... pass PASS 500 (133s) == sanity-hsm test 600: Changelog fields 'u=' and 'nid=' ========================================================== 22:37:30 (1789612650) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d600.sanity-hsm/f600.sanity-hsm] Got NID '192.168.203.48@tcp' lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 600 (21s) == sanity-hsm test 601: OPEN Changelog entry ============= 22:37:51 (1789612671) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001.4 10OPEN 02:37:59.867177281 2026.09.17 0x1 t=[0x240000bd3:0x1b:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.48@tcp m=r-- p=[0x200000bd2:0x84:0x0] lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 601 (19s) == sanity-hsm test 602: Changelog record CLOSE only if open+write or OPEN recorded ========================================================== 22:38:10 (1789612690) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl3 cl3' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0000.9 11CLOSE 02:38:19.275155799 2026.09.17 0x1 t=[0x2000032e2:0x1c:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0000.10 10OPEN 02:38:19.889362615 2026.09.17 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.48@tcp m=-w- lustre-MDT0000.11 13TRUNC 02:38:19.897890586 2026.09.17 0xe t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp p=[0x2000032e2:0x1b:0x0] lustre-MDT0000.12 11CLOSE 02:38:19.942648734 2026.09.17 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0000.12 11CLOSE 02:38:19.942648734 2026.09.17 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp mdd.lustre-MDT0000.changelog_mask=-OPEN mdd.lustre-MDT0001.changelog_mask=-OPEN lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records f lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0000.13 13TRUNC 02:38:22.921861095 2026.09.17 0xe t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp p=[0x2000032e2:0x1b:0x0] lustre-MDT0000.14 11CLOSE 02:38:22.952376356 2026.09.17 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0000.14 11CLOSE 02:38:22.952376356 2026.09.17 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001: Deregistered changelog user #3 lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 602 (24s) == sanity-hsm test 603: GETXATTR Changelog entry ========= 22:38:34 (1789612714) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl4 cl4' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0001: clear the changelog for cl4 of all records getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d603.sanity-hsm/f603.sanity-hsm user.xattr1="value1" lustre-MDT0001.11 23GXATR 02:38:43.867748021 2026.09.17 0x0 t=[0x240000bd3:0x1d:0x0] j=getfattr.0 ef=0x1b u=0:0 nid=192.168.203.48@tcp x=user.xattr1 lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0001: Deregistered changelog user #4 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 603 (20s) == sanity-hsm test 604: NOPEN Changelog entry ============ 22:38:54 (1789612734) mdd.lustre-MDT0000.changelog_deniednext=20 mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl5 cl5' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.19 24NOPEN 02:39:04.925858062 2026.09.17 0x1 t=[0x2000032e2:0x1e:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.48@tcp m=r-- p=[0x2000032e2:0x1d:0x0] Got NID '192.168.203.48@tcp' lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.20 24NOPEN 02:39:26.627513550 2026.09.17 0x1 t=[0x2000032e2:0x1e:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.48@tcp m=r-- Got NID '192.168.203.48@tcp' lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0001: Deregistered changelog user #5 lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 mdd.lustre-MDT0000.changelog_deniednext=60 PASS 604 (46s) == sanity-hsm test 605: Test OPEN and CLOSE rate limit in Changelogs ========================================================== 22:39:40 (1789612780) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl6 cl6' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000.24 10OPEN 02:39:50.077021907 2026.09.17 0x43 t=[0x2000032e2:0x20:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.48@tcp m=rw- lustre-MDT0000.24 10OPEN 02:39:50.077021907 2026.09.17 0x43 t=[0x2000032e2:0x20:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.48@tcp m=rw- lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000.25 10OPEN 02:39:51.580231113 2026.09.17 0x1 t=[0x2000032e2:0x20:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.48@tcp m=r-- lustre-MDT0000.26 11CLOSE 02:39:51.604142866 2026.09.17 0x1 t=[0x2000032e2:0x20:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0000.25 10OPEN 02:39:51.580231113 2026.09.17 0x1 t=[0x2000032e2:0x20:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.48@tcp m=r-- lustre-MDT0000.26 11CLOSE 02:39:51.604142866 2026.09.17 0x1 t=[0x2000032e2:0x20:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre/d605.sanity-hsm/f605.sanity-hsm] lustre-MDT0000.27 10OPEN 02:39:53.436092645 2026.09.17 0x1 t=[0x2000032e2:0x20:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.48@tcp m=r-- p=[0x2000032e2:0x19:0x0] lustre-MDT0000.28 11CLOSE 02:39:53.453308623 2026.09.17 0x1 t=[0x2000032e2:0x20:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.203.48@tcp lustre-MDT0000.27 10OPEN 02:39:53.436092645 2026.09.17 0x1 t=[0x2000032e2:0x20:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.48@tcp m=r-- p=[0x2000032e2:0x19:0x0] lustre-MDT0000.28 11CLOSE 02:39:53.453308623 2026.09.17 0x1 t=[0x2000032e2:0x20:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.203.48@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000.29 11CLOSE 02:39:54.356674460 2026.09.17 0x43 t=[0x2000032e2:0x20:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0000.29 11CLOSE 02:39:54.356674460 2026.09.17 0x43 t=[0x2000032e2:0x20:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.48@tcp lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001: Deregistered changelog user #6 lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 605 (26s) == sanity-hsm test 606: llog_reader groks changelog fields ========================================================== 22:40:06 (1789612806) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl7 cl7' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d606.sanity-hsm/f606.sanity-hsm] Stopping /mnt/lustre-mds1 (opts:) on oleg348-server rec #2 type=1064553b len=64 offset 8256 Header size : 8192 llh_size : 64 Time : Wed Sep 16 22:08:11 2026 Number of records: 1 cat_idx: 0 last_idx: 2 Flags: 0x2012 Target uuid : ----------------------- #02 (064)fid=[0x1:0x3b:0x0] path=oi.1/0x1:0x3b:0x0 Got UID/GID 500:500 Got NID 192.168.203.48@tcp Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg348-server: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg348-client: oleg348-server: ssh exited with exit code 1 Started lustre-MDT0000 lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0001: Deregistered changelog user #7 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 606 (59s) == sanity-hsm test 607a: release file migrated after archive (non-blocking migrate) ========================================================== 22:41:06 (1789612866) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.363571 s, 2.9 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 607a (13s) == sanity-hsm test 607b: Migrate should not change the HSM attribute of dirty files ========================================================== 22:41:19 (1789612879) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.396165 s, 2.6 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' llite.lustre-ffff9d38c6aac800.xattr_cache=0 llite.lustre-ffff9d38d8b9f000.xattr_cache=0 /mnt/lustre/d607b.sanity-hsm/f607b.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 llite.lustre-ffff9d38c6aac800.xattr_cache=1 llite.lustre-ffff9d38d8b9f000.xattr_cache=1 PASS 607b (10s) == sanity-hsm test 607c: 'lfs swap_layouts' should set dirty flag on HSM file ========================================================== 22:41:29 (1789612889) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.445426 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.502931 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.416853 s, 2.5 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-1' and '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-3': Operation not permitted (1) PASS 607c (16s) == sanity-hsm test 607d: Migrate should not set HSM dirty flag for an empty file ========================================================== 22:41:45 (1789612905) 0+0 records in 0+0 records out 0 bytes copied, 0.00773244 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 607d (11s) == sanity-hsm test 607e: release file migrated after archive (blocking migrate) ========================================================== 22:41:56 (1789612916) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.397319 s, 2.6 MB/s Starting copytool 'agt1' on 'oleg348-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 607e (16s) == sanity-hsm test complete, duration 5326 sec =========== 22:42:12 (1789612932) === sanity-hsm: start cleanup 22:42:14 (1789612934) === Stopping clients: oleg348-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg348-client.virtnet /mnt/lustre2 opts: === sanity-hsm: finish cleanup 22:42:57 (1789612977) ===