-----============= acceptance-small: sanity-hsm ============----- Wed Sep 30 12:08:38 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg352-server: /home/green/git/lustre-release/lustre/tests/except/sanity-hsm.ex - see MDS1_VERSION <= CLIENT_VERSION (34683432 <= 34683432) for LU-18924, go 100 105 excepting tests: === sanity-hsm: start setup 12:08:57 (1790784537) === Starting client oleg352-client.virtnet: -o user_xattr,flock 192.168.203.152@tcp:/lustre /mnt/lustre2 Mount client oleg352-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.203.152@tcp:/lustre /mnt/lustre2 Started clients oleg352-client.virtnet: 192.168.203.152@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) oleg352-client.virtnet: executing check_config_client /mnt/lustre oleg352-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg352-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff91728339d800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9172a0093800.idle_timeout=debug osc.lustre-OST0001-osc-ffff91728339d800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9172a0093800.idle_timeout=debug disable quota as required oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity-hsm: finish setup 12:09:37 (1790784577) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f8452] nobody:x:65534: Killing existing copytools on oleg352-client.virtnet with signal TERM pkill: pidfile not valid Try `pkill --help' for more information. Set HSM on and start Set sanity-hsm HSM policy == sanity-hsm test 1A: lfs hsm flags root/non-root access ========================================================== 12:10:06 (1790784606) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_clear] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--exists] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] lfs hsm_set: change hsm flags for '/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm' failed: Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] PASS 1A (9s) == sanity-hsm test 1a: mmap & cat a HSM released file ==== 12:10:15 (1790784615) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.286738 s, 3.7 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Verifying released state: PASS 1a (11s) == sanity-hsm test 1b: Archive, Release and Restore composite file ========================================================== 12:10:26 (1790784626) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0574685 s, 18.2 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm release /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm verify released state: pass restore /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm verify restored state: pass PASS 1b (11s) == sanity-hsm test 1c: Check setting archive-id in lfs hsm_set ========================================================== 12:10:37 (1790784637) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is still 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 33 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 16 PASS 1c (10s) == sanity-hsm test 1d: Archive, Release and Restore DoM file ========================================================== 12:10:47 (1790784647) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100986 s, 10.4 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm release /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm verify released state: pass restore /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm verify restored state: pass PASS 1d (13s) == sanity-hsm test 1e: Archive, Release and Restore SEL file ========================================================== 12:11:00 (1790784660) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0497713 s, 21.1 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm release /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm verify released state: pass restore /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass PASS 1e (14s) == sanity-hsm test 1f: DoM file release after restore ==== 12:11:14 (1790784674) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669726 s, 15.7 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm release /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm verify released state: pass restore /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass release again /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 verify released state: pass PASS 1f (14s) == sanity-hsm test 2: Check file dirtyness when doing setattr ========================================================== 12:11:28 (1790784688) 0+0 records in 0+0 records out 0 bytes copied, 0.00602797 s, 0.0 kB/s 0x200000403:0x8:0x0 PASS 2 (8s) == sanity-hsm test 3: Check file dirtyness when opening for write ========================================================== 12:11:36 (1790784696) Succeed in opening file "/mnt/lustre/d3.sanity-hsm/f3.sanity-hsm"(flags=O_WRONLY) 3+0 records in 3+0 records out 3+0 records in 3+0 records out PASS 3 (10s) == sanity-hsm test 4: Useless cancel must not be registered ========================================================== 12:11:46 (1790784706) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.246709 s, 4.3 MB/s PASS 4 (10s) == sanity-hsm test 8: Test default archive number ======== 12:11:57 (1790784717) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_8/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 8 (11s) == sanity-hsm test 9A: Use of explicit archive number, with dedicated copytool ========================================================== 12:12:08 (1790784728) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_9A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 3 "/mnt/lustre2"' oleg352-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg352-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=3 requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=3 requests=[current:0 ok:0 errors:0] on mds2 PASS 9A (29s) == sanity-hsm test 9a: Multiple remote agents ============ 12:12:37 (1790784757) SKIP: sanity-hsm test_9a Need 3 or more clients, have 1 SKIP 9a (4s) == sanity-hsm test 10a: Archive a file =================== 12:12:41 (1790784761) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 996: warning: command substitution: ignored null byte in input Verifying content Verifying hsm state Verifying archive number is 2 PASS 10a (10s) == sanity-hsm test 10b: Archive of non dirty file must work without doing request ========================================================== 12:12:51 (1790784771) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 10b (11s) == sanity-hsm test 10c: Check forbidden archive ========== 12:13:03 (1790784783) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d10c.sanity-hsm/f10c.sanity-hsm): Operation not permitted PASS 10c (7s) == sanity-hsm test 10d: Archive a file on the default archive id ========================================================== 12:13:09 (1790784789) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 10d (9s) == sanity-hsm test 11a: Import a file ==================== 12:13:19 (1790784799) lhsmtool_posix: 1790784801.117624 [22268]: action=1 src=d11a.sanity-hsm/f11a.sanity-hsm dst=/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790784801.122315 [22268]: importing '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1790784801.155830 [22268]: imported '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//0424/0x200000402:0x26:0x0'=='/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1790784801.155891 [22268]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying imported size 158=158 Verifying released pattern: raid0,released Verifying new fid 0x200000402:0x26:0x0 in archive PASS 11a (7s) == sanity-hsm test 11b: Import a deleted file using its FID ========================================================== 12:13:26 (1790784806) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_11b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790784810.478640 [22959]: action=1 src=0x200000403:0x11:0x0 dst=/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790784810.495103 [22959]: Resolving [0x200000403:0x11:0x0] to /tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0 lhsmtool_posix: 1790784810.495222 [22959]: importing '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0' lhsmtool_posix: 1790784810.584056 [22959]: imported '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//042a/0x200000402:0x28:0x0'=='/tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0' lhsmtool_posix: 1790784810.584957 [22959]: process finished, errs: 0 major, 0 minor, rc=0 (Success) /mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm: OK PASS 11b (10s) == sanity-hsm test 11c: Import a file to a directory with a pool ========================================================== 12:13:36 (1790784816) Creating new pool test_11c oleg352-server: Pool lustre.test_11c created Adding targets to pool oleg352-server: OST lustre-OST0001_UUID added to pool lustre.test_11c lhsmtool_posix: 1790784826.362223 [23675]: action=1 src=d11c.sanity-hsm/f11c.sanity-hsm dst=/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790784826.367334 [23675]: importing '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1790784826.399412 [23675]: imported '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//0405/0x240000402:0x7:0x0'=='/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1790784826.399468 [23675]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Destroy the created pools: test_11c lustre.test_11c oleg352-server: OST lustre-OST0001_UUID removed from pool lustre.test_11c oleg352-server: Pool lustre.test_11c destroyed PASS 11c (24s) == sanity-hsm test 12a: Restore an imported file explicitly ========================================================== 12:14:00 (1790784840) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790784842.586555 [24412]: action=1 src=d12a.sanity-hsm/f12a.sanity-hsm dst=/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790784842.610988 [24412]: importing '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1790784842.673185 [24412]: imported '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//0428/0x200000402:0x2a:0x0'=='/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1790784842.673931 [24412]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying file state: PASS 12a (8s) == sanity-hsm test 12b: Restore an imported file implicitly ========================================================== 12:14:08 (1790784848) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790784851.561662 [25119]: action=1 src=d12b.sanity-hsm/f12b.sanity-hsm dst=/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790784851.575677 [25119]: importing '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1790784851.642585 [25119]: imported '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//042e/0x200000402:0x2c:0x0'=='/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1790784851.643658 [25119]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying file state after restore: PASS 12b (10s) == sanity-hsm test 12c: Restore a file with stripe of 2 == 12:14:18 (1790784858) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.119928 s, 43.7 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d12c.sanity-hsm/f12c.sanity-hsm: OK PASS 12c (11s) == sanity-hsm test 12d: Restore of a non archived, non released file must work ========================================================== 12:14:29 (1790784869) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12d (9s) == sanity-hsm test 12e: Check forbidden restore ========== 12:14:38 (1790784878) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 Cannot send HSM request (use of /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm): Operation not permitted PASS 12e (10s) == sanity-hsm test 12f: Restore a released file explicitly ========================================================== 12:14:48 (1790784888) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying file state: PASS 12f (11s) == sanity-hsm test 12g: Restore a released file implicitly ========================================================== 12:14:59 (1790784899) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12g (10s) == sanity-hsm test 12h: Restore a released file implicitly from a second node ========================================================== 12:15:09 (1790784909) SKIP: sanity-hsm test_12h Need 2 or more clients, have 1 SKIP 12h (3s) == sanity-hsm test 12m: Archive/release/implicit restore ========================================================== 12:15:13 (1790784913) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12m/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' PASS 12m (11s) == sanity-hsm test 12n: Import/implicit restore/release == 12:15:23 (1790784923) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12n/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790784925.732248 [29978]: action=1 src=d12n.sanity-hsm/f12n.sanity-hsm dst=/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790784925.737235 [29978]: importing '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1790784925.781225 [29978]: imported '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//0438/0x200000402:0x3a:0x0'=='/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1790784925.786041 [29978]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12n (8s) == sanity-hsm test 12o: Layout-swap failure during Restore leaves file released ========================================================== 12:15:32 (1790784932) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12o/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' fail_loc=0x152 mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA diff: /mnt/lustre/d12o.sanity-hsm/f12o.sanity-hsm: No data available mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA fail_loc=0 mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 12o (37s) == sanity-hsm test 12p: implicit restore of a file on copytool mount point ========================================================== 12:16:10 (1790784970) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12p/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 12p (11s) == sanity-hsm test 12q: file attributes are refreshed after restore ========================================================== 12:16:22 (1790784982) Starting client: oleg352-client.virtnet: -o user_xattr,flock 192.168.203.152@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13745 s, 7.6 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Stopping client oleg352-client.virtnet /mnt/lustre3 (opts:) PASS 12q (21s) == sanity-hsm test 12r: lseek restores released file ===== 12:16:42 (1790785002) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12r/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12r (11s) == sanity-hsm test 12s: race between restore requests ==== 12:16:53 (1790785013) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12s/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x8000018b PASS 12s (15s) == sanity-hsm test 12t: Multiple parallel reads for a HSM imported file ========================================================== 12:17:08 (1790785028) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12t/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lhsmtool_posix: 1790785033.489853 [34146]: action=1 src=0x200000403:0x2b:0x0 dst=/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790785033.501999 [34146]: Resolving [0x200000403:0x2b:0x0] to /tmp/arc1/sanity-hsm.test_12t//0428/0x200000403:0x2b:0x0 lhsmtool_posix: 1790785033.507174 [34146]: importing '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//0428/0x200000403:0x2b:0x0' lhsmtool_posix: 1790785033.554376 [34146]: imported '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//044a/0x200000402:0x48:0x0'=='/tmp/arc1/sanity-hsm.test_12t//0428/0x200000403:0x2b:0x0' lhsmtool_posix: 1790785033.554453 [34146]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12t (12s) == sanity-hsm test 12u: Multiple reads on multiple HSM imported files in parallel ========================================================== 12:17:20 (1790785040) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12u/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790785057.126185 [35020]: action=1 src=0x200000403:0x2d:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0 mount_point=/mnt/lustre2 lhsmtool_posix: 1790785057.142255 [35020]: Resolving [0x200000403:0x2d:0x0] to /tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0 lhsmtool_posix: 1790785057.142342 [35020]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0' lhsmtool_posix: 1790785057.210971 [35020]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//0448/0x200000402:0x4a:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0' lhsmtool_posix: 1790785057.211029 [35020]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790785057.520216 [35049]: action=1 src=0x200000403:0x2e:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1 mount_point=/mnt/lustre2 lhsmtool_posix: 1790785057.546054 [35049]: Resolving [0x200000403:0x2e:0x0] to /tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0 lhsmtool_posix: 1790785057.546141 [35049]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0' lhsmtool_posix: 1790785057.593648 [35049]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//0449/0x200000402:0x4b:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0' lhsmtool_posix: 1790785057.593714 [35049]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790785057.911743 [35078]: action=1 src=0x200000403:0x2f:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2 mount_point=/mnt/lustre2 lhsmtool_posix: 1790785057.916687 [35078]: Resolving [0x200000403:0x2f:0x0] to /tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0 lhsmtool_posix: 1790785057.916818 [35078]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0' lhsmtool_posix: 1790785057.948329 [35078]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//044e/0x200000402:0x4c:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0' lhsmtool_posix: 1790785057.948384 [35078]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790785058.263000 [35107]: action=1 src=0x200000403:0x30:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3 mount_point=/mnt/lustre2 lhsmtool_posix: 1790785058.270483 [35107]: Resolving [0x200000403:0x30:0x0] to /tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0 lhsmtool_posix: 1790785058.270570 [35107]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0' lhsmtool_posix: 1790785058.321346 [35107]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//044f/0x200000402:0x4d:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0' lhsmtool_posix: 1790785058.321962 [35107]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790785058.642264 [35136]: action=1 src=0x200000403:0x31:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4 mount_point=/mnt/lustre2 lhsmtool_posix: 1790785058.654786 [35136]: Resolving [0x200000403:0x31:0x0] to /tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0 lhsmtool_posix: 1790785058.654865 [35136]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0' lhsmtool_posix: 1790785058.736499 [35136]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//044c/0x200000402:0x4e:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0' lhsmtool_posix: 1790785058.736640 [35136]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790785059.127603 [35165]: action=1 src=0x200000403:0x32:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5 mount_point=/mnt/lustre2 lhsmtool_posix: 1790785059.138269 [35165]: Resolving [0x200000403:0x32:0x0] to /tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0 lhsmtool_posix: 1790785059.138334 [35165]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0' lhsmtool_posix: 1790785059.238118 [35165]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//044d/0x200000402:0x4f:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0' lhsmtool_posix: 1790785059.238182 [35165]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790785059.593505 [35194]: action=1 src=0x200000403:0x33:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6 mount_point=/mnt/lustre2 lhsmtool_posix: 1790785059.598734 [35194]: Resolving [0x200000403:0x33:0x0] to /tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0 lhsmtool_posix: 1790785059.601005 [35194]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0' lhsmtool_posix: 1790785059.647257 [35194]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0452/0x200000402:0x50:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0' lhsmtool_posix: 1790785059.649466 [35194]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790785059.967964 [35223]: action=1 src=0x200000403:0x34:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7 mount_point=/mnt/lustre2 lhsmtool_posix: 1790785059.975098 [35223]: Resolving [0x200000403:0x34:0x0] to /tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0 lhsmtool_posix: 1790785059.978748 [35223]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0' lhsmtool_posix: 1790785060.033146 [35223]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0453/0x200000402:0x51:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0' lhsmtool_posix: 1790785060.033197 [35223]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790785060.395174 [35251]: action=1 src=0x200000403:0x35:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8 mount_point=/mnt/lustre2 lhsmtool_posix: 1790785060.406040 [35251]: Resolving [0x200000403:0x35:0x0] to /tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0 lhsmtool_posix: 1790785060.406118 [35251]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0' lhsmtool_posix: 1790785060.523935 [35251]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0450/0x200000402:0x52:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0' lhsmtool_posix: 1790785060.523990 [35251]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790785060.930277 [35281]: action=1 src=0x200000403:0x36:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9 mount_point=/mnt/lustre2 lhsmtool_posix: 1790785060.938480 [35281]: Resolving [0x200000403:0x36:0x0] to /tmp/arc1/sanity-hsm.test_12u//0435/0x200000403:0x36:0x0 lhsmtool_posix: 1790785060.938574 [35281]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0435/0x200000403:0x36:0x0' lhsmtool_posix: 1790785061.005362 [35281]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0451/0x200000402:0x53:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0435/0x200000403:0x36:0x0' lhsmtool_posix: 1790785061.006072 [35281]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12u (30s) == sanity-hsm test 13: Recursively import and restore a directory ========================================================== 12:17:50 (1790785070) lhsmtool_posix: 1790785078.608242 [37042]: action=1 src=subdir dst=/mnt/lustre/d13.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790785078.889362 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1790785078.970682 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//0457/0x200000402:0x55:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1790785078.971432 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1790785079.003303 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//0454/0x200000402:0x56:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1790785079.004027 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1790785079.033330 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//0455/0x200000402:0x57:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1790785079.033899 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1790785079.068333 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//045a/0x200000402:0x58:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1790785079.068532 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1790785079.114201 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//045b/0x200000402:0x59:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1790785079.115076 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1790785079.172218 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//0458/0x200000402:0x5a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1790785079.172494 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1790785079.223661 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//0459/0x200000402:0x5b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1790785079.224076 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1790785079.276031 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//045e/0x200000402:0x5c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1790785079.276282 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1790785079.341193 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//045f/0x200000402:0x5d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1790785079.341503 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1790785079.397745 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//045c/0x200000402:0x5e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1790785079.408242 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1790785079.477345 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//040c/0x240000402:0xe:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1790785079.477596 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1790785079.528953 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//040d/0x240000402:0xf:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1790785079.529196 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1790785079.581074 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//0412/0x240000402:0x10:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1790785079.581570 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1790785079.625262 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//0413/0x240000402:0x11:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1790785079.630731 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1790785079.671985 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//0410/0x240000402:0x12:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1790785079.672230 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1790785079.729556 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//0411/0x240000402:0x13:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1790785079.730031 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1790785079.774715 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//0416/0x240000402:0x14:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1790785079.775629 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1790785079.813067 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//0417/0x240000402:0x15:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1790785079.813711 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1790785079.882683 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//0414/0x240000402:0x16:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1790785079.884220 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1790785079.959868 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//0415/0x240000402:0x17:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1790785079.966292 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1790785080.019714 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//041b/0x240000402:0x19:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1790785080.020562 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1790785080.072974 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//0418/0x240000402:0x1a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1790785080.073214 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1790785080.159986 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//0419/0x240000402:0x1b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1790785080.160297 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1790785080.219066 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//041e/0x240000402:0x1c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1790785080.219307 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1790785080.290217 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//041f/0x240000402:0x1d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1790785080.290499 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1790785080.409098 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//041c/0x240000402:0x1e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1790785080.413113 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1790785080.473844 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//041d/0x240000402:0x1f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1790785080.474081 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1790785080.555291 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//0422/0x240000402:0x20:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1790785080.555514 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1790785080.641174 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//0423/0x240000402:0x21:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1790785080.641415 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1790785080.711407 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//0420/0x240000402:0x22:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1790785080.728665 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1790785080.969558 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//0462/0x200000402:0x60:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1790785080.971207 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1790785081.040274 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//0463/0x200000402:0x61:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1790785081.041088 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1790785081.118735 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//0460/0x200000402:0x62:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1790785081.119037 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1790785081.216735 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//0461/0x200000402:0x63:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1790785081.217270 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1790785081.318737 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//0466/0x200000402:0x64:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1790785081.319341 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1790785081.410546 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//0467/0x200000402:0x65:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1790785081.414408 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1790785081.497992 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//0464/0x200000402:0x66:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1790785081.499108 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1790785081.572102 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//0465/0x200000402:0x67:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1790785081.572345 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1790785081.612582 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//046a/0x200000402:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1790785081.613974 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1790785081.658064 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//046b/0x200000402:0x69:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1790785081.672863 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1790785081.800134 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//0426/0x240000402:0x24:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1790785081.804099 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1790785081.864480 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//0427/0x240000402:0x25:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1790785081.870470 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1790785081.940043 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//0424/0x240000402:0x26:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1790785081.940323 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1790785082.001328 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//0425/0x240000402:0x27:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1790785082.001600 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1790785082.053719 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//042a/0x240000402:0x28:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1790785082.054081 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1790785082.160189 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//042b/0x240000402:0x29:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1790785082.162859 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1790785082.235226 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//0428/0x240000402:0x2a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1790785082.235864 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1790785082.285578 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//0429/0x240000402:0x2b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1790785082.286223 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1790785082.367071 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//042e/0x240000402:0x2c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1790785082.367284 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1790785082.446130 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//042f/0x240000402:0x2d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1790785082.455502 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1790785082.513650 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//042d/0x240000402:0x2f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1790785082.514273 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1790785082.598944 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//0432/0x240000402:0x30:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1790785082.599207 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1790785082.677636 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//0433/0x240000402:0x31:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1790785082.681205 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1790785082.726509 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//0430/0x240000402:0x32:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1790785082.726836 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1790785082.835213 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//0431/0x240000402:0x33:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1790785082.835478 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1790785082.937535 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//0436/0x240000402:0x34:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1790785082.938348 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1790785083.097683 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//0437/0x240000402:0x35:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1790785083.098002 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1790785083.190725 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//0434/0x240000402:0x36:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1790785083.191053 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1790785083.284505 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//0435/0x240000402:0x37:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1790785083.284714 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1790785083.334416 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//043a/0x240000402:0x38:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1790785083.360194 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1790785083.516864 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//0469/0x200000402:0x6b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1790785083.517044 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1790785083.574308 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//046e/0x200000402:0x6c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1790785083.575311 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1790785083.673417 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//046f/0x200000402:0x6d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1790785083.673626 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1790785083.739455 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//046c/0x200000402:0x6e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1790785083.739721 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1790785083.842028 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//046d/0x200000402:0x6f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1790785083.842298 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1790785083.956456 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//0472/0x200000402:0x70:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1790785083.956733 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1790785084.056141 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//0473/0x200000402:0x71:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1790785084.056407 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1790785084.152997 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//0470/0x200000402:0x72:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1790785084.153262 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1790785084.246674 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//0471/0x200000402:0x73:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1790785084.247022 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1790785084.390700 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//0476/0x200000402:0x74:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1790785084.413956 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1790785084.551116 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//0438/0x240000402:0x3a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1790785084.553191 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1790785084.660117 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//0439/0x240000402:0x3b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1790785084.663019 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1790785084.775243 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//043e/0x240000402:0x3c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1790785084.776569 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1790785084.914969 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//043f/0x240000402:0x3d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1790785084.915617 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1790785084.986672 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//043c/0x240000402:0x3e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1790785084.988105 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1790785085.049641 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//043d/0x240000402:0x3f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1790785085.050143 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1790785085.113036 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//0442/0x240000402:0x40:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1790785085.114227 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1790785085.164660 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//0443/0x240000402:0x41:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1790785085.165566 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1790785085.264222 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//0440/0x240000402:0x42:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1790785085.264535 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1790785085.356696 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//0441/0x240000402:0x43:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1790785085.396529 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1790785085.533929 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//0474/0x200000402:0x76:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1790785085.534108 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1790785085.591382 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//0475/0x200000402:0x77:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1790785085.591751 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1790785085.659366 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//047a/0x200000402:0x78:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1790785085.661178 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1790785085.747283 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//047b/0x200000402:0x79:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1790785085.754455 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1790785085.815842 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//0478/0x200000402:0x7a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1790785085.816034 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1790785085.869930 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//0479/0x200000402:0x7b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1790785085.870110 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1790785085.898290 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//047e/0x200000402:0x7c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1790785085.898498 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1790785085.926699 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//047f/0x200000402:0x7d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1790785085.926935 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1790785085.954234 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//047c/0x200000402:0x7e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1790785085.954446 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1790785085.986357 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//047d/0x200000402:0x7f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1790785085.992497 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1790785086.024469 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//0447/0x240000402:0x45:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1790785086.024669 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1790785086.065621 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//0444/0x240000402:0x46:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1790785086.065862 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1790785086.091822 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//0445/0x240000402:0x47:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1790785086.092010 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1790785086.119456 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//044a/0x240000402:0x48:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1790785086.120306 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1790785086.158306 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//044b/0x240000402:0x49:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1790785086.158513 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1790785086.191204 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//0448/0x240000402:0x4a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1790785086.191878 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1790785086.219096 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//0449/0x240000402:0x4b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1790785086.219283 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1790785086.249305 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//044e/0x240000402:0x4c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1790785086.249505 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1790785086.280170 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//044f/0x240000402:0x4d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1790785086.281227 [37042]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1790785086.320144 [37042]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//044c/0x240000402:0x4e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1790785086.321196 [37042]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_13/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 13 (34s) == sanity-hsm test 14: Rebind archived file to a new fid ========================================================== 12:18:24 (1790785104) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_14/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145895 s, 7.2 MB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00449535 s, 0.0 kB/s rebind 0x200000403:0x69:0x0 to 0x200000403:0x6a:0x0 lhsmtool_posix: 1790785108.528242 [38006]: action=2 src=0x200000403:0x69:0x0 dst=0x200000403:0x6a:0x0 mount_point=/mnt/lustre2 lhsmtool_posix: 1790785108.530674 [38006]: rebind [0x200000403:0x69:0x0] to [0x200000403:0x6a:0x0] lhsmtool_posix: 1790785108.531285 [38006]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 14 (10s) == sanity-hsm test 15: Rebind a list of files ============ 12:18:34 (1790785114) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_15/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255585 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128061 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217764 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.282982 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195015 s, 5.4 MB/s pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 0+0 records in 0+0 records out 0 bytes copied, 0.0120785 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00404043 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00671498 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00496488 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00382924 s, 0.0 kB/s rebind list of files lhsmtool_posix: 1790785121.219794 [38792]: action=2 src=/tmp/tmp.8452 dst=(null) mount_point=/mnt/lustre2 lhsmtool_posix: 1790785121.221134 [38792]: rebind [0x200000403:0x6c:0x0] to [0x200000403:0x71:0x0] lhsmtool_posix: 1790785121.222139 [38792]: rebind [0x200000403:0x6d:0x0] to [0x200000403:0x72:0x0] lhsmtool_posix: 1790785121.222451 [38792]: rebind [0x200000403:0x6e:0x0] to [0x200000403:0x73:0x0] lhsmtool_posix: 1790785121.222749 [38792]: rebind [0x200000403:0x6f:0x0] to [0x200000403:0x74:0x0] lhsmtool_posix: 1790785121.223118 [38792]: rebind [0x200000403:0x70:0x0] to [0x200000403:0x75:0x0] lhsmtool_posix: 1790785121.223513 [38792]: 5 lines read from '/tmp/tmp.8452', 5 rebind successful lhsmtool_posix: 1790785121.223972 [38792]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 15 (18s) == sanity-hsm test 16: Test CT bandwith control option === 12:18:52 (1790785132) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_16/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 0.0551128 s, 381 MB/s Waiting 200s for 'SUCCEED' Waiting 190s for 'SUCCEED' Updated after 20s: want 'SUCCEED' got 'SUCCEED' PASS 16 (28s) == sanity-hsm test 20: Release is not permitted ========== 12:19:20 (1790785160) 0+0 records in 0+0 records out 0 bytes copied, 0.010094 s, 0.0 kB/s 0x240000403:0x41:0x0 Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted PASS 20 (7s) == sanity-hsm test 21: Simple release tests ============== 12:19:27 (1790785167) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_21/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201348 s, 5.2 MB/s 1+0 records in 1+0 records out 512 bytes copied, 0.159929 s, 3.2 kB/s PASS 21 (12s) == sanity-hsm test 22: Could not swap a release file ===== 12:19:40 (1790785180) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_22/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199661 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267806 s, 3.9 MB/s 0x200000403:0x7f:0x0 lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d22.sanity-hsm/test_swap' and '/mnt/lustre/d22.sanity-hsm/test_release': Operation not supported (95) PASS 22 (11s) == sanity-hsm test 23: Release does not change a/mtime (utime) ========================================================== 12:19:50 (1790785190) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_23/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270589 s, 3.9 MB/s PASS 23 (11s) == sanity-hsm test 24a: Archive, release, and restore does not change a/mtime (i/o) ========================================================== 12:20:01 (1790785201) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227855 s, 4.6 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s 192.168.203.152@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg352-client.virtnet /mnt/lustre (opts:) Starting client: oleg352-client.virtnet: -o user_xattr,flock 192.168.203.152@tcp:/lustre /mnt/lustre PASS 24a (20s) == sanity-hsm test 24b: root can archive, release, and restore user files ========================================================== 12:20:21 (1790785221) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214969 s, 4.9 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] /mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm: (0x00000009) exists archived, archive_id:1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [md5sum] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] PASS 24b (14s) == sanity-hsm test 24c: check that user,group,other request masks work ========================================================== 12:20:35 (1790785235) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.261633 s, 4.0 MB/s 0x200000403:0x88:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240376 s, 4.4 MB/s 0x200000403:0x89:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162166 s, 6.5 MB/s 0x200000403:0x8a:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] PASS 24c (24s) == sanity-hsm test 24d: check that read-only mounts are respected ========================================================== 12:20:59 (1790785259) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19476 s, 5.4 MB/s 0x200000403:0x8b:0x0 /mnt/lustre/d24d.sanity-hsm/f24d.sanity-hsm lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 76 0x4c 0x2c0000401 Starting client: oleg352-client.virtnet: -o user_xattr,flock 192.168.203.152@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.79048 s, 1.3 MB/s Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Stopping client oleg352-client.virtnet /mnt/lustre3 (opts:) PASS 24d (17s) == sanity-hsm test 24e: tar succeeds on HSM released files ========================================================== 12:21:16 (1790785276) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291977 s, 3.6 MB/s /mnt/lustre/d24e.sanity-hsm/f24e.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 tar: Removing leading `/' from member names PASS 24e (9s) == sanity-hsm test 24f: root can archive, release, and restore tar files ========================================================== 12:21:25 (1790785285) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm f24f.sanity-hsm f24f.sanity-hsm Sum0 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm, sum1 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm PASS 24f (9s) == sanity-hsm test 24g: write by non-owner still sets dirty ========================================================== 12:21:34 (1790785294) RUNAS = 'runas -u 500 -g 500' Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Please listen carefully as our options have changed. running as uid/gid/euid/egid 500/500/500/500, groups: 500 [tee] [/mnt/lustre/d24g.sanity-hsm/f24g.sanity-hsm] To be electrocuted by your telephone, press #. PASS 24g (7s) == sanity-hsm test 25a: Restore lost file (HS_LOST flag) from import ========================================================== 12:21:41 (1790785301) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790785303.801283 [47802]: action=1 src=d25a.sanity-hsm/f25a.sanity-hsm dst=/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790785303.809351 [47802]: importing '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1790785303.872866 [47802]: imported '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//0406/0x240000404:0x2:0x0'=='/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1790785303.872923 [47802]: process finished, errs: 0 major, 0 minor, rc=0 (Success) md5sum: /mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm: No data available PASS 25a (8s) == sanity-hsm test 25b: Restore lost file (HS_LOST flag) after release ========================================================== 12:21:49 (1790785309) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' md5sum: /mnt/lustre/d25b.sanity-hsm/f25b.sanity-hsm: No data available PASS 25b (9s) == sanity-hsm test 26A: Remove the archive of a valid file ========================================================== 12:21:58 (1790785318) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.0027683 s, 0.0 kB/s PASS 26A (9s) == sanity-hsm test 26a: Remove Archive On Last Unlink (RAoLU) policy ========================================================== 12:22:07 (1790785327) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 5s: want 'SUCCEED' got 'SUCCEED' PASS 26a (32s) == sanity-hsm test 26b: RAoLU policy when CDT off ======== 12:22:39 (1790785359) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' PASS 26b (26s) == sanity-hsm test 26c: RAoLU effective when file closed ========================================================== 12:23:05 (1790785385) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' multiop /mnt/lustre/d26c.sanity-hsm/f26c.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8452 Waiting 200s for 'SUCCEED' Updated after 5s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 10s: want 'SUCCEED' got 'SUCCEED' PASS 26c (44s) == sanity-hsm test 26d: RAoLU when Client eviction ======= 12:23:49 (1790785429) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254278 s, 4.1 MB/s multiop /mnt/lustre/d26d.sanity-hsm/f26d.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8452 Waiting 200s for 'SUCCEED' Updated after 4s: want 'SUCCEED' got 'SUCCEED' PASS 26d (30s) == sanity-hsm test 26e: RAoLU with a non-started coordinator ========================================================== 12:24:19 (1790785459) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.290765 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23622 s, 4.4 MB/s Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s Failing mds1 on oleg352-server Stopping /mnt/lustre-mds1 (opts:) on oleg352-server 12:24:55 (1790785495) shut down facet: mds1 facet_host: oleg352-server facet_failover_host: oleg352-server Failover mds1 to oleg352-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Started lustre-MDT0000 12:25:27 (1790785527) targets are mounted 12:25:28 (1790785528) facet_failover done oleg352-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Check cookie from RAoLU request (last: 1790784736, remove: 1790784737) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 26e (122s) == sanity-hsm test 27a: Remove the archive of an imported file (Operation not permitted) ========================================================== 12:26:22 (1790785582) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.65752 s, 24.7 MB/s lhsmtool_posix: 1790785587.827060 [54894]: action=1 src=d27a.sanity-hsm/f27a.sanity-hsm dst=/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790785587.844588 [54894]: importing '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1790785588.003960 [54894]: imported '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//0400/0x240000404:0x4:0x0'=='/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1790785588.004509 [54894]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Cannot send HSM request (use of /mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm): Operation not permitted PASS 27a (12s) == sanity-hsm test 27b: Remove the archive of a relased file (Operation not permitted) ========================================================== 12:26:34 (1790785594) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00547555 s, 0.0 kB/s Cannot send HSM request (use of /mnt/lustre/d27b.sanity-hsm/f27b.sanity-hsm): Operation not permitted PASS 27b (11s) == sanity-hsm test 28: Concurrent archive/file remove ==== 12:26:46 (1790785606) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_28/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00766215 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 28 (20s) == sanity-hsm test 29a: Tests --mntpath and --archive options ========================================================== 12:27:06 (1790785626) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 7 "/mnt/lustre"' Cannot send HSM request (use of /mnt/lustre): Invalid argument hsm: '/qwerty/uyt' is not a valid FID PASS 29a (7s) == sanity-hsm test 29b: Archive/delete/remove by FID from the archive. ========================================================== 12:27:14 (1790785634) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.295144 s, 3.6 MB/s PASS 29b (10s) == sanity-hsm test 29c: Archive/delete/remove by FID, using a file list. ========================================================== 12:27:23 (1790785643) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.431256 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.336288 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.398276 s, 2.6 MB/s PASS 29c (15s) == sanity-hsm test 29d: hsm_remove by FID with archive_id 0 for unlinked file cause ========================================================== 12:27:38 (1790785658) SKIP: sanity-hsm test_29d Need 3 or more clients, have 1 SKIP 29d (3s) == sanity-hsm test 30a: Restore at exec (import case) ==== 12:27:41 (1790785661) SKIP: sanity-hsm test_30a Need 2 or more clients, have 1 SKIP 30a (4s) == sanity-hsm test 30b: Restore at exec (release case) === 12:27:45 (1790785665) SKIP: sanity-hsm test_30b Need 2 or more clients, have 1 SKIP 30b (4s) == sanity-hsm test 30c: Update during exec of released file must fail ========================================================== 12:27:49 (1790785669) SKIP: sanity-hsm test_30c Need 2 or more clients, have 1 SKIP 30c (4s) == sanity-hsm test 31a: Import a large file and check size during restore ========================================================== 12:27:53 (1790785673) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.36845 s, 29.9 MB/s lhsmtool_posix: 1790785677.528591 [60043]: action=1 src=d31a.sanity-hsm/f31a.sanity-hsm dst=/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790785677.536491 [60043]: importing '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1790785677.630605 [60043]: imported '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//040c/0x200000407:0xb:0x0'=='/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1790785677.630665 [60043]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' size seen is right: 40894464 == 40894464 done PASS 31a (20s) == sanity-hsm test 31b: Restore a large unaligned file and check size during restore ========================================================== 12:28:13 (1790785693) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 1.02562 s, 38.0 MB/s Waiting 200s for 'SUCCEED' Updated after 3s: want 'SUCCEED' got 'SUCCEED' size seen is right: 39000000 == 39000000 done PASS 31b (24s) == sanity-hsm test 31c: Restore a large aligned file and check size during restore ========================================================== 12:28:37 (1790785717) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.00895 s, 40.5 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' size seen is right: 40894464 == 40894464 done PASS 31c (24s) == sanity-hsm test 33: Kill a restore waiting process ==== 12:29:01 (1790785741) 0+0 records in 0+0 records out 0 bytes copied, 0.0141679 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_33/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg352-client.virtnet Copytool is continued on oleg352-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2893: 62082 Terminated md5sum $f > /dev/null PASS 33 (12s) == sanity-hsm test 34: Remove file during restore ======== 12:29:13 (1790785753) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_34/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00801336 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg352-client.virtnet Copytool is continued on oleg352-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2925: kill: (62751) - No such process PASS 34 (12s) == sanity-hsm test 35: Overwrite file during restore ===== 12:29:25 (1790785765) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_35/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00628656 s, 0.0 kB/s Copytool is suspended on oleg352-client.virtnet Copytool is continued on oleg352-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2962: kill: (63427) - No such process PASS 35 (10s) == sanity-hsm test 36: Move file during restore ========== 12:29:35 (1790785775) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_36/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.0066622 s, 0.0 kB/s Copytool is suspended on oleg352-client.virtnet Copytool is continued on oleg352-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2998: kill: (64102) - No such process PASS 36 (11s) == sanity-hsm test 37: re-archive a dirty file =========== 12:29:46 (1790785786) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_37/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240098 s, 4.4 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0423335 s, 24.8 MB/s Waiting 200s for 'SUCCEED' Updated after 4s: want 'SUCCEED' got 'SUCCEED' PASS 37 (77s) == sanity-hsm test 40: Parallel archive requests ========= 12:31:03 (1790785863) oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1152921504606846976: Cannot allocate memory oleg352-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1152921504606846976': Cannot allocate memory pdsh@oleg352-client: oleg352-server: ssh exited with exit code 12 oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=1152921504606846976: Cannot allocate memory oleg352-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='1152921504606846976': Cannot allocate memory pdsh@oleg352-client: oleg352-server: ssh exited with exit code 12 set max_requests=1152921504606846976 failed with 12 [ 1635.652574] Lustre: lustre-MDT0000: No more memory to set HSM max_requests=1152921504606846976 (max request memory: 461MB, current total 6/975941): rc = -12 [ 1636.542925] Lustre: lustre-MDT0001: No more memory to set HSM max_requests=1152921504606846976 (max request memory: 461MB, current total 975941/975941): rc = -12 Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_40/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Wait for all requests being enqueued...100 archive requests submitted 100 archive requests submitted 100 archive requests submitted 100 archive requests submitted OK pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 PASS 40 (158s) == sanity-hsm test 50: Archive with large number of pending HSM actions ========================================================== 12:33:41 (1790786021) oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1000000: Cannot allocate memory oleg352-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1000000': Cannot allocate memory pdsh@oleg352-client: oleg352-server: ssh exited with exit code 12 oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=1000000: Cannot allocate memory oleg352-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='1000000': Cannot allocate memory pdsh@oleg352-client: oleg352-server: ssh exited with exit code 12 Filesystem Inodes IUsed IFree IUse% Mounted on 192.168.203.152@tcp:/lustre 236392 1320 235072 1% /mnt/lustre - create 1609 (time 1790786038.01 total 10.01 last 160.81) - create 3161 (time 1790786048.01 total 20.01 last 155.17) - create 4541 (time 1790786058.02 total 30.02 last 137.79) - create 6140 (time 1790786068.03 total 40.03 last 159.82) - create 7492 (time 1790786078.03 total 50.03 last 135.18) - create 9040 (time 1790786088.03 total 60.03 last 154.76) total: 10000 create in 66.59 seconds: 150.17 ops/second 0 - unlinked 0 (time 1790786184 ; total 0 ; last 0) total: 10000 unlinks in 66 seconds: 151.515152 unlinks/second Start Phase files_num: 10000 time: 85 - create 1558 (time 1790786263.65 total 10.00 last 155.75) - create 3272 (time 1790786273.65 total 20.01 last 171.33) - create 4880 (time 1790786283.66 total 30.01 last 160.76) - create 6443 (time 1790786293.66 total 40.01 last 156.24) - create 8009 (time 1790786303.67 total 50.02 last 156.49) - create 9632 (time 1790786313.68 total 60.03 last 162.11) - create 10000 (time 1790786316.21 total 62.56 last 145.34) - create 11544 (time 1790786326.21 total 72.57 last 154.34) - create 13066 (time 1790786336.22 total 82.57 last 152.17) - create 14656 (time 1790786346.23 total 92.58 last 158.78) - create 15793 (time 1790786356.23 total 102.59 last 113.68) - create 17441 (time 1790786366.23 total 112.59 last 164.78) - create 19048 (time 1790786376.24 total 122.59 last 160.67) total: 20000 create in 128.97 seconds: 155.07 ops/second - unlinked 0 (time 1790786532 ; total 1 ; last 1) - unlinked 10000 (time 1790786596 ; total 65 ; last 64) total: 20000 unlinks in 128 seconds: 156.250000 unlinks/second Middle Phase files_num: 20000 time: 145 - create 1717 (time 1790786673.72 total 10.03 last 171.24) - create 3307 (time 1790786683.72 total 20.03 last 159.00) - create 4763 (time 1790786693.73 total 30.03 last 145.54) - create 6446 (time 1790786703.73 total 40.03 last 168.30) - create 8086 (time 1790786713.73 total 50.04 last 163.93) - create 9409 (time 1790786723.74 total 60.04 last 132.18) total: 10000 create in 65.37 seconds: 152.98 ops/second 0 - unlinked 0 (time 1790786806 ; total 0 ; last 0) total: 10000 unlinks in 64 seconds: 156.250000 unlinks/second End Phase files_num: 10000 time: 73 0 mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 50 (863s) == sanity-hsm test 52: Opened for write file on an evicted client should be set dirty ========================================================== 12:48:04 (1790786884) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_52/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240998 s, 4.4 MB/s multiop /mnt/lustre/d52.sanity-hsm/f52.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8452 pdsh@oleg352-client: oleg352-client: ssh exited with exit code 5 PASS 52 (13s) == sanity-hsm test 53: Opened for read file on an evicted client should not be set dirty ========================================================== 12:48:17 (1790786897) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_53/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197358 s, 5.3 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' multiop /mnt/lustre/d53.sanity-hsm/f53.sanity-hsm vo_c TMPPIPE=/tmp/multiop_open_wait_pipe.8452 PASS 53 (12s) == sanity-hsm test 54: Write during an archive cancels it ========================================================== 12:48:29 (1790786909) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.867627 s, 45.0 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_54/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Waiting 200s for 'FAILED' Waiting 190s for 'FAILED' Waiting 170s for 'FAILED' Updated after 36s: want 'FAILED' got 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 54 (47s) == sanity-hsm test 55: Truncate during an archive cancels it ========================================================== 12:49:16 (1790786956) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.93254 s, 41.8 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_55/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 55 (15s) == sanity-hsm test 56: Setattr during an archive is ok === 12:49:32 (1790786972) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 1.55609 s, 25.1 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_56/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Waiting 170s for 'SUCCEED' Updated after 38s: want 'SUCCEED' got 'SUCCEED' PASS 56 (48s) == sanity-hsm test 57: Archive a file with dirty cache on another node ========================================================== 12:50:20 (1790787020) SKIP: sanity-hsm test_57 Need 2 or more clients, have 1 SKIP 57 (3s) == sanity-hsm test 58: Truncate a released file will trigger restore ========================================================== 12:50:23 (1790787023) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_58/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' truncate up from 1516 to 3032 Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate down from 1516 to 758 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate to 0 Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 PASS 58 (19s) == sanity-hsm test 59: Release stripeless file with non-zero size ========================================================== 12:50:42 (1790787042) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_59/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 59 (7s) == sanity-hsm test 60: Changing progress update interval from default ========================================================== 12:50:49 (1790787049) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.191506 s, 54.8 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_60/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 --update-interval 5 "/mnt/lustre2"' Waiting 100s for 'start copy' Expecting a progress update within 20 seconds... 5242880 bytes copied in 3 seconds. Wait for on going archive hsm action to complete Waiting 10s for 'copied' Updated after 3s: want 'copied' got 'copied' PASS 60 (18s) == sanity-hsm test 61: Waiting archive of a removed file should fail ========================================================== 12:51:07 (1790787067) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_61/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 61 (13s) == sanity-hsm test 70: Copytool logs JSON register/unregister events to FIFO ========================================================== 12:51:20 (1790787080) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_70/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --event-fifo /tmp/sanity-hsm.test_70.MybB/fifo "/mnt/lustre2"' Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s Register/Unregister events look OK. PASS 70 (9s) == sanity-hsm test 71: Copytool logs JSON archive events to FIFO ========================================================== 12:51:29 (1790787089) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_71/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_71.Hi6D/fifo "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240639 s, 4.4 MB/s Waiting 200s for 'SUCCEED' Archive events look OK. mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 71 (19s) == sanity-hsm test 72: Copytool logs JSON restore events to FIFO ========================================================== 12:51:48 (1790787108) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_72/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_72.v4YU/fifo "/mnt/lustre2"' 16+0 records in 16+0 records out 16000000 bytes (16 MB, 15 MiB) copied, 0.385233 s, 41.5 MB/s lhsmtool_posix: 1790787112.058599 [83128]: action=1 src=d72.sanity-hsm/f72.sanity-hsm dst=/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790787112.062944 [83128]: importing '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1790787112.101502 [83128]: imported '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//0bc5/0x200000bd2:0x17:0x0'=='/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1790787112.101564 [83128]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Restore events look OK. PASS 72 (15s) == sanity-hsm test 90: Archive/restore a file list ======= 12:52:03 (1790787123) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_90/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Waiting 200s for '' pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Waiting 200s for '' Changed after 2s: from 'lrh=[type=10680000 len=136 idx=2/504] fid=[0x200000403:0x252:0x0] dfid=[0x200000403:0x252:0x0] compound/cookie=0x0/0x6abd36d0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/505] fid=[0x200000403:0x253:0x0] dfid=[0x200000403:0x253:0x0] compound/cookie=0x0/0x6abd36d1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/506] fid=[0x200000403:0x254:0x0] dfid=[0x200000403:0x254:0x0] compound/cookie=0x0/0x6abd36d2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/507] fid=[0x200000403:0x255:0x0] dfid=[0x200000403:0x255:0x0] compound/cookie=0x0/0x6abd36d3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/508] fid=[0x200000403:0x256:0x0] dfid=[0x200000403:0x256:0x0] compound/cookie=0x0/0x6abd36d4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/509] fid=[0x200000403:0x257:0x0] dfid=[0x200000403:0x257:0x0] compound/cookie=0x0/0x6abd36d5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/510] fid=[0x200000403:0x258:0x0] dfid=[0x200000403:0x258:0x0] compound/cookie=0x0/0x6abd36d6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/511] fid=[0x200000403:0x259:0x0] dfid=[0x200000403:0x259:0x0] compound/cookie=0x0/0x6abd36d7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/512] fid=[0x200000403:0x25a:0x0] dfid=[0x200000403:0x25a:0x0] compound/cookie=0x0/0x6abd36d8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/513] fid=[0x200000403:0x25b:0x0] dfid=[0x200000403:0x25b:0x0] compound/cookie=0x0/0x6abd36d9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/514] fid=[0x200000403:0x25c:0x0] dfid=[0x200000403:0x25c:0x0] compound/cookie=0x0/0x6abd36da action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/515] fid=[0x200000403:0x25d:0x0] dfid=[0x200000403:0x25d:0x0] compound/cookie=0x0/0x6abd36db action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/516] fid=[0x200000403:0x25e:0x0] dfid=[0x200000403:0x25e:0x0] compound/cookie=0x0/0x6abd36dc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/517] fid=[0x200000403:0x25f:0x0] dfid=[0x200000403:0x25f:0x0] compound/cookie=0x0/0x6abd36dd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/518] fid=[0x200000403:0x260:0x0] dfid=[0x200000403:0x260:0x0] compound/cookie=0x0/0x6abd36de action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/519] fid=[0x200000403:0x261:0x0] dfid=[0x200000403:0x261:0x0] compound/cookie=0x0/0x6abd36df action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/520] fid=[0x200000403:0x262:0x0] dfid=[0x200000403:0x262:0x0] compound/cookie=0x0/0x6abd36e0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/521] fid=[0x200000403:0x263:0x0] dfid=[0x200000403:0x263:0x0] compound/cookie=0x0/0x6abd36e1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/522] fid=[0x200000403:0x264:0x0] dfid=[0x200000403:0x264:0x0] compound/cookie=0x0/0x6abd36e2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/523] fid=[0x200000403:0x265:0x0] dfid=[0x200000403:0x265:0x0] compound/cookie=0x0/0x6abd36e3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/524] fid=[0x200000403:0x266:0x0] dfid=[0x200000403:0x266:0x0] compound/cookie=0x0/0x6abd36e4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/525] fid=[0x200000403:0x267:0x0] dfid=[0x200000403:0x267:0x0] compound/cookie=0x0/0x6abd36e5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/526] fid=[0x200000403:0x268:0x0] dfid=[0x200000403:0x268:0x0] compound/cookie=0x0/0x6abd36e6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/527] fid=[0x200000403:0x269:0x0] dfid=[0x200000403:0x269:0x0] compound/cookie=0x0/0x6abd36e7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/528] fid=[0x200000403:0x26a:0x0] dfid=[0x200000403:0x26a:0x0] compound/cookie=0x0/0x6abd36e8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/529] fid=[0x200000403:0x26b:0x0] dfid=[0x200000403:0x26b:0x0] compound/cookie=0x0/0x6abd36e9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/530] fid=[0x200000403:0x26c:0x0] dfid=[0x200000403:0x26c:0x0] compound/cookie=0x0/0x6abd36ea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/531] fid=[0x200000403:0x26d:0x0] dfid=[0x200000403:0x26d:0x0] compound/cookie=0x0/0x6abd36eb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/532] fid=[0x200000403:0x26e:0x0] dfid=[0x200000403:0x26e:0x0] compound/cookie=0x0/0x6abd36ec action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/533] fid=[0x200000403:0x26f:0x0] dfid=[0x200000403:0x26f:0x0] compound/cookie=0x0/0x6abd36ed action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/534] fid=[0x200000403:0x270:0x0] dfid=[0x200000403:0x270:0x0] compound/cookie=0x0/0x6abd36ee action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/535] fid=[0x200000403:0x271:0x0] dfid=[0x200000403:0x271:0x0] compound/cookie=0x0/0x6abd36ef action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/536] fid=[0x200000403:0x272:0x0] dfid=[0x200000403:0x272:0x0] compound/cookie=0x0/0x6abd36f0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/537] fid=[0x200000403:0x273:0x0] dfid=[0x200000403:0x273:0x0] compound/cookie=0x0/0x6abd36f1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/538] fid=[0x200000403:0x274:0x0] dfid=[0x200000403:0x274:0x0] compound/cookie=0x0/0x6abd36f2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/539] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6abd36f3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/540] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6abd36f4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/541] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6abd36f5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/542] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6abd36f6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/543] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6abd36f7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/544] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6abd36f8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/545] fid=[0x200000403:0x27b:0x0] dfid=[0x200000403:0x27b:0x0] compound/cookie=0x0/0x6abd36f9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/546] fid=[0x200000403:0x27c:0x0] dfid=[0x200000403:0x27c:0x0] compound/cookie=0x0/0x6abd36fa action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/547] fid=[0x200000403:0x27d:0x0] dfid=[0x200000403:0x27d:0x0] compound/cookie=0x0/0x6abd36fb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/548] fid=[0x200000403:0x27e:0x0] dfid=[0x200000403:0x27e:0x0] compound/cookie=0x0/0x6abd36fc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/550] fid=[0x200000403:0x27f:0x0] dfid=[0x200000403:0x27f:0x0] compound/cookie=0x0/0x6abd36fd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/551] fid=[0x200000403:0x280:0x0] dfid=[0x200000403:0x280:0x0] compound/cookie=0x0/0x6abd36fe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/552] fid=[0x200000403:0x281:0x0] dfid=[0x200000403:0x281:0x0] compound/cookie=0x0/0x6abd36ff action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/553] fid=[0x200000403:0x282:0x0] dfid=[0x200000403:0x282:0x0] compound/cookie=0x0/0x6abd3700 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/554] fid=[0x200000403:0x283:0x0] dfid=[0x200000403:0x283:0x0] compound/cookie=0x0/0x6abd3701 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/555] fid=[0x200000403:0x284:0x0] dfid=[0x200000403:0x284:0x0] compound/cookie=0x0/0x6abd3702 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=2/516] fid=[0x200000403:0x25e:0x0] dfid=[0x200000403:0x25e:0x0] compound/cookie=0x0/0x6abd36dc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/517] fid=[0x200000403:0x25f:0x0] dfid=[0x200000403:0x25f:0x0] compound/cookie=0x0/0x6abd36dd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/518] fid=[0x200000403:0x260:0x0] dfid=[0x200000403:0x260:0x0] compound/cookie=0x0/0x6abd36de action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/519] fid=[0x200000403:0x261:0x0] dfid=[0x200000403:0x261:0x0] compound/cookie=0x0/0x6abd36df action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/520] fid=[0x200000403:0x262:0x0] dfid=[0x200000403:0x262:0x0] compound/cookie=0x0/0x6abd36e0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/521] fid=[0x200000403:0x263:0x0] dfid=[0x200000403:0x263:0x0] compound/cookie=0x0/0x6abd36e1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/522] fid=[0x200000403:0x264:0x0] dfid=[0x200000403:0x264:0x0] compound/cookie=0x0/0x6abd36e2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/523] fid=[0x200000403:0x265:0x0] dfid=[0x200000403:0x265:0x0] compound/cookie=0x0/0x6abd36e3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/524] fid=[0x200000403:0x266:0x0] dfid=[0x200000403:0x266:0x0] compound/cookie=0x0/0x6abd36e4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/525] fid=[0x200000403:0x267:0x0] dfid=[0x200000403:0x267:0x0] compound/cookie=0x0/0x6abd36e5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/526] fid=[0x200000403:0x268:0x0] dfid=[0x200000403:0x268:0x0] compound/cookie=0x0/0x6abd36e6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/527] fid=[0x200000403:0x269:0x0] dfid=[0x200000403:0x269:0x0] compound/cookie=0x0/0x6abd36e7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/528] fid=[0x200000403:0x26a:0x0] dfid=[0x200000403:0x26a:0x0] compound/cookie=0x0/0x6abd36e8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/529] fid=[0x200000403:0x26b:0x0] dfid=[0x200000403:0x26b:0x0] compound/cookie=0x0/0x6abd36e9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/530] fid=[0x200000403:0x26c:0x0] dfid=[0x200000403:0x26c:0x0] compound/cookie=0x0/0x6abd36ea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/531] fid=[0x200000403:0x26d:0x0] dfid=[0x200000403:0x26d:0x0] compound/cookie=0x0/0x6abd36eb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/532] fid=[0x200000403:0x26e:0x0] dfid=[0x200000403:0x26e:0x0] compound/cookie=0x0/0x6abd36ec action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/533] fid=[0x200000403:0x26f:0x0] dfid=[0x200000403:0x26f:0x0] compound/cookie=0x0/0x6abd36ed action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/534] fid=[0x200000403:0x270:0x0] dfid=[0x200000403:0x270:0x0] compound/cookie=0x0/0x6abd36ee action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/535] fid=[0x200000403:0x271:0x0] dfid=[0x200000403:0x271:0x0] compound/cookie=0x0/0x6abd36ef action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/536] fid=[0x200000403:0x272:0x0] dfid=[0x200000403:0x272:0x0] compound/cookie=0x0/0x6abd36f0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/537] fid=[0x200000403:0x273:0x0] dfid=[0x200000403:0x273:0x0] compound/cookie=0x0/0x6abd36f1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/538] fid=[0x200000403:0x274:0x0] dfid=[0x200000403:0x274:0x0] compound/cookie=0x0/0x6abd36f2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/539] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6abd36f3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/540] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6abd36f4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/541] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6abd36f5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/542] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6abd36f6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/543] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6abd36f7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/544] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6abd36f8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/545] fid=[0x200000403:0x27b:0x0] dfid=[0x200000403:0x27b:0x0] compound/cookie=0x0/0x6abd36f9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/546] fid=[0x200000403:0x27c:0x0] dfid=[0x200000403:0x27c:0x0] compound/cookie=0x0/0x6abd36fa action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/547] fid=[0x200000403:0x27d:0x0] dfid=[0x200000403:0x27d:0x0] compound/cookie=0x0/0x6abd36fb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/548] fid=[0x200000403:0x27e:0x0] dfid=[0x200000403:0x27e:0x0] compound/cookie=0x0/0x6abd36fc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/550] fid=[0x200000403:0x27f:0x0] dfid=[0x200000403:0x27f:0x0] compound/cookie=0x0/0x6abd36fd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/551] fid=[0x200000403:0x280:0x0] dfid=[0x200000403:0x280:0x0] compound/cookie=0x0/0x6abd36fe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/552] fid=[0x200000403:0x281:0x0] dfid=[0x200000403:0x281:0x0] compound/cookie=0x0/0x6abd36ff action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/553] fid=[0x200000403:0x282:0x0] dfid=[0x200000403:0x282:0x0] compound/cookie=0x0/0x6abd3700 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/554] fid=[0x200000403:0x283:0x0] dfid=[0x200000403:0x283:0x0] compound/cookie=0x0/0x6abd3701 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/555] fid=[0x200000403:0x284:0x0] dfid=[0x200000403:0x284:0x0] compound/cookie=0x0/0x6abd3702 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 4s: from 'lrh=[type=10680000 len=136 idx=2/516] fid=[0x200000403:0x25e:0x0] dfid=[0x200000403:0x25e:0x0] compound/cookie=0x0/0x6abd36dc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/517] fid=[0x200000403:0x25f:0x0] dfid=[0x200000403:0x25f:0x0] compound/cookie=0x0/0x6abd36dd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/518] fid=[0x200000403:0x260:0x0] dfid=[0x200000403:0x260:0x0] compound/cookie=0x0/0x6abd36de action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/519] fid=[0x200000403:0x261:0x0] dfid=[0x200000403:0x261:0x0] compound/cookie=0x0/0x6abd36df action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/520] fid=[0x200000403:0x262:0x0] dfid=[0x200000403:0x262:0x0] compound/cookie=0x0/0x6abd36e0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/521] fid=[0x200000403:0x263:0x0] dfid=[0x200000403:0x263:0x0] compound/cookie=0x0/0x6abd36e1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/522] fid=[0x200000403:0x264:0x0] dfid=[0x200000403:0x264:0x0] compound/cookie=0x0/0x6abd36e2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/523] fid=[0x200000403:0x265:0x0] dfid=[0x200000403:0x265:0x0] compound/cookie=0x0/0x6abd36e3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/524] fid=[0x200000403:0x266:0x0] dfid=[0x200000403:0x266:0x0] compound/cookie=0x0/0x6abd36e4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/525] fid=[0x200000403:0x267:0x0] dfid=[0x200000403:0x267:0x0] compound/cookie=0x0/0x6abd36e5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/526] fid=[0x200000403:0x268:0x0] dfid=[0x200000403:0x268:0x0] compound/cookie=0x0/0x6abd36e6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/527] fid=[0x200000403:0x269:0x0] dfid=[0x200000403:0x269:0x0] compound/cookie=0x0/0x6abd36e7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/528] fid=[0x200000403:0x26a:0x0] dfid=[0x200000403:0x26a:0x0] compound/cookie=0x0/0x6abd36e8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/529] fid=[0x200000403:0x26b:0x0] dfid=[0x200000403:0x26b:0x0] compound/cookie=0x0/0x6abd36e9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/530] fid=[0x200000403:0x26c:0x0] dfid=[0x200000403:0x26c:0x0] compound/cookie=0x0/0x6abd36ea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/531] fid=[0x200000403:0x26d:0x0] dfid=[0x200000403:0x26d:0x0] compound/cookie=0x0/0x6abd36eb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/532] fid=[0x200000403:0x26e:0x0] dfid=[0x200000403:0x26e:0x0] compound/cookie=0x0/0x6abd36ec action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/533] fid=[0x200000403:0x26f:0x0] dfid=[0x200000403:0x26f:0x0] compound/cookie=0x0/0x6abd36ed action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/534] fid=[0x200000403:0x270:0x0] dfid=[0x200000403:0x270:0x0] compound/cookie=0x0/0x6abd36ee action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/535] fid=[0x200000403:0x271:0x0] dfid=[0x200000403:0x271:0x0] compound/cookie=0x0/0x6abd36ef action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/536] fid=[0x200000403:0x272:0x0] dfid=[0x200000403:0x272:0x0] compound/cookie=0x0/0x6abd36f0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/537] fid=[0x200000403:0x273:0x0] dfid=[0x200000403:0x273:0x0] compound/cookie=0x0/0x6abd36f1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/538] fid=[0x200000403:0x274:0x0] dfid=[0x200000403:0x274:0x0] compound/cookie=0x0/0x6abd36f2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/539] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6abd36f3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/540] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6abd36f4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/541] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6abd36f5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/542] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6abd36f6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/543] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6abd36f7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/544] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6abd36f8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/545] fid=[0x200000403:0x27b:0x0] dfid=[0x200000403:0x27b:0x0] compound/cookie=0x0/0x6abd36f9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/546] fid=[0x200000403:0x27c:0x0] dfid=[0x200000403:0x27c:0x0] compound/cookie=0x0/0x6abd36fa action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/547] fid=[0x200000403:0x27d:0x0] dfid=[0x200000403:0x27d:0x0] compound/cookie=0x0/0x6abd36fb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/548] fid=[0x200000403:0x27e:0x0] dfid=[0x200000403:0x27e:0x0] compound/cookie=0x0/0x6abd36fc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/550] fid=[0x200000403:0x27f:0x0] dfid=[0x200000403:0x27f:0x0] compound/cookie=0x0/0x6abd36fd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/551] fid=[0x200000403:0x280:0x0] dfid=[0x200000403:0x280:0x0] compound/cookie=0x0/0x6abd36fe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/552] fid=[0x200000403:0x281:0x0] dfid=[0x200000403:0x281:0x0] compound/cookie=0x0/0x6abd36ff action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/553] fid=[0x200000403:0x282:0x0] dfid=[0x200000403:0x282:0x0] compound/cookie=0x0/0x6abd3700 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/554] fid=[0x200000403:0x283:0x0] dfid=[0x200000403:0x283:0x0] compound/cookie=0x0/0x6abd3701 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/555] fid=[0x200000403:0x284:0x0] dfid=[0x200000403:0x284:0x0] compound/cookie=0x0/0x6abd3702 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=2/531] fid=[0x200000403:0x26d:0x0] dfid=[0x200000403:0x26d:0x0] compound/cookie=0x0/0x6abd36eb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/532] fid=[0x200000403:0x26e:0x0] dfid=[0x200000403:0x26e:0x0] compound/cookie=0x0/0x6abd36ec action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/533] fid=[0x200000403:0x26f:0x0] dfid=[0x200000403:0x26f:0x0] compound/cookie=0x0/0x6abd36ed action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/534] fid=[0x200000403:0x270:0x0] dfid=[0x200000403:0x270:0x0] compound/cookie=0x0/0x6abd36ee action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/535] fid=[0x200000403:0x271:0x0] dfid=[0x200000403:0x271:0x0] compound/cookie=0x0/0x6abd36ef action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/536] fid=[0x200000403:0x272:0x0] dfid=[0x200000403:0x272:0x0] compound/cookie=0x0/0x6abd36f0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/537] fid=[0x200000403:0x273:0x0] dfid=[0x200000403:0x273:0x0] compound/cookie=0x0/0x6abd36f1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/538] fid=[0x200000403:0x274:0x0] dfid=[0x200000403:0x274:0x0] compound/cookie=0x0/0x6abd36f2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/539] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6abd36f3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/540] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6abd36f4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/541] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6abd36f5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/542] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6abd36f6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/543] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6abd36f7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/544] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6abd36f8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/545] fid=[0x200000403:0x27b:0x0] dfid=[0x200000403:0x27b:0x0] compound/cookie=0x0/0x6abd36f9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/546] fid=[0x200000403:0x27c:0x0] dfid=[0x200000403:0x27c:0x0] compound/cookie=0x0/0x6abd36fa action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/547] fid=[0x200000403:0x27d:0x0] dfid=[0x200000403:0x27d:0x0] compound/cookie=0x0/0x6abd36fb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/548] fid=[0x200000403:0x27e:0x0] dfid=[0x200000403:0x27e:0x0] compound/cookie=0x0/0x6abd36fc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/550] fid=[0x200000403:0x27f:0x0] dfid=[0x200000403:0x27f:0x0] compound/cookie=0x0/0x6abd36fd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/551] fid=[0x200000403:0x280:0x0] dfid=[0x200000403:0x280:0x0] compound/cookie=0x0/0x6abd36fe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/552] fid=[0x200000403:0x281:0x0] dfid=[0x200000403:0x281:0x0] compound/cookie=0x0/0x6abd36ff action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/553] fid=[0x200000403:0x282:0x0] dfid=[0x200000403:0x282:0x0] compound/cookie=0x0/0x6abd3700 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/554] fid=[0x200000403:0x283:0x0] dfid=[0x200000403:0x283:0x0] compound/cookie=0x0/0x6abd3701 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/555] fid=[0x200000403:0x284:0x0] dfid=[0x200000403:0x284:0x0] compound/cookie=0x0/0x6abd3702 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 6s: from 'lrh=[type=10680000 len=136 idx=2/531] fid=[0x200000403:0x26d:0x0] dfid=[0x200000403:0x26d:0x0] compound/cookie=0x0/0x6abd36eb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/532] fid=[0x200000403:0x26e:0x0] dfid=[0x200000403:0x26e:0x0] compound/cookie=0x0/0x6abd36ec action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/533] fid=[0x200000403:0x26f:0x0] dfid=[0x200000403:0x26f:0x0] compound/cookie=0x0/0x6abd36ed action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/534] fid=[0x200000403:0x270:0x0] dfid=[0x200000403:0x270:0x0] compound/cookie=0x0/0x6abd36ee action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/535] fid=[0x200000403:0x271:0x0] dfid=[0x200000403:0x271:0x0] compound/cookie=0x0/0x6abd36ef action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/536] fid=[0x200000403:0x272:0x0] dfid=[0x200000403:0x272:0x0] compound/cookie=0x0/0x6abd36f0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/537] fid=[0x200000403:0x273:0x0] dfid=[0x200000403:0x273:0x0] compound/cookie=0x0/0x6abd36f1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/538] fid=[0x200000403:0x274:0x0] dfid=[0x200000403:0x274:0x0] compound/cookie=0x0/0x6abd36f2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/539] fid=[0x200000403:0x275:0x0] dfid=[0x200000403:0x275:0x0] compound/cookie=0x0/0x6abd36f3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/540] fid=[0x200000403:0x276:0x0] dfid=[0x200000403:0x276:0x0] compound/cookie=0x0/0x6abd36f4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/541] fid=[0x200000403:0x277:0x0] dfid=[0x200000403:0x277:0x0] compound/cookie=0x0/0x6abd36f5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/542] fid=[0x200000403:0x278:0x0] dfid=[0x200000403:0x278:0x0] compound/cookie=0x0/0x6abd36f6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/543] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6abd36f7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/544] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6abd36f8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/545] fid=[0x200000403:0x27b:0x0] dfid=[0x200000403:0x27b:0x0] compound/cookie=0x0/0x6abd36f9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/546] fid=[0x200000403:0x27c:0x0] dfid=[0x200000403:0x27c:0x0] compound/cookie=0x0/0x6abd36fa action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/547] fid=[0x200000403:0x27d:0x0] dfid=[0x200000403:0x27d:0x0] compound/cookie=0x0/0x6abd36fb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/548] fid=[0x200000403:0x27e:0x0] dfid=[0x200000403:0x27e:0x0] compound/cookie=0x0/0x6abd36fc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/550] fid=[0x200000403:0x27f:0x0] dfid=[0x200000403:0x27f:0x0] compound/cookie=0x0/0x6abd36fd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/551] fid=[0x200000403:0x280:0x0] dfid=[0x200000403:0x280:0x0] compound/cookie=0x0/0x6abd36fe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/552] fid=[0x200000403:0x281:0x0] dfid=[0x200000403:0x281:0x0] compound/cookie=0x0/0x6abd36ff action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/553] fid=[0x200000403:0x282:0x0] dfid=[0x200000403:0x282:0x0] compound/cookie=0x0/0x6abd3700 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/554] fid=[0x200000403:0x283:0x0] dfid=[0x200000403:0x283:0x0] compound/cookie=0x0/0x6abd3701 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/555] fid=[0x200000403:0x284:0x0] dfid=[0x200000403:0x284:0x0] compound/cookie=0x0/0x6abd3702 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=2/543] fid=[0x200000403:0x279:0x0] dfid=[0x200000403:0x279:0x0] compound/cookie=0x0/0x6abd36f7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/544] fid=[0x200000403:0x27a:0x0] dfid=[0x200000403:0x27a:0x0] compound/cookie=0x0/0x6abd36f8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/545] fid=[0x200000403:0x27b:0x0] dfid=[0x200000403:0x27b:0x0] compound/cookie=0x0/0x6abd36f9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/546] fid=[0x200000403:0x27c:0x0] dfid=[0x200000403:0x27c:0x0] compound/cookie=0x0/0x6abd36fa action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/547] fid=[0x200000403:0x27d:0x0] dfid=[0x200000403:0x27d:0x0] compound/cookie=0x0/0x6abd36fb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/548] fid=[0x200000403:0x27e:0x0] dfid=[0x200000403:0x27e:0x0] compound/cookie=0x0/0x6abd36fc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/550] fid=[0x200000403:0x27f:0x0] dfid=[0x200000403:0x27f:0x0] compound/cookie=0x0/0x6abd36fd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/551] fid=[0x200000403:0x280:0x0] dfid=[0x200000403:0x280:0x0] compound/cookie=0x0/0x6abd36fe action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/552] fid=[0x200000403:0x281:0x0] dfid=[0x200000403:0x281:0x0] compound/cookie=0x0/0x6abd36ff action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/553] fid=[0x200000403:0x282:0x0] dfid=[0x200000403:0x282:0x0] compound/cookie=0x0/0x6abd3700 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/554] fid=[0x200000403:0x283:0x0] dfid=[0x200000403:0x283:0x0] compound/cookie=0x0/0x6abd3701 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/555] fid=[0x200000403:0x284:0x0] dfid=[0x200000403:0x284:0x0] compound/cookie=0x0/0x6abd3702 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 PASS 90 (94s) == sanity-hsm test 100: Set coordinator /proc tunables === 12:53:37 (1790787217) Testing mdt.lustre-MDT0000.hsm.loop_period Set loop_period to 0 must failed oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/loop_period=0: Invalid argument oleg352-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/loop_period'='0': Invalid argument pdsh@oleg352-client: oleg352-server: ssh exited with exit code 22 oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/loop_period=0: Invalid argument oleg352-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/loop_period'='0': Invalid argument pdsh@oleg352-client: oleg352-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.grace_delay Set grace_delay to 0 must failed oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/grace_delay=0: Invalid argument oleg352-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/grace_delay'='0': Invalid argument pdsh@oleg352-client: oleg352-server: ssh exited with exit code 22 oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/grace_delay=0: Invalid argument oleg352-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/grace_delay'='0': Invalid argument pdsh@oleg352-client: oleg352-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.active_request_timeout Set active_request_timeout to 0 must failed oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/active_request_timeout=0: Invalid argument oleg352-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/active_request_timeout'='0': Invalid argument pdsh@oleg352-client: oleg352-server: ssh exited with exit code 22 oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/active_request_timeout=0: Invalid argument oleg352-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/active_request_timeout'='0': Invalid argument pdsh@oleg352-client: oleg352-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.max_requests Set max_requests to 0 must failed oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=0: Invalid argument oleg352-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='0': Invalid argument pdsh@oleg352-client: oleg352-server: ssh exited with exit code 22 oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=0: Invalid argument oleg352-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='0': Invalid argument pdsh@oleg352-client: oleg352-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.default_archive_id Set default_archive_id to 0 must failed oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/default_archive_id=0: Invalid argument oleg352-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/default_archive_id'='0': Invalid argument pdsh@oleg352-client: oleg352-server: ssh exited with exit code 22 oleg352-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/default_archive_id=0: Invalid argument oleg352-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/default_archive_id'='0': Invalid argument pdsh@oleg352-client: oleg352-server: ssh exited with exit code 22 PASS 100 (47s) == sanity-hsm test 102: Verify coordinator control ======= 12:54:24 (1790787264) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 102 (21s) == sanity-hsm test 103: Purge all requests =============== 12:54:45 (1790787285) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Current requests pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 PASS 103 (15s) == sanity-hsm test 103a: Purge pending restore requests == 12:55:00 (1790787300) mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' (0x200000403:0x2cc:0x0|0x200000403:0x2cd:0x0|0x200000403:0x2ce:0x0|0x200000403:0x2cf:0x0|0x200000403:0x2d0:0x0|0x200000403:0x2d1:0x0|0x200000403:0x2d2:0x0|0x200000403:0x2d3:0x0).*action=ARCHIVE.*status=SUCCEED Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2d2:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2ce:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2cc:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2cf:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2d0:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2d1:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2d3:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x2cd:0x0': No data available mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 103a (19s) == sanity-hsm test 104: Copy tool data field ============= 12:55:19 (1790787319) 0+0 records in 0+0 records out 0 bytes copied, 0.00635011 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 104 (12s) == sanity-hsm test 105: Restart of coordinator =========== 12:55:31 (1790787331) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 105 (43s) == sanity-hsm test 106a: Copytool register/unregister ==== 12:56:14 (1790787374) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg352-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg352-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg352-client.virtnet: 89869 Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 uuid not found in agent list on mds2 Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg352-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg352-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 106a (34s) == sanity-hsm test 106b: Unregister agent after a copytool crash ========================================================== 12:56:48 (1790787408) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg352-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg352-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg352-client.virtnet: 91605 Killing existing copytools on oleg352-client.virtnet with signal KILL copytools stopped in 0s pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 uuid not found in agent list on mds2 Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 106b (27s) == sanity-hsm test 106c: Unregister copytool on client eviction ========================================================== 12:57:15 (1790787435) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg352-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg352-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg352-client.virtnet: 92839 oleg352-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg352-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg352-client.virtnet: executing get_client_uuid /mnt/lustre2 mdc.lustre-MDT0000-mdc-ffff9172a0093800.active=0 mdc.lustre-MDT0001-mdc-ffff9172a0093800.active=0 mdt.lustre-MDT0000.evict_client=9248f693-1680-4920-8f4e-79ced9d8cdd1 mdt.lustre-MDT0001.evict_client=9248f693-1680-4920-8f4e-79ced9d8cdd1 pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 uuid not found in agent list on mds2 mdc.lustre-MDT0000-mdc-ffff9172a0093800.active=1 mdc.lustre-MDT0001-mdc-ffff9172a0093800.active=1 oleg352-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 mdc.lustre-MDT0000-mdc-ffff9172a0093800.active=1 mdc.lustre-MDT0001-mdc-ffff9172a0093800.active=1 PASS 106c (54s) == sanity-hsm test 107: Copytool re-register after MDS restart ========================================================== 12:58:09 (1790787489) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_107/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Failing mds1 on oleg352-server Stopping /mnt/lustre-mds1 (opts:) on oleg352-server 12:58:19 (1790787499) shut down facet: mds1 facet_host: oleg352-server facet_failover_host: oleg352-server Failover mds1 to oleg352-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Started lustre-MDT0000 12:58:50 (1790787530) targets are mounted 12:58:50 (1790787530) facet_failover done oleg352-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 107 (55s) == sanity-hsm test 109: Policy display/change ============ 12:59:04 (1790787544) mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0000.hsm.policy=NRA NBR Next set_param must failed oleg352-server: error: set_param: setting /sys/kernel/debug/lustre/mdt/lustre-MDT0000/hsm/policy=wrong: Invalid argument oleg352-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/policy'='wrong': Invalid argument pdsh@oleg352-client: oleg352-server: ssh exited with exit code 22 Back to default policy mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 109 (23s) == sanity-hsm test 110a: Non blocking restore policy (import case) ========================================================== 12:59:27 (1790787567) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790787569.839968 [97450]: action=1 src=d110a.sanity-hsm/f110a.sanity-hsm dst=/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790787569.849427 [97450]: importing '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1790787569.910913 [97450]: imported '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//0bb7/0x200000bd2:0x65:0x0'=='/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1790787569.918055 [97450]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0001.hsm.policy=+NBR md5sum: /mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm: No data available Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 110a (14s) == sanity-hsm test 110b: Non blocking restore policy (release case) ========================================================== 12:59:41 (1790787581) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0001.hsm.policy=+NBR md5sum: /mnt/lustre/d110b.sanity-hsm/f110b.sanity-hsm: No data available Waiting 200s for 'SUCCEED' mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 110b (14s) == sanity-hsm test 111a: No retry policy (import case), restore will error ========================================================== 12:59:55 (1790787595) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790787599.196743 [98857]: action=1 src=d111a.sanity-hsm/f111a.sanity-hsm dst=/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790787599.211108 [98857]: importing '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1790787599.269105 [98857]: imported '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//0bb8/0x200000bd2:0x6a:0x0'=='/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1790787599.273333 [98857]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Remove from backend: 0x200000bd2:0x6a:0x0 = /tmp/arc1/sanity-hsm.test_111a/0bb8/0x200000bd2:0x6a:0x0 Waiting 200s for 'FAILED' Updated after 2s: want 'FAILED' got 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 111a (17s) == sanity-hsm test 111b: No retry policy (release case), restore will error ========================================================== 13:00:12 (1790787612) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Remove from backend: 0x200000bd3:0x7:0x0 = /tmp/arc1/sanity-hsm.test_111b/0bd4/0x200000bd3:0x7:0x0 mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 111b (13s) == sanity-hsm test 112: State of recorded request ======== 13:00:25 (1790787625) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_112/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled /mnt/lustre/d112.sanity-hsm/f112.sanity-hsm: ARCHIVE waiting (from 0 to EOF) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 112 (16s) == sanity-hsm test 113: wrong stat after restore ========= 13:00:41 (1790787641) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393896 s, 26.6 MB/s Starting client: oleg352-client.virtnet: -o user_xattr,flock 192.168.203.152@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_113/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Waiting 200s for 'SUCCEED' Verifying released state: multiop /mnt/lustre/d113.sanity-hsm/f113.sanity-hsm voO_WRONLY:O_APPEND:_w4c TMPPIPE=/tmp/multiop_open_wait_pipe.8452 File: /mnt/lustre2/d113.sanity-hsm/f113.sanity-hsm Size: 1048576 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115238860488715 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-09-30 13:00:43.000000000 -0400 Modify: 2026-09-30 13:00:43.000000000 -0400 Change: 2026-09-30 13:00:43.000000000 -0400 Birth: 2026-09-30 13:00:43.000000000 -0400 Stopping client oleg352-client.virtnet /mnt/lustre3 (opts:) PASS 113 (15s) == sanity-hsm test 114: Incompatible request does not set other requests as STARTED ========================================================== 13:00:56 (1790787656) 0+0 records in 0+0 records out 0 bytes copied, 0.00537455 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00330071 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_114/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 114 (16s) == sanity-hsm test 200: Register/Cancel archive ========== 13:01:12 (1790787672) 0+0 records in 0+0 records out 0 bytes copied, 0.01626 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_200/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg352-client.virtnet Copytool is continued on oleg352-client.virtnet PASS 200 (11s) == sanity-hsm test 201: Register/Cancel restore ========== 13:01:23 (1790787683) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_201/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.10973 s, 36.9 MB/s lhsmtool_posix: 1790787686.839367 [103265]: action=1 src=d201.sanity-hsm/f201.sanity-hsm dst=/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790787686.848891 [103265]: importing '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1790787686.903653 [103265]: imported '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//0ba6/0x200000bd2:0x74:0x0'=='/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1790787686.917457 [103265]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 201 (17s) == sanity-hsm test 202: Register/Cancel remove =========== 13:01:41 (1790787701) 0+0 records in 0+0 records out 0 bytes copied, 0.0102304 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_202/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg352-client.virtnet PASS 202 (9s) == sanity-hsm test 220A: Changelog for archive =========== 13:01:51 (1790787711) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.3 16HSM 17:02:00.558641247 2026.09.30 0x0 t=[0x200000bd3:0x13:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 220A (22s) == sanity-hsm test 220a: Changelog for failed archive ==== 13:02:12 (1790787732) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' Copytool is suspended on oleg352-client.virtnet Waiting 200s for 'STARTED' Copytool is continued on oleg352-client.virtnet lustre-MDT0000.6 16HSM 17:02:24.208065379 2026.09.30 0x2 t=[0x200000bd3:0x14:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 220a (22s) == sanity-hsm test 221: Changelog for archive canceled === 13:02:35 (1790787755) 0+0 records in 0+0 records out 0 bytes copied, 0.00377955 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_221/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl3 cl3' Copytool is suspended on oleg352-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Copytool is continued on oleg352-client.virtnet lustre-MDT0000.8 16HSM 17:02:49.030977511 2026.09.30 0x7d t=[0x200000bd3:0x16:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001: Deregistered changelog user #3 lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 221 (29s) == sanity-hsm test 222a: Changelog for explicit restore == 13:03:03 (1790787783) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790787785.676564 [107129]: action=1 src=d222a.sanity-hsm/f222a.sanity-hsm dst=/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790787785.694385 [107129]: importing '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1790787785.771192 [107129]: imported '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//0ba8/0x200000bd2:0x7a:0x0'=='/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1790787785.771260 [107129]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl4 cl4' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.11 16HSM 17:03:14.215936329 2026.09.30 0x80 t=[0x200000bd2:0x7a:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0001: Deregistered changelog user #4 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 222a (24s) == sanity-hsm test 222b: Changelog for implicit restore == 13:03:27 (1790787807) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl5 cl5' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 4ef076ad20e5d4a2f4729d6ed41a3bf9 /mnt/lustre/d222b.sanity-hsm/f222b.sanity-hsm lustre-MDT0000.20 16HSM 17:03:39.237584719 2026.09.30 0x80 t=[0x200000bd3:0x18:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0001: Deregistered changelog user #5 lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 PASS 222b (22s) == sanity-hsm test 222c: Changelog for failed explicit restore ========================================================== 13:03:49 (1790787829) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790787832.003844 [108775]: action=1 src=d222c.sanity-hsm/f222c.sanity-hsm dst=/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790787832.015316 [108775]: importing '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1790787832.078994 [108775]: imported '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//0bac/0x200000bd2:0x7e:0x0'=='/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1790787832.084020 [108775]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl6 cl6' Copytool is suspended on oleg352-client.virtnet Copytool is continued on oleg352-client.virtnet lustre-MDT0000.22 16HSM 17:03:59.284166572 2026.09.30 0x82 t=[0x200000bd2:0x7e:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001: Deregistered changelog user #6 lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 222c (19s) == sanity-hsm test 222d: Changelog for failed implicit restore ========================================================== 13:04:08 (1790787848) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl7 cl7' Remove from backend: 0x200000bd3:0x1a:0x0 = /tmp/arc1/sanity-hsm.test_222d/0bc9/0x200000bd3:0x1a:0x0 md5sum: /mnt/lustre/d222d.sanity-hsm/f222d.sanity-hsm: No data available lustre-MDT0000.29 16HSM 17:04:19.069369025 2026.09.30 0x82 t=[0x200000bd3:0x1a:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0001: Deregistered changelog user #7 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 222d (22s) == sanity-hsm test 223a: Changelog for restore canceled (import case) ========================================================== 13:04:31 (1790787871) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.774022 s, 52.8 MB/s mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl8 cl8' lhsmtool_posix: 1790787880.146322 [110518]: action=1 src=d223a.sanity-hsm/f223a.sanity-hsm dst=/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790787880.152574 [110518]: importing '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1790787880.254134 [110518]: imported '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//0b50/0x200000bd2:0x82:0x0'=='/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1790787880.254483 [110518]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Waiting 200s for 'SUCCEED' Waiting 190s for 'SUCCEED' Updated after 29s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.36 16HSM 17:05:11.743999040 2026.09.30 0xfd t=[0x200000bd2:0x82:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl8 of all records lustre-MDT0001: Deregistered changelog user #8 lustre-MDT0000: clear the changelog for cl8 of all records lustre-MDT0000: Deregistered changelog user #8 PASS 223a (53s) == sanity-hsm test 223b: Changelog for restore canceled (release case) ========================================================== 13:05:24 (1790787924) 0+0 records in 0+0 records out 0 bytes copied, 0.00698417 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl9 cl9' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg352-client.virtnet Copytool is continued on oleg352-client.virtnet lustre-MDT0000.43 16HSM 17:05:39.665215340 2026.09.30 0xfd t=[0x200000bd3:0x1d:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl9 of all records lustre-MDT0001: Deregistered changelog user #9 lustre-MDT0000: clear the changelog for cl9 of all records lustre-MDT0000: Deregistered changelog user #9 PASS 223b (26s) == sanity-hsm test 224A: Changelog for remove ============ 13:05:50 (1790787950) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl10 cl10' lustre-MDT0000.48 16HSM 17:06:00.366428313 2026.09.30 0x200 t=[0x200000bd3:0x1f:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl10 of all records lustre-MDT0001: Deregistered changelog user #10 lustre-MDT0000: clear the changelog for cl10 of all records lustre-MDT0000: Deregistered changelog user #10 PASS 224A (20s) == sanity-hsm test 224a: Changelog for failed remove ===== 13:06:10 (1790787970) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl11 cl11' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Remove from backend: 0x200000bd3:0x20:0x0 = /tmp/arc1/sanity-hsm.test_224a/0bf3/0x200000bd3:0x20:0x0 Copytool is suspended on oleg352-client.virtnet Copytool is continued on oleg352-client.virtnet lustre-MDT0000.53 16HSM 17:06:22.438218024 2026.09.30 0x202 t=[0x200000bd3:0x20:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl11 of all records lustre-MDT0001: Deregistered changelog user #11 lustre-MDT0000: clear the changelog for cl11 of all records lustre-MDT0000: Deregistered changelog user #11 PASS 224a (23s) == sanity-hsm test 225: Changelog for remove canceled ==== 13:06:33 (1790787993) Test disabled PASS 225 (6s) == sanity-hsm test 226: changelog for last rm/mv with exiting archive ========================================================== 13:06:39 (1790787999) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_226/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0x200000bd3:0x23:0x0 mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl12 cl12' Waiting 200s for 'SUCCEED' lustre-MDT0000.54 16HSM 17:06:47.057402244 2026.09.30 0x280 t=[0x200000bd3:0x21:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.55 16HSM 17:06:47.158329012 2026.09.30 0x280 t=[0x200000bd3:0x21:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0000.56 16HSM 17:06:47.159895364 2026.09.30 0x0 t=[0x200000bd3:0x21:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0000.57 16HSM 17:06:48.210011129 2026.09.30 0x280 t=[0x200000bd3:0x22:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.58 16HSM 17:06:48.317423237 2026.09.30 0x280 t=[0x200000bd3:0x22:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp p=[0x200000bd2:0x81:0x0] lustre-MDT0000.59 16HSM 17:06:48.317999879 2026.09.30 0x0 t=[0x200000bd3:0x22:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp p=[0x200000bd2:0x81:0x0] lustre-MDT0000.60 06UNLNK 17:06:49.992092649 2026.09.30 0x3 t=[0x200000bd3:0x21:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp p=[0x200000bd2:0x87:0x0] f226.sanity-hsm-1 lustre-MDT0000.60 06UNLNK 17:06:49.992092649 2026.09.30 0x3 t=[0x200000bd3:0x21:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp p=[0x200000bd2:0x87:0x0] f226.sanity-hsm-1 lustre-MDT0000.61 08RENME 17:06:50.370116260 2026.09.30 0x3 t=[0x200000bd3:0x22:0x0] j=mv.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp p=[0x200000bd2:0x87:0x0] f226.sanity-hsm-2 s=[0x200000bd3:0x23:0x0] sp=[0x200000bd2:0x87:0x0] f226.sanity-hsm-3 lustre-MDT0001: clear the changelog for cl12 of all records lustre-MDT0001: Deregistered changelog user #12 lustre-MDT0000: clear the changelog for cl12 of all records lustre-MDT0000: Deregistered changelog user #12 PASS 226 (20s) == sanity-hsm test 227: changelog when explicit setting of HSM flags ========================================================== 13:06:59 (1790788019) 0+0 records in 0+0 records out 0 bytes copied, 0.00394541 s, 0.0 kB/s mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl13 cl13' lustre-MDT0001: clear the changelog for cl13 of all records lustre-MDT0001: Deregistered changelog user #13 lustre-MDT0000: clear the changelog for cl13 of all records lustre-MDT0000: Deregistered changelog user #13 PASS 227 (18s) == sanity-hsm test 228: On released file, return extend to FIEMAP. For [cp,tar] --sparse ========================================================== 13:07:17 (1790788037) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_228/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0433563 s, 24.2 MB/s /mnt/lustre/f228.sanity-hsm: 1 extent found tar: Removing leading `/' from member names mnt/lustre/f228.sanity-hsm PASS 228 (10s) == sanity-hsm test 250: Coordinator max request ========== 13:07:27 (1790788047) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_250/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.0137295 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00334069 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00485031 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00402527 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.005878 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.008442 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg352-client.virtnet Copytool is continued on oleg352-client.virtnet PASS 250 (27s) == sanity-hsm test 251: Coordinator request timeout ====== 13:07:54 (1790788074) 0+0 records in 0+0 records out 0 bytes copied, 0.00420951 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_251/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg352-client.virtnet Waiting 200s for 'CANCELED' Updated after 2s: want 'CANCELED' got 'CANCELED' PASS 251 (25s) == sanity-hsm test 252: Timeout'ed running archive of a removed file should be canceled ========================================================== 13:08:19 (1790788099) 0+0 records in 0+0 records out 0 bytes copied, 0.00439854 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_252/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg352-client.virtnet Copytool is continued on oleg352-client.virtnet PASS 252 (15s) == sanity-hsm test 253: Check for wrong file size after release ========================================================== 13:08:34 (1790788114) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_253/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 10+0 records in 10+0 records out 10000000 bytes (10 MB, 9.5 MiB) copied, 0.253564 s, 39.4 MB/s fail_loc=0x807 Cannot send HSM request (use of /mnt/lustre/d253.sanity-hsm/f253.sanity-hsm): Invalid argument could not release file PASS 253 (8s) == sanity-hsm test 254a: Request counters are initialized to zero ========================================================== 13:08:42 (1790788122) PASS 254a (9s) == sanity-hsm test 254b: Request counters are correctly incremented and decremented ========================================================== 13:08:51 (1790788131) Will launch 56 requests of each type Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_254b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Checking archive requests Copytool is suspended on oleg352-client.virtnet Copytool is continued on oleg352-client.virtnet Checking restore requests Copytool is suspended on oleg352-client.virtnet Waiting 90s for '56' Updated after 2s: want '56' got '56' Copytool is continued on oleg352-client.virtnet Waiting 90s for '0' Updated after 2s: want '0' got '0' Checking remove requests Copytool is suspended on oleg352-client.virtnet Copytool is continued on oleg352-client.virtnet PASS 254b (37s) == sanity-hsm test 255: Copytool registration wakes the coordinator up ========================================================== 13:09:28 (1790788168) 0+0 records in 0+0 records out 0 bytes copied, 0.020704 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s 0+0 records in 0+0 records out 0 bytes copied, 0.0067899 s, 0.0 kB/s 0x200000bd3:0xa5:0x0 mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 255 (31s) == sanity-hsm test 256: HSM copytool register on DNE with stopped MDT must not panic ========================================================== 13:09:59 (1790788199) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=enabled PASS 256 (31s) == sanity-hsm test 260a: Restore request have priority over other requests ========================================================== 13:10:30 (1790788230) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225105 s, 4.7 MB/s 0x200000bd3:0xa6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21447 s, 4.9 MB/s 0x200000bd3:0xa7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1581 s, 6.6 MB/s 0x200000bd3:0xa8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213647 s, 4.9 MB/s 0x200000bd3:0xa9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239693 s, 4.4 MB/s 0x200000bd3:0xaa:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211459 s, 5.0 MB/s 0x200000bd3:0xab:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195962 s, 5.4 MB/s 0x200000bd3:0xac:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170401 s, 6.2 MB/s 0x200000bd3:0xad:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213849 s, 4.9 MB/s 0x200000bd3:0xae:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299453 s, 3.5 MB/s 0x200000bd3:0xaf:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253354 s, 4.1 MB/s 0x200000bd3:0xb0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159384 s, 6.6 MB/s 0x200000bd3:0xb1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240246 s, 4.4 MB/s 0x200000bd3:0xb2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140239 s, 7.5 MB/s 0x200000bd3:0xb3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256664 s, 4.1 MB/s 0x200000bd3:0xb4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175659 s, 6.0 MB/s 0x200000bd3:0xb5:0x0 Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260a (40s) == sanity-hsm test 260b: Restore request have priority over other requests ========================================================== 13:11:10 (1790788270) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276044 s, 3.8 MB/s 0x200000bd3:0xb7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.340497 s, 3.1 MB/s 0x200000bd3:0xb8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.307505 s, 3.4 MB/s 0x200000bd3:0xb9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288865 s, 3.6 MB/s 0x200000bd3:0xba:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.48534 s, 2.2 MB/s 0x200000bd3:0xbb:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220602 s, 4.8 MB/s 0x200000bd3:0xbc:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168116 s, 6.2 MB/s 0x200000bd3:0xbd:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141845 s, 7.4 MB/s 0x200000bd3:0xbe:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153875 s, 6.8 MB/s 0x200000bd3:0xbf:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14284 s, 7.3 MB/s 0x200000bd3:0xc0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197782 s, 5.3 MB/s 0x200000bd3:0xc1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199443 s, 5.3 MB/s 0x200000bd3:0xc2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273997 s, 3.8 MB/s 0x200000bd3:0xc3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225615 s, 4.6 MB/s 0x200000bd3:0xc4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.328139 s, 3.2 MB/s 0x200000bd3:0xc5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.296196 s, 3.5 MB/s 0x200000bd3:0xc6:0x0 Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260b (43s) == sanity-hsm test 260c: Requests are not reordered on the 'hot' path of the coordinator ========================================================== 13:11:53 (1790788313) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160325 s, 6.5 MB/s 0x200000bd3:0xc8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224776 s, 4.7 MB/s 0x200000bd3:0xc9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128257 s, 8.2 MB/s 0x200000bd3:0xca:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105928 s, 9.9 MB/s 0x200000bd3:0xcb:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143131 s, 7.3 MB/s 0x200000bd3:0xcc:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190476 s, 5.5 MB/s 0x200000bd3:0xcd:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171892 s, 6.1 MB/s 0x200000bd3:0xce:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210584 s, 5.0 MB/s 0x200000bd3:0xcf:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225505 s, 4.6 MB/s 0x200000bd3:0xd0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194424 s, 5.4 MB/s 0x200000bd3:0xd1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157626 s, 6.7 MB/s 0x200000bd3:0xd2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159151 s, 6.6 MB/s 0x200000bd3:0xd3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209469 s, 5.0 MB/s 0x200000bd3:0xd4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151775 s, 6.9 MB/s 0x200000bd3:0xd5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177223 s, 5.9 MB/s 0x200000bd3:0xd6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151693 s, 6.9 MB/s 0x200000bd3:0xd7:0x0 Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE RESTORE ARCHIVE PASS 260c (47s) == sanity-hsm test 261: Report 0 bytes size after HSM release ========================================================== 13:12:40 (1790788360) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_261/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0282173 s, 290 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x00000009) exists archived, archive_id:2 /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x0000000d) released exists archived, archive_id:2 PASS 261 (12s) == sanity-hsm test 262: The client should return 1 block for HSM released files ========================================================== 13:12:52 (1790788372) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_262/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0171774 s, 477 kB/s PASS 262 (11s) == sanity-hsm test 300: On disk coordinator state kept between MDT umount/mount ========================================================== 13:13:03 (1790788383) Stop coordinator and remove coordinator state at mount mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown Failing mds1 on oleg352-server Stopping /mnt/lustre-mds1 (opts:) on oleg352-server 13:13:15 (1790788395) shut down facet: mds1 facet_host: oleg352-server facet_failover_host: oleg352-server Failover mds1 to oleg352-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Started lustre-MDT0000 13:13:37 (1790788417) targets are mounted 13:13:37 (1790788417) facet_failover done oleg352-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Set coordinator start at mount, and start coordinator Failing mds1 on oleg352-server Stopping /mnt/lustre-mds1 (opts:) on oleg352-server 13:14:18 (1790788458) shut down facet: mds1 facet_host: oleg352-server facet_failover_host: oleg352-server Failover mds1 to oleg352-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Started lustre-MDT0000 13:14:43 (1790788483) targets are mounted 13:14:43 (1790788483) facet_failover done oleg352-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 300 (116s) == sanity-hsm test 301: HSM tunnable are persistent ====== 13:14:59 (1790788499) Failing mds1 on oleg352-server Stopping /mnt/lustre-mds1 (opts:) on oleg352-server 13:15:08 (1790788508) shut down facet: mds1 facet_host: oleg352-server facet_failover_host: oleg352-server Failover mds1 to oleg352-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Started lustre-MDT0000 13:15:32 (1790788532) targets are mounted 13:15:32 (1790788532) facet_failover done oleg352-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 301 (52s) == sanity-hsm test 302: HSM tunnable are persistent when CDT is off ========================================================== 13:15:51 (1790788551) mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown Failing mds1 on oleg352-server Stopping /mnt/lustre-mds1 (opts:) on oleg352-server 13:16:04 (1790788564) shut down facet: mds1 facet_host: oleg352-server facet_failover_host: oleg352-server Failover mds1 to oleg352-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Started lustre-MDT0000 13:16:30 (1790788590) targets are mounted 13:16:30 (1790788590) facet_failover done oleg352-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds2 on oleg352-server Stopping /mnt/lustre-mds2 (opts:) on oleg352-server 13:16:46 (1790788606) shut down facet: mds2 facet_host: oleg352-server facet_failover_host: oleg352-server Failover mds2 to oleg352-server mount facets: mds2 Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Started lustre-MDT0001 13:17:11 (1790788631) targets are mounted 13:17:11 (1790788631) facet_failover done oleg352-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 302 (100s) == sanity-hsm test 400: Single request is sent to the right MDT ========================================================== 13:17:31 (1790788651) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_400/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175264 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222443 s, 4.7 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' archive successful on mdt0 Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' archive successful on mdt1 PASS 400 (14s) == sanity-hsm test 401: Compound requests split and sent to their respective MDTs ========================================================== 13:17:46 (1790788666) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_401/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244329 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225484 s, 4.7 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' archive successful on mdt0 archive successful on mdt1 PASS 401 (14s) == sanity-hsm test 402a: Copytool start fails if all MDTs are inactive ========================================================== 13:18:00 (1790788680) deactivate lustre-MDT0000-mdc-ffff917292395000 on oleg352-client.virtnet deactivate lustre-MDT0001-mdc-ffff917292395000 on oleg352-client.virtnet deactivate lustre-MDT0000-mdc-ffff9172a0093800 on oleg352-client.virtnet deactivate lustre-MDT0001-mdc-ffff9172a0093800 on oleg352-client.virtnet Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Failed to start copytool agt1 on 'oleg352-client.virtnet' pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 uuid not found in agent list on mds2 activate lustre-MDT0000-mdc-ffff917292395000 on oleg352-client.virtnet activate lustre-MDT0001-mdc-ffff917292395000 on oleg352-client.virtnet activate lustre-MDT0000-mdc-ffff9172a0093800 on oleg352-client.virtnet activate lustre-MDT0001-mdc-ffff9172a0093800 on oleg352-client.virtnet PASS 402a (11s) == sanity-hsm test 402b: CDT must retry request upon slow start of CT ========================================================== 13:18:11 (1790788691) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x14d fail_loc=0 Waiting 200s for 'SUCCEED' Updated after 9s: want 'SUCCEED' got 'SUCCEED' PASS 402b (30s) == sanity-hsm test 403: Copytool starts with inactive MDT and register on reconnect ========================================================== 13:18:41 (1790788721) deactivate lustre-MDT0001-mdc-ffff917292395000 on oleg352-client.virtnet deactivate lustre-MDT0001-mdc-ffff9172a0093800 on oleg352-client.virtnet Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_403/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg352-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg352-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 uuid not found in agent list on mds2 oleg352-client.virtnet: 137610 activate lustre-MDT0001-mdc-ffff917292395000 on oleg352-client.virtnet activate lustre-MDT0001-mdc-ffff9172a0093800 on oleg352-client.virtnet found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=9248f693-1680-4920-8f4e-79ced9d8cdd1 archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 403 (24s) == sanity-hsm test 404: Inactive MDT does not block requests for active MDTs ========================================================== 13:19:05 (1790788745) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_404/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15935 s, 6.6 MB/s deactivate lustre-MDT0001-mdc-ffff917292395000 on oleg352-client.virtnet deactivate lustre-MDT0001-mdc-ffff9172a0093800 on oleg352-client.virtnet archive successful on mdt0 activate lustre-MDT0001-mdc-ffff917292395000 on oleg352-client.virtnet activate lustre-MDT0001-mdc-ffff9172a0093800 on oleg352-client.virtnet PASS 404 (9s) == sanity-hsm test 405: archive and release under striped directory ========================================================== 13:19:14 (1790788754) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_405/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845316 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665058 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114401 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0990057 s, 10.6 MB/s archive successful on 0x240000bd2:0x1:0x0 archive successful on 0x240000bd2:0x2:0x0 archive successful on 0x240000bd2:0x3:0x0 archive successful on 0x2000032e3:0x6:0x0 PASS 405 (21s) == sanity-hsm test 406: attempting to migrate HSM archived files is safe ========================================================== 13:19:35 (1790788775) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213561 s, 4.9 MB/s old fid 0x2000032e3:0x9:0x0 Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_406/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lfs migrate: /mnt/lustre/d406.sanity-hsm/f406.sanity-hsm migrate failed: Operation not supported (95) lfs migrate: cb_migrate_mdt_fini: error completing migration of /mnt/lustre/d406.sanity-hsm: Directory not empty (39) Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' new fid 0x240000bd3:0x7:0x0 debug=+trace Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 406 (18s) == sanity-hsm test 407: Check for double RESTORE records in llog ========================================================== 13:19:53 (1790788793) 0+0 records in 0+0 records out 0 bytes copied, 0.00347021 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_407/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_val=5 fail_loc=0x164 Copytool is suspended on oleg352-client.virtnet mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=4/4] fid=[0x2000032e3:0x3:0x0] dfid=[0x2000032e3:0x3:0x0] compound/cookie=0x0/0x6abd43ec action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x6abd43ed action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x6abd43ee action=RESTORE archive#=3 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6abd43ef action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6abd43f0 action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6abd43f1 action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6abd43f2 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] Failing mds1 on oleg352-server Stopping /mnt/lustre-mds1 (opts:) on oleg352-server 13:20:06 (1790788806) shut down facet: mds1 facet_host: oleg352-server facet_failover_host: oleg352-server Failover mds1 to oleg352-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Started lustre-MDT0000 13:20:32 (1790788832) targets are mounted 13:20:32 (1790788832) facet_failover done mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=4/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x6abd43ee action=RESTORE archive#=3 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6abd43ef action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6abd43f0 action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6abd43f1 action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6abd43f2 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/11] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6abd43f3 action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] oleg352-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec fail_loc=0 mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=4/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6abd43ef action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6abd43f0 action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6abd43f1 action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6abd43f2 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/11] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6abd43f3 action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] Copytool is continued on oleg352-client.virtnet d41d8cd98f00b204e9800998ecf8427e /mnt/lustre/d407.sanity-hsm/f407.sanity-hsm d41d8cd98f00b204e9800998ecf8427e /mnt/lustre2/d407.sanity-hsm/f407.sanity-hsm pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 PASS 407 (59s) == sanity-hsm test 408: Verify fiemap on release file ==== 13:20:52 (1790788852) Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_408/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.18615 s, 352 kB/s SKIP: sanity-hsm test_408 ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 408 (4s) == sanity-hsm test 409a: Coordinator should not stop when in use ========================================================== 13:20:56 (1790788856) 0+0 records in 0+0 records out 0 bytes copied, 0.0119354 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_val=5 fail_loc=0x164 cat: /mnt/lustre/d409a.sanity-hsm/f409a.sanity-hsm: No data available mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 409a (31s) == sanity-hsm test 409b: getattr released file with CDT stopped after remount ========================================================== 13:21:27 (1790788887) 0+0 records in 0+0 records out 0 bytes copied, 0.017429 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s Failing mds1 on oleg352-server Stopping /mnt/lustre-mds1 (opts:) on oleg352-server 13:22:00 (1790788920) shut down facet: mds1 facet_host: oleg352-server facet_failover_host: oleg352-server Failover mds1 to oleg352-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Started lustre-MDT0000 13:22:23 (1790788943) targets are mounted 13:22:23 (1790788943) facet_failover done oleg352-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec File: /mnt/lustre/d409b.sanity-hsm/f409b.sanity-hsm Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115406632648722 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-09-30 13:21:29.000000000 -0400 Modify: 2026-09-30 13:21:29.000000000 -0400 Change: 2026-09-30 13:21:29.000000000 -0400 Birth: 2026-09-30 13:21:30.000000000 -0400 PASS 409b (95s) == sanity-hsm test 410: lfs data_version -s allows release of force-archived file ========================================================== 13:23:02 (1790788982) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213608 s, 4.9 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_410/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d410.sanity-hsm/f410.sanity-hsm): Operation not permitted PASS 410 (6s) == sanity-hsm test 411: hsm_ops rbac role ================ 13:23:08 (1790788988) On MGS 192.168.203.152, active = nodemap.active=1 On MGS 192.168.203.152, test_411.trusted_nodemap = nodemap.test_411.trusted_nodemap=1 Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /mnt/lustre/d411.sanity-hsm/fileA: (0x00000000) Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg352-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' On MGS 192.168.203.152, test_411.rbac = nodemap.test_411.rbac=file_perms,dne_ops,quota_ops,byfid_ops,chlg_ops,fscrypt_admin,server_upcall,ignore_root_prjquota,immutable_flags,local_admin,pool_quota_ops,lqa_quota_ops,projid_set,foreign_ops /mnt/lustre/d411.sanity-hsm/fileB: (0x00000000) Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied lfs hsm_set: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied lfs hsm_clear: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied On MGS 192.168.203.152, active = nodemap.active=0 oleg352-server: error: nodemap 'test_411' does not exist oleg352-server: nodemap_del: Invalid argument pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 On MGS 192.168.203.152, active = nodemap.active=0 PASS 411 (43s) == sanity-hsm test 500: various LLAPI HSM tests ========== 13:23:51 (1790789031) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d500.sanity-hsm test1 @1790789101: Register/unregister copytool 2000x to check for leaks ...... pass test2 @1790789101: Re/un-register copytool multiple times without error ....... pass llapi_hsm_test: NULL archive numbers llapi_hsm_test: NULL archive numbers llapi_hsm_test: -1 requested when archive id >= 0 is supported test3 @1790789101: Pass bad parameters to llapi_hsm_copytool_register() ....... pass test4 @1790789101: Bad parameters to llapi_hsm_copytool_unregister() .......... pass test5 @1790789101: Test llapi_hsm_copytool_recv() in non-blocking mode ........ pass test6 @1790789101: Test llapi_hsm_copytool_recv() with bogus parameters ....... pass test7 @1790789101: Test event polling (without actual traffic) ................ pass test50 @1790789101: Test llapi_hsm_state_get()/get_fd() ....................... pass test51 @1790789102: Test llapi_hsm_state_set_fd() ............................. pass test52 @1790789102: Test llapi_hsm_current_action() ........................... pass test100 @1790789103: Simple archive creation with no progress reported. ....... pass test101 @1790789109: Simple archive creation with progress every byte. ........ pass test102 @1790789116: Archive creation with progress every byte, backwards ..... pass test103 @1790789117: Archive creation with a single progress report ........... pass test104 @1790789118: Archive creation with two progress reports ............... pass test105 @1790789120: Archive creation with one bogus progress report .......... pass test106 @1790789121: Archive creation with one empty progress report .......... pass test107 @1790789122: Archive creation with one bogus progress report .......... pass test108 @1790789127: Archive creation with same progress report each time ..... pass test109 @1790789128: Archive creation with one report, with large number ...... pass test110 @1790789129: Archive with 10 progress reports, checking progress ...... pass test111 @1790789130: Archive with 10 reports in reverse, checking progress .... pass test112 @1790789131: Archive with 10 reports, duplicated, check progress ...... pass test113 @1790789133: Archive with 9 reports, with 20% overlapping coverage .... pass PASS 500 (106s) == sanity-hsm test 600: Changelog fields 'u=' and 'nid=' ========================================================== 13:25:37 (1790789137) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d600.sanity-hsm/f600.sanity-hsm] Got NID '192.168.203.52@tcp' lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 600 (19s) == sanity-hsm test 601: OPEN Changelog entry ============= 13:25:56 (1790789156) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001.4 10OPEN 17:26:05.938685258 2026.09.30 0x1 t=[0x240000bd3:0x1b:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.52@tcp m=r-- p=[0x200000bd2:0x81:0x0] lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 601 (19s) == sanity-hsm test 602: Changelog record CLOSE only if open+write or OPEN recorded ========================================================== 13:26:15 (1790789175) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl3 cl3' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0000.9 11CLOSE 17:26:24.529254497 2026.09.30 0x1 t=[0x2000032e2:0x1c:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0000.10 10OPEN 17:26:25.007043449 2026.09.30 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.52@tcp m=-w- lustre-MDT0000.11 13TRUNC 17:26:25.019071984 2026.09.30 0xe t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp p=[0x2000032e2:0x1b:0x0] lustre-MDT0000.12 11CLOSE 17:26:25.068949337 2026.09.30 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0000.12 11CLOSE 17:26:25.068949337 2026.09.30 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp mdd.lustre-MDT0000.changelog_mask=-OPEN mdd.lustre-MDT0001.changelog_mask=-OPEN lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records f lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0000.13 13TRUNC 17:26:27.375883042 2026.09.30 0xe t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp p=[0x2000032e2:0x1b:0x0] lustre-MDT0000.14 11CLOSE 17:26:27.393115814 2026.09.30 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0000.14 11CLOSE 17:26:27.393115814 2026.09.30 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001: Deregistered changelog user #3 lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 602 (21s) == sanity-hsm test 603: GETXATTR Changelog entry ========= 13:26:36 (1790789196) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl4 cl4' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0001: clear the changelog for cl4 of all records getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d603.sanity-hsm/f603.sanity-hsm user.xattr1="value1" lustre-MDT0001.11 23GXATR 17:26:45.695643594 2026.09.30 0x0 t=[0x240000bd3:0x1d:0x0] j=getfattr.0 ef=0x1b u=0:0 nid=192.168.203.52@tcp x=user.xattr1 lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0001: Deregistered changelog user #4 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 603 (20s) == sanity-hsm test 604: NOPEN Changelog entry ============ 13:26:56 (1790789216) mdd.lustre-MDT0000.changelog_deniednext=20 mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl5 cl5' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.19 24NOPEN 17:27:08.119644092 2026.09.30 0x1 t=[0x2000032e2:0x1e:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.52@tcp m=r-- p=[0x2000032e2:0x1b:0x0] Got NID '192.168.203.52@tcp' lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.20 24NOPEN 17:27:29.678723803 2026.09.30 0x1 t=[0x2000032e2:0x1e:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.52@tcp m=r-- Got NID '192.168.203.52@tcp' lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0001: Deregistered changelog user #5 lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 mdd.lustre-MDT0000.changelog_deniednext=60 PASS 604 (45s) == sanity-hsm test 605: Test OPEN and CLOSE rate limit in Changelogs ========================================================== 13:27:41 (1790789261) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl6 cl6' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000.24 10OPEN 17:27:49.584168439 2026.09.30 0x43 t=[0x2000032e2:0x20:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.52@tcp m=rw- p=[0x200000bd2:0x86:0x0] lustre-MDT0000.24 10OPEN 17:27:49.584168439 2026.09.30 0x43 t=[0x2000032e2:0x20:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.203.52@tcp m=rw- p=[0x200000bd2:0x86:0x0] lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000.25 10OPEN 17:27:50.861090133 2026.09.30 0x1 t=[0x2000032e2:0x20:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.52@tcp m=r-- p=[0x2000032e2:0x1f:0x0] lustre-MDT0000.26 11CLOSE 17:27:50.894550541 2026.09.30 0x1 t=[0x2000032e2:0x20:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0000.25 10OPEN 17:27:50.861090133 2026.09.30 0x1 t=[0x2000032e2:0x20:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.203.52@tcp m=r-- p=[0x2000032e2:0x1f:0x0] lustre-MDT0000.26 11CLOSE 17:27:50.894550541 2026.09.30 0x1 t=[0x2000032e2:0x20:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre/d605.sanity-hsm/f605.sanity-hsm] lustre-MDT0000.27 10OPEN 17:27:52.512340078 2026.09.30 0x1 t=[0x2000032e2:0x20:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.52@tcp m=r-- p=[0x2000032e2:0x1b:0x0] lustre-MDT0000.28 11CLOSE 17:27:52.525546633 2026.09.30 0x1 t=[0x2000032e2:0x20:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.203.52@tcp lustre-MDT0000.27 10OPEN 17:27:52.512340078 2026.09.30 0x1 t=[0x2000032e2:0x20:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.203.52@tcp m=r-- p=[0x2000032e2:0x1b:0x0] lustre-MDT0000.28 11CLOSE 17:27:52.525546633 2026.09.30 0x1 t=[0x2000032e2:0x20:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.203.52@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000.29 11CLOSE 17:27:53.131645520 2026.09.30 0x43 t=[0x2000032e2:0x20:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0000.29 11CLOSE 17:27:53.131645520 2026.09.30 0x43 t=[0x2000032e2:0x20:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.203.52@tcp lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001: Deregistered changelog user #6 lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 605 (21s) == sanity-hsm test 606: llog_reader groks changelog fields ========================================================== 13:28:02 (1790789282) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl7 cl7' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d606.sanity-hsm/f606.sanity-hsm] Stopping /mnt/lustre-mds1 (opts:) on oleg352-server rec #2 type=1064553b len=64 offset 8256 Header size : 8192 llh_size : 64 Time : Wed Sep 30 12:58:45 2026 Number of records: 1 cat_idx: 0 last_idx: 2 Flags: 0x2012 Target uuid : ----------------------- #02 (064)fid=[0x1:0x3a:0x0] path=oi.1/0x1:0x3a:0x0 Got UID/GID 500:500 Got NID 192.168.203.52@tcp Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Started lustre-MDT0000 lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0001: Deregistered changelog user #7 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 606 (66s) == sanity-hsm test 607a: release file migrated after archive (non-blocking migrate) ========================================================== 13:29:08 (1790789348) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.242964 s, 4.3 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 607a (8s) == sanity-hsm test 607b: Migrate should not change the HSM attribute of dirty files ========================================================== 13:29:17 (1790789357) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262276 s, 4.0 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' llite.lustre-ffff917292395000.xattr_cache=0 llite.lustre-ffff9172a0093800.xattr_cache=0 /mnt/lustre/d607b.sanity-hsm/f607b.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 llite.lustre-ffff917292395000.xattr_cache=1 llite.lustre-ffff9172a0093800.xattr_cache=1 PASS 607b (10s) == sanity-hsm test 607c: 'lfs swap_layouts' should set dirty flag on HSM file ========================================================== 13:29:27 (1790789367) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318175 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266812 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.338109 s, 3.1 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-1' and '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-3': Operation not permitted (1) PASS 607c (11s) == sanity-hsm test 607d: Migrate should not set HSM dirty flag for an empty file ========================================================== 13:29:39 (1790789379) 0+0 records in 0+0 records out 0 bytes copied, 0.0110849 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 607d (13s) == sanity-hsm test 607e: release file migrated after archive (blocking migrate) ========================================================== 13:29:51 (1790789391) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.302204 s, 3.5 MB/s Starting copytool 'agt1' on 'oleg352-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 607e (11s) == sanity-hsm test complete, duration 4882 sec =========== 13:30:02 (1790789402) === sanity-hsm: start cleanup 13:30:04 (1790789404) === Stopping clients: oleg352-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg352-client.virtnet /mnt/lustre2 opts: === sanity-hsm: finish cleanup 13:30:37 (1790789437) ===