-----============= acceptance-small: sanity-hsm ============----- Fri Oct 2 01:26:02 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky use SANITY_HSM_ONLY='' SANITY_HSM_EXCEPT='' oleg436-server: /home/green/git/lustre-release/lustre/tests/except/sanity-hsm.ex - see MDS1_VERSION <= CLIENT_VERSION (34683650 <= 34683650) for LU-18924, go 100 105 excepting tests: === sanity-hsm: start setup 01:26:22 (1790918782) === Starting client oleg436-client.virtnet: -o user_xattr,flock 192.168.204.136@tcp:/lustre /mnt/lustre2 Mount client oleg436-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.204.136@tcp:/lustre /mnt/lustre2 Started clients oleg436-client.virtnet: 192.168.204.136@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) oleg436-client.virtnet: executing check_config_client /mnt/lustre oleg436-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg436-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9dd748a00800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9dd7515db000.idle_timeout=debug osc.lustre-OST0001-osc-ffff9dd748a00800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9dd7515db000.idle_timeout=debug disable quota as required oleg436-server: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity-hsm: finish setup 01:27:01 (1790918821) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f8412] nobody:x:65534: Killing existing copytools on oleg436-client.virtnet with signal TERM pkill: pidfile not valid Try `pkill --help' for more information. Set HSM on and start Set sanity-hsm HSM policy == sanity-hsm test 1A: lfs hsm flags root/non-root access ========================================================== 01:27:29 (1790918849) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_clear] [--norelease] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_set] [--exists] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] lfs hsm_set: change hsm flags for '/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm' failed: Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1A.sanity-hsm/f1A.sanity-hsm] PASS 1A (7s) == sanity-hsm test 1a: mmap & cat a HSM released file ==== 01:27:36 (1790918856) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.310102 s, 3.4 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Verifying released state: PASS 1a (11s) == sanity-hsm test 1b: Archive, Release and Restore composite file ========================================================== 01:27:47 (1790918867) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102826 s, 10.2 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm release /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm verify released state: pass restore /mnt/lustre/d1b.sanity-hsm/f1b.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass PASS 1b (13s) == sanity-hsm test 1c: Check setting archive-id in lfs hsm_set ========================================================== 01:28:00 (1790918880) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is still 32 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 33 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d1c.sanity-hsm/f1c.sanity-hsm] verifying archive number is 16 PASS 1c (7s) == sanity-hsm test 1d: Archive, Release and Restore DoM file ========================================================== 01:28:08 (1790918888) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0744334 s, 14.1 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' release /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm verify released state: pass restore /mnt/lustre/d1d.sanity-hsm/f1d.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass PASS 1d (17s) == sanity-hsm test 1e: Archive, Release and Restore SEL file ========================================================== 01:28:24 (1790918904) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0572571 s, 18.3 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm release /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm verify released state: pass restore /mnt/lustre/d1e.sanity-hsm/f1e.sanity-hsm Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' verify restored state: pass PASS 1e (12s) == sanity-hsm test 1f: DoM file release after restore ==== 01:28:37 (1790918917) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700835 s, 15.0 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_1f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' archive /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm release /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm verify released state: pass restore /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm verify restored state: pass release again /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm /mnt/lustre/d1f.sanity-hsm/f1f.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 verify released state: pass PASS 1f (10s) == sanity-hsm test 2: Check file dirtyness when doing setattr ========================================================== 01:28:48 (1790918928) 0+0 records in 0+0 records out 0 bytes copied, 0.00460149 s, 0.0 kB/s 0x200000403:0x8:0x0 PASS 2 (7s) == sanity-hsm test 3: Check file dirtyness when opening for write ========================================================== 01:28:56 (1790918936) Succeed in opening file "/mnt/lustre/d3.sanity-hsm/f3.sanity-hsm"(flags=O_WRONLY) 3+0 records in 3+0 records out 3+0 records in 3+0 records out PASS 3 (9s) == sanity-hsm test 4: Useless cancel must not be registered ========================================================== 01:29:04 (1790918944) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319656 s, 3.3 MB/s PASS 4 (9s) == sanity-hsm test 8: Test default archive number ======== 01:29:13 (1790918953) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_8/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 8 (10s) == sanity-hsm test 9A: Use of explicit archive number, with dedicated copytool ========================================================== 01:29:23 (1790918963) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_9A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 3 "/mnt/lustre2"' oleg436-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg436-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=3 requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=3 requests=[current:0 ok:0 errors:0] on mds2 PASS 9A (27s) == sanity-hsm test 9a: Multiple remote agents ============ 01:29:50 (1790918990) SKIP: sanity-hsm test_9a Need 3 or more clients, have 1 SKIP 9a (3s) == sanity-hsm test 10a: Archive a file =================== 01:29:54 (1790918994) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 996: warning: command substitution: ignored null byte in input Verifying content Verifying hsm state Verifying archive number is 2 PASS 10a (8s) == sanity-hsm test 10b: Archive of non dirty file must work without doing request ========================================================== 01:30:02 (1790919002) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 10b (10s) == sanity-hsm test 10c: Check forbidden archive ========== 01:30:12 (1790919012) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d10c.sanity-hsm/f10c.sanity-hsm): Operation not permitted PASS 10c (7s) == sanity-hsm test 10d: Archive a file on the default archive id ========================================================== 01:30:19 (1790919019) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_10d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 10d (10s) == sanity-hsm test 11a: Import a file ==================== 01:30:29 (1790919029) lhsmtool_posix: 1790919031.963361 [22219]: action=1 src=d11a.sanity-hsm/f11a.sanity-hsm dst=/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790919031.973845 [22219]: importing '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1790919032.053208 [22219]: imported '/mnt/lustre/d11a.sanity-hsm/f11a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11a//0425/0x200000402:0x27:0x0'=='/tmp/arc1/sanity-hsm.test_11a//d11a.sanity-hsm/f11a.sanity-hsm' lhsmtool_posix: 1790919032.053266 [22219]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying imported size 158=158 Verifying released pattern: raid0,released Verifying new fid 0x200000402:0x27:0x0 in archive PASS 11a (8s) == sanity-hsm test 11b: Import a deleted file using its FID ========================================================== 01:30:37 (1790919037) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_11b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lhsmtool_posix: 1790919043.633730 [22919]: action=1 src=0x200000403:0x11:0x0 dst=/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790919043.669035 [22919]: Resolving [0x200000403:0x11:0x0] to /tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0 lhsmtool_posix: 1790919043.676809 [22919]: importing '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0' lhsmtool_posix: 1790919043.779618 [22919]: imported '/mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11b//042b/0x200000402:0x29:0x0'=='/tmp/arc1/sanity-hsm.test_11b//0412/0x200000403:0x11:0x0' lhsmtool_posix: 1790919043.779695 [22919]: process finished, errs: 0 major, 0 minor, rc=0 (Success) /mnt/lustre/d11b.sanity-hsm/f11b.sanity-hsm: OK PASS 11b (12s) == sanity-hsm test 11c: Import a file to a directory with a pool ========================================================== 01:30:50 (1790919050) Creating new pool test_11c oleg436-server: Pool lustre.test_11c created Adding targets to pool oleg436-server: OST lustre-OST0001_UUID added to pool lustre.test_11c lhsmtool_posix: 1790919062.070994 [23634]: action=1 src=d11c.sanity-hsm/f11c.sanity-hsm dst=/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790919062.090948 [23634]: importing '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1790919062.199829 [23634]: imported '/mnt/lustre/d11c.sanity-hsm/f11c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_11c//0407/0x240000402:0x5:0x0'=='/tmp/arc1/sanity-hsm.test_11c//d11c.sanity-hsm/f11c.sanity-hsm' lhsmtool_posix: 1790919062.199911 [23634]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Destroy the created pools: test_11c lustre.test_11c oleg436-server: OST lustre-OST0001_UUID removed from pool lustre.test_11c oleg436-server: Pool lustre.test_11c destroyed PASS 11c (34s) == sanity-hsm test 12a: Restore an imported file explicitly ========================================================== 01:31:23 (1790919083) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790919086.666133 [24373]: action=1 src=d12a.sanity-hsm/f12a.sanity-hsm dst=/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790919086.672370 [24373]: importing '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1790919086.720502 [24373]: imported '/mnt/lustre/d12a.sanity-hsm/f12a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12a//042f/0x200000402:0x2d:0x0'=='/tmp/arc1/sanity-hsm.test_12a//d12a.sanity-hsm/f12a.sanity-hsm' lhsmtool_posix: 1790919086.720596 [24373]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying file state: PASS 12a (11s) == sanity-hsm test 12b: Restore an imported file implicitly ========================================================== 01:31:34 (1790919094) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790919097.208058 [25080]: action=1 src=d12b.sanity-hsm/f12b.sanity-hsm dst=/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790919097.213851 [25080]: importing '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1790919097.288646 [25080]: imported '/mnt/lustre/d12b.sanity-hsm/f12b.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12b//042d/0x200000402:0x2f:0x0'=='/tmp/arc1/sanity-hsm.test_12b//d12b.sanity-hsm/f12b.sanity-hsm' lhsmtool_posix: 1790919097.288707 [25080]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Verifying file state after restore: PASS 12b (10s) == sanity-hsm test 12c: Restore a file with stripe of 2 == 01:31:45 (1790919105) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.180374 s, 29.1 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d12c.sanity-hsm/f12c.sanity-hsm: OK PASS 12c (12s) == sanity-hsm test 12d: Restore of a non archived, non released file must work ========================================================== 01:31:57 (1790919117) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12d (12s) == sanity-hsm test 12e: Check forbidden restore ========== 01:32:09 (1790919129) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 Cannot send HSM request (use of /mnt/lustre/d12e.sanity-hsm/f12e.sanity-hsm): Operation not permitted PASS 12e (13s) == sanity-hsm test 12f: Restore a released file explicitly ========================================================== 01:32:22 (1790919142) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying file state: PASS 12f (12s) == sanity-hsm test 12g: Restore a released file implicitly ========================================================== 01:32:34 (1790919154) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12g (9s) == sanity-hsm test 12h: Restore a released file implicitly from a second node ========================================================== 01:32:44 (1790919164) SKIP: sanity-hsm test_12h Need 2 or more clients, have 1 SKIP 12h (4s) == sanity-hsm test 12m: Archive/release/implicit restore ========================================================== 01:32:47 (1790919167) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12m/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12m (10s) == sanity-hsm test 12n: Import/implicit restore/release == 01:32:58 (1790919178) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12n/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790919181.060105 [29932]: action=1 src=d12n.sanity-hsm/f12n.sanity-hsm dst=/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790919181.073193 [29932]: importing '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1790919181.114968 [29932]: imported '/mnt/lustre/d12n.sanity-hsm/f12n.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12n//043c/0x200000402:0x3e:0x0'=='/tmp/arc1/sanity-hsm.test_12n//d12n.sanity-hsm/f12n.sanity-hsm' lhsmtool_posix: 1790919181.115053 [29932]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12n (10s) == sanity-hsm test 12o: Layout-swap failure during Restore leaves file released ========================================================== 01:33:08 (1790919188) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12o/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x152 mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA diff: /mnt/lustre/d12o.sanity-hsm/f12o.sanity-hsm: No data available mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA fail_loc=0 mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 12o (36s) == sanity-hsm test 12p: implicit restore of a file on copytool mount point ========================================================== 01:33:44 (1790919224) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12p/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12p (13s) == sanity-hsm test 12q: file attributes are refreshed after restore ========================================================== 01:33:57 (1790919237) Starting client: oleg436-client.virtnet: -o user_xattr,flock 192.168.204.136@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226905 s, 4.6 MB/s Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12q/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Stopping client oleg436-client.virtnet /mnt/lustre3 (opts:) PASS 12q (19s) == sanity-hsm test 12r: lseek restores released file ===== 01:34:17 (1790919257) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12r/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 12r (11s) == sanity-hsm test 12s: race between restore requests ==== 01:34:28 (1790919268) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12s/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x8000018b PASS 12s (18s) == sanity-hsm test 12t: Multiple parallel reads for a HSM imported file ========================================================== 01:34:47 (1790919287) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12t/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790919291.046149 [34068]: action=1 src=0x200000403:0x2a:0x0 dst=/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790919291.054873 [34068]: Resolving [0x200000403:0x2a:0x0] to /tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0 lhsmtool_posix: 1790919291.059079 [34068]: importing '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0' lhsmtool_posix: 1790919291.119295 [34068]: imported '/mnt/lustre/d12t.sanity-hsm/f12t.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_12t//044e/0x200000402:0x4c:0x0'=='/tmp/arc1/sanity-hsm.test_12t//0429/0x200000403:0x2a:0x0' lhsmtool_posix: 1790919291.119370 [34068]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12t (12s) == sanity-hsm test 12u: Multiple reads on multiple HSM imported files in parallel ========================================================== 01:34:58 (1790919298) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_12u/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790919316.972631 [34944]: action=1 src=0x200000403:0x2c:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0 mount_point=/mnt/lustre2 lhsmtool_posix: 1790919316.984117 [34944]: Resolving [0x200000403:0x2c:0x0] to /tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0 lhsmtool_posix: 1790919316.984249 [34944]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0' lhsmtool_posix: 1790919317.041766 [34944]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.0' from '/tmp/arc1/sanity-hsm.test_12u//044c/0x200000402:0x4e:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042f/0x200000403:0x2c:0x0' lhsmtool_posix: 1790919317.041844 [34944]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790919317.667959 [34973]: action=1 src=0x200000403:0x2d:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1 mount_point=/mnt/lustre2 lhsmtool_posix: 1790919317.674140 [34973]: Resolving [0x200000403:0x2d:0x0] to /tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0 lhsmtool_posix: 1790919317.674480 [34973]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0' lhsmtool_posix: 1790919317.734249 [34973]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.1' from '/tmp/arc1/sanity-hsm.test_12u//044d/0x200000402:0x4f:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042e/0x200000403:0x2d:0x0' lhsmtool_posix: 1790919317.735186 [34973]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790919318.188024 [35002]: action=1 src=0x200000403:0x2e:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2 mount_point=/mnt/lustre2 lhsmtool_posix: 1790919318.200535 [35002]: Resolving [0x200000403:0x2e:0x0] to /tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0 lhsmtool_posix: 1790919318.201085 [35002]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0' lhsmtool_posix: 1790919318.262510 [35002]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.2' from '/tmp/arc1/sanity-hsm.test_12u//0452/0x200000402:0x50:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042d/0x200000403:0x2e:0x0' lhsmtool_posix: 1790919318.262785 [35002]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790919318.900250 [35031]: action=1 src=0x200000403:0x2f:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3 mount_point=/mnt/lustre2 lhsmtool_posix: 1790919318.913498 [35031]: Resolving [0x200000403:0x2f:0x0] to /tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0 lhsmtool_posix: 1790919318.913613 [35031]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0' lhsmtool_posix: 1790919318.971836 [35031]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.3' from '/tmp/arc1/sanity-hsm.test_12u//0453/0x200000402:0x51:0x0'=='/tmp/arc1/sanity-hsm.test_12u//042c/0x200000403:0x2f:0x0' lhsmtool_posix: 1790919318.971893 [35031]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790919319.508605 [35060]: action=1 src=0x200000403:0x30:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4 mount_point=/mnt/lustre2 lhsmtool_posix: 1790919319.524712 [35060]: Resolving [0x200000403:0x30:0x0] to /tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0 lhsmtool_posix: 1790919319.524818 [35060]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0' lhsmtool_posix: 1790919319.572927 [35060]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.4' from '/tmp/arc1/sanity-hsm.test_12u//0450/0x200000402:0x52:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0433/0x200000403:0x30:0x0' lhsmtool_posix: 1790919319.573773 [35060]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790919320.110179 [35089]: action=1 src=0x200000403:0x31:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5 mount_point=/mnt/lustre2 lhsmtool_posix: 1790919320.123167 [35089]: Resolving [0x200000403:0x31:0x0] to /tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0 lhsmtool_posix: 1790919320.123319 [35089]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0' lhsmtool_posix: 1790919320.173058 [35089]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.5' from '/tmp/arc1/sanity-hsm.test_12u//0451/0x200000402:0x53:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0432/0x200000403:0x31:0x0' lhsmtool_posix: 1790919320.173127 [35089]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790919320.629842 [35119]: action=1 src=0x200000403:0x32:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6 mount_point=/mnt/lustre2 lhsmtool_posix: 1790919320.637063 [35119]: Resolving [0x200000403:0x32:0x0] to /tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0 lhsmtool_posix: 1790919320.638019 [35119]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0' lhsmtool_posix: 1790919320.686898 [35119]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.6' from '/tmp/arc1/sanity-hsm.test_12u//0456/0x200000402:0x54:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0431/0x200000403:0x32:0x0' lhsmtool_posix: 1790919320.686970 [35119]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790919321.202836 [35148]: action=1 src=0x200000403:0x33:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7 mount_point=/mnt/lustre2 lhsmtool_posix: 1790919321.216728 [35148]: Resolving [0x200000403:0x33:0x0] to /tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0 lhsmtool_posix: 1790919321.217990 [35148]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0' lhsmtool_posix: 1790919321.280876 [35148]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.7' from '/tmp/arc1/sanity-hsm.test_12u//0457/0x200000402:0x55:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0430/0x200000403:0x33:0x0' lhsmtool_posix: 1790919321.280958 [35148]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790919321.867933 [35177]: action=1 src=0x200000403:0x34:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8 mount_point=/mnt/lustre2 lhsmtool_posix: 1790919321.899586 [35177]: Resolving [0x200000403:0x34:0x0] to /tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0 lhsmtool_posix: 1790919321.899718 [35177]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0' lhsmtool_posix: 1790919322.001981 [35177]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.8' from '/tmp/arc1/sanity-hsm.test_12u//0454/0x200000402:0x56:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0437/0x200000403:0x34:0x0' lhsmtool_posix: 1790919322.010948 [35177]: process finished, errs: 0 major, 0 minor, rc=0 (Success) lhsmtool_posix: 1790919322.640322 [35206]: action=1 src=0x200000403:0x35:0x0 dst=/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9 mount_point=/mnt/lustre2 lhsmtool_posix: 1790919322.662269 [35206]: Resolving [0x200000403:0x35:0x0] to /tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0 lhsmtool_posix: 1790919322.662383 [35206]: importing '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0' lhsmtool_posix: 1790919322.739883 [35206]: imported '/mnt/lustre/d12u.sanity-hsm/f12u.sanity-hsm.9' from '/tmp/arc1/sanity-hsm.test_12u//0455/0x200000402:0x57:0x0'=='/tmp/arc1/sanity-hsm.test_12u//0436/0x200000403:0x35:0x0' lhsmtool_posix: 1790919322.739959 [35206]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 12u (37s) == sanity-hsm test 13: Recursively import and restore a directory ========================================================== 01:35:36 (1790919336) lhsmtool_posix: 1790919350.077080 [36967]: action=1 src=subdir dst=/mnt/lustre/d13.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790919350.333286 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1790919350.466831 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.10' from '/tmp/arc1/sanity-hsm.test_13//040a/0x240000402:0x8:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.10' lhsmtool_posix: 1790919350.467437 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1790919350.524502 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.9' from '/tmp/arc1/sanity-hsm.test_13//040b/0x240000402:0x9:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.9' lhsmtool_posix: 1790919350.524769 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1790919350.576369 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.8' from '/tmp/arc1/sanity-hsm.test_13//0408/0x240000402:0xa:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.8' lhsmtool_posix: 1790919350.577462 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1790919350.617947 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.7' from '/tmp/arc1/sanity-hsm.test_13//0409/0x240000402:0xb:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.7' lhsmtool_posix: 1790919350.619106 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1790919350.658026 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.6' from '/tmp/arc1/sanity-hsm.test_13//040e/0x240000402:0xc:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.6' lhsmtool_posix: 1790919350.658270 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1790919350.708059 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.5' from '/tmp/arc1/sanity-hsm.test_13//040f/0x240000402:0xd:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.5' lhsmtool_posix: 1790919350.708306 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1790919350.768285 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.4' from '/tmp/arc1/sanity-hsm.test_13//040c/0x240000402:0xe:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.4' lhsmtool_posix: 1790919350.768611 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1790919350.825657 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.3' from '/tmp/arc1/sanity-hsm.test_13//040d/0x240000402:0xf:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.3' lhsmtool_posix: 1790919350.826068 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1790919350.919295 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.2' from '/tmp/arc1/sanity-hsm.test_13//0412/0x240000402:0x10:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.2' lhsmtool_posix: 1790919350.920300 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1790919350.987300 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.10/file.1' from '/tmp/arc1/sanity-hsm.test_13//0413/0x240000402:0x11:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.10/file.1' lhsmtool_posix: 1790919351.038489 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1790919351.161731 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.10' from '/tmp/arc1/sanity-hsm.test_13//0411/0x240000402:0x13:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.10' lhsmtool_posix: 1790919351.161965 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1790919351.196965 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.9' from '/tmp/arc1/sanity-hsm.test_13//0416/0x240000402:0x14:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.9' lhsmtool_posix: 1790919351.197226 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1790919351.261381 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.8' from '/tmp/arc1/sanity-hsm.test_13//0417/0x240000402:0x15:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.8' lhsmtool_posix: 1790919351.261789 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1790919351.329690 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.7' from '/tmp/arc1/sanity-hsm.test_13//0414/0x240000402:0x16:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.7' lhsmtool_posix: 1790919351.336148 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1790919351.406777 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.6' from '/tmp/arc1/sanity-hsm.test_13//0415/0x240000402:0x17:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.6' lhsmtool_posix: 1790919351.412780 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1790919351.505343 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.5' from '/tmp/arc1/sanity-hsm.test_13//041a/0x240000402:0x18:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.5' lhsmtool_posix: 1790919351.507583 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1790919351.558573 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.4' from '/tmp/arc1/sanity-hsm.test_13//041b/0x240000402:0x19:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.4' lhsmtool_posix: 1790919351.558808 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1790919351.624242 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.3' from '/tmp/arc1/sanity-hsm.test_13//0418/0x240000402:0x1a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.3' lhsmtool_posix: 1790919351.628558 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1790919351.701883 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.2' from '/tmp/arc1/sanity-hsm.test_13//0419/0x240000402:0x1b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.2' lhsmtool_posix: 1790919351.711164 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1790919351.795740 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.9/file.1' from '/tmp/arc1/sanity-hsm.test_13//041e/0x240000402:0x1c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.9/file.1' lhsmtool_posix: 1790919351.831076 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1790919351.974198 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.10' from '/tmp/arc1/sanity-hsm.test_13//041c/0x240000402:0x1e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.10' lhsmtool_posix: 1790919351.989090 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1790919352.102157 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.9' from '/tmp/arc1/sanity-hsm.test_13//041d/0x240000402:0x1f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.9' lhsmtool_posix: 1790919352.103814 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1790919352.179834 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.8' from '/tmp/arc1/sanity-hsm.test_13//0422/0x240000402:0x20:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.8' lhsmtool_posix: 1790919352.180186 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1790919352.277877 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.7' from '/tmp/arc1/sanity-hsm.test_13//0423/0x240000402:0x21:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.7' lhsmtool_posix: 1790919352.278479 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1790919352.381111 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.6' from '/tmp/arc1/sanity-hsm.test_13//0420/0x240000402:0x22:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.6' lhsmtool_posix: 1790919352.381358 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1790919352.450287 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.5' from '/tmp/arc1/sanity-hsm.test_13//0421/0x240000402:0x23:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.5' lhsmtool_posix: 1790919352.451177 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1790919352.496649 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.4' from '/tmp/arc1/sanity-hsm.test_13//0426/0x240000402:0x24:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.4' lhsmtool_posix: 1790919352.497125 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1790919352.560893 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.3' from '/tmp/arc1/sanity-hsm.test_13//0427/0x240000402:0x25:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.3' lhsmtool_posix: 1790919352.561832 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1790919352.602197 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.2' from '/tmp/arc1/sanity-hsm.test_13//0424/0x240000402:0x26:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.2' lhsmtool_posix: 1790919352.604927 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1790919352.659268 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.8/file.1' from '/tmp/arc1/sanity-hsm.test_13//0425/0x240000402:0x27:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.8/file.1' lhsmtool_posix: 1790919352.666281 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1790919352.712005 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.10' from '/tmp/arc1/sanity-hsm.test_13//0459/0x200000402:0x5b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.10' lhsmtool_posix: 1790919352.712233 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1790919352.818723 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.9' from '/tmp/arc1/sanity-hsm.test_13//045e/0x200000402:0x5c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.9' lhsmtool_posix: 1790919352.819221 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1790919352.875884 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.8' from '/tmp/arc1/sanity-hsm.test_13//045f/0x200000402:0x5d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.8' lhsmtool_posix: 1790919352.876793 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1790919352.927283 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.7' from '/tmp/arc1/sanity-hsm.test_13//045c/0x200000402:0x5e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.7' lhsmtool_posix: 1790919352.928182 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1790919352.975618 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.6' from '/tmp/arc1/sanity-hsm.test_13//045d/0x200000402:0x5f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.6' lhsmtool_posix: 1790919352.975841 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1790919353.018608 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.5' from '/tmp/arc1/sanity-hsm.test_13//0462/0x200000402:0x60:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.5' lhsmtool_posix: 1790919353.018831 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1790919353.065794 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.4' from '/tmp/arc1/sanity-hsm.test_13//0463/0x200000402:0x61:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.4' lhsmtool_posix: 1790919353.066617 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1790919353.114153 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.3' from '/tmp/arc1/sanity-hsm.test_13//0460/0x200000402:0x62:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.3' lhsmtool_posix: 1790919353.115321 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1790919353.161720 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.2' from '/tmp/arc1/sanity-hsm.test_13//0461/0x200000402:0x63:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.2' lhsmtool_posix: 1790919353.162282 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1790919353.245912 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.7/file.1' from '/tmp/arc1/sanity-hsm.test_13//0466/0x200000402:0x64:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.7/file.1' lhsmtool_posix: 1790919353.260486 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1790919353.320329 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.10' from '/tmp/arc1/sanity-hsm.test_13//0464/0x200000402:0x66:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.10' lhsmtool_posix: 1790919353.320686 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1790919353.379640 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.9' from '/tmp/arc1/sanity-hsm.test_13//0465/0x200000402:0x67:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.9' lhsmtool_posix: 1790919353.380363 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1790919353.451571 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.8' from '/tmp/arc1/sanity-hsm.test_13//046a/0x200000402:0x68:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.8' lhsmtool_posix: 1790919353.452385 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1790919353.541860 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.7' from '/tmp/arc1/sanity-hsm.test_13//046b/0x200000402:0x69:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.7' lhsmtool_posix: 1790919353.542274 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1790919353.673223 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.6' from '/tmp/arc1/sanity-hsm.test_13//0468/0x200000402:0x6a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.6' lhsmtool_posix: 1790919353.674759 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1790919353.759832 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.5' from '/tmp/arc1/sanity-hsm.test_13//0469/0x200000402:0x6b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.5' lhsmtool_posix: 1790919353.763559 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1790919353.874342 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.4' from '/tmp/arc1/sanity-hsm.test_13//046e/0x200000402:0x6c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.4' lhsmtool_posix: 1790919353.875033 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1790919353.971626 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.3' from '/tmp/arc1/sanity-hsm.test_13//046f/0x200000402:0x6d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.3' lhsmtool_posix: 1790919353.971850 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1790919354.114391 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.2' from '/tmp/arc1/sanity-hsm.test_13//046c/0x200000402:0x6e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.2' lhsmtool_posix: 1790919354.114744 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1790919354.242780 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.6/file.1' from '/tmp/arc1/sanity-hsm.test_13//046d/0x200000402:0x6f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.6/file.1' lhsmtool_posix: 1790919354.264482 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1790919354.492656 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.10' from '/tmp/arc1/sanity-hsm.test_13//042b/0x240000402:0x29:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.10' lhsmtool_posix: 1790919354.492885 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1790919354.547877 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.9' from '/tmp/arc1/sanity-hsm.test_13//0428/0x240000402:0x2a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.9' lhsmtool_posix: 1790919354.548288 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1790919354.600748 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.8' from '/tmp/arc1/sanity-hsm.test_13//0429/0x240000402:0x2b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.8' lhsmtool_posix: 1790919354.603347 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1790919354.706849 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.7' from '/tmp/arc1/sanity-hsm.test_13//042e/0x240000402:0x2c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.7' lhsmtool_posix: 1790919354.707143 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1790919354.842587 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.6' from '/tmp/arc1/sanity-hsm.test_13//042f/0x240000402:0x2d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.6' lhsmtool_posix: 1790919354.842818 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1790919354.957765 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.5' from '/tmp/arc1/sanity-hsm.test_13//042c/0x240000402:0x2e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.5' lhsmtool_posix: 1790919354.962192 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1790919355.062046 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.4' from '/tmp/arc1/sanity-hsm.test_13//042d/0x240000402:0x2f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.4' lhsmtool_posix: 1790919355.067661 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1790919355.184230 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.3' from '/tmp/arc1/sanity-hsm.test_13//0432/0x240000402:0x30:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.3' lhsmtool_posix: 1790919355.184646 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1790919355.302848 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.2' from '/tmp/arc1/sanity-hsm.test_13//0433/0x240000402:0x31:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.2' lhsmtool_posix: 1790919355.303096 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1790919355.401012 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.5/file.1' from '/tmp/arc1/sanity-hsm.test_13//0430/0x240000402:0x32:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.5/file.1' lhsmtool_posix: 1790919355.411543 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1790919355.500086 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.10' from '/tmp/arc1/sanity-hsm.test_13//0473/0x200000402:0x71:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.10' lhsmtool_posix: 1790919355.504316 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1790919355.580323 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.9' from '/tmp/arc1/sanity-hsm.test_13//0470/0x200000402:0x72:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.9' lhsmtool_posix: 1790919355.581066 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1790919355.665595 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.8' from '/tmp/arc1/sanity-hsm.test_13//0471/0x200000402:0x73:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.8' lhsmtool_posix: 1790919355.665836 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1790919355.781816 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.7' from '/tmp/arc1/sanity-hsm.test_13//0476/0x200000402:0x74:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.7' lhsmtool_posix: 1790919355.787232 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1790919355.876601 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.6' from '/tmp/arc1/sanity-hsm.test_13//0477/0x200000402:0x75:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.6' lhsmtool_posix: 1790919355.876817 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1790919356.014824 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.5' from '/tmp/arc1/sanity-hsm.test_13//0474/0x200000402:0x76:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.5' lhsmtool_posix: 1790919356.015213 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1790919356.116355 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.4' from '/tmp/arc1/sanity-hsm.test_13//0475/0x200000402:0x77:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.4' lhsmtool_posix: 1790919356.124614 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1790919356.235991 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.3' from '/tmp/arc1/sanity-hsm.test_13//047a/0x200000402:0x78:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.3' lhsmtool_posix: 1790919356.252052 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1790919356.356065 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.2' from '/tmp/arc1/sanity-hsm.test_13//047b/0x200000402:0x79:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.2' lhsmtool_posix: 1790919356.360130 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1790919356.523606 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.4/file.1' from '/tmp/arc1/sanity-hsm.test_13//0478/0x200000402:0x7a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.4/file.1' lhsmtool_posix: 1790919356.545574 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1790919356.660719 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.10' from '/tmp/arc1/sanity-hsm.test_13//047e/0x200000402:0x7c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.10' lhsmtool_posix: 1790919356.662876 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1790919356.752944 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.9' from '/tmp/arc1/sanity-hsm.test_13//047f/0x200000402:0x7d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.9' lhsmtool_posix: 1790919356.753297 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1790919356.872289 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.8' from '/tmp/arc1/sanity-hsm.test_13//047c/0x200000402:0x7e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.8' lhsmtool_posix: 1790919356.878116 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1790919356.949164 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.7' from '/tmp/arc1/sanity-hsm.test_13//047d/0x200000402:0x7f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.7' lhsmtool_posix: 1790919356.950833 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1790919357.007305 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.6' from '/tmp/arc1/sanity-hsm.test_13//0482/0x200000402:0x80:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.6' lhsmtool_posix: 1790919357.011221 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1790919357.081159 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.5' from '/tmp/arc1/sanity-hsm.test_13//0483/0x200000402:0x81:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.5' lhsmtool_posix: 1790919357.082479 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1790919357.153999 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.4' from '/tmp/arc1/sanity-hsm.test_13//0480/0x200000402:0x82:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.4' lhsmtool_posix: 1790919357.156493 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1790919357.309051 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.3' from '/tmp/arc1/sanity-hsm.test_13//0481/0x200000402:0x83:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.3' lhsmtool_posix: 1790919357.309967 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1790919357.366016 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.2' from '/tmp/arc1/sanity-hsm.test_13//0486/0x200000402:0x84:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.2' lhsmtool_posix: 1790919357.366398 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1790919357.470209 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.3/file.1' from '/tmp/arc1/sanity-hsm.test_13//0487/0x200000402:0x85:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.3/file.1' lhsmtool_posix: 1790919357.484262 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1790919357.606449 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.10' from '/tmp/arc1/sanity-hsm.test_13//0485/0x200000402:0x87:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.10' lhsmtool_posix: 1790919357.611787 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1790919357.738524 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.9' from '/tmp/arc1/sanity-hsm.test_13//048a/0x200000402:0x88:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.9' lhsmtool_posix: 1790919357.738830 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1790919357.800846 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.8' from '/tmp/arc1/sanity-hsm.test_13//048b/0x200000402:0x89:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.8' lhsmtool_posix: 1790919357.801660 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1790919357.953595 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.7' from '/tmp/arc1/sanity-hsm.test_13//0488/0x200000402:0x8a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.7' lhsmtool_posix: 1790919357.953847 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1790919358.048041 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.6' from '/tmp/arc1/sanity-hsm.test_13//0489/0x200000402:0x8b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.6' lhsmtool_posix: 1790919358.056451 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1790919358.152371 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.5' from '/tmp/arc1/sanity-hsm.test_13//048e/0x200000402:0x8c:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.5' lhsmtool_posix: 1790919358.159317 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1790919358.283576 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.4' from '/tmp/arc1/sanity-hsm.test_13//048f/0x200000402:0x8d:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.4' lhsmtool_posix: 1790919358.283935 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1790919358.373645 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.3' from '/tmp/arc1/sanity-hsm.test_13//048c/0x200000402:0x8e:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.3' lhsmtool_posix: 1790919358.374319 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1790919358.479808 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.2' from '/tmp/arc1/sanity-hsm.test_13//048d/0x200000402:0x8f:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.2' lhsmtool_posix: 1790919358.481223 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1790919358.561633 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.2/file.1' from '/tmp/arc1/sanity-hsm.test_13//0492/0x200000402:0x90:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.2/file.1' lhsmtool_posix: 1790919358.578677 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1790919358.648080 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.10' from '/tmp/arc1/sanity-hsm.test_13//0490/0x200000402:0x92:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.10' lhsmtool_posix: 1790919358.649117 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1790919358.710396 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.9' from '/tmp/arc1/sanity-hsm.test_13//0491/0x200000402:0x93:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.9' lhsmtool_posix: 1790919358.710870 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1790919358.759391 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.8' from '/tmp/arc1/sanity-hsm.test_13//0496/0x200000402:0x94:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.8' lhsmtool_posix: 1790919358.761597 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1790919358.865976 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.7' from '/tmp/arc1/sanity-hsm.test_13//0497/0x200000402:0x95:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.7' lhsmtool_posix: 1790919358.866221 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1790919358.954564 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.6' from '/tmp/arc1/sanity-hsm.test_13//0494/0x200000402:0x96:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.6' lhsmtool_posix: 1790919358.959870 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1790919359.004169 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.5' from '/tmp/arc1/sanity-hsm.test_13//0495/0x200000402:0x97:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.5' lhsmtool_posix: 1790919359.004721 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1790919359.048621 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.4' from '/tmp/arc1/sanity-hsm.test_13//049a/0x200000402:0x98:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.4' lhsmtool_posix: 1790919359.049336 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1790919359.132234 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.3' from '/tmp/arc1/sanity-hsm.test_13//049b/0x200000402:0x99:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.3' lhsmtool_posix: 1790919359.132918 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1790919359.208272 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.2' from '/tmp/arc1/sanity-hsm.test_13//0498/0x200000402:0x9a:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.2' lhsmtool_posix: 1790919359.209284 [36967]: importing '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1790919359.284660 [36967]: imported '/mnt/lustre/d13.sanity-hsm/subdir/dir.1/file.1' from '/tmp/arc1/sanity-hsm.test_13//0499/0x200000402:0x9b:0x0'=='/tmp/arc1/sanity-hsm.test_13//subdir/dir.1/file.1' lhsmtool_posix: 1790919359.284819 [36967]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_13/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 13 (50s) == sanity-hsm test 14: Rebind archived file to a new fid ========================================================== 01:36:26 (1790919386) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_14/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.261282 s, 4.0 MB/s 0+0 records in 0+0 records out 0 bytes copied, 0.0105089 s, 0.0 kB/s rebind 0x200000403:0x7d:0x0 to 0x200000403:0x7e:0x0 lhsmtool_posix: 1790919391.249004 [37935]: action=2 src=0x200000403:0x7d:0x0 dst=0x200000403:0x7e:0x0 mount_point=/mnt/lustre2 lhsmtool_posix: 1790919391.250817 [37935]: rebind [0x200000403:0x7d:0x0] to [0x200000403:0x7e:0x0] lhsmtool_posix: 1790919391.251716 [37935]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 14 (12s) == sanity-hsm test 15: Rebind a list of files ============ 01:36:38 (1790919398) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_15/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.261824 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319111 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.390307 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222065 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353965 s, 3.0 MB/s pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 0+0 records in 0+0 records out 0 bytes copied, 0.00529708 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.0124923 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.010144 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.015435 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00448524 s, 0.0 kB/s rebind list of files lhsmtool_posix: 1790919408.316724 [38722]: action=2 src=/tmp/tmp.8412 dst=(null) mount_point=/mnt/lustre2 lhsmtool_posix: 1790919408.323204 [38722]: rebind [0x200000403:0x81:0x0] to [0x200000403:0x86:0x0] lhsmtool_posix: 1790919408.324280 [38722]: rebind [0x200000403:0x82:0x0] to [0x200000403:0x87:0x0] lhsmtool_posix: 1790919408.324892 [38722]: rebind [0x200000403:0x83:0x0] to [0x200000403:0x88:0x0] lhsmtool_posix: 1790919408.324951 [38722]: rebind [0x200000403:0x84:0x0] to [0x200000403:0x89:0x0] lhsmtool_posix: 1790919408.325006 [38722]: rebind [0x200000403:0x85:0x0] to [0x200000403:0x8a:0x0] lhsmtool_posix: 1790919408.325086 [38722]: 5 lines read from '/tmp/tmp.8412', 5 rebind successful lhsmtool_posix: 1790919408.325110 [38722]: process finished, errs: 0 major, 0 minor, rc=0 (Success) PASS 15 (25s) == sanity-hsm test 16: Test CT bandwith control option === 01:37:03 (1790919423) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_16/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 0.0996976 s, 210 MB/s Waiting 200s for 'SUCCEED' Waiting 190s for 'SUCCEED' Updated after 21s: want 'SUCCEED' got 'SUCCEED' PASS 16 (34s) == sanity-hsm test 20: Release is not permitted ========== 01:37:37 (1790919457) 0+0 records in 0+0 records out 0 bytes copied, 0.0106614 s, 0.0 kB/s 0x200000403:0x92:0x0 Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d20.sanity-hsm/f20.sanity-hsm): Operation not permitted PASS 20 (9s) == sanity-hsm test 21: Simple release tests ============== 01:37:46 (1790919466) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_21/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2425 s, 4.3 MB/s 1+0 records in 1+0 records out 512 bytes copied, 0.23482 s, 2.2 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 21 (15s) == sanity-hsm test 22: Could not swap a release file ===== 01:38:01 (1790919481) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_22/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214468 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194507 s, 5.4 MB/s 0x200000403:0x95:0x0 lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d22.sanity-hsm/test_swap' and '/mnt/lustre/d22.sanity-hsm/test_release': Operation not supported (95) PASS 22 (12s) == sanity-hsm test 23: Release does not change a/mtime (utime) ========================================================== 01:38:13 (1790919493) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_23/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.378204 s, 2.8 MB/s PASS 23 (12s) == sanity-hsm test 24a: Archive, release, and restore does not change a/mtime (i/o) ========================================================== 01:38:25 (1790919505) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352438 s, 3.0 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s 192.168.204.136@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg436-client.virtnet /mnt/lustre (opts:) Starting client: oleg436-client.virtnet: -o user_xattr,flock 192.168.204.136@tcp:/lustre /mnt/lustre PASS 24a (19s) == sanity-hsm test 24b: root can archive, release, and restore user files ========================================================== 01:38:44 (1790919524) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273326 s, 3.8 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_state] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] /mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm: (0x00000009) exists archived, archive_id:1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [md5sum] [/mnt/lustre/d24b.sanity-hsm/f24b.sanity-hsm] PASS 24b (16s) == sanity-hsm test 24c: check that user,group,other request masks work ========================================================== 01:39:00 (1790919540) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320584 s, 3.3 MB/s 0x200000403:0x9c:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134319 s, 7.8 MB/s 0x200000403:0x9d:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.330762 s, 3.2 MB/s 0x200000403:0x9e:0x0 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] Cannot send HSM request (use of /mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm): Operation not permitted running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [hsm_archive] [/mnt/lustre/d24c.sanity-hsm/f24c.sanity-hsm] PASS 24c (25s) == sanity-hsm test 24d: check that read-only mounts are respected ========================================================== 01:39:25 (1790919565) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251373 s, 4.2 MB/s 0x200000403:0xa0:0x0 /mnt/lustre/d24d.sanity-hsm/f24d.sanity-hsm lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 86 0x56 0x2c0000401 Starting client: oleg436-client.virtnet: -o user_xattr,flock 192.168.204.136@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 1.5066 s, 696 kB/s Cannot send HSM request (use of /mnt/lustre2/d24d.sanity-hsm/f24d.sanity-hsm): Read-only file system Stopping client oleg436-client.virtnet /mnt/lustre3 (opts:) PASS 24d (22s) == sanity-hsm test 24e: tar succeeds on HSM released files ========================================================== 01:39:47 (1790919587) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.329579 s, 3.2 MB/s /mnt/lustre/d24e.sanity-hsm/f24e.sanity-hsm: (0x0000000d) released exists archived, archive_id:1 tar: Removing leading `/' from member names PASS 24e (13s) == sanity-hsm test 24f: root can archive, release, and restore tar files ========================================================== 01:40:00 (1790919600) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24f/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm f24f.sanity-hsm f24f.sanity-hsm Sum0 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm, sum1 = 54fb6627dbaa37721048e4549db3224d /mnt/lustre/d24f.sanity-hsm/f24f.sanity-hsm PASS 24f (12s) == sanity-hsm test 24g: write by non-owner still sets dirty ========================================================== 01:40:12 (1790919612) RUNAS = 'runas -u 500 -g 500' Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_24g/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Please listen carefully as our options have changed. running as uid/gid/euid/egid 500/500/500/500, groups: 500 [tee] [/mnt/lustre/d24g.sanity-hsm/f24g.sanity-hsm] To be electrocuted by your telephone, press #. PASS 24g (12s) == sanity-hsm test 25a: Restore lost file (HS_LOST flag) from import ========================================================== 01:40:25 (1790919625) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790919629.230139 [47737]: action=1 src=d25a.sanity-hsm/f25a.sanity-hsm dst=/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790919629.250354 [47737]: importing '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1790919629.412259 [47737]: imported '/mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_25a//0411/0x200000405:0x14:0x0'=='/tmp/arc1/sanity-hsm.test_25a//d25a.sanity-hsm/f25a.sanity-hsm' lhsmtool_posix: 1790919629.412354 [47737]: process finished, errs: 0 major, 0 minor, rc=0 (Success) md5sum: /mnt/lustre/d25a.sanity-hsm/f25a.sanity-hsm: No data available PASS 25a (13s) == sanity-hsm test 25b: Restore lost file (HS_LOST flag) after release ========================================================== 01:40:39 (1790919639) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_25b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' md5sum: /mnt/lustre/d25b.sanity-hsm/f25b.sanity-hsm: No data available PASS 25b (12s) == sanity-hsm test 26A: Remove the archive of a valid file ========================================================== 01:40:50 (1790919650) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00761942 s, 0.0 kB/s PASS 26A (11s) == sanity-hsm test 26a: Remove Archive On Last Unlink (RAoLU) policy ========================================================== 01:41:02 (1790919662) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 26a (32s) == sanity-hsm test 26b: RAoLU policy when CDT off ======== 01:41:35 (1790919695) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 26b (28s) == sanity-hsm test 26c: RAoLU effective when file closed ========================================================== 01:42:04 (1790919724) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' multiop /mnt/lustre/d26c.sanity-hsm/f26c.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8412 Waiting 200s for 'SUCCEED' Updated after 5s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 6s: want 'SUCCEED' got 'SUCCEED' PASS 26c (44s) == sanity-hsm test 26d: RAoLU when Client eviction ======= 01:42:48 (1790919768) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.32069 s, 3.3 MB/s multiop /mnt/lustre/d26d.sanity-hsm/f26d.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8412 Waiting 200s for 'SUCCEED' Updated after 10s: want 'SUCCEED' got 'SUCCEED' PASS 26d (42s) == sanity-hsm test 26e: RAoLU with a non-started coordinator ========================================================== 01:43:30 (1790919810) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276175 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267582 s, 3.9 MB/s Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s Failing mds1 on oleg436-server Stopping /mnt/lustre-mds1 (opts:) on oleg436-server 01:44:06 (1790919846) shut down facet: mds1 facet_host: oleg436-server facet_failover_host: oleg436-server Failover mds1 to oleg436-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg436-server: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 Started lustre-MDT0000 01:44:40 (1790919880) targets are mounted 01:44:40 (1790919880) facet_failover done oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Check cookie from RAoLU request (last: 1790919000, remove: 1790919001) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_26e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 26e (127s) == sanity-hsm test 27a: Remove the archive of an imported file (Operation not permitted) ========================================================== 01:45:37 (1790919937) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.34893 s, 30.3 MB/s lhsmtool_posix: 1790919942.079740 [54806]: action=1 src=d27a.sanity-hsm/f27a.sanity-hsm dst=/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790919942.087009 [54806]: importing '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1790919942.326360 [54806]: imported '/mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_27a//0407/0x240000404:0x3:0x0'=='/tmp/arc1/sanity-hsm.test_27a//d27a.sanity-hsm/f27a.sanity-hsm' lhsmtool_posix: 1790919942.326828 [54806]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Cannot send HSM request (use of /mnt/lustre/d27a.sanity-hsm/f27a.sanity-hsm): Operation not permitted PASS 27a (11s) == sanity-hsm test 27b: Remove the archive of a relased file (Operation not permitted) ========================================================== 01:45:48 (1790919948) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_27b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00670351 s, 0.0 kB/s Cannot send HSM request (use of /mnt/lustre/d27b.sanity-hsm/f27b.sanity-hsm): Operation not permitted PASS 27b (10s) == sanity-hsm test 28: Concurrent archive/file remove ==== 01:45:58 (1790919958) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_28/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00646726 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 28 (19s) == sanity-hsm test 29a: Tests --mntpath and --archive options ========================================================== 01:46:17 (1790919977) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 7 "/mnt/lustre"' Cannot send HSM request (use of /mnt/lustre): Invalid argument hsm: '/qwerty/uyt' is not a valid FID PASS 29a (9s) == sanity-hsm test 29b: Archive/delete/remove by FID from the archive. ========================================================== 01:46:26 (1790919986) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.309189 s, 3.4 MB/s PASS 29b (10s) == sanity-hsm test 29c: Archive/delete/remove by FID, using a file list. ========================================================== 01:46:36 (1790919996) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_29c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27624 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182448 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152549 s, 6.9 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 29c (20s) == sanity-hsm test 29d: hsm_remove by FID with archive_id 0 for unlinked file cause ========================================================== 01:46:56 (1790920016) SKIP: sanity-hsm test_29d Need 3 or more clients, have 1 SKIP 29d (4s) == sanity-hsm test 30a: Restore at exec (import case) ==== 01:47:00 (1790920020) SKIP: sanity-hsm test_30a Need 2 or more clients, have 1 SKIP 30a (5s) == sanity-hsm test 30b: Restore at exec (release case) === 01:47:05 (1790920025) SKIP: sanity-hsm test_30b Need 2 or more clients, have 1 SKIP 30b (5s) == sanity-hsm test 30c: Update during exec of released file must fail ========================================================== 01:47:10 (1790920030) SKIP: sanity-hsm test_30c Need 2 or more clients, have 1 SKIP 30c (4s) == sanity-hsm test 31a: Import a large file and check size during restore ========================================================== 01:47:14 (1790920034) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.00712 s, 40.6 MB/s lhsmtool_posix: 1790920038.813966 [59968]: action=1 src=d31a.sanity-hsm/f31a.sanity-hsm dst=/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790920038.826541 [59968]: importing '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1790920038.929164 [59968]: imported '/mnt/lustre/d31a.sanity-hsm/f31a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_31a//040d/0x200000407:0xa:0x0'=='/tmp/arc1/sanity-hsm.test_31a//d31a.sanity-hsm/f31a.sanity-hsm' lhsmtool_posix: 1790920038.929228 [59968]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' size seen is right: 40894464 == 40894464 done PASS 31a (22s) == sanity-hsm test 31b: Restore a large unaligned file and check size during restore ========================================================== 01:47:36 (1790920056) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 1.15257 s, 33.8 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' size seen is right: 39000000 == 39000000 done PASS 31b (24s) == sanity-hsm test 31c: Restore a large aligned file and check size during restore ========================================================== 01:48:00 (1790920080) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_31c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.1695 s, 35.0 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' size seen is right: 40894464 == 40894464 done PASS 31c (25s) == sanity-hsm test 33: Kill a restore waiting process ==== 01:48:25 (1790920105) 0+0 records in 0+0 records out 0 bytes copied, 0.0107986 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_33/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2893: 62002 Terminated md5sum $f > /dev/null PASS 33 (13s) == sanity-hsm test 34: Remove file during restore ======== 01:48:38 (1790920118) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_34/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.0140076 s, 0.0 kB/s Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2925: kill: (62663) - No such process PASS 34 (12s) == sanity-hsm test 35: Overwrite file during restore ===== 01:48:50 (1790920130) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_35/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00852452 s, 0.0 kB/s Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2962: kill: (63341) - No such process PASS 35 (11s) == sanity-hsm test 36: Move file during restore ========== 01:49:01 (1790920141) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_36/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.00368327 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet /home/green/git/lustre-release/lustre/tests/sanity-hsm.sh: line 2998: kill: (64024) - No such process PASS 36 (13s) == sanity-hsm test 37: re-archive a dirty file =========== 01:49:14 (1790920154) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_37/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.26517 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597663 s, 17.5 MB/s PASS 37 (72s) == sanity-hsm test 40: Parallel archive requests ========= 01:50:27 (1790920227) oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1152921504606846976: Cannot allocate memory oleg436-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1152921504606846976': Cannot allocate memory pdsh@oleg436-client: oleg436-server: ssh exited with exit code 12 oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=1152921504606846976: Cannot allocate memory oleg436-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='1152921504606846976': Cannot allocate memory pdsh@oleg436-client: oleg436-server: ssh exited with exit code 12 set max_requests=1152921504606846976 failed with 12 [ 1757.018856] Lustre: lustre-MDT0000: No more memory to set HSM max_requests=1152921504606846976 (max request memory: 461MB, current total 6/975941): rc = -12 [ 1757.944519] Lustre: lustre-MDT0001: No more memory to set HSM max_requests=1152921504606846976 (max request memory: 461MB, current total 975941/975941): rc = -12 Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_40/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Wait for all requests being enqueued...100 archive requests submitted 100 archive requests submitted 100 archive requests submitted 100 archive requests submitted OK pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 PASS 40 (172s) == sanity-hsm test 50: Archive with large number of pending HSM actions ========================================================== 01:53:18 (1790920398) oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=1000000: Cannot allocate memory oleg436-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='1000000': Cannot allocate memory pdsh@oleg436-client: oleg436-server: ssh exited with exit code 12 oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=1000000: Cannot allocate memory oleg436-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='1000000': Cannot allocate memory pdsh@oleg436-client: oleg436-server: ssh exited with exit code 12 Filesystem Inodes IUsed IFree IUse% Mounted on 192.168.204.136@tcp:/lustre 236390 1318 235072 1% /mnt/lustre - create 1227 (time 1790920416.22 total 10.00 last 122.64) - create 2310 (time 1790920426.22 total 20.01 last 108.23) - create 3582 (time 1790920436.23 total 30.01 last 127.15) - create 4779 (time 1790920446.23 total 40.02 last 119.65) - create 6067 (time 1790920456.24 total 50.02 last 128.75) - create 7096 (time 1790920466.24 total 60.02 last 102.89) - create 8132 (time 1790920476.24 total 70.03 last 103.56) - create 9209 (time 1790920486.24 total 80.03 last 107.66) total: 10000 create in 87.45 seconds: 114.35 ops/second 0 - unlinked 0 (time 1790920590 ; total 0 ; last 0) total: 10000 unlinks in 82 seconds: 121.951218 unlinks/second Start Phase files_num: 10000 time: 90 - create 1390 (time 1790920688.20 total 10.00 last 138.94) - create 2689 (time 1790920698.20 total 20.01 last 129.88) - create 3910 (time 1790920708.21 total 30.01 last 121.98) - create 5030 (time 1790920718.21 total 40.02 last 111.99) - create 6074 (time 1790920728.21 total 50.02 last 104.39) - create 7165 (time 1790920738.21 total 60.02 last 109.07) - create 8663 (time 1790920748.22 total 70.03 last 149.69) - create 9969 (time 1790920758.22 total 80.03 last 130.57) - create 11246 (time 1790920768.23 total 90.03 last 127.67) - create 12340 (time 1790920778.23 total 100.03 last 109.38) - create 13590 (time 1790920788.23 total 110.03 last 125.00) - create 14775 (time 1790920798.23 total 120.04 last 118.45) - create 16048 (time 1790920808.25 total 130.06 last 127.03) - create 17270 (time 1790920818.26 total 140.06 last 122.13) - create 18274 (time 1790920828.28 total 150.08 last 100.24) - create 19436 (time 1790920838.30 total 160.11 last 115.86) total: 20000 create in 164.10 seconds: 121.87 ops/second - unlinked 0 (time 1790921032 ; total 0 ; last 0) - unlinked 10000 (time 1790921122 ; total 90 ; last 90) total: 20000 unlinks in 176 seconds: 113.636360 unlinks/second Middle Phase files_num: 20000 time: 184 - create 1355 (time 1790921222.84 total 10.00 last 135.49) - create 2546 (time 1790921232.85 total 20.00 last 119.06) - create 3751 (time 1790921242.85 total 30.01 last 120.43) - create 5010 (time 1790921252.87 total 40.03 last 125.71) - create 6202 (time 1790921262.88 total 50.03 last 119.13) - create 7274 (time 1790921272.88 total 60.03 last 107.17) - create 8523 (time 1790921282.88 total 70.04 last 124.88) - create 9902 (time 1790921292.88 total 80.04 last 137.85) total: 10000 create in 80.73 seconds: 123.87 ops/second 0 - unlinked 0 (time 1790921397 ; total 0 ; last 0) total: 10000 unlinks in 88 seconds: 113.636360 unlinks/second End Phase files_num: 10000 time: 97 0 mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 50 (1109s) == sanity-hsm test 52: Opened for write file on an evicted client should be set dirty ========================================================== 02:11:47 (1790921507) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_52/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.589198 s, 1.8 MB/s multiop /mnt/lustre/d52.sanity-hsm/f52.sanity-hsm vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.8412 pdsh@oleg436-client: oleg436-client: ssh exited with exit code 5 PASS 52 (16s) == sanity-hsm test 53: Opened for read file on an evicted client should not be set dirty ========================================================== 02:12:03 (1790921523) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_53/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264729 s, 4.0 MB/s multiop /mnt/lustre/d53.sanity-hsm/f53.sanity-hsm vo_c TMPPIPE=/tmp/multiop_open_wait_pipe.8412 pdsh@oleg436-client: oleg436-client: ssh exited with exit code 5 PASS 53 (15s) == sanity-hsm test 54: Write during an archive cancels it ========================================================== 02:12:18 (1790921538) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 1.30799 s, 29.8 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_54/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Waiting 200s for 'FAILED' Updated after 35s: want 'FAILED' got 'FAILED' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 54 (51s) == sanity-hsm test 55: Truncate during an archive cancels it ========================================================== 02:13:09 (1790921589) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 0.991068 s, 39.4 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_55/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 55 (13s) == sanity-hsm test 56: Setattr during an archive is ok === 02:13:22 (1790921602) 39+0 records in 39+0 records out 39000000 bytes (39 MB, 37 MiB) copied, 1.18682 s, 32.9 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_56/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 37s: want 'SUCCEED' got 'SUCCEED' PASS 56 (47s) == sanity-hsm test 57: Archive a file with dirty cache on another node ========================================================== 02:14:09 (1790921649) SKIP: sanity-hsm test_57 Need 2 or more clients, have 1 SKIP 57 (3s) == sanity-hsm test 58: Truncate a released file will trigger restore ========================================================== 02:14:12 (1790921652) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_58/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' truncate up from 1516 to 3032 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate down from 1516 to 758 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 truncate to 0 /mnt/lustre/d58.sanity-hsm/f58.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:2 PASS 58 (19s) == sanity-hsm test 59: Release stripeless file with non-zero size ========================================================== 02:14:31 (1790921671) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_59/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 59 (9s) == sanity-hsm test 60: Changing progress update interval from default ========================================================== 02:14:40 (1790921680) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.246464 s, 42.5 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_60/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 --update-interval 5 "/mnt/lustre2"' Waiting 100s for 'start copy' Expecting a progress update within 20 seconds... 5242880 bytes copied in 2 seconds. Wait for on going archive hsm action to complete Waiting 10s for 'copied' Updated after 4s: want 'copied' got 'copied' PASS 60 (21s) == sanity-hsm test 61: Waiting archive of a removed file should fail ========================================================== 02:15:01 (1790921701) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_61/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 61 (18s) == sanity-hsm test 70: Copytool logs JSON register/unregister events to FIFO ========================================================== 02:15:19 (1790921719) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_70/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --event-fifo /tmp/sanity-hsm.test_70.Pawr/fifo "/mnt/lustre2"' Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s Register/Unregister events look OK. PASS 70 (11s) == sanity-hsm test 71: Copytool logs JSON archive events to FIFO ========================================================== 02:15:30 (1790921730) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_71/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_71.mvtQ/fifo "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.249742 s, 4.2 MB/s Waiting 200s for 'SUCCEED' Updated after 3s: want 'SUCCEED' got 'SUCCEED' Archive events look OK. mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 71 (25s) == sanity-hsm test 72: Copytool logs JSON restore events to FIFO ========================================================== 02:15:55 (1790921755) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_72/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --update-interval 5 --event-fifo /tmp/sanity-hsm.test_72.YW8w/fifo "/mnt/lustre2"' 16+0 records in 16+0 records out 16000000 bytes (16 MB, 15 MiB) copied, 0.608645 s, 26.3 MB/s lhsmtool_posix: 1790921760.551914 [83090]: action=1 src=d72.sanity-hsm/f72.sanity-hsm dst=/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790921760.573118 [83090]: importing '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1790921760.640619 [83090]: imported '/mnt/lustre/d72.sanity-hsm/f72.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_72//0bca/0x200000bd2:0x18:0x0'=='/tmp/arc1/sanity-hsm.test_72//d72.sanity-hsm/f72.sanity-hsm' lhsmtool_posix: 1790921760.640679 [83090]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Verifying released state: Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Restore events look OK. PASS 72 (22s) == sanity-hsm test 90: Archive/restore a file list ======= 02:16:17 (1790921777) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_90/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Waiting 200s for '' pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 Waiting 200s for '' Changed after 2s: from 'lrh=[type=10680000 len=136 idx=2/98] fid=[0x200000403:0xd9:0x0] dfid=[0x200000403:0xd9:0x0] compound/cookie=0x0/0x6abf41b9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/100] fid=[0x200000403:0xdb:0x0] dfid=[0x200000403:0xdb:0x0] compound/cookie=0x0/0x6abf41bb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/101] fid=[0x200000403:0xdc:0x0] dfid=[0x200000403:0xdc:0x0] compound/cookie=0x0/0x6abf41bc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/102] fid=[0x200000403:0xdd:0x0] dfid=[0x200000403:0xdd:0x0] compound/cookie=0x0/0x6abf41bd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/103] fid=[0x200000403:0xde:0x0] dfid=[0x200000403:0xde:0x0] compound/cookie=0x0/0x6abf41be action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/104] fid=[0x200000403:0xdf:0x0] dfid=[0x200000403:0xdf:0x0] compound/cookie=0x0/0x6abf41bf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/105] fid=[0x200000403:0xe0:0x0] dfid=[0x200000403:0xe0:0x0] compound/cookie=0x0/0x6abf41c0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/106] fid=[0x200000403:0xe1:0x0] dfid=[0x200000403:0xe1:0x0] compound/cookie=0x0/0x6abf41c1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/107] fid=[0x200000403:0xe2:0x0] dfid=[0x200000403:0xe2:0x0] compound/cookie=0x0/0x6abf41c2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/108] fid=[0x200000403:0xe3:0x0] dfid=[0x200000403:0xe3:0x0] compound/cookie=0x0/0x6abf41c3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/109] fid=[0x200000403:0xe4:0x0] dfid=[0x200000403:0xe4:0x0] compound/cookie=0x0/0x6abf41c4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/110] fid=[0x200000403:0xe5:0x0] dfid=[0x200000403:0xe5:0x0] compound/cookie=0x0/0x6abf41c5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/111] fid=[0x200000403:0xe6:0x0] dfid=[0x200000403:0xe6:0x0] compound/cookie=0x0/0x6abf41c6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/112] fid=[0x200000403:0xe7:0x0] dfid=[0x200000403:0xe7:0x0] compound/cookie=0x0/0x6abf41c7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/113] fid=[0x200000403:0xe8:0x0] dfid=[0x200000403:0xe8:0x0] compound/cookie=0x0/0x6abf41c8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/114] fid=[0x200000403:0xe9:0x0] dfid=[0x200000403:0xe9:0x0] compound/cookie=0x0/0x6abf41c9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/115] fid=[0x200000403:0xea:0x0] dfid=[0x200000403:0xea:0x0] compound/cookie=0x0/0x6abf41ca action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/116] fid=[0x200000403:0xeb:0x0] dfid=[0x200000403:0xeb:0x0] compound/cookie=0x0/0x6abf41cb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/117] fid=[0x200000403:0xec:0x0] dfid=[0x200000403:0xec:0x0] compound/cookie=0x0/0x6abf41cc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/118] fid=[0x200000403:0xed:0x0] dfid=[0x200000403:0xed:0x0] compound/cookie=0x0/0x6abf41cd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/119] fid=[0x200000403:0xee:0x0] dfid=[0x200000403:0xee:0x0] compound/cookie=0x0/0x6abf41ce action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/120] fid=[0x200000403:0xef:0x0] dfid=[0x200000403:0xef:0x0] compound/cookie=0x0/0x6abf41cf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/121] fid=[0x200000403:0xf0:0x0] dfid=[0x200000403:0xf0:0x0] compound/cookie=0x0/0x6abf41d0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/123] fid=[0x200000403:0xf1:0x0] dfid=[0x200000403:0xf1:0x0] compound/cookie=0x0/0x6abf41d1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/124] fid=[0x200000403:0xf2:0x0] dfid=[0x200000403:0xf2:0x0] compound/cookie=0x0/0x6abf41d2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/125] fid=[0x200000403:0xf3:0x0] dfid=[0x200000403:0xf3:0x0] compound/cookie=0x0/0x6abf41d3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/126] fid=[0x200000403:0xf4:0x0] dfid=[0x200000403:0xf4:0x0] compound/cookie=0x0/0x6abf41d4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/127] fid=[0x200000403:0xf5:0x0] dfid=[0x200000403:0xf5:0x0] compound/cookie=0x0/0x6abf41d5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/128] fid=[0x200000403:0xf6:0x0] dfid=[0x200000403:0xf6:0x0] compound/cookie=0x0/0x6abf41d6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/129] fid=[0x200000403:0xf7:0x0] dfid=[0x200000403:0xf7:0x0] compound/cookie=0x0/0x6abf41d7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/130] fid=[0x200000403:0xf8:0x0] dfid=[0x200000403:0xf8:0x0] compound/cookie=0x0/0x6abf41d8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/131] fid=[0x200000403:0xf9:0x0] dfid=[0x200000403:0xf9:0x0] compound/cookie=0x0/0x6abf41d9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/132] fid=[0x200000403:0xfa:0x0] dfid=[0x200000403:0xfa:0x0] compound/cookie=0x0/0x6abf41da action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/133] fid=[0x200000403:0xfb:0x0] dfid=[0x200000403:0xfb:0x0] compound/cookie=0x0/0x6abf41db action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/134] fid=[0x200000403:0xfc:0x0] dfid=[0x200000403:0xfc:0x0] compound/cookie=0x0/0x6abf41dc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/135] fid=[0x200000403:0xfd:0x0] dfid=[0x200000403:0xfd:0x0] compound/cookie=0x0/0x6abf41dd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/136] fid=[0x200000403:0xfe:0x0] dfid=[0x200000403:0xfe:0x0] compound/cookie=0x0/0x6abf41de action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/137] fid=[0x200000403:0xff:0x0] dfid=[0x200000403:0xff:0x0] compound/cookie=0x0/0x6abf41df action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/138] fid=[0x200000403:0x100:0x0] dfid=[0x200000403:0x100:0x0] compound/cookie=0x0/0x6abf41e0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/139] fid=[0x200000403:0x101:0x0] dfid=[0x200000403:0x101:0x0] compound/cookie=0x0/0x6abf41e1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/140] fid=[0x200000403:0x102:0x0] dfid=[0x200000403:0x102:0x0] compound/cookie=0x0/0x6abf41e2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/141] fid=[0x200000403:0x103:0x0] dfid=[0x200000403:0x103:0x0] compound/cookie=0x0/0x6abf41e3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/142] fid=[0x200000403:0x104:0x0] dfid=[0x200000403:0x104:0x0] compound/cookie=0x0/0x6abf41e4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/143] fid=[0x200000403:0x105:0x0] dfid=[0x200000403:0x105:0x0] compound/cookie=0x0/0x6abf41e5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/144] fid=[0x200000403:0x106:0x0] dfid=[0x200000403:0x106:0x0] compound/cookie=0x0/0x6abf41e6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/145] fid=[0x200000403:0x107:0x0] dfid=[0x200000403:0x107:0x0] compound/cookie=0x0/0x6abf41e7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/146] fid=[0x200000403:0x108:0x0] dfid=[0x200000403:0x108:0x0] compound/cookie=0x0/0x6abf41e8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/147] fid=[0x200000403:0x109:0x0] dfid=[0x200000403:0x109:0x0] compound/cookie=0x0/0x6abf41e9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/148] fid=[0x200000403:0x10a:0x0] dfid=[0x200000403:0x10a:0x0] compound/cookie=0x0/0x6abf41ea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=2/109] fid=[0x200000403:0xe4:0x0] dfid=[0x200000403:0xe4:0x0] compound/cookie=0x0/0x6abf41c4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/110] fid=[0x200000403:0xe5:0x0] dfid=[0x200000403:0xe5:0x0] compound/cookie=0x0/0x6abf41c5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/111] fid=[0x200000403:0xe6:0x0] dfid=[0x200000403:0xe6:0x0] compound/cookie=0x0/0x6abf41c6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/112] fid=[0x200000403:0xe7:0x0] dfid=[0x200000403:0xe7:0x0] compound/cookie=0x0/0x6abf41c7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/113] fid=[0x200000403:0xe8:0x0] dfid=[0x200000403:0xe8:0x0] compound/cookie=0x0/0x6abf41c8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/114] fid=[0x200000403:0xe9:0x0] dfid=[0x200000403:0xe9:0x0] compound/cookie=0x0/0x6abf41c9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/115] fid=[0x200000403:0xea:0x0] dfid=[0x200000403:0xea:0x0] compound/cookie=0x0/0x6abf41ca action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/116] fid=[0x200000403:0xeb:0x0] dfid=[0x200000403:0xeb:0x0] compound/cookie=0x0/0x6abf41cb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/117] fid=[0x200000403:0xec:0x0] dfid=[0x200000403:0xec:0x0] compound/cookie=0x0/0x6abf41cc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/118] fid=[0x200000403:0xed:0x0] dfid=[0x200000403:0xed:0x0] compound/cookie=0x0/0x6abf41cd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/119] fid=[0x200000403:0xee:0x0] dfid=[0x200000403:0xee:0x0] compound/cookie=0x0/0x6abf41ce action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/120] fid=[0x200000403:0xef:0x0] dfid=[0x200000403:0xef:0x0] compound/cookie=0x0/0x6abf41cf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/121] fid=[0x200000403:0xf0:0x0] dfid=[0x200000403:0xf0:0x0] compound/cookie=0x0/0x6abf41d0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/123] fid=[0x200000403:0xf1:0x0] dfid=[0x200000403:0xf1:0x0] compound/cookie=0x0/0x6abf41d1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/124] fid=[0x200000403:0xf2:0x0] dfid=[0x200000403:0xf2:0x0] compound/cookie=0x0/0x6abf41d2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/125] fid=[0x200000403:0xf3:0x0] dfid=[0x200000403:0xf3:0x0] compound/cookie=0x0/0x6abf41d3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/126] fid=[0x200000403:0xf4:0x0] dfid=[0x200000403:0xf4:0x0] compound/cookie=0x0/0x6abf41d4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/127] fid=[0x200000403:0xf5:0x0] dfid=[0x200000403:0xf5:0x0] compound/cookie=0x0/0x6abf41d5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/128] fid=[0x200000403:0xf6:0x0] dfid=[0x200000403:0xf6:0x0] compound/cookie=0x0/0x6abf41d6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/129] fid=[0x200000403:0xf7:0x0] dfid=[0x200000403:0xf7:0x0] compound/cookie=0x0/0x6abf41d7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/130] fid=[0x200000403:0xf8:0x0] dfid=[0x200000403:0xf8:0x0] compound/cookie=0x0/0x6abf41d8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/131] fid=[0x200000403:0xf9:0x0] dfid=[0x200000403:0xf9:0x0] compound/cookie=0x0/0x6abf41d9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/132] fid=[0x200000403:0xfa:0x0] dfid=[0x200000403:0xfa:0x0] compound/cookie=0x0/0x6abf41da action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/133] fid=[0x200000403:0xfb:0x0] dfid=[0x200000403:0xfb:0x0] compound/cookie=0x0/0x6abf41db action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/134] fid=[0x200000403:0xfc:0x0] dfid=[0x200000403:0xfc:0x0] compound/cookie=0x0/0x6abf41dc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/135] fid=[0x200000403:0xfd:0x0] dfid=[0x200000403:0xfd:0x0] compound/cookie=0x0/0x6abf41dd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/136] fid=[0x200000403:0xfe:0x0] dfid=[0x200000403:0xfe:0x0] compound/cookie=0x0/0x6abf41de action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/137] fid=[0x200000403:0xff:0x0] dfid=[0x200000403:0xff:0x0] compound/cookie=0x0/0x6abf41df action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/138] fid=[0x200000403:0x100:0x0] dfid=[0x200000403:0x100:0x0] compound/cookie=0x0/0x6abf41e0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/139] fid=[0x200000403:0x101:0x0] dfid=[0x200000403:0x101:0x0] compound/cookie=0x0/0x6abf41e1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/140] fid=[0x200000403:0x102:0x0] dfid=[0x200000403:0x102:0x0] compound/cookie=0x0/0x6abf41e2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/141] fid=[0x200000403:0x103:0x0] dfid=[0x200000403:0x103:0x0] compound/cookie=0x0/0x6abf41e3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/142] fid=[0x200000403:0x104:0x0] dfid=[0x200000403:0x104:0x0] compound/cookie=0x0/0x6abf41e4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/143] fid=[0x200000403:0x105:0x0] dfid=[0x200000403:0x105:0x0] compound/cookie=0x0/0x6abf41e5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/144] fid=[0x200000403:0x106:0x0] dfid=[0x200000403:0x106:0x0] compound/cookie=0x0/0x6abf41e6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/145] fid=[0x200000403:0x107:0x0] dfid=[0x200000403:0x107:0x0] compound/cookie=0x0/0x6abf41e7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/146] fid=[0x200000403:0x108:0x0] dfid=[0x200000403:0x108:0x0] compound/cookie=0x0/0x6abf41e8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/147] fid=[0x200000403:0x109:0x0] dfid=[0x200000403:0x109:0x0] compound/cookie=0x0/0x6abf41e9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/148] fid=[0x200000403:0x10a:0x0] dfid=[0x200000403:0x10a:0x0] compound/cookie=0x0/0x6abf41ea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 4s: from 'lrh=[type=10680000 len=136 idx=2/109] fid=[0x200000403:0xe4:0x0] dfid=[0x200000403:0xe4:0x0] compound/cookie=0x0/0x6abf41c4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/110] fid=[0x200000403:0xe5:0x0] dfid=[0x200000403:0xe5:0x0] compound/cookie=0x0/0x6abf41c5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/111] fid=[0x200000403:0xe6:0x0] dfid=[0x200000403:0xe6:0x0] compound/cookie=0x0/0x6abf41c6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/112] fid=[0x200000403:0xe7:0x0] dfid=[0x200000403:0xe7:0x0] compound/cookie=0x0/0x6abf41c7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/113] fid=[0x200000403:0xe8:0x0] dfid=[0x200000403:0xe8:0x0] compound/cookie=0x0/0x6abf41c8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/114] fid=[0x200000403:0xe9:0x0] dfid=[0x200000403:0xe9:0x0] compound/cookie=0x0/0x6abf41c9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/115] fid=[0x200000403:0xea:0x0] dfid=[0x200000403:0xea:0x0] compound/cookie=0x0/0x6abf41ca action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/116] fid=[0x200000403:0xeb:0x0] dfid=[0x200000403:0xeb:0x0] compound/cookie=0x0/0x6abf41cb action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/117] fid=[0x200000403:0xec:0x0] dfid=[0x200000403:0xec:0x0] compound/cookie=0x0/0x6abf41cc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/118] fid=[0x200000403:0xed:0x0] dfid=[0x200000403:0xed:0x0] compound/cookie=0x0/0x6abf41cd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/119] fid=[0x200000403:0xee:0x0] dfid=[0x200000403:0xee:0x0] compound/cookie=0x0/0x6abf41ce action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/120] fid=[0x200000403:0xef:0x0] dfid=[0x200000403:0xef:0x0] compound/cookie=0x0/0x6abf41cf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/121] fid=[0x200000403:0xf0:0x0] dfid=[0x200000403:0xf0:0x0] compound/cookie=0x0/0x6abf41d0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/123] fid=[0x200000403:0xf1:0x0] dfid=[0x200000403:0xf1:0x0] compound/cookie=0x0/0x6abf41d1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/124] fid=[0x200000403:0xf2:0x0] dfid=[0x200000403:0xf2:0x0] compound/cookie=0x0/0x6abf41d2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/125] fid=[0x200000403:0xf3:0x0] dfid=[0x200000403:0xf3:0x0] compound/cookie=0x0/0x6abf41d3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/126] fid=[0x200000403:0xf4:0x0] dfid=[0x200000403:0xf4:0x0] compound/cookie=0x0/0x6abf41d4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/127] fid=[0x200000403:0xf5:0x0] dfid=[0x200000403:0xf5:0x0] compound/cookie=0x0/0x6abf41d5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/128] fid=[0x200000403:0xf6:0x0] dfid=[0x200000403:0xf6:0x0] compound/cookie=0x0/0x6abf41d6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/129] fid=[0x200000403:0xf7:0x0] dfid=[0x200000403:0xf7:0x0] compound/cookie=0x0/0x6abf41d7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/130] fid=[0x200000403:0xf8:0x0] dfid=[0x200000403:0xf8:0x0] compound/cookie=0x0/0x6abf41d8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/131] fid=[0x200000403:0xf9:0x0] dfid=[0x200000403:0xf9:0x0] compound/cookie=0x0/0x6abf41d9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/132] fid=[0x200000403:0xfa:0x0] dfid=[0x200000403:0xfa:0x0] compound/cookie=0x0/0x6abf41da action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/133] fid=[0x200000403:0xfb:0x0] dfid=[0x200000403:0xfb:0x0] compound/cookie=0x0/0x6abf41db action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/134] fid=[0x200000403:0xfc:0x0] dfid=[0x200000403:0xfc:0x0] compound/cookie=0x0/0x6abf41dc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/135] fid=[0x200000403:0xfd:0x0] dfid=[0x200000403:0xfd:0x0] compound/cookie=0x0/0x6abf41dd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/136] fid=[0x200000403:0xfe:0x0] dfid=[0x200000403:0xfe:0x0] compound/cookie=0x0/0x6abf41de action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/137] fid=[0x200000403:0xff:0x0] dfid=[0x200000403:0xff:0x0] compound/cookie=0x0/0x6abf41df action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/138] fid=[0x200000403:0x100:0x0] dfid=[0x200000403:0x100:0x0] compound/cookie=0x0/0x6abf41e0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/139] fid=[0x200000403:0x101:0x0] dfid=[0x200000403:0x101:0x0] compound/cookie=0x0/0x6abf41e1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/140] fid=[0x200000403:0x102:0x0] dfid=[0x200000403:0x102:0x0] compound/cookie=0x0/0x6abf41e2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/141] fid=[0x200000403:0x103:0x0] dfid=[0x200000403:0x103:0x0] compound/cookie=0x0/0x6abf41e3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/142] fid=[0x200000403:0x104:0x0] dfid=[0x200000403:0x104:0x0] compound/cookie=0x0/0x6abf41e4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/143] fid=[0x200000403:0x105:0x0] dfid=[0x200000403:0x105:0x0] compound/cookie=0x0/0x6abf41e5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/144] fid=[0x200000403:0x106:0x0] dfid=[0x200000403:0x106:0x0] compound/cookie=0x0/0x6abf41e6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/145] fid=[0x200000403:0x107:0x0] dfid=[0x200000403:0x107:0x0] compound/cookie=0x0/0x6abf41e7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/146] fid=[0x200000403:0x108:0x0] dfid=[0x200000403:0x108:0x0] compound/cookie=0x0/0x6abf41e8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/147] fid=[0x200000403:0x109:0x0] dfid=[0x200000403:0x109:0x0] compound/cookie=0x0/0x6abf41e9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/148] fid=[0x200000403:0x10a:0x0] dfid=[0x200000403:0x10a:0x0] compound/cookie=0x0/0x6abf41ea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=2/118] fid=[0x200000403:0xed:0x0] dfid=[0x200000403:0xed:0x0] compound/cookie=0x0/0x6abf41cd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/119] fid=[0x200000403:0xee:0x0] dfid=[0x200000403:0xee:0x0] compound/cookie=0x0/0x6abf41ce action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/120] fid=[0x200000403:0xef:0x0] dfid=[0x200000403:0xef:0x0] compound/cookie=0x0/0x6abf41cf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/121] fid=[0x200000403:0xf0:0x0] dfid=[0x200000403:0xf0:0x0] compound/cookie=0x0/0x6abf41d0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/123] fid=[0x200000403:0xf1:0x0] dfid=[0x200000403:0xf1:0x0] compound/cookie=0x0/0x6abf41d1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/124] fid=[0x200000403:0xf2:0x0] dfid=[0x200000403:0xf2:0x0] compound/cookie=0x0/0x6abf41d2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/125] fid=[0x200000403:0xf3:0x0] dfid=[0x200000403:0xf3:0x0] compound/cookie=0x0/0x6abf41d3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/126] fid=[0x200000403:0xf4:0x0] dfid=[0x200000403:0xf4:0x0] compound/cookie=0x0/0x6abf41d4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/127] fid=[0x200000403:0xf5:0x0] dfid=[0x200000403:0xf5:0x0] compound/cookie=0x0/0x6abf41d5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/128] fid=[0x200000403:0xf6:0x0] dfid=[0x200000403:0xf6:0x0] compound/cookie=0x0/0x6abf41d6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/129] fid=[0x200000403:0xf7:0x0] dfid=[0x200000403:0xf7:0x0] compound/cookie=0x0/0x6abf41d7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/130] fid=[0x200000403:0xf8:0x0] dfid=[0x200000403:0xf8:0x0] compound/cookie=0x0/0x6abf41d8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/131] fid=[0x200000403:0xf9:0x0] dfid=[0x200000403:0xf9:0x0] compound/cookie=0x0/0x6abf41d9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/132] fid=[0x200000403:0xfa:0x0] dfid=[0x200000403:0xfa:0x0] compound/cookie=0x0/0x6abf41da action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/133] fid=[0x200000403:0xfb:0x0] dfid=[0x200000403:0xfb:0x0] compound/cookie=0x0/0x6abf41db action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/134] fid=[0x200000403:0xfc:0x0] dfid=[0x200000403:0xfc:0x0] compound/cookie=0x0/0x6abf41dc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/135] fid=[0x200000403:0xfd:0x0] dfid=[0x200000403:0xfd:0x0] compound/cookie=0x0/0x6abf41dd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/136] fid=[0x200000403:0xfe:0x0] dfid=[0x200000403:0xfe:0x0] compound/cookie=0x0/0x6abf41de action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/137] fid=[0x200000403:0xff:0x0] dfid=[0x200000403:0xff:0x0] compound/cookie=0x0/0x6abf41df action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/138] fid=[0x200000403:0x100:0x0] dfid=[0x200000403:0x100:0x0] compound/cookie=0x0/0x6abf41e0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/139] fid=[0x200000403:0x101:0x0] dfid=[0x200000403:0x101:0x0] compound/cookie=0x0/0x6abf41e1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/140] fid=[0x200000403:0x102:0x0] dfid=[0x200000403:0x102:0x0] compound/cookie=0x0/0x6abf41e2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/141] fid=[0x200000403:0x103:0x0] dfid=[0x200000403:0x103:0x0] compound/cookie=0x0/0x6abf41e3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/142] fid=[0x200000403:0x104:0x0] dfid=[0x200000403:0x104:0x0] compound/cookie=0x0/0x6abf41e4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/143] fid=[0x200000403:0x105:0x0] dfid=[0x200000403:0x105:0x0] compound/cookie=0x0/0x6abf41e5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/144] fid=[0x200000403:0x106:0x0] dfid=[0x200000403:0x106:0x0] compound/cookie=0x0/0x6abf41e6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/145] fid=[0x200000403:0x107:0x0] dfid=[0x200000403:0x107:0x0] compound/cookie=0x0/0x6abf41e7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/146] fid=[0x200000403:0x108:0x0] dfid=[0x200000403:0x108:0x0] compound/cookie=0x0/0x6abf41e8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/147] fid=[0x200000403:0x109:0x0] dfid=[0x200000403:0x109:0x0] compound/cookie=0x0/0x6abf41e9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/148] fid=[0x200000403:0x10a:0x0] dfid=[0x200000403:0x10a:0x0] compound/cookie=0x0/0x6abf41ea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 6s: from 'lrh=[type=10680000 len=136 idx=2/118] fid=[0x200000403:0xed:0x0] dfid=[0x200000403:0xed:0x0] compound/cookie=0x0/0x6abf41cd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/119] fid=[0x200000403:0xee:0x0] dfid=[0x200000403:0xee:0x0] compound/cookie=0x0/0x6abf41ce action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/120] fid=[0x200000403:0xef:0x0] dfid=[0x200000403:0xef:0x0] compound/cookie=0x0/0x6abf41cf action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/121] fid=[0x200000403:0xf0:0x0] dfid=[0x200000403:0xf0:0x0] compound/cookie=0x0/0x6abf41d0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/123] fid=[0x200000403:0xf1:0x0] dfid=[0x200000403:0xf1:0x0] compound/cookie=0x0/0x6abf41d1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/124] fid=[0x200000403:0xf2:0x0] dfid=[0x200000403:0xf2:0x0] compound/cookie=0x0/0x6abf41d2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/125] fid=[0x200000403:0xf3:0x0] dfid=[0x200000403:0xf3:0x0] compound/cookie=0x0/0x6abf41d3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/126] fid=[0x200000403:0xf4:0x0] dfid=[0x200000403:0xf4:0x0] compound/cookie=0x0/0x6abf41d4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/127] fid=[0x200000403:0xf5:0x0] dfid=[0x200000403:0xf5:0x0] compound/cookie=0x0/0x6abf41d5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/128] fid=[0x200000403:0xf6:0x0] dfid=[0x200000403:0xf6:0x0] compound/cookie=0x0/0x6abf41d6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/129] fid=[0x200000403:0xf7:0x0] dfid=[0x200000403:0xf7:0x0] compound/cookie=0x0/0x6abf41d7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/130] fid=[0x200000403:0xf8:0x0] dfid=[0x200000403:0xf8:0x0] compound/cookie=0x0/0x6abf41d8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/131] fid=[0x200000403:0xf9:0x0] dfid=[0x200000403:0xf9:0x0] compound/cookie=0x0/0x6abf41d9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/132] fid=[0x200000403:0xfa:0x0] dfid=[0x200000403:0xfa:0x0] compound/cookie=0x0/0x6abf41da action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/133] fid=[0x200000403:0xfb:0x0] dfid=[0x200000403:0xfb:0x0] compound/cookie=0x0/0x6abf41db action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/134] fid=[0x200000403:0xfc:0x0] dfid=[0x200000403:0xfc:0x0] compound/cookie=0x0/0x6abf41dc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/135] fid=[0x200000403:0xfd:0x0] dfid=[0x200000403:0xfd:0x0] compound/cookie=0x0/0x6abf41dd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/136] fid=[0x200000403:0xfe:0x0] dfid=[0x200000403:0xfe:0x0] compound/cookie=0x0/0x6abf41de action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/137] fid=[0x200000403:0xff:0x0] dfid=[0x200000403:0xff:0x0] compound/cookie=0x0/0x6abf41df action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/138] fid=[0x200000403:0x100:0x0] dfid=[0x200000403:0x100:0x0] compound/cookie=0x0/0x6abf41e0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/139] fid=[0x200000403:0x101:0x0] dfid=[0x200000403:0x101:0x0] compound/cookie=0x0/0x6abf41e1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/140] fid=[0x200000403:0x102:0x0] dfid=[0x200000403:0x102:0x0] compound/cookie=0x0/0x6abf41e2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/141] fid=[0x200000403:0x103:0x0] dfid=[0x200000403:0x103:0x0] compound/cookie=0x0/0x6abf41e3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/142] fid=[0x200000403:0x104:0x0] dfid=[0x200000403:0x104:0x0] compound/cookie=0x0/0x6abf41e4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/143] fid=[0x200000403:0x105:0x0] dfid=[0x200000403:0x105:0x0] compound/cookie=0x0/0x6abf41e5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/144] fid=[0x200000403:0x106:0x0] dfid=[0x200000403:0x106:0x0] compound/cookie=0x0/0x6abf41e6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/145] fid=[0x200000403:0x107:0x0] dfid=[0x200000403:0x107:0x0] compound/cookie=0x0/0x6abf41e7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/146] fid=[0x200000403:0x108:0x0] dfid=[0x200000403:0x108:0x0] compound/cookie=0x0/0x6abf41e8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/147] fid=[0x200000403:0x109:0x0] dfid=[0x200000403:0x109:0x0] compound/cookie=0x0/0x6abf41e9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/148] fid=[0x200000403:0x10a:0x0] dfid=[0x200000403:0x10a:0x0] compound/cookie=0x0/0x6abf41ea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=2/131] fid=[0x200000403:0xf9:0x0] dfid=[0x200000403:0xf9:0x0] compound/cookie=0x0/0x6abf41d9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/132] fid=[0x200000403:0xfa:0x0] dfid=[0x200000403:0xfa:0x0] compound/cookie=0x0/0x6abf41da action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/133] fid=[0x200000403:0xfb:0x0] dfid=[0x200000403:0xfb:0x0] compound/cookie=0x0/0x6abf41db action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/134] fid=[0x200000403:0xfc:0x0] dfid=[0x200000403:0xfc:0x0] compound/cookie=0x0/0x6abf41dc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/135] fid=[0x200000403:0xfd:0x0] dfid=[0x200000403:0xfd:0x0] compound/cookie=0x0/0x6abf41dd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/136] fid=[0x200000403:0xfe:0x0] dfid=[0x200000403:0xfe:0x0] compound/cookie=0x0/0x6abf41de action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/137] fid=[0x200000403:0xff:0x0] dfid=[0x200000403:0xff:0x0] compound/cookie=0x0/0x6abf41df action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/138] fid=[0x200000403:0x100:0x0] dfid=[0x200000403:0x100:0x0] compound/cookie=0x0/0x6abf41e0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/139] fid=[0x200000403:0x101:0x0] dfid=[0x200000403:0x101:0x0] compound/cookie=0x0/0x6abf41e1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/140] fid=[0x200000403:0x102:0x0] dfid=[0x200000403:0x102:0x0] compound/cookie=0x0/0x6abf41e2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/141] fid=[0x200000403:0x103:0x0] dfid=[0x200000403:0x103:0x0] compound/cookie=0x0/0x6abf41e3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/142] fid=[0x200000403:0x104:0x0] dfid=[0x200000403:0x104:0x0] compound/cookie=0x0/0x6abf41e4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/143] fid=[0x200000403:0x105:0x0] dfid=[0x200000403:0x105:0x0] compound/cookie=0x0/0x6abf41e5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/144] fid=[0x200000403:0x106:0x0] dfid=[0x200000403:0x106:0x0] compound/cookie=0x0/0x6abf41e6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/145] fid=[0x200000403:0x107:0x0] dfid=[0x200000403:0x107:0x0] compound/cookie=0x0/0x6abf41e7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/146] fid=[0x200000403:0x108:0x0] dfid=[0x200000403:0x108:0x0] compound/cookie=0x0/0x6abf41e8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/147] fid=[0x200000403:0x109:0x0] dfid=[0x200000403:0x109:0x0] compound/cookie=0x0/0x6abf41e9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/148] fid=[0x200000403:0x10a:0x0] dfid=[0x200000403:0x10a:0x0] compound/cookie=0x0/0x6abf41ea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' Changed after 9s: from 'lrh=[type=10680000 len=136 idx=2/131] fid=[0x200000403:0xf9:0x0] dfid=[0x200000403:0xf9:0x0] compound/cookie=0x0/0x6abf41d9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/132] fid=[0x200000403:0xfa:0x0] dfid=[0x200000403:0xfa:0x0] compound/cookie=0x0/0x6abf41da action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/133] fid=[0x200000403:0xfb:0x0] dfid=[0x200000403:0xfb:0x0] compound/cookie=0x0/0x6abf41db action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/134] fid=[0x200000403:0xfc:0x0] dfid=[0x200000403:0xfc:0x0] compound/cookie=0x0/0x6abf41dc action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/135] fid=[0x200000403:0xfd:0x0] dfid=[0x200000403:0xfd:0x0] compound/cookie=0x0/0x6abf41dd action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/136] fid=[0x200000403:0xfe:0x0] dfid=[0x200000403:0xfe:0x0] compound/cookie=0x0/0x6abf41de action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/137] fid=[0x200000403:0xff:0x0] dfid=[0x200000403:0xff:0x0] compound/cookie=0x0/0x6abf41df action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/138] fid=[0x200000403:0x100:0x0] dfid=[0x200000403:0x100:0x0] compound/cookie=0x0/0x6abf41e0 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/139] fid=[0x200000403:0x101:0x0] dfid=[0x200000403:0x101:0x0] compound/cookie=0x0/0x6abf41e1 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/140] fid=[0x200000403:0x102:0x0] dfid=[0x200000403:0x102:0x0] compound/cookie=0x0/0x6abf41e2 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/141] fid=[0x200000403:0x103:0x0] dfid=[0x200000403:0x103:0x0] compound/cookie=0x0/0x6abf41e3 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/142] fid=[0x200000403:0x104:0x0] dfid=[0x200000403:0x104:0x0] compound/cookie=0x0/0x6abf41e4 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/143] fid=[0x200000403:0x105:0x0] dfid=[0x200000403:0x105:0x0] compound/cookie=0x0/0x6abf41e5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/144] fid=[0x200000403:0x106:0x0] dfid=[0x200000403:0x106:0x0] compound/cookie=0x0/0x6abf41e6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/145] fid=[0x200000403:0x107:0x0] dfid=[0x200000403:0x107:0x0] compound/cookie=0x0/0x6abf41e7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/146] fid=[0x200000403:0x108:0x0] dfid=[0x200000403:0x108:0x0] compound/cookie=0x0/0x6abf41e8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/147] fid=[0x200000403:0x109:0x0] dfid=[0x200000403:0x109:0x0] compound/cookie=0x0/0x6abf41e9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/148] fid=[0x200000403:0x10a:0x0] dfid=[0x200000403:0x10a:0x0] compound/cookie=0x0/0x6abf41ea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' to 'lrh=[type=10680000 len=136 idx=2/143] fid=[0x200000403:0x105:0x0] dfid=[0x200000403:0x105:0x0] compound/cookie=0x0/0x6abf41e5 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/144] fid=[0x200000403:0x106:0x0] dfid=[0x200000403:0x106:0x0] compound/cookie=0x0/0x6abf41e6 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/145] fid=[0x200000403:0x107:0x0] dfid=[0x200000403:0x107:0x0] compound/cookie=0x0/0x6abf41e7 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] lrh=[type=10680000 len=136 idx=2/146] fid=[0x200000403:0x108:0x0] dfid=[0x200000403:0x108:0x0] compound/cookie=0x0/0x6abf41e8 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/147] fid=[0x200000403:0x109:0x0] dfid=[0x200000403:0x109:0x0] compound/cookie=0x0/0x6abf41e9 action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[] lrh=[type=10680000 len=136 idx=2/148] fid=[0x200000403:0x10a:0x0] dfid=[0x200000403:0x10a:0x0] compound/cookie=0x0/0x6abf41ea action=RESTORE archive#=1 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=WAITING data=[]' pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 PASS 90 (101s) == sanity-hsm test 100: Set coordinator /proc tunables === 02:17:59 (1790921879) Testing mdt.lustre-MDT0000.hsm.loop_period Set loop_period to 0 must failed oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/loop_period=0: Invalid argument oleg436-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/loop_period'='0': Invalid argument pdsh@oleg436-client: oleg436-server: ssh exited with exit code 22 oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/loop_period=0: Invalid argument oleg436-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/loop_period'='0': Invalid argument pdsh@oleg436-client: oleg436-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.grace_delay Set grace_delay to 0 must failed oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/grace_delay=0: Invalid argument oleg436-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/grace_delay'='0': Invalid argument pdsh@oleg436-client: oleg436-server: ssh exited with exit code 22 oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/grace_delay=0: Invalid argument oleg436-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/grace_delay'='0': Invalid argument pdsh@oleg436-client: oleg436-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.active_request_timeout Set active_request_timeout to 0 must failed oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/active_request_timeout=0: Invalid argument oleg436-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/active_request_timeout'='0': Invalid argument pdsh@oleg436-client: oleg436-server: ssh exited with exit code 22 oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/active_request_timeout=0: Invalid argument oleg436-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/active_request_timeout'='0': Invalid argument pdsh@oleg436-client: oleg436-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.max_requests Set max_requests to 0 must failed oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/max_requests=0: Invalid argument oleg436-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/max_requests'='0': Invalid argument pdsh@oleg436-client: oleg436-server: ssh exited with exit code 22 oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/max_requests=0: Invalid argument oleg436-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/max_requests'='0': Invalid argument pdsh@oleg436-client: oleg436-server: ssh exited with exit code 22 Testing mdt.lustre-MDT0000.hsm.default_archive_id Set default_archive_id to 0 must failed oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0000/hsm/default_archive_id=0: Invalid argument oleg436-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/default_archive_id'='0': Invalid argument pdsh@oleg436-client: oleg436-server: ssh exited with exit code 22 oleg436-server: error: set_param: setting /sys/fs/lustre/mdt/lustre-MDT0001/hsm/default_archive_id=0: Invalid argument oleg436-server: error: set_param: setting 'mdt/lustre-MDT0001/hsm/default_archive_id'='0': Invalid argument pdsh@oleg436-client: oleg436-server: ssh exited with exit code 22 PASS 100 (63s) == sanity-hsm test 102: Verify coordinator control ======= 02:19:03 (1790921943) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 102 (27s) == sanity-hsm test 103: Purge all requests =============== 02:19:30 (1790921970) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge Current requests pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 PASS 103 (21s) == sanity-hsm test 103a: Purge pending restore requests == 02:19:51 (1790921991) mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_103a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' (0x200000403:0x13e:0x0|0x200000403:0x13f:0x0|0x200000403:0x140:0x0|0x200000403:0x141:0x0|0x200000403:0x142:0x0|0x200000403:0x143:0x0|0x200000403:0x144:0x0|0x200000403:0x145:0x0).*action=ARCHIVE.*status=SUCCEED Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s cat: '/mnt/lustre/.lustre/fid/0x200000403:0x13f:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x140:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x143:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x141:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x13e:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x145:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x142:0x0': No data available cat: '/mnt/lustre/.lustre/fid/0x200000403:0x144:0x0': No data available mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 103a (27s) == sanity-hsm test 104: Copy tool data field ============= 02:20:18 (1790922018) 0+0 records in 0+0 records out 0 bytes copied, 0.0124604 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 104 (17s) == sanity-hsm test 105: Restart of coordinator =========== 02:20:35 (1790922035) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled mdt.lustre-MDT0000.hsm_control=purge mdt.lustre-MDT0001.hsm_control=purge PASS 105 (44s) == sanity-hsm test 106a: Copytool register/unregister ==== 02:21:19 (1790922079) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg436-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg436-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg436-client.virtnet: 89893 Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 uuid not found in agent list on mds2 Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg436-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg436-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 106a (49s) == sanity-hsm test 106b: Unregister agent after a copytool crash ========================================================== 02:22:08 (1790922128) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg436-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg436-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg436-client.virtnet: 91632 Killing existing copytools on oleg436-client.virtnet with signal KILL copytools stopped in 0s pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 uuid not found in agent list on mds2 Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 106b (30s) == sanity-hsm test 106c: Unregister copytool on client eviction ========================================================== 02:22:38 (1790922158) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_106c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg436-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg436-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 oleg436-client.virtnet: 92865 oleg436-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg436-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg436-client.virtnet: executing get_client_uuid /mnt/lustre2 mdc.lustre-MDT0000-mdc-ffff9dd7515db000.active=0 mdc.lustre-MDT0001-mdc-ffff9dd7515db000.active=0 mdt.lustre-MDT0000.evict_client=640cc026-fd80-495d-b9e7-b5da3242e14f mdt.lustre-MDT0001.evict_client=640cc026-fd80-495d-b9e7-b5da3242e14f pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 uuid not found in agent list on mds2 mdc.lustre-MDT0000-mdc-ffff9dd7515db000.active=1 mdc.lustre-MDT0001-mdc-ffff9dd7515db000.active=1 oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 mdc.lustre-MDT0000-mdc-ffff9dd7515db000.active=1 mdc.lustre-MDT0001-mdc-ffff9dd7515db000.active=1 PASS 106c (62s) == sanity-hsm test 107: Copytool re-register after MDS restart ========================================================== 02:23:40 (1790922220) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_107/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Failing mds1 on oleg436-server Stopping /mnt/lustre-mds1 (opts:) on oleg436-server 02:23:51 (1790922231) shut down facet: mds1 facet_host: oleg436-server facet_failover_host: oleg436-server Failover mds1 to oleg436-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg436-server: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 Started lustre-MDT0000 02:24:26 (1790922266) targets are mounted 02:24:26 (1790922266) facet_failover done oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 107 (67s) == sanity-hsm test 109: Policy display/change ============ 02:24:47 (1790922287) mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0000.hsm.policy=NRA NBR Next set_param must failed oleg436-server: error: set_param: setting /sys/kernel/debug/lustre/mdt/lustre-MDT0000/hsm/policy=wrong: Invalid argument oleg436-server: error: set_param: setting 'mdt/lustre-MDT0000/hsm/policy'='wrong': Invalid argument pdsh@oleg436-client: oleg436-server: ssh exited with exit code 22 Back to default policy mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 109 (28s) == sanity-hsm test 110a: Non blocking restore policy (import case) ========================================================== 02:25:15 (1790922315) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790922318.692683 [97475]: action=1 src=d110a.sanity-hsm/f110a.sanity-hsm dst=/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790922318.707681 [97475]: importing '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1790922318.828310 [97475]: imported '/mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_110a//0bb7/0x200000bd2:0x65:0x0'=='/tmp/arc1/sanity-hsm.test_110a//d110a.sanity-hsm/f110a.sanity-hsm' lhsmtool_posix: 1790922318.828394 [97475]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0001.hsm.policy=+NBR md5sum: /mnt/lustre/d110a.sanity-hsm/f110a.sanity-hsm: No data available mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 110a (16s) == sanity-hsm test 110b: Non blocking restore policy (release case) ========================================================== 02:25:31 (1790922331) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_110b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NBR mdt.lustre-MDT0001.hsm.policy=+NBR md5sum: /mnt/lustre/d110b.sanity-hsm/f110b.sanity-hsm: No data available Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' mdt.lustre-MDT0000.hsm.policy=-NBR mdt.lustre-MDT0001.hsm.policy=-NBR PASS 110b (18s) == sanity-hsm test 111a: No retry policy (import case), restore will error ========================================================== 02:25:50 (1790922350) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790922352.812169 [98873]: action=1 src=d111a.sanity-hsm/f111a.sanity-hsm dst=/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790922352.821965 [98873]: importing '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1790922352.901357 [98873]: imported '/mnt/lustre/d111a.sanity-hsm/f111a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_111a//0bb8/0x200000bd2:0x6a:0x0'=='/tmp/arc1/sanity-hsm.test_111a//d111a.sanity-hsm/f111a.sanity-hsm' lhsmtool_posix: 1790922352.903191 [98873]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Remove from backend: 0x200000bd2:0x6a:0x0 = /tmp/arc1/sanity-hsm.test_111a/0bb8/0x200000bd2:0x6a:0x0 mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 111a (16s) == sanity-hsm test 111b: No retry policy (release case), restore will error ========================================================== 02:26:05 (1790922365) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_111b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm.policy=+NRA mdt.lustre-MDT0001.hsm.policy=+NRA Remove from backend: 0x200000bd3:0x7:0x0 = /tmp/arc1/sanity-hsm.test_111b/0bd4/0x200000bd3:0x7:0x0 mdt.lustre-MDT0000.hsm.policy=-NRA mdt.lustre-MDT0001.hsm.policy=-NRA PASS 111b (14s) == sanity-hsm test 112: State of recorded request ======== 02:26:20 (1790922380) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_112/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled /mnt/lustre/d112.sanity-hsm/f112.sanity-hsm: ARCHIVE waiting (from 0 to EOF) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 112 (17s) == sanity-hsm test 113: wrong stat after restore ========= 02:26:38 (1790922398) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0668544 s, 15.7 MB/s Starting client: oleg436-client.virtnet: -o user_xattr,flock 192.168.204.136@tcp:/lustre /mnt/lustre3 Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_113/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre3"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Verifying released state: multiop /mnt/lustre/d113.sanity-hsm/f113.sanity-hsm voO_WRONLY:O_APPEND:_w4c TMPPIPE=/tmp/multiop_open_wait_pipe.8412 File: /mnt/lustre2/d113.sanity-hsm/f113.sanity-hsm Size: 1048576 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115238860488715 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-10-02 02:26:39.000000000 -0400 Modify: 2026-10-02 02:26:39.000000000 -0400 Change: 2026-10-02 02:26:39.000000000 -0400 Birth: 2026-10-02 02:26:40.000000000 -0400 Stopping client oleg436-client.virtnet /mnt/lustre3 (opts:) PASS 113 (18s) == sanity-hsm test 114: Incompatible request does not set other requests as STARTED ========================================================== 02:26:56 (1790922416) 0+0 records in 0+0 records out 0 bytes copied, 0.0036963 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00728974 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_114/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 114 (18s) == sanity-hsm test 200: Register/Cancel archive ========== 02:27:14 (1790922434) 0+0 records in 0+0 records out 0 bytes copied, 0.0107357 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_200/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet PASS 200 (12s) == sanity-hsm test 201: Register/Cancel restore ========== 02:27:27 (1790922447) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_201/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 1.33278 s, 30.7 MB/s lhsmtool_posix: 1790922450.917604 [103271]: action=1 src=d201.sanity-hsm/f201.sanity-hsm dst=/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790922450.925241 [103271]: importing '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1790922450.995810 [103271]: imported '/mnt/lustre/d201.sanity-hsm/f201.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_201//0ba1/0x200000bd2:0x73:0x0'=='/tmp/arc1/sanity-hsm.test_201//d201.sanity-hsm/f201.sanity-hsm' lhsmtool_posix: 1790922450.995958 [103271]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 201 (19s) == sanity-hsm test 202: Register/Cancel remove =========== 02:27:46 (1790922466) 0+0 records in 0+0 records out 0 bytes copied, 0.00868555 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_202/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg436-client.virtnet PASS 202 (11s) == sanity-hsm test 220A: Changelog for archive =========== 02:27:58 (1790922478) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' lustre-MDT0000.3 16HSM 06:28:07.138145079 2026.10.02 0x0 t=[0x200000bd3:0x13:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 220A (20s) == sanity-hsm test 220a: Changelog for failed archive ==== 02:28:18 (1790922498) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_220a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet lustre-MDT0000.6 16HSM 06:28:27.596650459 2026.10.02 0x2 t=[0x200000bd3:0x14:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 220a (22s) == sanity-hsm test 221: Changelog for archive canceled === 02:28:40 (1790922520) 0+0 records in 0+0 records out 0 bytes copied, 0.0123783 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_221/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl3 cl3' Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet lustre-MDT0000.8 16HSM 06:28:52.803533952 2026.10.02 0x7d t=[0x200000bd3:0x16:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001: Deregistered changelog user #3 lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 221 (28s) == sanity-hsm test 222a: Changelog for explicit restore == 02:29:08 (1790922548) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790922552.420799 [107112]: action=1 src=d222a.sanity-hsm/f222a.sanity-hsm dst=/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790922552.467346 [107112]: importing '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1790922552.556313 [107112]: imported '/mnt/lustre/d222a.sanity-hsm/f222a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222a//0ba8/0x200000bd2:0x7a:0x0'=='/tmp/arc1/sanity-hsm.test_222a//d222a.sanity-hsm/f222a.sanity-hsm' lhsmtool_posix: 1790922552.556393 [107112]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl4 cl4' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.11 16HSM 06:29:20.836555770 2026.10.02 0x80 t=[0x200000bd2:0x7a:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0001: Deregistered changelog user #4 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 222a (26s) == sanity-hsm test 222b: Changelog for implicit restore == 02:29:34 (1790922574) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl5 cl5' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' 4ef076ad20e5d4a2f4729d6ed41a3bf9 /mnt/lustre/d222b.sanity-hsm/f222b.sanity-hsm lustre-MDT0000.20 16HSM 06:29:46.098528595 2026.10.02 0x80 t=[0x200000bd3:0x18:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0001: Deregistered changelog user #5 lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 PASS 222b (25s) == sanity-hsm test 222c: Changelog for failed explicit restore ========================================================== 02:29:59 (1790922599) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lhsmtool_posix: 1790922602.384963 [108760]: action=1 src=d222c.sanity-hsm/f222c.sanity-hsm dst=/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790922602.393338 [108760]: importing '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1790922602.482962 [108760]: imported '/mnt/lustre/d222c.sanity-hsm/f222c.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_222c//0bad/0x200000bd2:0x7f:0x0'=='/tmp/arc1/sanity-hsm.test_222c//d222c.sanity-hsm/f222c.sanity-hsm' lhsmtool_posix: 1790922602.487560 [108760]: process finished, errs: 0 major, 0 minor, rc=0 (Success) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl6 cl6' Copytool is suspended on oleg436-client.virtnet Waiting 200s for 'STARTED' Updated after 2s: want 'STARTED' got 'STARTED' Copytool is continued on oleg436-client.virtnet lustre-MDT0000.22 16HSM 06:30:11.776384006 2026.10.02 0x82 t=[0x200000bd2:0x7f:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp p=[0x200000bd2:0x76:0x0] lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001: Deregistered changelog user #6 lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 222c (23s) == sanity-hsm test 222d: Changelog for failed implicit restore ========================================================== 02:30:23 (1790922623) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_222d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl7 cl7' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Remove from backend: 0x200000bd3:0x1a:0x0 = /tmp/arc1/sanity-hsm.test_222d/0bc9/0x200000bd3:0x1a:0x0 md5sum: /mnt/lustre/d222d.sanity-hsm/f222d.sanity-hsm: No data available lustre-MDT0000.29 16HSM 06:30:35.809551776 2026.10.02 0x82 t=[0x200000bd3:0x1a:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp p=[0x200000bd2:0x7d:0x0] lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0001: Deregistered changelog user #7 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 222d (26s) == sanity-hsm test 223a: Changelog for restore canceled (import case) ========================================================== 02:30:49 (1790922649) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' 39+0 records in 39+0 records out 40894464 bytes (41 MB, 39 MiB) copied, 0.934783 s, 43.7 MB/s mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl8 cl8' lhsmtool_posix: 1790922660.504569 [110523]: action=1 src=d223a.sanity-hsm/f223a.sanity-hsm dst=/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm mount_point=/mnt/lustre2 lhsmtool_posix: 1790922660.519123 [110523]: importing '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1790922660.659796 [110523]: imported '/mnt/lustre/d223a.sanity-hsm/f223a.sanity-hsm' from '/tmp/arc1/sanity-hsm.test_223a//0b51/0x200000bd2:0x83:0x0'=='/tmp/arc1/sanity-hsm.test_223a//d223a.sanity-hsm/f223a.sanity-hsm' lhsmtool_posix: 1790922660.659877 [110523]: process finished, errs: 0 major, 0 minor, rc=0 (Success) Waiting 200s for 'SUCCEED' Updated after 27s: want 'SUCCEED' got 'SUCCEED' lustre-MDT0000.36 16HSM 06:31:31.925387588 2026.10.02 0xfd t=[0x200000bd2:0x83:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0001: clear the changelog for cl8 of all records lustre-MDT0001: Deregistered changelog user #8 lustre-MDT0000: clear the changelog for cl8 of all records lustre-MDT0000: Deregistered changelog user #8 PASS 223a (59s) == sanity-hsm test 223b: Changelog for restore canceled (release case) ========================================================== 02:31:48 (1790922708) 0+0 records in 0+0 records out 0 bytes copied, 0.0139487 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_223b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --bandwidth 1 "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl9 cl9' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet lustre-MDT0000.43 16HSM 06:32:03.352956534 2026.10.02 0xfd t=[0x200000bd3:0x1e:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp p=[0x200000bd2:0x82:0x0] lustre-MDT0001: clear the changelog for cl9 of all records lustre-MDT0001: Deregistered changelog user #9 lustre-MDT0000: clear the changelog for cl9 of all records lustre-MDT0000: Deregistered changelog user #9 PASS 223b (29s) == sanity-hsm test 224A: Changelog for remove ============ 02:32:17 (1790922737) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224A/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl10 cl10' lustre-MDT0000.48 16HSM 06:32:30.267948837 2026.10.02 0x200 t=[0x200000bd3:0x20:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0001: clear the changelog for cl10 of all records lustre-MDT0001: Deregistered changelog user #10 lustre-MDT0000: clear the changelog for cl10 of all records lustre-MDT0000: Deregistered changelog user #10 PASS 224A (30s) == sanity-hsm test 224a: Changelog for failed remove ===== 02:32:47 (1790922767) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_224a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl11 cl11' Remove from backend: 0x200000bd3:0x21:0x0 = /tmp/arc1/sanity-hsm.test_224a/0bf2/0x200000bd3:0x21:0x0 Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet lustre-MDT0000.53 16HSM 06:33:00.779249309 2026.10.02 0x202 t=[0x200000bd3:0x21:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0001: clear the changelog for cl11 of all records lustre-MDT0001: Deregistered changelog user #11 lustre-MDT0000: clear the changelog for cl11 of all records lustre-MDT0000: Deregistered changelog user #11 PASS 224a (29s) == sanity-hsm test 225: Changelog for remove canceled ==== 02:33:16 (1790922796) Test disabled PASS 225 (8s) == sanity-hsm test 226: changelog for last rm/mv with exiting archive ========================================================== 02:33:24 (1790922804) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_226/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0x200000bd3:0x24:0x0 mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl12 cl12' lustre-MDT0000.54 16HSM 06:33:36.626191193 2026.10.02 0x280 t=[0x200000bd3:0x22:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.55 16HSM 06:33:36.768595543 2026.10.02 0x280 t=[0x200000bd3:0x22:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp p=[0x200000bd2:0x76:0x0] lustre-MDT0000.56 16HSM 06:33:36.769431056 2026.10.02 0x0 t=[0x200000bd3:0x22:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp p=[0x200000bd2:0x76:0x0] lustre-MDT0000.57 16HSM 06:33:37.842420982 2026.10.02 0x280 t=[0x200000bd3:0x23:0x0] ef=0x13 u=0:0 nid=0@<0:0> lustre-MDT0000.58 16HSM 06:33:38.024098508 2026.10.02 0x280 t=[0x200000bd3:0x23:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0000.59 16HSM 06:33:38.028016198 2026.10.02 0x0 t=[0x200000bd3:0x23:0x0] j=lhsmtool_posix.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0000.60 06UNLNK 06:33:38.624180143 2026.10.02 0x3 t=[0x200000bd3:0x22:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp p=[0x200000bd2:0x88:0x0] f226.sanity-hsm-1 lustre-MDT0000.60 06UNLNK 06:33:38.624180143 2026.10.02 0x3 t=[0x200000bd3:0x22:0x0] j=rm.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp p=[0x200000bd2:0x88:0x0] f226.sanity-hsm-1 lustre-MDT0000.61 08RENME 06:33:39.298024319 2026.10.02 0x3 t=[0x200000bd3:0x23:0x0] j=mv.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp p=[0x200000bd2:0x88:0x0] f226.sanity-hsm-2 s=[0x200000bd3:0x24:0x0] sp=[0x200000bd2:0x88:0x0] f226.sanity-hsm-3 lustre-MDT0001: clear the changelog for cl12 of all records lustre-MDT0001: Deregistered changelog user #12 lustre-MDT0000: clear the changelog for cl12 of all records lustre-MDT0000: Deregistered changelog user #12 PASS 226 (28s) == sanity-hsm test 227: changelog when explicit setting of HSM flags ========================================================== 02:33:52 (1790922832) 0+0 records in 0+0 records out 0 bytes copied, 0.00458582 s, 0.0 kB/s mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl13 cl13' lustre-MDT0001: clear the changelog for cl13 of all records lustre-MDT0001: Deregistered changelog user #13 lustre-MDT0000: clear the changelog for cl13 of all records lustre-MDT0000: Deregistered changelog user #13 PASS 227 (27s) == sanity-hsm test 228: On released file, return extend to FIEMAP. For [cp,tar] --sparse ========================================================== 02:34:20 (1790922860) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_228/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106074 s, 9.9 MB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' /mnt/lustre/f228.sanity-hsm: 1 extent found tar: Removing leading `/' from member names mnt/lustre/f228.sanity-hsm PASS 228 (16s) == sanity-hsm test 250: Coordinator max request ========== 02:34:37 (1790922877) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_250/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 0+0 records in 0+0 records out 0 bytes copied, 0.0098601 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00960817 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00680894 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00457073 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.0161574 s, 0.0 kB/s 0+0 records in 0+0 records out 0 bytes copied, 0.00457243 s, 0.0 kB/s Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet PASS 250 (40s) == sanity-hsm test 251: Coordinator request timeout ====== 02:35:16 (1790922916) 0+0 records in 0+0 records out 0 bytes copied, 0.0110262 s, 0.0 kB/s mdt.lustre-MDT0000.hsm_control=disabled mdt.lustre-MDT0001.hsm_control=disabled mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_251/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg436-client.virtnet Waiting 200s for 'STARTED' Updated after 3s: want 'STARTED' got 'STARTED' PASS 251 (34s) == sanity-hsm test 252: Timeout'ed running archive of a removed file should be canceled ========================================================== 02:35:50 (1790922950) 0+0 records in 0+0 records out 0 bytes copied, 0.00721251 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_252/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet PASS 252 (25s) == sanity-hsm test 253: Check for wrong file size after release ========================================================== 02:36:15 (1790922975) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_253/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 10+0 records in 10+0 records out 10000000 bytes (10 MB, 9.5 MiB) copied, 0.321934 s, 31.1 MB/s fail_loc=0x807 Cannot send HSM request (use of /mnt/lustre/d253.sanity-hsm/f253.sanity-hsm): Invalid argument could not release file PASS 253 (12s) == sanity-hsm test 254a: Request counters are initialized to zero ========================================================== 02:36:27 (1790922987) PASS 254a (12s) == sanity-hsm test 254b: Request counters are correctly incremented and decremented ========================================================== 02:36:39 (1790922999) Will launch 33 requests of each type Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_254b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Checking archive requests Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet Checking restore requests Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet Waiting 90s for '0' Updated after 3s: want '0' got '0' Checking remove requests Copytool is suspended on oleg436-client.virtnet Copytool is continued on oleg436-client.virtnet PASS 254b (43s) == sanity-hsm test 255: Copytool registration wakes the coordinator up ========================================================== 02:37:23 (1790923043) 0+0 records in 0+0 records out 0 bytes copied, 0.0119433 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s 0+0 records in 0+0 records out 0 bytes copied, 0.0131802 s, 0.0 kB/s 0x200000bd3:0x79:0x0 mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_255/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' PASS 255 (38s) == sanity-hsm test 256: HSM copytool register on DNE with stopped MDT must not panic ========================================================== 02:38:02 (1790923082) mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=enabled PASS 256 (34s) == sanity-hsm test 260a: Restore request have priority over other requests ========================================================== 02:38:35 (1790923115) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300048 s, 3.5 MB/s 0x200000bd3:0x7a:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276582 s, 3.8 MB/s 0x200000bd3:0x7b:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.242288 s, 4.3 MB/s 0x200000bd3:0x7c:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.33242 s, 3.2 MB/s 0x200000bd3:0x7d:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.316033 s, 3.3 MB/s 0x200000bd3:0x7e:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243513 s, 4.3 MB/s 0x200000bd3:0x7f:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19071 s, 5.5 MB/s 0x200000bd3:0x80:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288147 s, 3.6 MB/s 0x200000bd3:0x81:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.283332 s, 3.7 MB/s 0x200000bd3:0x82:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27139 s, 3.9 MB/s 0x200000bd3:0x83:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253946 s, 4.1 MB/s 0x200000bd3:0x84:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.355155 s, 3.0 MB/s 0x200000bd3:0x85:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233065 s, 4.5 MB/s 0x200000bd3:0x86:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154243 s, 6.8 MB/s 0x200000bd3:0x87:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218766 s, 4.8 MB/s 0x200000bd3:0x88:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22136 s, 4.7 MB/s 0x200000bd3:0x89:0x0 Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' ARCHIVE ARCHIVE RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260a (50s) == sanity-hsm test 260b: Restore request have priority over other requests ========================================================== 02:39:25 (1790923165) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291962 s, 3.6 MB/s 0x200000bd3:0x8b:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.414562 s, 2.5 MB/s 0x200000bd3:0x8c:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268031 s, 3.9 MB/s 0x200000bd3:0x8d:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186616 s, 5.6 MB/s 0x200000bd3:0x8e:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262406 s, 4.0 MB/s 0x200000bd3:0x8f:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239444 s, 4.4 MB/s 0x200000bd3:0x90:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28708 s, 3.7 MB/s 0x200000bd3:0x91:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.342887 s, 3.1 MB/s 0x200000bd3:0x92:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230998 s, 4.5 MB/s 0x200000bd3:0x93:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226183 s, 4.6 MB/s 0x200000bd3:0x94:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.361693 s, 2.9 MB/s 0x200000bd3:0x95:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.332651 s, 3.2 MB/s 0x200000bd3:0x96:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.327165 s, 3.2 MB/s 0x200000bd3:0x97:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.425136 s, 2.5 MB/s 0x200000bd3:0x98:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255851 s, 4.1 MB/s 0x200000bd3:0x99:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.378056 s, 2.8 MB/s 0x200000bd3:0x9a:0x0 Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' RESTORE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE PASS 260b (51s) == sanity-hsm test 260c: Requests are not reordered on the 'hot' path of the coordinator ========================================================== 02:40:16 (1790923216) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161207 s, 6.5 MB/s 0x200000bd3:0x9c:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321077 s, 3.3 MB/s 0x200000bd3:0x9d:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.311317 s, 3.4 MB/s 0x200000bd3:0x9e:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37635 s, 2.8 MB/s 0x200000bd3:0x9f:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.303867 s, 3.5 MB/s 0x200000bd3:0xa0:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.460983 s, 2.3 MB/s 0x200000bd3:0xa1:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.339412 s, 3.1 MB/s 0x200000bd3:0xa2:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.212746 s, 4.9 MB/s 0x200000bd3:0xa3:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231536 s, 4.5 MB/s 0x200000bd3:0xa4:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185703 s, 5.6 MB/s 0x200000bd3:0xa5:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25434 s, 4.1 MB/s 0x200000bd3:0xa6:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.374123 s, 2.8 MB/s 0x200000bd3:0xa7:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.265094 s, 4.0 MB/s 0x200000bd3:0xa8:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300092 s, 3.5 MB/s 0x200000bd3:0xa9:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264703 s, 4.0 MB/s 0x200000bd3:0xaa:0x0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208809 s, 5.0 MB/s 0x200000bd3:0xab:0x0 Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 3s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_260c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid --archive 2 "/mnt/lustre2"' ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE ARCHIVE RESTORE PASS 260c (65s) == sanity-hsm test 261: Report 0 bytes size after HSM release ========================================================== 02:41:21 (1790923281) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_261/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0236709 s, 346 kB/s /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x00000009) exists archived, archive_id:2 /mnt/lustre/d261.sanity-hsm/f261.sanity-hsm: (0x0000000d) released exists archived, archive_id:2 PASS 261 (12s) == sanity-hsm test 262: The client should return 1 block for HSM released files ========================================================== 02:41:33 (1790923293) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_262/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0188963 s, 434 kB/s PASS 262 (11s) == sanity-hsm test 300: On disk coordinator state kept between MDT umount/mount ========================================================== 02:41:44 (1790923304) Stop coordinator and remove coordinator state at mount mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown Failing mds1 on oleg436-server Stopping /mnt/lustre-mds1 (opts:) on oleg436-server 02:42:00 (1790923320) shut down facet: mds1 facet_host: oleg436-server facet_failover_host: oleg436-server Failover mds1 to oleg436-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg436-server: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 Started lustre-MDT0000 02:42:33 (1790923353) targets are mounted 02:42:33 (1790923353) facet_failover done oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Set coordinator start at mount, and start coordinator Failing mds1 on oleg436-server Stopping /mnt/lustre-mds1 (opts:) on oleg436-server 02:43:19 (1790923399) shut down facet: mds1 facet_host: oleg436-server facet_failover_host: oleg436-server Failover mds1 to oleg436-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg436-server: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 Started lustre-MDT0000 02:43:53 (1790923433) targets are mounted 02:43:53 (1790923433) facet_failover done oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 300 (149s) == sanity-hsm test 301: HSM tunnable are persistent ====== 02:44:13 (1790923453) Failing mds1 on oleg436-server Stopping /mnt/lustre-mds1 (opts:) on oleg436-server 02:44:23 (1790923463) shut down facet: mds1 facet_host: oleg436-server facet_failover_host: oleg436-server Failover mds1 to oleg436-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg436-server: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 Started lustre-MDT0000 02:44:47 (1790923487) targets are mounted 02:44:47 (1790923487) facet_failover done oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 301 (55s) == sanity-hsm test 302: HSM tunnable are persistent when CDT is off ========================================================== 02:45:09 (1790923509) mdt.lustre-MDT0000.hsm_control=shutdown mdt.lustre-MDT0001.hsm_control=shutdown Failing mds1 on oleg436-server Stopping /mnt/lustre-mds1 (opts:) on oleg436-server 02:45:22 (1790923522) shut down facet: mds1 facet_host: oleg436-server facet_failover_host: oleg436-server Failover mds1 to oleg436-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg436-server: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 Started lustre-MDT0000 02:45:58 (1790923558) targets are mounted 02:45:58 (1790923558) facet_failover done oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds2 on oleg436-server Stopping /mnt/lustre-mds2 (opts:) on oleg436-server 02:46:15 (1790923575) shut down facet: mds2 facet_host: oleg436-server facet_failover_host: oleg436-server Failover mds2 to oleg436-server mount facets: mds2 Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg436-server: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 Started lustre-MDT0001 02:46:42 (1790923602) targets are mounted 02:46:42 (1790923602) facet_failover done oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec PASS 302 (115s) == sanity-hsm test 400: Single request is sent to the right MDT ========================================================== 02:47:04 (1790923624) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_400/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285834 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181847 s, 5.8 MB/s archive successful on mdt0 archive successful on mdt1 PASS 400 (11s) == sanity-hsm test 401: Compound requests split and sent to their respective MDTs ========================================================== 02:47:15 (1790923635) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_401/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196212 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190377 s, 5.5 MB/s archive successful on mdt0 archive successful on mdt1 PASS 401 (11s) == sanity-hsm test 402a: Copytool start fails if all MDTs are inactive ========================================================== 02:47:27 (1790923647) deactivate lustre-MDT0000-mdc-ffff9dd74715d800 on oleg436-client.virtnet deactivate lustre-MDT0001-mdc-ffff9dd74715d800 on oleg436-client.virtnet deactivate lustre-MDT0000-mdc-ffff9dd7515db000 on oleg436-client.virtnet deactivate lustre-MDT0001-mdc-ffff9dd7515db000 on oleg436-client.virtnet Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Failed to start copytool agt1 on 'oleg436-client.virtnet' pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 uuid not found in agent list on mds1 pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 uuid not found in agent list on mds2 activate lustre-MDT0000-mdc-ffff9dd74715d800 on oleg436-client.virtnet activate lustre-MDT0001-mdc-ffff9dd74715d800 on oleg436-client.virtnet activate lustre-MDT0000-mdc-ffff9dd7515db000 on oleg436-client.virtnet activate lustre-MDT0001-mdc-ffff9dd7515db000 on oleg436-client.virtnet PASS 402a (11s) == sanity-hsm test 402b: CDT must retry request upon slow start of CT ========================================================== 02:47:37 (1790923657) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_402b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_loc=0x14d fail_loc=0 Waiting 200s for 'SUCCEED' Updated after 8s: want 'SUCCEED' got 'SUCCEED' PASS 402b (30s) == sanity-hsm test 403: Copytool starts with inactive MDT and register on reconnect ========================================================== 02:48:07 (1790923687) deactivate lustre-MDT0001-mdc-ffff9dd74715d800 on oleg436-client.virtnet deactivate lustre-MDT0001-mdc-ffff9dd7515db000 on oleg436-client.virtnet Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_403/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' oleg436-client.virtnet: executing pgrep --pidfile=/var/run/lhsmtool_posix.pid --list-full hsmtool oleg436-client.virtnet: executing get_client_uuid /mnt/lustre2 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 uuid not found in agent list on mds2 oleg436-client.virtnet: 137210 activate lustre-MDT0001-mdc-ffff9dd74715d800 on oleg436-client.virtnet activate lustre-MDT0001-mdc-ffff9dd7515db000 on oleg436-client.virtnet found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds1 found agent uuid=640cc026-fd80-495d-b9e7-b5da3242e14f archive_id=ANY requests=[current:0 ok:0 errors:0] on mds2 PASS 403 (19s) == sanity-hsm test 404: Inactive MDT does not block requests for active MDTs ========================================================== 02:48:27 (1790923707) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_404/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180882 s, 5.8 MB/s deactivate lustre-MDT0001-mdc-ffff9dd74715d800 on oleg436-client.virtnet deactivate lustre-MDT0001-mdc-ffff9dd7515db000 on oleg436-client.virtnet archive successful on mdt0 activate lustre-MDT0001-mdc-ffff9dd74715d800 on oleg436-client.virtnet activate lustre-MDT0001-mdc-ffff9dd7515db000 on oleg436-client.virtnet PASS 404 (8s) == sanity-hsm test 405: archive and release under striped directory ========================================================== 02:48:36 (1790923716) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_405/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.075016 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597254 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0678514 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0995153 s, 10.5 MB/s archive successful on 0x240000bd2:0x1:0x0 archive successful on 0x240000bd2:0x2:0x0 archive successful on 0x240000bd2:0x3:0x0 archive successful on 0x2000032e3:0x6:0x0 PASS 405 (17s) == sanity-hsm test 406: attempting to migrate HSM archived files is safe ========================================================== 02:48:53 (1790923733) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23992 s, 4.4 MB/s old fid 0x2000032e3:0x9:0x0 Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_406/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' lfs migrate: /mnt/lustre/d406.sanity-hsm/f406.sanity-hsm migrate failed: Operation not supported (95) lfs migrate: cb_migrate_mdt_fini: error completing migration of /mnt/lustre/d406.sanity-hsm: Directory not empty (39) Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' new fid 0x240000bd3:0x7:0x0 Waiting 200s for 'SUCCEED' debug=+trace PASS 406 (17s) == sanity-hsm test 407: Check for double RESTORE records in llog ========================================================== 02:49:10 (1790923750) 0+0 records in 0+0 records out 0 bytes copied, 0.0118473 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_407/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' fail_val=5 fail_loc=0x164 Copytool is suspended on oleg436-client.virtnet mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=4/4] fid=[0x2000032e3:0x3:0x0] dfid=[0x2000032e3:0x3:0x0] compound/cookie=0x0/0x6abf5322 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/5] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x6:0x0] compound/cookie=0x0/0x6abf5323 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x6abf5324 action=RESTORE archive#=3 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6abf5325 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6abf5326 action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6abf5327 action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6abf5328 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] Failing mds1 on oleg436-server Stopping /mnt/lustre-mds1 (opts:) on oleg436-server 02:49:22 (1790923762) shut down facet: mds1 facet_host: oleg436-server facet_failover_host: oleg436-server Failover mds1 to oleg436-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg436-server: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 Started lustre-MDT0000 02:49:47 (1790923787) targets are mounted 02:49:47 (1790923787) facet_failover done mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=4/6] fid=[0x2000032e3:0x6:0x0] dfid=[0x2000032e3:0x7:0x0] compound/cookie=0x0/0x6abf5324 action=RESTORE archive#=3 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6abf5325 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6abf5326 action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6abf5327 action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6abf5328 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/11] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6abf5329 action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec fail_loc=0 mdt.lustre-MDT0000.hsm.actions= lrh=[type=10680000 len=136 idx=4/7] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6abf5325 action=ARCHIVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/8] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0xa:0x0] compound/cookie=0x0/0x6abf5326 action=RESTORE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/9] fid=[0x2000032e3:0x9:0x0] dfid=[0x2000032e3:0x9:0x0] compound/cookie=0x0/0x6abf5327 action=REMOVE archive#=3 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/10] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6abf5328 action=ARCHIVE archive#=2 flags=0x0 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=SUCCEED data=[] lrh=[type=10680000 len=136 idx=4/11] fid=[0x2000032e3:0xc:0x0] dfid=[0x2000032e3:0xc:0x0] compound/cookie=0x0/0x6abf5329 action=RESTORE archive#=2 flags=0x4 extent=0x0-0xffffffffffffffff gid=0x0 datalen=0 status=STARTED data=[] Copytool is continued on oleg436-client.virtnet d41d8cd98f00b204e9800998ecf8427e /mnt/lustre/d407.sanity-hsm/f407.sanity-hsm d41d8cd98f00b204e9800998ecf8427e /mnt/lustre2/d407.sanity-hsm/f407.sanity-hsm pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 PASS 407 (55s) == sanity-hsm test 408: Verify fiemap on release file ==== 02:50:06 (1790923806) Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_408/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.31973 s, 205 kB/s SKIP: sanity-hsm test_408 ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 408 (5s) == sanity-hsm test 409a: Coordinator should not stop when in use ========================================================== 02:50:11 (1790923811) 0+0 records in 0+0 records out 0 bytes copied, 0.00429613 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' fail_val=5 fail_loc=0x164 mdt.lustre-MDT0000.hsm_control=shutdown cat: /mnt/lustre/d409a.sanity-hsm/f409a.sanity-hsm: No data available mdt.lustre-MDT0001.hsm_control=shutdown mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' mdt.lustre-MDT0000.hsm_control=enabled mdt.lustre-MDT0001.hsm_control=enabled PASS 409a (32s) == sanity-hsm test 409b: getattr released file with CDT stopped after remount ========================================================== 02:50:43 (1790923843) 0+0 records in 0+0 records out 0 bytes copied, 0.00798488 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_409b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s Failing mds1 on oleg436-server Stopping /mnt/lustre-mds1 (opts:) on oleg436-server 02:51:17 (1790923877) shut down facet: mds1 facet_host: oleg436-server facet_failover_host: oleg436-server Failover mds1 to oleg436-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg436-server: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 Started lustre-MDT0000 02:51:50 (1790923910) targets are mounted 02:51:50 (1790923910) facet_failover done oleg436-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec File: /mnt/lustre/d409b.sanity-hsm/f409b.sanity-hsm Size: 0 Blocks: 0 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115406632648722 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-10-02 02:50:45.000000000 -0400 Modify: 2026-10-02 02:50:45.000000000 -0400 Change: 2026-10-02 02:50:45.000000000 -0400 Birth: 2026-10-02 02:50:45.000000000 -0400 PASS 409b (109s) == sanity-hsm test 410: lfs data_version -s allows release of force-archived file ========================================================== 02:52:32 (1790923952) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.363523 s, 2.9 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_410/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Cannot send HSM request (use of /mnt/lustre/d410.sanity-hsm/f410.sanity-hsm): Operation not permitted PASS 410 (8s) == sanity-hsm test 411: hsm_ops rbac role ================ 02:52:40 (1790923960) On MGS 192.168.204.136, active = nodemap.active=1 On MGS 192.168.204.136, test_411.trusted_nodemap = nodemap.test_411.trusted_nodemap=1 Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' /mnt/lustre/d411.sanity-hsm/fileA: (0x00000000) Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Killing existing copytools on oleg436-client.virtnet with signal TERM copytools stopped in 0s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_411/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' On MGS 192.168.204.136, test_411.rbac = nodemap.test_411.rbac=file_perms,dne_ops,quota_ops,byfid_ops,chlg_ops,fscrypt_admin,server_upcall,ignore_root_prjquota,immutable_flags,local_admin,pool_quota_ops,lqa_quota_ops,projid_set,foreign_ops /mnt/lustre/d411.sanity-hsm/fileB: (0x00000000) Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Operation not permitted Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied Cannot send HSM request (use of /mnt/lustre/d411.sanity-hsm/fileB): Permission denied lfs hsm_set: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied lfs hsm_clear: change hsm flags for '/mnt/lustre/d411.sanity-hsm/fileB' failed: Permission denied On MGS 192.168.204.136, active = nodemap.active=0 oleg436-server: error: nodemap 'test_411' does not exist oleg436-server: nodemap_del: Invalid argument pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 On MGS 192.168.204.136, active = nodemap.active=0 PASS 411 (50s) == sanity-hsm test 500: various LLAPI HSM tests ========== 02:53:30 (1790924010) striped dir -i0 -c2 -H crush2 /mnt/lustre/d500.sanity-hsm test1 @1790924106: Register/unregister copytool 2000x to check for leaks ...... pass test2 @1790924106: Re/un-register copytool multiple times without error ....... pass llapi_hsm_test: NULL archive numbers llapi_hsm_test: NULL archive numbers llapi_hsm_test: -1 requested when archive id >= 0 is supported test3 @1790924106: Pass bad parameters to llapi_hsm_copytool_register() ....... pass test4 @1790924106: Bad parameters to llapi_hsm_copytool_unregister() .......... pass test5 @1790924107: Test llapi_hsm_copytool_recv() in non-blocking mode ........ pass test6 @1790924107: Test llapi_hsm_copytool_recv() with bogus parameters ....... pass test7 @1790924107: Test event polling (without actual traffic) ................ pass test50 @1790924107: Test llapi_hsm_state_get()/get_fd() ....................... pass test51 @1790924108: Test llapi_hsm_state_set_fd() ............................. pass test52 @1790924108: Test llapi_hsm_current_action() ........................... pass test100 @1790924109: Simple archive creation with no progress reported. ....... pass test101 @1790924116: Simple archive creation with progress every byte. ........ pass test102 @1790924123: Archive creation with progress every byte, backwards ..... pass test103 @1790924124: Archive creation with a single progress report ........... pass test104 @1790924125: Archive creation with two progress reports ............... pass test105 @1790924127: Archive creation with one bogus progress report .......... pass test106 @1790924128: Archive creation with one empty progress report .......... pass test107 @1790924129: Archive creation with one bogus progress report .......... pass test108 @1790924135: Archive creation with same progress report each time ..... pass test109 @1790924137: Archive creation with one report, with large number ...... pass test110 @1790924138: Archive with 10 progress reports, checking progress ...... pass test111 @1790924139: Archive with 10 reports in reverse, checking progress .... pass test112 @1790924140: Archive with 10 reports, duplicated, check progress ...... pass test113 @1790924142: Archive with 9 reports, with 20% overlapping coverage .... pass PASS 500 (137s) == sanity-hsm test 600: Changelog fields 'u=' and 'nid=' ========================================================== 02:55:47 (1790924147) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl1 cl1' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d600.sanity-hsm/f600.sanity-hsm] Got NID '192.168.204.36@tcp' lustre-MDT0001: clear the changelog for cl1 of all records lustre-MDT0001: Deregistered changelog user #1 lustre-MDT0000: clear the changelog for cl1 of all records lustre-MDT0000: Deregistered changelog user #1 PASS 600 (20s) == sanity-hsm test 601: OPEN Changelog entry ============= 02:56:07 (1790924167) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl2 cl2' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0000.4 10OPEN 06:56:17.576384279 2026.10.02 0x1 t=[0x2000032e2:0x1a:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.204.36@tcp m=r-- lustre-MDT0001: clear the changelog for cl2 of all records lustre-MDT0001: Deregistered changelog user #2 lustre-MDT0000: clear the changelog for cl2 of all records lustre-MDT0000: Deregistered changelog user #2 PASS 601 (21s) == sanity-hsm test 602: Changelog record CLOSE only if open+write or OPEN recorded ========================================================== 02:56:28 (1790924188) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl3 cl3' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0000.10 11CLOSE 06:56:37.349978505 2026.10.02 0x1 t=[0x2000032e2:0x1c:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0000.11 10OPEN 06:56:37.774190536 2026.10.02 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.204.36@tcp m=-w- lustre-MDT0000.12 13TRUNC 06:56:37.781070354 2026.10.02 0xe t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp p=[0x2000032e2:0x1b:0x0] lustre-MDT0000.13 11CLOSE 06:56:37.806402575 2026.10.02 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0000.13 11CLOSE 06:56:37.806402575 2026.10.02 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp mdd.lustre-MDT0000.changelog_mask=-OPEN mdd.lustre-MDT0001.changelog_mask=-OPEN lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records f lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0000.14 13TRUNC 06:56:39.999966051 2026.10.02 0xe t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp p=[0x2000032e2:0x1b:0x0] lustre-MDT0000.15 11CLOSE 06:56:40.016939251 2026.10.02 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0000.15 11CLOSE 06:56:40.016939251 2026.10.02 0x242 t=[0x2000032e2:0x1c:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0001: clear the changelog for cl3 of all records lustre-MDT0001: Deregistered changelog user #3 lustre-MDT0000: clear the changelog for cl3 of all records lustre-MDT0000: Deregistered changelog user #3 PASS 602 (23s) == sanity-hsm test 603: GETXATTR Changelog entry ========= 02:56:51 (1790924211) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl4 cl4' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0001: clear the changelog for cl4 of all records getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/d603.sanity-hsm/f603.sanity-hsm user.xattr1="value1" lustre-MDT0001.10 23GXATR 06:57:00.544967046 2026.10.02 0x0 t=[0x240000bd3:0x1d:0x0] j=getfattr.0 ef=0x1b u=0:0 nid=192.168.204.36@tcp x=user.xattr1 lustre-MDT0001: clear the changelog for cl4 of all records lustre-MDT0001: Deregistered changelog user #4 lustre-MDT0000: clear the changelog for cl4 of all records lustre-MDT0000: Deregistered changelog user #4 PASS 603 (20s) == sanity-hsm test 604: NOPEN Changelog entry ============ 02:57:11 (1790924231) mdd.lustre-MDT0000.changelog_deniednext=20 mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl5 cl5' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.20 24NOPEN 06:57:24.277815610 2026.10.02 0x1 t=[0x2000032e2:0x1e:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.204.36@tcp m=r-- p=[0x2000032e2:0x1b:0x0] Got NID '192.168.204.36@tcp' lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0001: clear the changelog for cl5 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm] cat: /mnt/lustre2/d604.sanity-hsm/f604.sanity-hsm: Permission denied lustre-MDT0000.21 24NOPEN 06:57:45.698373191 2026.10.02 0x1 t=[0x2000032e2:0x1e:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.204.36@tcp m=r-- Got NID '192.168.204.36@tcp' lustre-MDT0001: clear the changelog for cl5 of all records lustre-MDT0001: Deregistered changelog user #5 lustre-MDT0000: clear the changelog for cl5 of all records lustre-MDT0000: Deregistered changelog user #5 mdd.lustre-MDT0000.changelog_deniednext=60 PASS 604 (47s) == sanity-hsm test 605: Test OPEN and CLOSE rate limit in Changelogs ========================================================== 02:57:58 (1790924278) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl6 cl6' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000.25 10OPEN 06:58:07.746620949 2026.10.02 0x43 t=[0x2000032e2:0x20:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.204.36@tcp m=rw- p=[0x240000bd3:0x1c:0x0] lustre-MDT0000.25 10OPEN 06:58:07.746620949 2026.10.02 0x43 t=[0x2000032e2:0x20:0x0] j=bash.0 ef=0x17 u=0:0 nid=192.168.204.36@tcp m=rw- p=[0x240000bd3:0x1c:0x0] lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000.26 10OPEN 06:58:09.493358692 2026.10.02 0x1 t=[0x2000032e2:0x20:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.204.36@tcp m=r-- p=[0x240000bd3:0x1c:0x0] lustre-MDT0000.27 11CLOSE 06:58:09.557063813 2026.10.02 0x1 t=[0x2000032e2:0x20:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0000.26 10OPEN 06:58:09.493358692 2026.10.02 0x1 t=[0x2000032e2:0x20:0x0] j=cat.0 ef=0x17 u=0:0 nid=192.168.204.36@tcp m=r-- p=[0x240000bd3:0x1c:0x0] lustre-MDT0000.27 11CLOSE 06:58:09.557063813 2026.10.02 0x1 t=[0x2000032e2:0x20:0x0] j=cat.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records running as uid/gid/euid/egid 500/500/500/500, groups: 500 [cat] [/mnt/lustre/d605.sanity-hsm/f605.sanity-hsm] lustre-MDT0000.28 10OPEN 06:58:11.197410307 2026.10.02 0x1 t=[0x2000032e2:0x20:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.204.36@tcp m=r-- p=[0x2000032e2:0x1d:0x0] lustre-MDT0000.29 11CLOSE 06:58:11.213487820 2026.10.02 0x1 t=[0x2000032e2:0x20:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.204.36@tcp lustre-MDT0000.28 10OPEN 06:58:11.197410307 2026.10.02 0x1 t=[0x2000032e2:0x20:0x0] j=cat.500 ef=0x17 u=500:500 nid=192.168.204.36@tcp m=r-- p=[0x2000032e2:0x1d:0x0] lustre-MDT0000.29 11CLOSE 06:58:11.213487820 2026.10.02 0x1 t=[0x2000032e2:0x20:0x0] j=cat.500 ef=0x13 u=500:500 nid=192.168.204.36@tcp lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0000.30 11CLOSE 06:58:12.059699496 2026.10.02 0x43 t=[0x2000032e2:0x20:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0000.30 11CLOSE 06:58:12.059699496 2026.10.02 0x43 t=[0x2000032e2:0x20:0x0] j=bash.0 ef=0x13 u=0:0 nid=192.168.204.36@tcp lustre-MDT0001: clear the changelog for cl6 of all records lustre-MDT0001: Deregistered changelog user #6 lustre-MDT0000: clear the changelog for cl6 of all records lustre-MDT0000: Deregistered changelog user #6 PASS 605 (26s) == sanity-hsm test 606: llog_reader groks changelog fields ========================================================== 02:58:24 (1790924304) mdd.lustre-MDT0000.changelog_mask=+hsm mdd.lustre-MDT0001.changelog_mask=+hsm Registered 2 changelog users: 'cl7 cl7' mdd.lustre-MDT0000.changelog_mask=ALL mdd.lustre-MDT0001.changelog_mask=ALL running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d606.sanity-hsm/f606.sanity-hsm] Stopping /mnt/lustre-mds1 (opts:) on oleg436-server rec #2 type=1064553b len=64 offset 8256 Header size : 8192 llh_size : 64 Time : Fri Oct 2 02:24:18 2026 Number of records: 1 cat_idx: 0 last_idx: 2 Flags: 0x2012 Target uuid : ----------------------- #02 (064)fid=[0x1:0x3a:0x0] path=oi.1/0x1:0x3a:0x0 Got UID/GID 500:500 Got NID 192.168.204.36@tcp Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg436-server: oleg436-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg436-client: oleg436-server: ssh exited with exit code 1 Started lustre-MDT0000 lustre-MDT0001: clear the changelog for cl7 of all records lustre-MDT0001: Deregistered changelog user #7 lustre-MDT0000: clear the changelog for cl7 of all records lustre-MDT0000: Deregistered changelog user #7 PASS 606 (68s) == sanity-hsm test 607a: release file migrated after archive (non-blocking migrate) ========================================================== 02:59:32 (1790924372) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.397513 s, 2.6 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607a/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 607a (14s) == sanity-hsm test 607b: Migrate should not change the HSM attribute of dirty files ========================================================== 02:59:46 (1790924386) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.387609 s, 2.7 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607b/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' llite.lustre-ffff9dd74715d800.xattr_cache=0 llite.lustre-ffff9dd7515db000.xattr_cache=0 /mnt/lustre/d607b.sanity-hsm/f607b.sanity-hsm: (0x0000000b) exists dirty archived, archive_id:1 llite.lustre-ffff9dd74715d800.xattr_cache=1 llite.lustre-ffff9dd7515db000.xattr_cache=1 PASS 607b (12s) == sanity-hsm test 607c: 'lfs swap_layouts' should set dirty flag on HSM file ========================================================== 02:59:58 (1790924398) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.48776 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264465 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.500465 s, 2.1 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607c/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' lfs swap_layouts: error: cannot swap layout between '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-1' and '/mnt/lustre/d607c.sanity-hsm/f607c.sanity-hsm-3': Operation not permitted (1) PASS 607c (14s) == sanity-hsm test 607d: Migrate should not set HSM dirty flag for an empty file ========================================================== 03:00:12 (1790924412) 0+0 records in 0+0 records out 0 bytes copied, 0.00754249 s, 0.0 kB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607d/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 607d (13s) == sanity-hsm test 607e: release file migrated after archive (blocking migrate) ========================================================== 03:00:25 (1790924425) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267317 s, 3.9 MB/s Starting copytool 'agt1' on 'oleg436-client.virtnet' with cmdline 'lhsmtool_posix --archive-format=v2 --hsm-root=/tmp/arc1/sanity-hsm.test_607e/ --daemon --pid-file=/var/run/lhsmtool_posix.pid "/mnt/lustre2"' Waiting 200s for 'SUCCEED' Updated after 2s: want 'SUCCEED' got 'SUCCEED' PASS 607e (14s) == sanity-hsm test complete, duration 5675 sec =========== 03:00:39 (1790924439) === sanity-hsm: start cleanup 03:00:40 (1790924440) === Stopping clients: oleg436-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg436-client.virtnet /mnt/lustre2 opts: === sanity-hsm: finish cleanup 03:01:18 (1790924478) ===