-----============= acceptance-small: sanity-flr ============----- Fri Oct 24 01:41:54 EDT 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg110-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanity-flr.*ex': No such file or directory excepting tests: 6 201 44c === sanity-flr: start setup 01:42:12 (1761284532) === oleg110-client.virtnet: executing check_config_client /mnt/lustre oleg110-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg110-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff992e88681800.idle_timeout=debug osc.lustre-OST0001-osc-ffff992e88681800.idle_timeout=debug disable quota as required oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 === sanity-flr: finish setup 01:42:39 (1761284559) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f8189] == sanity-flr test 0a: lfs mirror create with -N option == 01:42:40 (1761284560) PASS 0a (9s) == sanity-flr test 0b: lfs mirror create plain layout mirrors ========================================================== 01:42:49 (1761284569) SKIP: sanity-flr test_0b need >= 4 OSTs SKIP 0b (3s) == sanity-flr test 0c: lfs mirror create composite layout mirrors ========================================================== 01:42:52 (1761284572) SKIP: sanity-flr test_0c need >= 4 OSTs SKIP 0c (2s) == sanity-flr test 0d: lfs mirror extend with -N option == 01:42:54 (1761284574) PASS 0d (16s) == sanity-flr test 0e: lfs mirror extend plain layout mirrors ========================================================== 01:43:10 (1761284590) SKIP: sanity-flr test_0e need >= 4 OSTs SKIP 0e (2s) == sanity-flr test 0f: lfs mirror extend composite layout mirrors ========================================================== 01:43:12 (1761284592) SKIP: sanity-flr test_0f need >= 4 OSTs SKIP 0f (2s) == sanity-flr test 0g: lfs mirror create flags support === 01:43:14 (1761284594) error: lfs create: --flags must be specified with --mirror-count|-N option Create a mirrored file. usage: lfs mirror create --mirror-count|-N[MIRROR_COUNT] [SETSTRIPE_OPTIONS] ... FILENAME|DIRECTORY ... lfs commands used to manage files with mirrored components: lfs mirror create - create a mirrored file or directory lfs mirror extend - add mirror(s) to an existing file lfs mirror split - split a mirror from an existing mirrored file lfs mirror resync - resynchronize out-of-sync mirrored file(s) lfs mirror read - read a mirror content of a mirrored file lfs mirror write - write to a mirror of a mirrored file lfs mirror copy - copy a mirror to other mirror(s) of a file lfs mirror verify - verify mirrored file(s) lfs create: component flag 'foo' not supported Create a mirrored file. usage: lfs mirror create --mirror-count|-N[MIRROR_COUNT] [SETSTRIPE_OPTIONS] ... FILENAME|DIRECTORY ... lfs commands used to manage files with mirrored components: lfs mirror create - create a mirrored file or directory lfs mirror extend - add mirror(s) to an existing file lfs mirror split - split a mirror from an existing mirrored file lfs mirror resync - resynchronize out-of-sync mirrored file(s) lfs mirror read - read a mirror content of a mirrored file lfs mirror write - write to a mirror of a mirrored file lfs mirror copy - copy a mirror to other mirror(s) of a file lfs mirror verify - verify mirrored file(s) lfs create: unsupported mirror flags: stale lfs create: unsupported mirror flags: prefer,init lfs create: inverted flags are not supported Create a mirrored file. usage: lfs mirror create --mirror-count|-N[MIRROR_COUNT] [SETSTRIPE_OPTIONS] ... FILENAME|DIRECTORY ... lfs commands used to manage files with mirrored components: lfs mirror create - create a mirrored file or directory lfs mirror extend - add mirror(s) to an existing file lfs mirror split - split a mirror from an existing mirrored file lfs mirror resync - resynchronize out-of-sync mirrored file(s) lfs mirror read - read a mirror content of a mirrored file lfs mirror write - write to a mirror of a mirrored file lfs mirror copy - copy a mirror to other mirror(s) of a file lfs mirror verify - verify mirrored file(s) osc.lustre-OST0000-osc-ffff992e88681800.stats=clear osc.lustre-OST0001-osc-ffff992e88681800.stats=clear PASS 0g (5s) == sanity-flr test 0h: set, clear and test flags for FLR files ========================================================== 01:43:19 (1761284599) striped dir -i0 -c2 -H all_char /mnt/lustre/d0h.sanity-flr lfs setstripe: component flag 'foo' not supported Create a file with specified striping/composite layout, or set the default layout on an existing directory. Usage: setstripe [--component-add|--component-del|--delete|-d] [--comp-set --comp-id|-I COMP_ID|--comp-flags=COMP_FLAGS] [--component-end|-E END_OFFSET] [--copy=SOURCE_LAYOUT_FILE]|--yaml|-y YAML_TEMPLATE_FILE] [--extension-size|--ext-size|-z EXT_SIZE] [--help|-h] [--foreign=FOREIGN_TYPE --xattr|-x LAYOUT] [--layout|-L PATTERN] [--mode FILE_MODE] [--mirror-count|-N[MIRROR_COUNT]] [--ost|-o OST_INDEX[,OST_INDEX,...]] [--overstripe-count|-C STRIPE_COUNT] [--pool|-p POOL_NAME] [--stripe-count|-c STRIPE_COUNT] [--stripe-index|-i START_OST_IDX] [--stripe-size|-S STRIPE_SIZE] FILENAME|DIRECTORY lfs: component flag 'foo' not supported error: getstripe bad component flags 'prefer,foo'. List the layout pattern for a given file or files in a directory or recursively for all files in a directory tree. Usage: getstripe [--ost|-O OST_NAME] [--quiet|-q] [--verbose|-v] [--stripe-count|-c] [--stripe-index|-i] [--fid|-F] [--pool|-p] [--stripe-size|-S] [--directory|-d] [--mdt-index|-m] [--recursive|-r] [--raw|-R] [--layout|-L] [--generation|-g] [--yaml|-y] [--help|-h] [--hex-idx] [--component-id|-I[=COMP_ID]] [--component-flags[=COMP_FLAGS]] [--component-count] [--extension-size|--ext-size|-z] [--component-start[=[+-]START_OFFSET]] [--component-end|-E[[+-]END_OFFSET]] [[!] --mirror-index=[+-]MIRROR_INDEX | [!] --mirror-id=[+-]MIRROR_ID] [--mirror-count|-N] [--no-follow] FILENAME|DIRECTORY lfs setstripe: cannot clear 'stale' flags from component. Please use lfs-mirror-resync(1) instead PASS 0h (7s) == sanity-flr test 0j: test lfs mirror read/write commands ========================================================== 01:43:26 (1761284606) PASS 0j (4s) == sanity-flr test 0k: mirroring a file in directory with default FLR layout ========================================================== 01:43:31 (1761284611) SKIP: sanity-flr test_0k need >= 3 OSTs SKIP 0k (2s) == sanity-flr test 1: create components with setstripe options ========================================================== 01:43:33 (1761284613) lfs mirror mirror: cannot get UNLOCK lease, ext 4 for [0x200000402:0x1e:0x0]: Numerical result out of range (34) error: lfs extend: /mnt/lustre/f1.sanity-flr: cannot merge layout: Numerical result out of range PASS 1 (5s) == sanity-flr test 2: create components from existing files ========================================================== 01:43:39 (1761284619) PASS 2 (5s) == sanity-flr test 3: create components from files located on different MDTs ========================================================== 01:43:43 (1761284623) PASS 3 (4s) == sanity-flr test 4: Make sure mirror attributes can be inhertied from directory ========================================================== 01:43:47 (1761284627) striped dir -i0 -c2 -H crush /mnt/lustre/d4.sanity-flr striped dir -i0 -c2 -H all_char /mnt/lustre/d4.sanity-flr/td error: lfs extend: /mnt/lustre/d4.sanity-flr: cannot get the last init comp: Is a directory PASS 4 (5s) == sanity-flr test 5: Make sure init size work for mirrored layout ========================================================== 01:43:52 (1761284632) /mnt/lustre/f5.sanity-flr has type file OK /mnt/lustre/f5.sanity-flr has size 12345 OK /mnt/lustre/f5.sanity-flr has type file OK /mnt/lustre/f5.sanity-flr has size 12345 OK PASS 5 (4s) SKIP: sanity-flr test_6a skipping ALWAYS excluded test 6a (base 6) SKIP: sanity-flr test_6b skipping ALWAYS excluded test 6b (base 6) == sanity-flr test 7: Create mirror with DoM component === 01:43:58 (1761284638) PASS 7 (4s) == sanity-flr test 21: glimpse should report accurate i_blocks ========================================================== 01:44:02 (1761284642) 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 1.04116 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541009 s, 19.4 MB/s PASS 21 (6s) == sanity-flr test 22: no glimpse to OSTs for READ_ONLY files ========================================================== 01:44:08 (1761284648) 17+0 records in 17+0 records out 17825792 bytes (18 MB, 17 MiB) copied, 0.244883 s, 72.8 MB/s PASS 22 (5s) == sanity-flr test 31: make sure glimpse request can be retried ========================================================== 01:44:13 (1761284653) Stopping /mnt/lustre-ost1 (opts:) on oleg110-server oleg110-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff992e88681800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff992e88681800.ost_server_uuid in DISCONN state after 1 sec Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg110-server: oleg110-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg110-server: *.lustre-OST0000.recovery_status status: COMPLETE Stopping /mnt/lustre-ost2 (opts:) on oleg110-server oleg110-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0001-osc-ffff992e88681800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff992e88681800.ost_server_uuid in DISCONN state after 1 sec Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Started lustre-OST0001 affected facets: ost2 oleg110-server: oleg110-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg110-server: *.lustre-OST0001.recovery_status status: COMPLETE PASS 31 (44s) == sanity-flr test 32: data should be mirrored to newly created mirror ========================================================== 01:44:57 (1761284697) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.289649 s, 36.2 MB/s Stopping /mnt/lustre-ost1 (opts:) on oleg110-server oleg110-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff992e88681800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff992e88681800.ost_server_uuid in DISCONN state after 0 sec /mnt/lustre/f32.sanity-flr has type file OK /mnt/lustre/f32.sanity-flr has size 10485760 OK reading file from the 2nd mirror and verify checksum Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg110-server: oleg110-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg110-server: *.lustre-OST0000.recovery_status status: COMPLETE PASS 32 (23s) == sanity-flr test 33a: read can choose available mirror to read ========================================================== 01:45:20 (1761284720) error: lfs extend: /mnt/lustre/f33a.sanity-flr: file busy or contents don't match: Invalid argument /mnt/lustre/f33a.sanity-flr has type file OK /mnt/lustre/f33a.sanity-flr has size 500 OK reading file (data can be provided by any ost)... Stopping /mnt/lustre-ost1 (opts:) on oleg110-server oleg110-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff992e88681800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff992e88681800.ost_server_uuid in DISCONN state after 0 sec reading file (data should be provided by ost2)... Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg110-server: oleg110-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg110-server: *.lustre-OST0000.recovery_status status: COMPLETE Stopping /mnt/lustre-ost2 (opts:) on oleg110-server oleg110-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0001-osc-ffff992e88681800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff992e88681800.ost_server_uuid in DISCONN state after 0 sec /mnt/lustre/f33a.sanity-flr has type file OK /mnt/lustre/f33a.sanity-flr has size 500 OK reading file (data should be provided by ost1)... Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Started lustre-OST0001 affected facets: ost2 oleg110-server: oleg110-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg110-server: *.lustre-OST0001.recovery_status status: COMPLETE PASS 33a (59s) == sanity-flr test 33b: avoid reading from unhealthy mirror ========================================================== 01:46:19 (1761284779) 25+0 records in 25+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 2.09636 s, 25.0 MB/s -rw-r--r-- 1 root root 52428800 Oct 24 01:46 /mnt/lustre/f33b.sanity-flr reading file (data can be provided by any ost)... real 0m0.807s user 0m0.001s sys 0m0.147s Stopping /mnt/lustre-ost1 (opts:) on oleg110-server oleg110-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff992e88681800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff992e88681800.ost_server_uuid in DISCONN state after 0 sec reading file (data should be provided by ost2)... real 0m0.840s user 0m0.002s sys 0m0.187s Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg110-server: oleg110-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg110-server: *.lustre-OST0000.recovery_status status: COMPLETE Stopping /mnt/lustre-ost2 (opts:) on oleg110-server oleg110-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0001-osc-ffff992e88681800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff992e88681800.ost_server_uuid in DISCONN state after 1 sec reading file (data should be provided by ost1)... real 0m0.824s user 0m0.000s sys 0m0.172s Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Started lustre-OST0001 affected facets: ost2 oleg110-server: oleg110-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg110-server: *.lustre-OST0001.recovery_status status: COMPLETE oleg110-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff992e88681800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff992e88681800.ost_server_uuid in FULL state after 0 sec PASS 33b (59s) == sanity-flr test 33c: keep reading among unhealthy mirrors ========================================================== 01:47:18 (1761284838) SKIP: sanity-flr test_33c need >= 3 OSTs SKIP 33c (2s) == sanity-flr test 34a: read mirrored file with multiple stripes ========================================================== 01:47:20 (1761284840) SKIP: sanity-flr test_34a need >= 4 OSTs SKIP 34a (2s) == sanity-flr test 34b: read mirrored file with multiple components ========================================================== 01:47:22 (1761284842) SKIP: sanity-flr test_34b need >= 4 OSTs SKIP 34b (2s) == sanity-flr test 35: allow to write to mirrored files == 01:47:24 (1761284844) PASS 35 (4s) == sanity-flr test 36a: write to mirrored files ========== 01:47:28 (1761284848) 12+0 records in 12+0 records out 12582912 bytes (13 MB, 12 MiB) copied, 0.341978 s, 36.8 MB/s fail_val=10 fail_loc=0x1A01 fail_loc=0 PASS 36a (9s) == sanity-flr test 36b: write should not hung when prefered mirror is stale ========================================================== 01:47:37 (1761284857) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022498 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162618 s, 64.5 MB/s PASS 36b (5s) == sanity-flr test 36c: change prefer mirror during write shouldn't hung ========================================================== 01:47:42 (1761284862) PASS 36c (6s) == sanity-flr test 36d: write/punch FLR file update OST layout version ========================================================== 01:47:48 (1761284868) ** create /mnt/lustre/f36d.sanity-flr ** mirror extend /mnt/lustre/f36d.sanity-flr (1/2) flv=2 olv=0 ** mirror extend /mnt/lustre/f36d.sanity-flr (2/2) flv=3 olv=0 ** write /mnt/lustre/f36d.sanity-flr (1/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0020524 s, 499 kB/s flv=4 olv=4 ** write /mnt/lustre/f36d.sanity-flr (2/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00233037 s, 439 kB/s flv=4 olv=4 ** resync /mnt/lustre/f36d.sanity-flr flv=6 olv=6 ** truncate /mnt/lustre/f36d.sanity-flr (1/2) flv=7 olv=7 ** truncate /mnt/lustre/f36d.sanity-flr (2/2) flv=7 olv=7 ** resync /mnt/lustre/f36d.sanity-flr flv=9 olv=9 ** write /mnt/lustre/f36d.sanity-flr (1/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00138383 s, 740 kB/s flv=10 olv=10 ** write /mnt/lustre/f36d.sanity-flr (2/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00259646 s, 394 kB/s flv=10 olv=10 PASS 36d (7s) == sanity-flr test 37: mirror I/O API verification ======= 01:47:55 (1761284875) create test files with size 15738551 .. be563a85536e63ce692171ad67f9d0a9 - 0a76e2fc036af10b5d45b878f5ac8861 - 79599a314ea120227a7ef9cfdac6e6df - 3 Verifying mirror read .. Verifying mirror write .. Verifying mirror copy .. mirror copied successfully: 2 3 Stopping client oleg110-client.virtnet /mnt/lustre (opts:) Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre PASS 37 (14s) == sanity-flr test 38: resync ============================ 01:48:09 (1761284889) 4 mirror IDs: 1 2 3 4 mirror_io: 391: hit by error injection mirror_io: 391: hit by error injection PASS 38 (9s) == sanity-flr test 39: check FLR+PFL (a.k.a. PFLR) creation ========================================================== 01:48:18 (1761284898) PASS 39 (5s) == sanity-flr test 40: PFLR rdonly state instantiation check ========================================================== 01:48:23 (1761284903) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0520839 s, 20.1 MB/s composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 760 lcm_flags: wp lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 7 components: - lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 0 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - l_ost_idx: 1 l_fid: 0x2c0000401:0x35:0x0 - lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - l_ost_idx: 1 l_fid: 0x2c0000401:0x36:0x0 - lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: prefer lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - l_ost_idx: 0 l_fid: 0x280000401:0x39:0x0 - lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: 0 lcme_extent.e_start: 1048576 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131079 lcme_mirror_id: 2 lcme_flags: 0 lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222558 s, 47.1 MB/s composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 760 lcm_flags: wp lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 7 components: - lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 0 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - l_ost_idx: 0 l_fid: 0x280000401:0x3a:0x0 - lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - l_ost_idx: 0 l_fid: 0x280000401:0x3b:0x0 - lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: prefer lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - l_ost_idx: 1 l_fid: 0x2c0000401:0x37:0x0 - lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: 0 lcme_extent.e_start: 1048576 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131079 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 PASS 40 (7s) == sanity-flr test 41: lfs mirror resync check =========== 01:48:30 (1761284910) **create two FLR files /mnt/lustre/f41.sanity-flr /mnt/lustre/f41.sanity-flr-1 **verify files be RDONLY 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0686192 s, 30.6 MB/s 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.230746 s, 18.2 MB/s **verify files be WRITE_PENDING **verify files have stale component **full resync **verify /mnt/lustre/f41.sanity-flr-1 data consistency in all mirrors **verify files be RDONLY **verify files do not contain stale component **write /mnt/lustre/f41.sanity-flr-1 for partial resync test 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.121091 s, 17.3 MB/s **only resync mirror 2 **resync mirror 2 again **verify /mnt/lustre/f41.sanity-flr-1 contains stale component **full resync /mnt/lustre/f41.sanity-flr-1 **full resync /mnt/lustre/f41.sanity-flr-1 again **verify /mnt/lustre/f41.sanity-flr-1 does not contain stale component PASS 41 (8s) == sanity-flr test 42: lfs mirror verify ================= 01:48:39 (1761284919) SKIP: sanity-flr test_42 need >= 4 OSTs SKIP 42 (3s) == sanity-flr test 43a: mirror pick on write ============= 01:48:41 (1761284921) SKIP: sanity-flr test_43a needs >= 3 OSTs SKIP 43a (3s) == sanity-flr test 43b: allow writing to multiple preferred mirror file ========================================================== 01:48:44 (1761284924) striped dir -i1 -c2 -H all_char /mnt/lustre/d43b.sanity-flr ** write to /mnt/lustre/d43b.sanity-flr/f43b.sanity-flr 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294892 s, 35.6 MB/s ** resync /mnt/lustre/d43b.sanity-flr/f43b.sanity-flr PASS 43b (7s) == sanity-flr test 44a: lfs mirror split check =========== 01:48:51 (1761284931) 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.270023 s, 11.6 MB/s PASS 44a (8s) == sanity-flr test 44b: mirror split does not create stale file ========================================================== 01:49:00 (1761284940) /mnt/lustre/d44b.sanity-flr/f44b.sanity-flr lcm_layout_gen: 2 lcm_mirror_count: 2 lcm_entry_count: 2 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x40:0x0] } lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0x45:0x0] } split mirror_id 1 id 65537 /mnt/lustre/d44b.sanity-flr/f44b.sanity-flr lcm_layout_gen: 5 lcm_mirror_count: 1 lcm_entry_count: 1 lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0x45:0x0] } split mirror_id 2 id 131073 lfs split: '/mnt/lustre/d44b.sanity-flr/f44b.sanity-flr' is not a FLR file. Cannot resync file make sure there's no stale comp in the file PASS 44b (6s) SKIP: sanity-flr test_44c skipping ALWAYS excluded test 44c == sanity-flr test 44d: lfs mirror split does not break size ========================================================== 01:49:07 (1761284947) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.216111 s, 48.5 MB/s ** before mirror ops, file size=10485760 ** after mirror extend, file size=10485760 lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000403:0x7:0x0]: Invalid argument (22) ** after mirror split, file size=10485760 Stopping client oleg110-client.virtnet /mnt/lustre (opts:) Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre ** after mirror split & remount, file size=10485760 PASS 44d (8s) == sanity-flr test 44e: basic FLR SOM tests + disable SOM ========================================================== 01:49:15 (1761284955) striped dir -i0 -c2 -H crush2 /mnt/lustre/d44e.sanity-flr llite.lustre-ffff992e882eb800.xattr_cache=0 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.228629 s, 45.9 MB/s ** before mirror resync, file size=10485760 file: /mnt/lustre/d44e.sanity-flr/f44e.sanity-flr size: 10485760 blocks: 40960 flags: 1 Stopping client oleg110-client.virtnet /mnt/lustre (opts:) Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre mdt.lustre-MDT0000.enable_strict_som=0 mdt.lustre-MDT0001.enable_strict_som=0 file: /mnt/lustre/d44e.sanity-flr/f44e.sanity-flr size: 10485760 blocks: 40960 flags: 1 llite.lustre-ffff992e85e8f000.xattr_cache=1 PASS 44e (10s) == sanity-flr test 45: Verify setstripe/getstripe with YAML with FLR file ========================================================== 01:49:25 (1761284965) striped dir -i1 -c2 -H all_char /mnt/lustre/d45.sanity-flr 1. FLR file: getstripe --yaml /mnt/lustre/d45.sanity-flr/f45.sanity-flr 1. FLR file: setstripe --yaml=/mnt/lustre/d45.sanity-flr/template /mnt/lustre/d45.sanity-flr/f45.sanity-flr.copy 1. FLR file: compare 2. FLR file with flags: getstripe --yaml /mnt/lustre/d45.sanity-flr/f45.sanity-flr 2. FLR file with flags: setstripe --yaml=/mnt/lustre/d45.sanity-flr/template /mnt/lustre/d45.sanity-flr/f45.sanity-flr.copy 2. FLR file with flags: compare PASS 45 (29s) == sanity-flr test 46: Verify setstripe --copy option ==== 01:49:54 (1761284994) striped dir -i0 -c2 -H all_char /mnt/lustre/d46.sanity-flr ** 1. PFL file ** 2. plain file ** 3. FLR file ** 4. PFL dir striped dir -i0 -c2 -H crush /mnt/lustre/d46.sanity-flr/dir striped dir -i0 -c2 -H crush2 /mnt/lustre/d46.sanity-flr/dir.copy ** 5. plain dir ** 6. FLR dir PASS 46 (26s) == sanity-flr test 47: Verify mirror obj alloc =========== 01:50:20 (1761285020) SKIP: sanity-flr test_47 needs >= 3 OSTs SKIP 47 (2s) == sanity-flr test 48: Verify snapshot mirror ============ 01:50:22 (1761285022) ** create 2 mirrors FLR file /mnt/lustre/f48.sanity-flr ** write it 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.144214 s, 21.8 MB/s ** resync the file ** snapshot mirror 2 ** write it again 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.0902643 s, 34.9 MB/s ** resync it again ** verify mirror 2 doesn't change original checksum: 2cf0f89e4bd10fde82779063c8032ffb - mirror 1 checksum: 35444878fbe33ac40b7959c971ac56b0 - mirror 2 checksum: 2cf0f89e4bd10fde82779063c8032ffb - ** mirror 2 stripe info /mnt/lustre/f48.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 448 lcm_flags: wp lcm_layout_gen: 10 lcm_mirror_count: 2 lcm_entry_count: 4 components: - lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale,nosync lcme_timestamp: 'Fri Oct 24 01:50:24 2025' lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lcme_offset: 336 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x9 lmm_fid: [0x200000405:0x9:0x0] lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x4b:0x0] } - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,stale,nosync lcme_timestamp: 'Fri Oct 24 01:50:24 2025' lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lcme_offset: 392 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x9 lmm_fid: [0x200000405:0x9:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x4c:0x0] } ** resync mirror 2 ** verify mirror 2 resync-ed original checksum: 2cf0f89e4bd10fde82779063c8032ffb - mirror 1 checksum: 35444878fbe33ac40b7959c971ac56b0 - mirror 2 checksum: 35444878fbe33ac40b7959c971ac56b0 - ** mirror 2 stripe info /mnt/lustre/f48.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 448 lcm_flags: ro lcm_layout_gen: 12 lcm_mirror_count: 2 lcm_entry_count: 4 components: - lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,nosync lcme_timestamp: 'Fri Oct 24 01:50:24 2025' lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lcme_offset: 336 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x9 lmm_fid: [0x200000405:0x9:0x0] lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x4b:0x0] } - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,nosync lcme_timestamp: 'Fri Oct 24 01:50:24 2025' lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lcme_offset: 392 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x9 lmm_fid: [0x200000405:0x9:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x4c:0x0] } PASS 48 (6s) == sanity-flr test 49a: FIEMAP upon FLR file ============= 01:50:28 (1761285028) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198922 s, 52.7 MB/s Filesystem type is: bd00bd0 File size of /mnt/lustre/f49a.sanity-flr is 1048576 (1024 blocks of 1024 bytes) ext: device_logical: physical_offset: length: dev: flags: 0: 0.. 1023: 310272.. 311295: 1024: 0001: net,eof 1: 0.. 1023: 308224.. 309247: 1024: 0000: last,net,eof /mnt/lustre/f49a.sanity-flr: 2 extents found FIEMAP on /mnt/lustre/f49a.sanity-flr succeeded PASS 49a (4s) == sanity-flr test 50A: mirror split update layout generation ========================================================== 01:50:32 (1761285032) Failing mds1 on oleg110-server Stopping /mnt/lustre-mds1 (opts:) on oleg110-server 01:50:42 (1761285042) shut down facet: mds1 facet_host: oleg110-server facet_failover_host: oleg110-server Failover mds1 to oleg110-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Started lustre-MDT0000 01:51:03 (1761285063) targets are mounted 01:51:03 (1761285063) facet_failover done oleg110-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0xf:0x0]: Invalid argument (22) /mnt/lustre/d50A.sanity-flr/f50A.sanity-flr layout generation from 1 to 2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330209 s, 31.8 MB/s /mnt/lustre/d50A.sanity-flr/f50A.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 136 lcm_flags: 0 lcm_layout_gen: 2 lcm_mirror_count: 1 lcm_entry_count: 1 components: - lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lcme_offset: 80 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0xf lmm_fid: [0x200000405:0xf:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x4e:0x0] } PASS 50A (44s) == sanity-flr test 50a: mirror extend/copy preserves sparseness ========================================================== 01:51:16 (1761285076) ** create striped file /mnt/lustre/d50a.sanity-flr/f50a.sanity-flr ** write 1st data chunk at 1M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0390388 s, 525 kB/s ** write 2nd data chunk at 2M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0242835 s, 843 kB/s ** create hole at the file end ** verify sparseness first data offset: 1044480 hole at the end: 3500000 ** extend the file with new mirror ** verify mirror #2 sparseness first data offset: 1044480 hole at the end: 3500000 ** copy mirror #2 to mirror #1 mirror copied successfully: 1 ** verify mirror #1 sparseness again first data offset: 1044480 hole at the end: 3500000 ** final consumed blocks: 104 PASS 50a (8s) == sanity-flr test 50b: mirror rsync handles sparseness == 01:51:24 (1761285084) ** create mirrored file /mnt/lustre/d50b.sanity-flr/f50b.sanity-flr ** write data chunk at 1M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0406626 s, 504 kB/s ** create hole at the file end ** verify sparseness first data offset: 1044480 hole at the end: 3500000 ** resync mirror #2 to mirror #1 ** consumed blocks: 96 ** truncate file down ** write data chunk at 2M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0105202 s, 1.9 MB/s ** resync mirror #2 to mirror #1 with nosync 1st component ** resync mirror #2 to mirror #1 again ** final consumed blocks: 96 PASS 50b (8s) == sanity-flr test 50c: punch_hole/mmap_write stale other mirrors ========================================================== 01:51:32 (1761285092) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d50c.sanity-flr 4+0 records in 4+0 records out 16384 bytes (16 kB, 16 KiB) copied, 0.0066093 s, 2.5 MB/s PASS 50c (6s) == sanity-flr test 50d: mirror rsync keep holes ========== 01:51:38 (1761285098) ** create mirrored file /mnt/lustre/d50d.sanity-flr/f50d.sanity-flr ** write data chunk at 1M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0253972 s, 806 kB/s ** create hole at the file start ** verify sparseness first data offset: 1048576 ** resync mirror #2 ** verify sparseness of mirror #2 first data offset: 1048576 PASS 50d (7s) == sanity-flr test 60a: mirror extend sets correct size on sparse file ========================================================== 01:51:45 (1761285105) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0111413 s, 368 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0351833 s, 116 kB/s PASS 60a (6s) == sanity-flr test 60b: mirror merge/split cancel client's in-memory layout gen ========================================================== 01:51:51 (1761285111) striped dir -i0 -c2 -H crush2 /mnt/lustre/d60b.sanity-flr lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1b:0x0]: Invalid argument (22) PASS 60b (17s) == sanity-flr test 61a: mirror extend and migrate preserve timestamps ========================================================== 01:52:08 (1761285128) mdd.lustre-MDT0000.atime_diff=1 mdd.lustre-MDT0001.atime_diff=1 create /mnt/lustre/d61a.sanity-flr/f61a.sanity-flr create /mnt/lustre/d61a.sanity-flr/f61a.sanity-flr-2 sleep 5 seconds, then cat f61a.sanity-flr XXX sleep 5 seconds, then re-write f61a.sanity-flr sleep 5 seconds mirror merge f61a.sanity-flr-2 to f61a.sanity-flr and test timestamps mirror extend f61a.sanity-flr and test timestamps migrate f61a.sanity-flr and test timestamps normal user migrate f61a.sanity-flr and test timestamps running as uid/gid/euid/egid 500/500/500/500, groups: 0 [/home/green/git/lustre-release/lustre/utils/lfs] [migrate] [-n] [/mnt/lustre/d61a.sanity-flr/f61a.sanity-flr] mdd.lustre-MDT0000.atime_diff=60 mdd.lustre-MDT0001.atime_diff=60 PASS 61a (25s) == sanity-flr test 61b: mirror extend and split preserve timestamps ========================================================== 01:52:33 (1761285153) create /mnt/lustre/d61b.sanity-flr/f61b.sanity-flr sleep 5 seconds, then cat f61b.sanity-flr XXX sleep 5 seconds, then re-write f61b.sanity-flr sleep 5 seconds, then test timestamps mirror extend f61b.sanity-flr and test timestamps mirror split f61b.sanity-flr and test timestamps lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x16:0x0]: Invalid argument (22) normal user mirror extend f61b.sanity-flr and test timestamps running as uid/gid/euid/egid 500/500/500/500, groups: 0 [/home/green/git/lustre-release/lustre/utils/lfs] [mirror] [extend] [-N] [-c1] [-i1] [/mnt/lustre/d61b.sanity-flr/f61b.sanity-flr] PASS 61b (20s) == sanity-flr test 61c: mirror resync preserves timestamps ========================================================== 01:52:54 (1761285174) create /mnt/lustre/d61c.sanity-flr/f61c.sanity-flr sleep 5 seconds, then cat f61c.sanity-flr XXX sleep 5 seconds, then mirror extend f61c.sanity-flr and write it sleep 5 seconds, then resync f61c.sanity-flr and test timestamps normal user resync f61c.sanity-flr and test timestamps running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [mirror] [resync] [/mnt/lustre/d61c.sanity-flr/f61c.sanity-flr] PASS 61c (21s) == sanity-flr test 62: read/write with unknown type of mirror ========================================================== 01:53:15 (1761285195) create mirror file with unknown magic fail_loc=0x1426 fail_val=2 create mirror file with unknown pattern fail_loc=0x1427 fail_val=1 PASS 62 (5s) == sanity-flr test 70: mirror create and split race ====== 01:53:20 (1761285200) striped dir -i0 -c2 -H crush2 /mnt/lustre/d70.sanity-flr mirror create pid 55868 mirror split pid 55869 mirror create and split race for 60 seconds, should not crash /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 55868 Killed while true; do rm -f $tf; $LFS mirror create -N -E 1M -c -1 -E eof -N $tf; echo xxxx > $tf; done (wd: ~) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 55869 Killed while true; do $LFS mirror split -d --mirror-id=1 $tf &> /dev/null; done (wd: ~) PASS 70 (64s) == sanity-flr test 70a: flr mode fsx test ================ 01:54:25 (1761285265) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d70a.sanity-flr Seed set to 5266 skipping zero size read 000002 1761285266.395316 mapwrite 0x0123c3 thru 0x019653 (0x7291 bytes) 000003 1761285266.447732 mapread 0x0042f9 thru 0x0126f3 (0xe3fb bytes) 000004 1761285266.457932 read 0x007d31 thru 0x013699 (0xb969 bytes) fallocating to largest ever: 0x24949 000005 1761285266.458960 fallocate 0x017474 thru 0x024948 (0xd4d5 bytes) 000006 1761285266.463138 read 0x01472c thru 0x017971 (0x3246 bytes) truncating to largest ever: 0x24a3d 000007 1761285266.464069 trunc from 0x019654 to 0x024a3c (0xb3e9 bytes) 000008 1761285266.479654 read 0x00e143 thru 0x01c5cc (0xe48a bytes) 000010 1761285266.491156 mapread 0x01eb16 thru 0x024a3c (0x5f27 bytes) 000011 1761285266.492058 trunc from 0x0016a8 to 0x00861b (0x6f74 bytes) 000012 1761285266.501616 mapread 0x00438f thru 0x004840 (0x4b2 bytes) 000013 1761285266.511731 read 0x00f981 thru 0x01a033 (0xa6b3 bytes) 000015 1761285266.512905 trunc from 0x00b269 to 0x01b014 (0xfdac bytes) 000016 1761285266.517355 punch from 0x005847 to 0x014e26 (0xf5e0 bytes) fallocating to largest ever: 0x33743 000018 1761285266.524250 fallocate 0x0307d1 thru 0x033742 (0x2f72 bytes) 000019 1761285266.529372 trunc from 0x033743 to 0x000dbb (0x32987 bytes) 000020 1761285266.542329 trunc from 0x00040c to 0x000dbb (0x9b0 bytes) 000022 1761285266.548985 mapwrite 0x00bdd7 thru 0x00e25a (0x2484 bytes) 000023 1761285266.600184 trunc from 0x00e25b to 0x012c93 (0x4a39 bytes) 000024 1761285266.622145 write 0x004092 thru 0x0138a0 (0xf80f bytes) fallocating to largest ever: 0x3b7f3 000025 1761285266.624040 fallocate 0x034b9f thru 0x03b7f2 (0x6c54 bytes) fallocating to largest ever: 0x40000 000027 1761285266.637262 fallocate 0x03441c thru 0x03ffff (0xbbe4 bytes) 000028 1761285266.647505 read 0x00bcc8 thru 0x00c1aa (0x4e3 bytes) 000029 1761285266.648239 mapread 0x00935d thru 0x00a8bd (0x1561 bytes) 000030 1761285266.663717 MIRROR_EXTEND to 2 mirrors 000031 1761285267.084286 MIRROR_RESYNC 2 mirrors 000032 1761285267.240879 write 0x016997 thru 0x02086b (0x9ed5 bytes) 000034 1761285267.282994 read 0x004e40 thru 0x00d36d (0x852e bytes) 000035 1761285267.295336 read 0x0040d3 thru 0x0121e3 (0xe111 bytes) 000036 1761285267.309735 trunc from 0x0016aa to 0x00eedf (0xd836 bytes) 000037 1761285267.320700 trunc from 0x02086c to 0x024cc3 (0x4458 bytes) 000039 1761285267.364768 MIRROR_RESYNC 2 mirrors 000040 1761285267.860161 trunc from 0x006879 to 0x007347 (0xacf bytes) 000041 1761285267.893559 write 0x03bda8 thru 0x03ffff (0x4258 bytes) 000042 1761285267.901338 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000043 1761285268.046111 trunc from 0x040000 to 0x0192a8 (0x26d57 bytes) 000044 1761285268.086350 punch from 0x003eab to 0x0110bd (0xd213 bytes) 000045 1761285268.092973 mapwrite 0x008b92 thru 0x00cea0 (0x430f bytes) 000046 1761285268.139425 mapread 0x00caf9 thru 0x01407c (0x7584 bytes) 000047 1761285268.151083 punch from 0x015a2e to 0x0186ca (0x2c9d bytes) 000048 1761285268.156922 mapwrite 0x02fee9 thru 0x03abb9 (0xacd1 bytes) 000049 1761285268.198339 punch from 0x0075c1 to 0x00aa1e (0x345e bytes) 000050 1761285268.202343 mapwrite 0x025058 thru 0x02e833 (0x97dc bytes) 000051 1761285268.228240 mapwrite 0x02d08d thru 0x033962 (0x68d6 bytes) 000052 1761285268.240403 read 0x0078bd thru 0x010604 (0x8d48 bytes) 000054 1761285268.245361 fallocate 0x03a3c5 thru 0x03d949 (0x3585 bytes) 000055 1761285268.250910 mapwrite 0x009081 thru 0x00be80 (0x2e00 bytes) 000056 1761285268.264305 write 0x00c1cd thru 0x016adb (0xa90f bytes) 000057 1761285268.271217 write 0x015e28 thru 0x020413 (0xa5ec bytes) 000059 1761285268.272862 write 0x006442 thru 0x0107c8 (0xa387 bytes) 000060 1761285268.275949 trunc from 0x03abba to 0x016859 (0x24360 bytes) 000062 1761285268.301992 mapwrite 0x00c7ba thru 0x01160a (0x4e51 bytes) 000063 1761285268.325209 fallocate 0x02c000 thru 0x02c4bc (0x4bd bytes) 000065 1761285268.335393 trunc from 0x01685a to 0x034fff (0x1e7a6 bytes) 000066 1761285268.351556 mapwrite 0x003881 thru 0x01160a (0xdd8a bytes) 000068 1761285268.376255 trunc from 0x024851 to 0x02f8be (0xb06e bytes) 000069 1761285268.382906 fallocate 0x028a35 thru 0x037757 (0xed23 bytes) 000071 1761285268.390097 fallocate 0x011a5c thru 0x0196b7 (0x7c5c bytes) 000072 1761285268.402101 mapwrite 0x013d82 thru 0x01a299 (0x6518 bytes) 000073 1761285268.448140 MIRROR_EXTEND to 2 mirrors 000074 1761285268.942843 fallocate 0x024bf0 thru 0x031e76 (0xd287 bytes) 000075 1761285268.977743 fallocate 0x00e387 thru 0x0189e5 (0xa65f bytes) 000076 1761285268.992059 MIRROR_EXTEND to 3 mirrors 000077 1761285269.402372 punch from 0x00aa6a to 0x015873 (0xae0a bytes) 000078 1761285269.446845 fallocate 0x00e517 thru 0x0187f5 (0xa2df bytes) 000079 1761285269.453231 trunc from 0x035000 to 0x013a35 (0x215ca bytes) 000080 1761285269.481527 mapread 0x008378 thru 0x013a35 (0xb6be bytes) 000081 1761285269.511988 mapwrite 0x00e7cc thru 0x00e9ee (0x223 bytes) 000083 1761285269.523700 mapread 0x0082a8 thru 0x01175b (0x94b4 bytes) 000084 1761285269.524565 fallocate 0x0366a1 thru 0x03caa7 (0x6407 bytes) 000085 1761285269.534220 trunc from 0x03caa8 to 0x014ec5 (0x27be2 bytes) 000086 1761285269.553317 MIRROR_EXTEND to 4 mirrors 000087 1761285270.164071 write 0x020516 thru 0x02fd9c (0xf887 bytes) 000088 1761285270.208227 MIRROR_RESYNC 4 mirrors 000089 1761285271.193139 write 0x02c4be thru 0x031bac (0x56ef bytes) 000090 1761285271.219320 trunc from 0x010adc to 0x018c22 (0x8147 bytes) 000092 1761285271.229387 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000093 1761285271.429547 trunc from 0x028220 to 0x031bac (0x998d bytes) 000095 1761285271.434890 mapread 0x011b3c thru 0x01fb37 (0xdffc bytes) 000096 1761285271.478565 punch from 0x015b0c to 0x01ebf9 (0x90ee bytes) 000098 1761285271.497344 trunc from 0x031bad to 0x013b7a (0x1e032 bytes) 000099 1761285271.529848 trunc from 0x013b7b to 0x020709 (0xcb8f bytes) 000100 1761285271.556710 punch from 0x003972 to 0x00bdd3 (0x8462 bytes) 000101 1761285271.563256 read 0x011af7 thru 0x01d2f6 (0xb800 bytes) 000102 1761285271.578447 write 0x01c8f2 thru 0x0259a3 (0x90b2 bytes) 000104 1761285271.585583 MIRROR_RESYNC 3 mirrors 000105 1761285272.462196 read 0x02213d thru 0x0259a3 (0x3867 bytes) 000106 1761285272.473852 write 0x01459b thru 0x01ac03 (0x6669 bytes) 000107 1761285272.524596 MIRROR_EXTEND to 4 mirrors 000108 1761285273.263387 read 0x01d010 thru 0x024323 (0x7314 bytes) 000110 1761285273.296778 read 0x0010f7 thru 0x008004 (0x6f0e bytes) 000111 1761285273.323644 mapread 0x023fa9 thru 0x024050 (0xa8 bytes) 000112 1761285273.358577 read 0x0230f7 thru 0x0259a3 (0x28ad bytes) 000113 1761285273.371297 MIRROR_RESYNC 4 mirrors 000114 1761285274.259358 trunc from 0x0053b5 to 0x00b52e (0x617a bytes) 000115 1761285274.300438 trunc from 0x0259a4 to 0x022e49 (0x2b5a bytes) 000116 1761285274.328623 fallocate 0x00ff6d thru 0x016bcd (0x6c61 bytes) 000118 1761285274.340732 write 0x03e0c7 thru 0x03ffff (0x1f39 bytes) 000119 1761285274.343658 read 0x01761c thru 0x01de1c (0x6801 bytes) 000120 1761285274.368712 mapwrite 0x02c80e thru 0x0342f9 (0x7aec bytes) 000121 1761285274.420735 read 0x01b14a thru 0x021de8 (0x6c9f bytes) 000122 1761285274.434706 write 0x013a96 thru 0x022916 (0xee81 bytes) 000123 1761285274.437258 read 0x00019a thru 0x009eb9 (0x9d20 bytes) 000124 1761285274.438073 trunc from 0x040000 to 0x025d78 (0x1a287 bytes) 000125 1761285274.457615 trunc from 0x005e24 to 0x00882d (0x2a0a bytes) 000126 1761285274.464123 trunc from 0x0019c8 to 0x00299f (0xfd8 bytes) 000127 1761285274.472080 trunc from 0x016192 to 0x017015 (0xe84 bytes) 000128 1761285274.488198 trunc from 0x0200f1 to 0x025d78 (0x5c88 bytes) 000129 1761285274.503671 MIRROR_RESYNC 4 mirrors 000130 1761285275.568190 write 0x01559d thru 0x0162a3 (0xd07 bytes) 000131 1761285275.597247 fallocate 0x03577c thru 0x03ffff (0xa884 bytes) 000132 1761285275.601268 write 0x02aca8 thru 0x02adc1 (0x11a bytes) 000133 1761285275.613118 mapwrite 0x02235c thru 0x02275d (0x402 bytes) 000134 1761285275.666887 trunc from 0x040000 to 0x0286a2 (0x1795d bytes) 000136 1761285275.692748 trunc from 0x0286a3 to 0x03f94a (0x172a8 bytes) 000137 1761285275.706443 mapwrite 0x01e998 thru 0x0231a2 (0x480b bytes) 000138 1761285275.722417 punch from 0x013e5c to 0x020f5a (0xd0ff bytes) 000139 1761285275.727902 punch from 0x008c5d to 0x015f4f (0xd2f3 bytes) 000140 1761285275.733696 fallocate 0x00fdba thru 0x011baa (0x1df1 bytes) 000141 1761285275.739862 trunc from 0x003913 to 0x00d3cb (0x9ab9 bytes) 000142 1761285275.746857 mapread 0x002e9e thru 0x00ceef (0xa052 bytes) 000143 1761285275.760710 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000144 1761285276.015143 mapread 0x01d6fb thru 0x025c19 (0x851f bytes) 000146 1761285276.036825 trunc from 0x03daac to 0x03f94a (0x1e9f bytes) 000147 1761285276.040724 fallocate 0x026321 thru 0x02cdfb (0x6adb bytes) 000149 1761285276.047436 trunc from 0x035a42 to 0x03c78e (0x6d4d bytes) 000150 1761285276.054575 fallocate 0x029a02 thru 0x0369f7 (0xcff6 bytes) 000152 1761285276.059300 mapwrite 0x00b74c thru 0x00c7ff (0x10b4 bytes) 000153 1761285276.080996 trunc from 0x02958b to 0x034fdd (0xba53 bytes) 000154 1761285276.086915 fallocate 0x017065 thru 0x018e24 (0x1dc0 bytes) 000156 1761285276.092578 trunc from 0x004f32 to 0x00560f (0x6de bytes) 000158 1761285276.100119 fallocate 0x02942c thru 0x029847 (0x41c bytes) 000159 1761285276.119078 MIRROR_RESYNC 3 mirrors 000160 1761285276.927975 fallocate 0x01909c thru 0x01bd56 (0x2cbb bytes) 000161 1761285276.967900 write 0x035a7a thru 0x03ff7b (0xa502 bytes) 000162 1761285276.984449 write 0x014f01 thru 0x01f671 (0xa771 bytes) 000163 1761285276.998908 MIRROR_EXTEND to 4 mirrors 000164 1761285277.617975 write 0x03048f thru 0x03a447 (0x9fb9 bytes) 000165 1761285277.676241 mapread 0x0026ec thru 0x0049fc (0x2311 bytes) 000167 1761285277.689206 mapread 0x00b72f thru 0x015d5a (0xa62c bytes) 000168 1761285277.696631 write 0x0114bb thru 0x014a00 (0x3546 bytes) 000169 1761285277.697580 trunc from 0x00e96f to 0x012273 (0x3905 bytes) 000170 1761285277.723137 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000171 1761285277.895636 write 0x0053fc thru 0x00c076 (0x6c7b bytes) 000172 1761285277.914314 trunc from 0x03ff7c to 0x019862 (0x26719 bytes) 000173 1761285277.938623 fallocate 0x037d8d thru 0x03ffff (0x8273 bytes) 000174 1761285277.945437 write 0x01e844 thru 0x029968 (0xb125 bytes) 000175 1761285277.965073 fallocate 0x02707d thru 0x02858b (0x150f bytes) 000176 1761285277.973193 mapread 0x01dfef thru 0x02a5fd (0xc60f bytes) 000177 1761285277.982964 mapwrite 0x01bf0d thru 0x027782 (0xb876 bytes) 000178 1761285278.004337 fallocate 0x003f88 thru 0x00ec02 (0xac7b bytes) 000180 1761285278.011347 trunc from 0x040000 to 0x012f5e (0x2d0a1 bytes) 000181 1761285278.033148 mapread 0x00ed89 thru 0x012f5e (0x41d6 bytes) 000182 1761285278.044105 punch from 0x001fde to 0x00caf9 (0xab1c bytes) 000183 1761285278.048450 trunc from 0x012f5f to 0x03b155 (0x281f7 bytes) 000184 1761285278.067419 write 0x015764 thru 0x023271 (0xdb0e bytes) 000185 1761285278.080540 write 0x0319b5 thru 0x03e1aa (0xc7f6 bytes) 000186 1761285278.090257 punch from 0x02d07e to 0x038b57 (0xbada bytes) 000187 1761285278.107662 MIRROR_EXTEND to 4 mirrors 000188 1761285278.730106 write 0x00d714 thru 0x0136a8 (0x5f95 bytes) 000189 1761285278.775990 MIRROR_EXTEND to 5 mirrors 000190 1761285279.434229 mapwrite 0x023c3f thru 0x02974f (0x5b11 bytes) 000191 1761285279.549297 write 0x01e15e thru 0x02a655 (0xc4f8 bytes) 000192 1761285279.555107 fallocate 0x03cb00 thru 0x03ffff (0x3500 bytes) 000193 1761285279.565446 MIRROR_EXTEND to 6 mirrors 000194 1761285280.132731 MIRROR_EXTEND to 7 mirrors 000195 1761285280.683956 read 0x033c74 thru 0x03e1aa (0xa537 bytes) 000196 1761285280.693664 fallocate 0x013e8e thru 0x019326 (0x5499 bytes) 000197 1761285280.749781 punch from 0x0237a7 to 0x029fa7 (0x6801 bytes) 000199 1761285280.755897 fallocate 0x01abe8 thru 0x021330 (0x6749 bytes) 000200 1761285280.762620 trunc from 0x03e1ab to 0x00b3cf (0x32ddb bytes) 000201 1761285280.782046 trunc from 0x00b3d0 to 0x00d6ef (0x2320 bytes) 000202 1761285280.797976 MIRROR_RESYNC 7 mirrors 000203 1761285281.570832 punch from 0x0093b0 to 0x00b16a (0x1dbb bytes) 000204 1761285281.608385 mapwrite 0x01c2d5 thru 0x01e110 (0x1e3c bytes) 000205 1761285281.667191 write 0x0271ab thru 0x02f7a6 (0x85fc bytes) 000206 1761285281.668650 fallocate 0x00575d thru 0x00ed62 (0x9606 bytes) 000207 1761285281.674650 write 0x00dad2 thru 0x014152 (0x6681 bytes) 000208 1761285281.684441 read 0x01095f thru 0x01b8e7 (0xaf89 bytes) 000209 1761285281.691781 punch from 0x00abea to 0x00ac10 (0x27 bytes) 000210 1761285281.696662 punch from 0x018c27 to 0x028550 (0xf92a bytes) 000211 1761285281.712136 fallocate 0x018d70 thru 0x028640 (0xf8d1 bytes) 000212 1761285281.718039 trunc from 0x02f7a7 to 0x01eeac (0x108fa bytes) 000213 1761285281.735433 trunc from 0x01eead to 0x00ed45 (0x10167 bytes) 000214 1761285281.755140 trunc from 0x00ed46 to 0x02f70a (0x209c5 bytes) 000215 1761285281.768129 mapwrite 0x02ddcd thru 0x02de4f (0x83 bytes) 000216 1761285281.788893 trunc from 0x00b7d0 to 0x015fe5 (0xa816 bytes) 000217 1761285281.792687 punch from 0x0236ed to 0x02c216 (0x8b2a bytes) 000218 1761285281.799651 mapread 0x0226d6 thru 0x02f70a (0xd035 bytes) 000219 1761285281.816700 trunc from 0x01d2d2 to 0x028e56 (0xbb85 bytes) 000220 1761285281.820247 mapwrite 0x03671f thru 0x03ffff (0x98e1 bytes) 000221 1761285281.875747 punch from 0x038cff to 0x03ffff (0x7301 bytes) 000222 1761285281.882191 punch from 0x00e9e2 to 0x00f24e (0x86d bytes) 000223 1761285281.890227 fallocate 0x00d634 thru 0x01190b (0x42d8 bytes) 000225 1761285281.894584 trunc from 0x040000 to 0x01899a (0x27665 bytes) 000226 1761285281.914107 read 0x015bff thru 0x01899a (0x2d9c bytes) 000227 1761285281.925004 trunc from 0x002273 to 0x005402 (0x3190 bytes) 000228 1761285281.933510 mapread 0x00b63e thru 0x0184ab (0xce6e bytes) 000229 1761285281.935315 mapwrite 0x012b77 thru 0x016f94 (0x441e bytes) 000230 1761285281.967854 mapwrite 0x025f20 thru 0x029887 (0x3968 bytes) 000231 1761285282.025035 mapread 0x000911 thru 0x00de84 (0xd574 bytes) 000232 1761285282.044589 trunc from 0x029888 to 0x012234 (0x17653 bytes) 000233 1761285282.073000 MIRROR_SPLIT mirror 1 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000234 1761285282.264809 mapwrite 0x02fd7c thru 0x03c9c4 (0xcc49 bytes) 000235 1761285282.339038 trunc from 0x02a513 to 0x032b9e (0x868c bytes) 000236 1761285282.358747 MIRROR_SPLIT mirror 3 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000237 1761285282.578411 trunc from 0x03c9c5 to 0x0119b3 (0x2b011 bytes) 000238 1761285282.592126 trunc from 0x009506 to 0x0119b3 (0x84ae bytes) 000239 1761285282.597748 fallocate 0x0130f9 thru 0x01ddb7 (0xacbf bytes) 000240 1761285282.604286 mapwrite 0x03bcfa thru 0x03ffff (0x4306 bytes) 000241 1761285282.662671 MIRROR_EXTEND to 6 mirrors 000242 1761285283.092328 read 0x0352f2 thru 0x03ffff (0xad0e bytes) 000243 1761285283.115433 read 0x03c8ae thru 0x03ffff (0x3752 bytes) 000244 1761285283.123619 trunc from 0x021c64 to 0x02c0bd (0xa45a bytes) 000245 1761285283.154120 MIRROR_EXTEND to 7 mirrors 000246 1761285283.639034 read 0x007db0 thru 0x014e1d (0xd06e bytes) 000247 1761285283.650314 read 0x00dbbe thru 0x00fb52 (0x1f95 bytes) 000248 1761285283.650757 read 0x016365 thru 0x016967 (0x603 bytes) 000251 1761285283.660985 trunc from 0x040000 to 0x009d3d (0x362c2 bytes) 000252 1761285283.701590 MIRROR_RESYNC 7 mirrors 000253 1761285284.371285 mapwrite 0x036f80 thru 0x03ffff (0x9080 bytes) 000254 1761285284.467276 read 0x02d72d thru 0x038f0b (0xb7df bytes) 000255 1761285284.482723 MIRROR_EXTEND to 8 mirrors 000256 1761285285.148818 punch from 0x012857 to 0x01ec35 (0xc3df bytes) 000257 1761285285.221699 trunc from 0x03fb35 to 0x03ffff (0x4cb bytes) 000260 1761285285.230268 mapread 0x031e8f thru 0x03f02c (0xd19e bytes) 000261 1761285285.250742 mapwrite 0x0121b5 thru 0x01eab0 (0xc8fc bytes) 000262 1761285285.279603 punch from 0x027774 to 0x0291d3 (0x1a60 bytes) 000263 1761285285.286607 trunc from 0x0154ee to 0x01ecf4 (0x9807 bytes) 000264 1761285285.292977 write 0x015b85 thru 0x0222b9 (0xc735 bytes) 000265 1761285285.310370 mapread 0x010144 thru 0x0177aa (0x7667 bytes) 000266 1761285285.311986 write 0x0102dd thru 0x01bb22 (0xb846 bytes) 000267 1761285285.313899 mapread 0x00ffce thru 0x011abb (0x1aee bytes) 000268 1761285285.315568 write 0x037b95 thru 0x03ae4f (0x32bb bytes) 000269 1761285285.317656 mapwrite 0x02c4c0 thru 0x03a533 (0xe074 bytes) 000270 1761285285.347311 punch from 0x024779 to 0x033f68 (0xf7f0 bytes) 000271 1761285285.356176 punch from 0x0294d0 to 0x036075 (0xcba6 bytes) 000272 1761285285.362413 mapread 0x015673 thru 0x016358 (0xce6 bytes) 000273 1761285285.363545 punch from 0x03da62 to 0x03ffff (0x259e bytes) 000274 1761285285.370056 write 0x006991 thru 0x008723 (0x1d93 bytes) 000276 1761285285.377965 punch from 0x01feae to 0x02e8a0 (0xe9f3 bytes) 000277 1761285285.393150 mapread 0x019609 thru 0x01cd76 (0x376e bytes) 000278 1761285285.393947 read 0x01893e thru 0x025ef5 (0xd5b8 bytes) 000279 1761285285.405382 fallocate 0x03bcd9 thru 0x03ffff (0x4327 bytes) 000280 1761285285.412088 trunc from 0x040000 to 0x02a24e (0x15db1 bytes) 000281 1761285285.484608 mapread 0x00ef8e thru 0x01620a (0x727d bytes) 000282 1761285285.486420 mapwrite 0x03dc71 thru 0x03ffff (0x238f bytes) 000283 1761285285.553628 mapwrite 0x02f63f thru 0x037f52 (0x8914 bytes) 000284 1761285285.597210 read 0x03a49d thru 0x03ffff (0x5b63 bytes) 000285 1761285285.597852 fallocate 0x00aa09 thru 0x0124bc (0x7ab4 bytes) 000286 1761285285.609095 fallocate 0x02b35e thru 0x02f5aa (0x424d bytes) 000287 1761285285.614947 trunc from 0x040000 to 0x006f1c (0x390e3 bytes) 000288 1761285285.635455 fallocate 0x015734 thru 0x01ffb4 (0xa881 bytes) 000289 1761285285.645064 fallocate 0x028d1a thru 0x034708 (0xb9ef bytes) 000290 1761285285.654021 fallocate 0x03a6d5 thru 0x03ffff (0x592b bytes) 000291 1761285285.661183 punch from 0x00a578 to 0x00af36 (0x9bf bytes) 000292 1761285285.670382 read 0x00db8a thru 0x01ca70 (0xeee7 bytes) 000293 1761285285.684226 trunc from 0x040000 to 0x02821a (0x17de5 bytes) 000294 1761285285.712380 write 0x01d7f3 thru 0x0291ea (0xb9f8 bytes) 000295 1761285285.713724 read 0x00727e thru 0x01289a (0xb61d bytes) 000296 1761285285.714316 trunc from 0x0291eb to 0x035b22 (0xc938 bytes) 000297 1761285285.754532 write 0x03c754 thru 0x03ffff (0x38ac bytes) 000298 1761285285.756862 punch from 0x024565 to 0x02debc (0x9958 bytes) 000299 1761285285.777096 MIRROR_SPLIT mirror 4 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000301 1761285285.986058 trunc from 0x03aac7 to 0x03ffff (0x5539 bytes) 000302 1761285286.000822 MIRROR_RESYNC 7 mirrors 000303 1761285287.444597 punch from 0x03bb59 to 0x03ffff (0x44a7 bytes) 000304 1761285287.503431 read 0x03b5d4 thru 0x03e80c (0x3239 bytes) 000305 1761285287.509045 read 0x027673 thru 0x034f04 (0xd892 bytes) 000306 1761285287.548801 MIRROR_SPLIT mirror 3 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000308 1761285287.789313 fallocate 0x035974 thru 0x039ca7 (0x4334 bytes) 000309 1761285287.796836 read 0x005e95 thru 0x013020 (0xd18c bytes) 000310 1761285287.804211 trunc from 0x040000 to 0x03d440 (0x2bbf bytes) 000311 1761285287.822680 punch from 0x0095ad to 0x0190fa (0xfb4e bytes) 000312 1761285287.828073 fallocate 0x0122b4 thru 0x012a15 (0x762 bytes) 000313 1761285287.832950 write 0x03c0c8 thru 0x03ffff (0x3f38 bytes) 000314 1761285287.837227 fallocate 0x0041c5 thru 0x00ac1e (0x6a5a bytes) 000316 1761285287.851461 trunc from 0x040000 to 0x0210a7 (0x1ef58 bytes) 000317 1761285287.868972 trunc from 0x01884e to 0x0210a7 (0x885a bytes) 000318 1761285287.879730 write 0x0192d6 thru 0x01b666 (0x2391 bytes) 000319 1761285287.900307 read 0x0111ef thru 0x020672 (0xf484 bytes) 000320 1761285287.909264 trunc from 0x0096ab to 0x00ece2 (0x5638 bytes) 000321 1761285287.920403 MIRROR_RESYNC 6 mirrors 000322 1761285289.456965 trunc from 0x0210a8 to 0x03eb5d (0x1dab6 bytes) 000323 1761285289.529192 trunc from 0x03eb5e to 0x03c6f7 (0x2466 bytes) 000324 1761285289.573381 trunc from 0x02852f to 0x02ffce (0x7aa0 bytes) 000325 1761285289.580862 read 0x003110 thru 0x003a3d (0x92e bytes) 000327 1761285289.588119 read 0x01f958 thru 0x027c2b (0x82d4 bytes) 000328 1761285289.603248 trunc from 0x025bdb to 0x02b190 (0x55b6 bytes) 000329 1761285289.612919 fallocate 0x036569 thru 0x03d548 (0x6fe0 bytes) 000330 1761285289.621050 read 0x02fd93 thru 0x039f40 (0xa1ae bytes) 000331 1761285289.622542 read 0x02e056 thru 0x03595d (0x7908 bytes) 000332 1761285289.625337 write 0x028fe4 thru 0x02ce04 (0x3e21 bytes) 000333 1761285289.645794 MIRROR_EXTEND to 7 mirrors 000334 1761285290.642770 mapwrite 0x022515 thru 0x02faaa (0xd596 bytes) 000335 1761285290.787928 mapwrite 0x0238bb thru 0x02676e (0x2eb4 bytes) 000336 1761285290.816140 write 0x03dc8e thru 0x03ffff (0x2372 bytes) 000337 1761285290.835397 punch from 0x01a227 to 0x01ade2 (0xbbc bytes) 000338 1761285290.839440 mapread 0x02e181 thru 0x033456 (0x52d6 bytes) 000339 1761285290.840390 mapread 0x02be6b thru 0x02df90 (0x2126 bytes) 000341 1761285290.841633 trunc from 0x02e25a to 0x02e8b5 (0x65c bytes) 000342 1761285290.850169 trunc from 0x029db9 to 0x02b45f (0x16a7 bytes) 000343 1761285290.857077 mapwrite 0x02a48a thru 0x02d795 (0x330c bytes) 000345 1761285290.884684 write 0x00ffe0 thru 0x017d5b (0x7d7c bytes) 000346 1761285290.896875 mapread 0x008273 thru 0x01507a (0xce08 bytes) 000347 1761285290.897862 punch from 0x0068ea to 0x0096ef (0x2e06 bytes) 000348 1761285290.912336 MIRROR_RESYNC 7 mirrors 000349 1761285292.887877 mapwrite 0x03116e thru 0x03e65d (0xd4f0 bytes) 000350 1761285292.988022 MIRROR_RESYNC 7 mirrors 000351 1761285294.985069 mapwrite 0x0219c1 thru 0x02a2a1 (0x88e1 bytes) 000353 1761285295.107599 mapwrite 0x02a05e thru 0x0369c4 (0xc967 bytes) 000354 1761285295.166050 trunc from 0x0335fe to 0x03ffff (0xca02 bytes) 000355 1761285295.174548 trunc from 0x03b92d to 0x03ffff (0x46d3 bytes) 000356 1761285295.180401 mapwrite 0x005aaa thru 0x011def (0xc346 bytes) 000357 1761285295.239952 write 0x019f70 thru 0x0255e5 (0xb676 bytes) 000358 1761285295.241524 fallocate 0x01e7e8 thru 0x020603 (0x1e1c bytes) 000359 1761285295.247501 trunc from 0x0214e2 to 0x02f4a7 (0xdfc6 bytes) 000360 1761285295.267689 trunc from 0x040000 to 0x0232c8 (0x1cd37 bytes) 000361 1761285295.298948 mapread 0x007399 thru 0x010021 (0x8c89 bytes) 000362 1761285295.301546 mapwrite 0x00c5f5 thru 0x01a7ca (0xe1d6 bytes) 000363 1761285295.353935 trunc from 0x0232c9 to 0x015444 (0xde84 bytes) 000364 1761285295.379383 punch from 0x0066ed to 0x015444 (0xed58 bytes) 000365 1761285295.397519 write 0x0301ff thru 0x030dd4 (0xbd6 bytes) 000366 1761285295.398791 mapread 0x0046dd thru 0x00fde5 (0xb709 bytes) 000367 1761285295.422174 trunc from 0x02958a to 0x030dd4 (0x784b bytes) 000368 1761285295.442757 mapread 0x006a95 thru 0x0160b5 (0xf621 bytes) 000369 1761285295.464583 MIRROR_RESYNC 7 mirrors 000370 1761285297.322434 fallocate 0x0194d1 thru 0x01a517 (0x1047 bytes) 000371 1761285297.367392 write 0x0235e3 thru 0x03059b (0xcfb9 bytes) 000372 1761285297.376094 punch from 0x00a1da to 0x00cdb5 (0x2bdc bytes) 000373 1761285297.388757 MIRROR_EXTEND to 8 mirrors 000374 1761285298.019838 trunc from 0x007190 to 0x016a69 (0xf8da bytes) 000375 1761285298.067602 punch from 0x0035cf to 0x004f85 (0x19b7 bytes) 000376 1761285298.083754 MIRROR_SPLIT mirror 6 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000377 1761285298.275747 fallocate 0x027a77 thru 0x02da4f (0x5fd9 bytes) 000378 1761285298.280207 mapread 0x01a149 thru 0x0289bf (0xe877 bytes) 000379 1761285298.307526 mapwrite 0x01bfbb thru 0x0245dc (0x8622 bytes) 000380 1761285298.333453 punch from 0x024bc7 to 0x02f0b3 (0xa4ed bytes) 000381 1761285298.354187 MIRROR_EXTEND to 8 mirrors 000382 1761285298.963080 fallocate 0x01644b thru 0x01ab83 (0x4739 bytes) 000383 1761285299.004018 trunc from 0x00972e to 0x017dce (0xe6a1 bytes) 000384 1761285299.009360 write 0x03b7a9 thru 0x03ffff (0x4857 bytes) 000385 1761285299.017641 fallocate 0x010581 thru 0x0121be (0x1c3e bytes) 000387 1761285299.029223 trunc from 0x028f4c to 0x02b79a (0x284f bytes) 000388 1761285299.035016 mapread 0x0009bf thru 0x006f0f (0x6551 bytes) 000389 1761285299.058020 punch from 0x00e637 to 0x01c8cf (0xe299 bytes) 000390 1761285299.065988 fallocate 0x02bbaf thru 0x0320ea (0x653c bytes) 000391 1761285299.071279 read 0x028df1 thru 0x02f883 (0x6a93 bytes) 000392 1761285299.087128 fallocate 0x0034c8 thru 0x0115a3 (0xe0dc bytes) 000393 1761285299.099776 write 0x032c47 thru 0x0382f8 (0x56b2 bytes) 000394 1761285299.118343 MIRROR_EXTEND to 9 mirrors 000396 1761285299.754180 MIRROR_SPLIT mirror 1 to 8 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000397 1761285300.131798 read 0x02f8a0 thru 0x03d51b (0xdc7c bytes) 000398 1761285300.156532 trunc from 0x040000 to 0x006748 (0x398b7 bytes) 000399 1761285300.256193 fallocate 0x029456 thru 0x02b1ab (0x1d56 bytes) 000402 1761285300.261797 trunc from 0x0049a3 to 0x006748 (0x1da6 bytes) 000403 1761285300.266950 trunc from 0x006749 to 0x02e735 (0x27fed bytes) 000404 1761285300.291242 mapwrite 0x028a2e thru 0x034eaf (0xc482 bytes) 000405 1761285300.347585 fallocate 0x02e1b9 thru 0x036629 (0x8471 bytes) 000406 1761285300.352656 fallocate 0x0163ff thru 0x016eaa (0xaac bytes) 000407 1761285300.356382 trunc from 0x02efbd to 0x036629 (0x766d bytes) 000408 1761285300.362119 mapread 0x012c16 thru 0x014fba (0x23a5 bytes) 000409 1761285300.376944 mapwrite 0x00db92 thru 0x01ba67 (0xded6 bytes) 000410 1761285300.399201 write 0x01d1d0 thru 0x0235bf (0x63f0 bytes) 000411 1761285300.400358 write 0x028023 thru 0x02e51c (0x64fa bytes) 000413 1761285300.409001 trunc from 0x013724 to 0x0146d0 (0xfad bytes) 000414 1761285300.417360 punch from 0x02a33a to 0x02a6e6 (0x3ad bytes) 000415 1761285300.430212 punch from 0x001d7e to 0x004f7c (0x31ff bytes) 000416 1761285300.435431 fallocate 0x0241af thru 0x02b43a (0x728c bytes) 000418 1761285300.443853 trunc from 0x019754 to 0x0232e2 (0x9b8f bytes) 000419 1761285300.448948 read 0x03393e thru 0x036629 (0x2cec bytes) 000420 1761285300.460188 trunc from 0x03662a to 0x03213f (0x44ea bytes) 000421 1761285300.475875 mapwrite 0x00d418 thru 0x01432d (0x6f16 bytes) 000422 1761285300.492331 punch from 0x0298da to 0x02f2e1 (0x5a08 bytes) 000423 1761285300.502830 read 0x01355f thru 0x018bd5 (0x5677 bytes) 000424 1761285300.504858 trunc from 0x001d9b to 0x0111ab (0xf411 bytes) 000425 1761285300.513913 fallocate 0x017039 thru 0x018dbe (0x1d86 bytes) 000426 1761285300.521395 read 0x030da2 thru 0x03213f (0x139e bytes) 000427 1761285300.521930 mapwrite 0x0262ac thru 0x028f9e (0x2cf3 bytes) 000428 1761285300.535535 mapwrite 0x0017f3 thru 0x00ec0c (0xd41a bytes) 000429 1761285300.564190 mapwrite 0x021d3e thru 0x02b525 (0x97e8 bytes) 000430 1761285300.586553 mapwrite 0x014164 thru 0x0197c4 (0x5661 bytes) 000431 1761285300.603424 read 0x030cd7 thru 0x03213f (0x1469 bytes) 000432 1761285300.604502 trunc from 0x032140 to 0x01a49a (0x17ca5 bytes) 000433 1761285300.626861 write 0x028e23 thru 0x0368d1 (0xdaaf bytes) 000434 1761285300.628508 trunc from 0x0368d2 to 0x02022d (0x166a4 bytes) 000435 1761285300.647416 fallocate 0x01236d thru 0x02002e (0xdcc2 bytes) 000436 1761285300.657520 trunc from 0x007cbd to 0x00c28b (0x45cf bytes) 000437 1761285300.668603 fallocate 0x01c536 thru 0x02a7e1 (0xe2ac bytes) 000438 1761285300.684096 MIRROR_SPLIT mirror 3 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000439 1761285300.901415 fallocate 0x00fcb7 thru 0x0169ea (0x6d34 bytes) 000441 1761285300.911953 trunc from 0x011388 to 0x018a3a (0x76b3 bytes) 000442 1761285300.916921 trunc from 0x00932d to 0x0130c6 (0x9d9a bytes) 000443 1761285300.923591 fallocate 0x03c282 thru 0x03ffff (0x3d7e bytes) 000444 1761285300.928658 mapread 0x005970 thru 0x00fd21 (0xa3b2 bytes) 000445 1761285300.953730 trunc from 0x02a7e2 to 0x00c4d1 (0x1e310 bytes) 000446 1761285300.993284 punch from 0x000230 to 0x00b702 (0xb4d3 bytes) 000447 1761285300.999749 write 0x00093f thru 0x005f26 (0x55e8 bytes) 000448 1761285301.014568 punch from 0x001fb9 to 0x00c4d1 (0xa519 bytes) 000449 1761285301.036294 mapwrite 0x00c1f4 thru 0x01c1a4 (0xffb1 bytes) 000450 1761285301.135313 mapread 0x0186e7 thru 0x01c1a4 (0x3abe bytes) 000451 1761285301.136176 punch from 0x011075 to 0x01c1a4 (0xb130 bytes) 000452 1761285301.145151 mapread 0x014817 thru 0x01c1a4 (0x798e bytes) 000453 1761285301.155660 mapread 0x00a661 thru 0x017ced (0xd68d bytes) 000454 1761285301.156627 mapread 0x01aa87 thru 0x01c1a4 (0x171e bytes) 000455 1761285301.157328 punch from 0x013449 to 0x01c1a4 (0x8d5c bytes) 000456 1761285301.163216 trunc from 0x01c1a5 to 0x0092d5 (0x12ecf bytes) 000457 1761285301.190714 read 0x001e57 thru 0x0092d5 (0x747f bytes) 000458 1761285301.202365 trunc from 0x0092d6 to 0x028881 (0x1f5ac bytes) 000459 1761285301.226349 mapwrite 0x0071b5 thru 0x00d171 (0x5fbd bytes) 000460 1761285301.261132 trunc from 0x004911 to 0x00cedd (0x85cd bytes) 000461 1761285301.268192 fallocate 0x019484 thru 0x01da97 (0x4614 bytes) 000462 1761285301.283348 mapwrite 0x021baf thru 0x02c0b3 (0xa505 bytes) 000463 1761285301.348689 write 0x023901 thru 0x02551f (0x1c1f bytes) 000464 1761285301.349686 trunc from 0x028d4d to 0x02c0b3 (0x3367 bytes) 000465 1761285301.354762 fallocate 0x0287a9 thru 0x030002 (0x785a bytes) 000466 1761285301.362664 trunc from 0x02c0b4 to 0x039eba (0xde07 bytes) 000467 1761285301.384750 read 0x028287 thru 0x02b049 (0x2dc3 bytes) 000468 1761285301.393565 write 0x0082de thru 0x009110 (0xe33 bytes) 000469 1761285301.404845 MIRROR_RESYNC 7 mirrors 000470 1761285303.266906 MIRROR_EXTEND to 8 mirrors 000472 1761285304.013288 MIRROR_SPLIT mirror 4 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000473 1761285304.331221 read 0x0131e6 thru 0x01a713 (0x752e bytes) 000474 1761285304.339656 mapread 0x00bebd thru 0x00df55 (0x2099 bytes) 000475 1761285304.357293 trunc from 0x039ebb to 0x001b0f (0x383ab bytes) 000476 1761285304.403939 write 0x00088f thru 0x001b1e (0x1290 bytes) 000477 1761285304.408745 read 0x0001fb thru 0x001b1e (0x1924 bytes) 000478 1761285304.409340 write 0x00bd14 thru 0x00d9eb (0x1cd8 bytes) 000479 1761285304.410314 punch from 0x008ba9 to 0x00d9eb (0x4e43 bytes) 000480 1761285304.429340 punch from 0x003e67 to 0x00d107 (0x92a1 bytes) 000481 1761285304.434794 trunc from 0x00d9ec to 0x024b0b (0x17120 bytes) 000483 1761285304.463382 MIRROR_RESYNC 7 mirrors 000484 1761285305.680415 trunc from 0x021cb9 to 0x024b0b (0x2e53 bytes) 000485 1761285305.719383 fallocate 0x012905 thru 0x01eebf (0xc5bb bytes) 000486 1761285305.724164 trunc from 0x024b0c to 0x036dae (0x122a3 bytes) 000487 1761285305.742276 fallocate 0x023459 thru 0x02693b (0x34e3 bytes) 000489 1761285305.752373 MIRROR_RESYNC 7 mirrors 000490 1761285306.851724 trunc from 0x017981 to 0x022ffa (0xb67a bytes) 000491 1761285306.885170 read 0x036059 thru 0x036dae (0xd56 bytes) 000492 1761285306.890125 mapwrite 0x0358d1 thru 0x03f153 (0x9883 bytes) 000493 1761285306.960400 fallocate 0x013dea thru 0x023741 (0xf958 bytes) 000494 1761285306.966358 trunc from 0x02195d to 0x025b5a (0x41fe bytes) 000495 1761285306.973937 write 0x02a9c1 thru 0x034fa2 (0xa5e2 bytes) 000496 1761285306.991177 trunc from 0x03f154 to 0x028cc4 (0x1648f bytes) 000497 1761285307.017370 mapwrite 0x0267f1 thru 0x0291d6 (0x29e6 bytes) 000498 1761285307.069892 MIRROR_SPLIT mirror 3 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000499 1761285307.307665 mapread 0x006fbf thru 0x00ecc8 (0x7d0a bytes) 000500 1761285307.337904 write 0x01fce7 thru 0x027afd (0x7e17 bytes) 000501 1761285307.353354 fallocate 0x03ebcc thru 0x03ffff (0x1434 bytes) 000502 1761285307.360823 read 0x01401d thru 0x01498c (0x970 bytes) 000503 1761285307.366931 write 0x00af6c thru 0x015f7f (0xb014 bytes) 000504 1761285307.374024 mapwrite 0x03a412 thru 0x03ffff (0x5bee bytes) 000505 1761285307.410826 trunc from 0x040000 to 0x00b1b9 (0x34e46 bytes) 000506 1761285307.431705 mapread 0x000be2 thru 0x0077b3 (0x6bd2 bytes) 000507 1761285307.441685 punch from 0x006edb to 0x00b1b9 (0x42df bytes) 000508 1761285307.447210 read 0x002e2b thru 0x0036f8 (0x8ce bytes) 000509 1761285307.447747 trunc from 0x00b1ba to 0x026a44 (0x1b88b bytes) 000510 1761285307.475636 trunc from 0x026a45 to 0x0170c6 (0xf97e bytes) 000511 1761285307.494655 mapread 0x001ce0 thru 0x0024c6 (0x7e7 bytes) 000512 1761285307.495562 fallocate 0x00a3d9 thru 0x010917 (0x653f bytes) 000513 1761285307.500283 trunc from 0x013a0e to 0x0170c6 (0x36b9 bytes) 000514 1761285307.512120 MIRROR_RESYNC 6 mirrors 000515 1761285308.582094 trunc from 0x010f33 to 0x011570 (0x63e bytes) 000516 1761285308.611061 punch from 0x000de0 to 0x00f872 (0xea93 bytes) 000517 1761285308.617173 write 0x020060 thru 0x02f5e8 (0xf589 bytes) 000518 1761285308.618901 punch from 0x013f3f to 0x01df04 (0x9fc6 bytes) 000519 1761285308.623354 punch from 0x00cc1d to 0x01a6c7 (0xdaab bytes) 000520 1761285308.634014 mapread 0x010241 thru 0x01e086 (0xde46 bytes) 000521 1761285308.667937 write 0x01aff7 thru 0x01affa (0x4 bytes) 000522 1761285308.673221 MIRROR_SPLIT mirror 3 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000523 1761285308.896848 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000524 1761285309.110285 trunc from 0x0071a1 to 0x011ac5 (0xa925 bytes) 000525 1761285309.114563 read 0x01df7e thru 0x026adf (0x8b62 bytes) 000527 1761285309.122723 trunc from 0x02f5e9 to 0x027272 (0x8376 bytes) 000528 1761285309.148328 mapread 0x0142b9 thru 0x0228a6 (0xe5ee bytes) 000529 1761285309.174994 write 0x00048c thru 0x007332 (0x6ea7 bytes) 000530 1761285309.184596 mapwrite 0x01f8a7 thru 0x02a998 (0xb0f2 bytes) 000531 1761285309.222068 write 0x03a29e thru 0x03ffff (0x5d62 bytes) 000532 1761285309.223362 trunc from 0x0362c2 to 0x03ef42 (0x8c81 bytes) 000533 1761285309.238984 punch from 0x00e3cc to 0x01acaf (0xc8e4 bytes) 000535 1761285309.255827 punch from 0x01f611 to 0x025fa6 (0x6996 bytes) 000536 1761285309.259777 punch from 0x011d10 to 0x0134cb (0x17bc bytes) 000537 1761285309.264811 mapwrite 0x0187b4 thru 0x02378f (0xafdc bytes) 000538 1761285309.302249 write 0x014900 thru 0x022277 (0xd978 bytes) 000539 1761285309.303653 fallocate 0x0141ef thru 0x019e8e (0x5ca0 bytes) 000540 1761285309.310175 trunc from 0x040000 to 0x00e5c7 (0x31a38 bytes) 000541 1761285309.324784 trunc from 0x00e5c8 to 0x031308 (0x22d41 bytes) 000542 1761285309.341140 MIRROR_EXTEND to 5 mirrors 000543 1761285309.796541 mapwrite 0x03cff5 thru 0x03ffff (0x300b bytes) 000544 1761285309.889059 punch from 0x035aac to 0x03877d (0x2cd2 bytes) 000545 1761285309.893600 fallocate 0x038a40 thru 0x03ffff (0x75c0 bytes) 000546 1761285309.898345 mapread 0x033fde thru 0x03ee6a (0xae8d bytes) 000547 1761285309.905535 mapread 0x03a276 thru 0x03ffff (0x5d8a bytes) 000548 1761285309.906329 punch from 0x00b73b to 0x00df6f (0x2835 bytes) 000549 1761285309.909914 mapread 0x02a08b thru 0x0327dc (0x8752 bytes) 000550 1761285309.918784 mapwrite 0x0159cf thru 0x016bb1 (0x11e3 bytes) 000551 1761285309.937298 write 0x016a5e thru 0x02632b (0xf8ce bytes) 000553 1761285309.939237 trunc from 0x031112 to 0x033d66 (0x2c55 bytes) 000554 1761285309.943675 trunc from 0x040000 to 0x03fdb3 (0x24c bytes) 000556 1761285309.966596 read 0x0105de thru 0x01db04 (0xd527 bytes) 000557 1761285309.973161 MIRROR_SPLIT mirror 4 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000558 1761285310.186834 mapread 0x03960c thru 0x03fdb3 (0x67a8 bytes) 000560 1761285310.215864 write 0x01fc04 thru 0x0293e0 (0x97dd bytes) 000561 1761285310.241220 mapwrite 0x01d8f8 thru 0x0245a8 (0x6cb1 bytes) 000562 1761285310.282221 MIRROR_RESYNC 4 mirrors 000563 1761285311.042279 mapwrite 0x035c51 thru 0x03f6e4 (0x9a94 bytes) 000564 1761285311.138174 punch from 0x0312e6 to 0x03fdb3 (0xeace bytes) 000566 1761285311.143984 trunc from 0x017fc9 to 0x02713f (0xf177 bytes) 000567 1761285311.150299 mapwrite 0x003b06 thru 0x009414 (0x590f bytes) 000568 1761285311.182434 trunc from 0x000040 to 0x0003ac (0x36d bytes) 000570 1761285311.190699 mapread 0x0323c9 thru 0x03bcea (0x9922 bytes) 000571 1761285311.210805 mapread 0x029788 thru 0x039443 (0xfcbc bytes) 000572 1761285311.218618 read 0x01f255 thru 0x022afd (0x38a9 bytes) 000573 1761285311.224314 fallocate 0x01d518 thru 0x02bf9e (0xea87 bytes) 000574 1761285311.229741 punch from 0x02ec61 to 0x0384fa (0x989a bytes) 000575 1761285311.234230 trunc from 0x03fdb4 to 0x0194f2 (0x268c1 bytes) 000576 1761285311.269236 fallocate 0x03bf7b thru 0x03ffff (0x4085 bytes) 000577 1761285311.273431 punch from 0x01e26a to 0x0242dd (0x6074 bytes) 000578 1761285311.281730 trunc from 0x00745f to 0x012872 (0xb414 bytes) 000579 1761285311.293338 fallocate 0x016583 thru 0x023e59 (0xd8d7 bytes) 000580 1761285311.298833 write 0x018850 thru 0x01d289 (0x4a3a bytes) 000581 1761285311.307083 punch from 0x029d1c to 0x03068e (0x6973 bytes) 000582 1761285311.317051 write 0x00add9 thru 0x017b36 (0xcd5e bytes) 000583 1761285311.326003 fallocate 0x00b3fa thru 0x013d5b (0x8962 bytes) 000584 1761285311.335052 mapread 0x0195d7 thru 0x02271b (0x9145 bytes) 000585 1761285311.340690 mapread 0x00ca9d thru 0x00fec5 (0x3429 bytes) 000586 1761285311.341524 trunc from 0x032219 to 0x03ffff (0xdde7 bytes) 000587 1761285311.353136 MIRROR_EXTEND to 5 mirrors 000588 1761285311.855332 trunc from 0x040000 to 0x0195e8 (0x26a17 bytes) 000589 1761285311.928053 fallocate 0x03d8c2 thru 0x03ffff (0x273e bytes) 000590 1761285311.935686 trunc from 0x030634 to 0x03fad5 (0xf4a2 bytes) 000591 1761285311.939736 mapread 0x00559c thru 0x009795 (0x41fa bytes) 000592 1761285311.962513 write 0x03cca8 thru 0x03ea6c (0x1dc5 bytes) 000593 1761285311.984382 mapwrite 0x00aac1 thru 0x016aff (0xc03f bytes) 000594 1761285312.026611 punch from 0x030bcb to 0x037ca4 (0x70da bytes) 000595 1761285312.035662 read 0x01da1a thru 0x024d76 (0x735d bytes) 000596 1761285312.045395 write 0x034cae thru 0x03ffff (0xb352 bytes) 000597 1761285312.055635 trunc from 0x040000 to 0x009a17 (0x365e8 bytes) 000599 1761285312.082354 mapread 0x0037e5 thru 0x009a17 (0x6233 bytes) 000600 1761285312.084717 write 0x00091d thru 0x00daa3 (0xd187 bytes) 000601 1761285312.096152 mapwrite 0x008f49 thru 0x015d60 (0xce18 bytes) 000602 1761285312.148233 mapwrite 0x00c8b7 thru 0x00f061 (0x27ab bytes) 000603 1761285312.165088 mapread 0x0002d5 thru 0x00ba3d (0xb769 bytes) 000604 1761285312.166047 mapread 0x00e6bc thru 0x015d60 (0x76a5 bytes) 000605 1761285312.166859 mapread 0x0047ff thru 0x00fa08 (0xb20a bytes) 000606 1761285312.167668 write 0x010f18 thru 0x01b3e6 (0xa4cf bytes) 000607 1761285312.172566 mapread 0x008977 thru 0x01019a (0x7824 bytes) 000608 1761285312.195565 MIRROR_RESYNC 5 mirrors 000609 1761285312.822399 punch from 0x009ffd to 0x00cfea (0x2fee bytes) 000610 1761285312.864519 fallocate 0x01b161 thru 0x023ca1 (0x8b41 bytes) 000611 1761285312.870196 mapread 0x013bc9 thru 0x01b3e6 (0x781e bytes) 000612 1761285312.887787 trunc from 0x000298 to 0x002c2a (0x2993 bytes) 000613 1761285312.897623 trunc from 0x01b3e7 to 0x03e314 (0x22f2e bytes) 000614 1761285312.958322 trunc from 0x03e315 to 0x032bb8 (0xb75c bytes) 000615 1761285312.993395 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000618 1761285313.244912 fallocate 0x016570 thru 0x0200c5 (0x9b56 bytes) 000619 1761285313.252977 MIRROR_EXTEND to 5 mirrors 000620 1761285313.962336 trunc from 0x021f81 to 0x02ceb7 (0xaf37 bytes) 000621 1761285314.021988 write 0x014e0d thru 0x023887 (0xea7b bytes) 000622 1761285314.037506 fallocate 0x00d3ce thru 0x00f09a (0x1ccd bytes) 000623 1761285314.048329 read 0x010e37 thru 0x01327b (0x2445 bytes) 000624 1761285314.059774 MIRROR_SPLIT mirror 3 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000625 1761285314.310966 punch from 0x020f0e to 0x02ea53 (0xdb46 bytes) 000627 1761285314.317628 punch from 0x027dd9 to 0x02ee52 (0x707a bytes) 000629 1761285314.322973 mapread 0x00aa12 thru 0x00e094 (0x3683 bytes) 000630 1761285314.343671 punch from 0x01ed67 to 0x02b3a1 (0xc63b bytes) 000631 1761285314.350101 punch from 0x019d65 to 0x02586c (0xbb08 bytes) 000632 1761285314.358790 mapread 0x00d3e7 thru 0x014647 (0x7261 bytes) 000633 1761285314.364656 MIRROR_EXTEND to 5 mirrors 000634 1761285315.049098 trunc from 0x032bb9 to 0x037742 (0x4b8a bytes) 000635 1761285315.131284 fallocate 0x01d87c thru 0x02698f (0x9114 bytes) 000636 1761285315.139379 fallocate 0x03fcc3 thru 0x03ffff (0x33d bytes) 000637 1761285315.149108 fallocate 0x0211b4 thru 0x02ee61 (0xdcae bytes) 000638 1761285315.154994 mapread 0x007af7 thru 0x009b57 (0x2061 bytes) 000639 1761285315.176420 MIRROR_SPLIT mirror 4 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000640 1761285315.403269 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000641 1761285315.599640 trunc from 0x040000 to 0x03e5da (0x1a25 bytes) 000642 1761285315.636800 punch from 0x030d5c to 0x037015 (0x62ba bytes) 000643 1761285315.644139 MIRROR_RESYNC 3 mirrors 000644 1761285316.565149 trunc from 0x03e5db to 0x032e52 (0xb788 bytes) 000645 1761285316.606967 write 0x017d3d thru 0x02119b (0x945f bytes) 000646 1761285316.617377 trunc from 0x01cd77 to 0x023eca (0x7154 bytes) 000647 1761285316.635211 trunc from 0x01e062 to 0x02c4b9 (0xe458 bytes) 000648 1761285316.641602 fallocate 0x00e373 thru 0x01060b (0x2299 bytes) 000649 1761285316.649456 read 0x00ae77 thru 0x019fd0 (0xf15a bytes) 000650 1761285316.667052 trunc from 0x032e53 to 0x01fe13 (0x1303f bytes) 000651 1761285316.691262 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000653 1761285316.928745 MIRROR_EXTEND to 3 mirrors 000654 1761285317.661532 punch from 0x011830 to 0x01d80d (0xbfde bytes) 000655 1761285317.693632 trunc from 0x007589 to 0x01350f (0xbf87 bytes) 000656 1761285317.700922 trunc from 0x01fe14 to 0x03e8a2 (0x1ea8f bytes) 000657 1761285317.737967 read 0x019e27 thru 0x02501c (0xb1f6 bytes) 000658 1761285317.751628 trunc from 0x03ad76 to 0x03e8a2 (0x3b2d bytes) 000659 1761285317.756732 fallocate 0x012ee9 thru 0x020c53 (0xdd6b bytes) 000660 1761285317.762175 read 0x01756b thru 0x026958 (0xf3ee bytes) 000661 1761285317.777085 mapwrite 0x018736 thru 0x027831 (0xf0fc bytes) 000662 1761285317.800371 write 0x0313f4 thru 0x039b4d (0x875a bytes) 000663 1761285317.801560 mapwrite 0x03981f thru 0x03fec2 (0x66a4 bytes) 000664 1761285317.842744 trunc from 0x03fec3 to 0x03dc28 (0x229a bytes) 000665 1761285317.862845 read 0x01f907 thru 0x02dbcd (0xe2c7 bytes) 000666 1761285317.863702 fallocate 0x033994 thru 0x03da33 (0xa0a0 bytes) 000667 1761285317.884682 MIRROR_EXTEND to 4 mirrors 000668 1761285318.693948 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000669 1761285319.047730 write 0x03a340 thru 0x03e66a (0x432b bytes) 000670 1761285319.101551 trunc from 0x03e66b to 0x021708 (0x1cf62 bytes) 000672 1761285319.127552 trunc from 0x017f60 to 0x01bcc6 (0x3d67 bytes) 000673 1761285319.135118 trunc from 0x01b76e to 0x021708 (0x5f9b bytes) 000675 1761285319.142300 read 0x001d91 thru 0x0093d9 (0x7649 bytes) 000676 1761285319.163360 trunc from 0x021709 to 0x014eb5 (0xc853 bytes) 000677 1761285319.184032 trunc from 0x014eb6 to 0x005e17 (0xf09e bytes) 000678 1761285319.213395 trunc from 0x001298 to 0x005e17 (0x4b80 bytes) 000680 1761285319.220221 fallocate 0x03c8d3 thru 0x03d2a9 (0x9d7 bytes) 000681 1761285319.238872 read 0x002116 thru 0x004e42 (0x2d2d bytes) 000682 1761285319.246341 trunc from 0x005e18 to 0x02365c (0x1d845 bytes) 000683 1761285319.262817 punch from 0x007dff to 0x00be86 (0x4088 bytes) 000685 1761285319.270026 trunc from 0x02365d to 0x0160ed (0xd56f bytes) 000686 1761285319.286567 fallocate 0x00f1de thru 0x011713 (0x2536 bytes) 000687 1761285319.296534 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000689 1761285319.538282 trunc from 0x011e82 to 0x0160ed (0x426c bytes) 000690 1761285319.545128 mapwrite 0x026312 thru 0x02ba10 (0x56ff bytes) 000691 1761285319.624102 read 0x015fd5 thru 0x01bb7e (0x5baa bytes) 000692 1761285319.634082 trunc from 0x02ba11 to 0x006483 (0x2558d bytes) 000693 1761285319.658894 read 0x001bac thru 0x006483 (0x48d8 bytes) 000694 1761285319.669217 trunc from 0x006484 to 0x02c3b8 (0x25f35 bytes) 000695 1761285319.687358 read 0x018b44 thru 0x01f61d (0x6ada bytes) 000697 1761285319.698300 trunc from 0x02b178 to 0x02c3b8 (0x1241 bytes) 000698 1761285319.705872 read 0x01ab68 thru 0x01edb8 (0x4251 bytes) 000699 1761285319.715672 MIRROR_EXTEND to 3 mirrors 000700 1761285320.433591 write 0x030799 thru 0x031e5d (0x16c5 bytes) 000701 1761285320.473150 trunc from 0x00348c to 0x00c004 (0x8b79 bytes) 000703 1761285320.479356 mapwrite 0x029850 thru 0x033e1b (0xa5cc bytes) 000704 1761285320.538910 mapread 0x030e96 thru 0x033e1b (0x2f86 bytes) 000705 1761285320.539849 punch from 0x031ca3 to 0x033e1b (0x2179 bytes) 000706 1761285320.546251 punch from 0x00422e to 0x00a402 (0x61d5 bytes) 000707 1761285320.551212 read 0x02465f thru 0x02b3fc (0x6d9e bytes) 000708 1761285320.551902 punch from 0x022b94 to 0x0265e7 (0x3a54 bytes) 000709 1761285320.557664 punch from 0x01a6b9 to 0x01af8e (0x8d6 bytes) 000710 1761285320.564541 punch from 0x032437 to 0x033e1b (0x19e5 bytes) 000711 1761285320.569357 trunc from 0x02fe2b to 0x032e46 (0x301c bytes) 000713 1761285320.577786 trunc from 0x033e1c to 0x023913 (0x10508 bytes) 000714 1761285320.602609 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000715 1761285320.843962 MIRROR_RESYNC 2 mirrors 000716 1761285321.512971 write 0x004d02 thru 0x005c47 (0xf46 bytes) 000718 1761285321.564448 trunc from 0x023914 to 0x00817a (0x1b799 bytes) 000719 1761285321.600104 read 0x003993 thru 0x004911 (0xf7f bytes) 000720 1761285321.613576 read 0x006867 thru 0x00817a (0x1914 bytes) 000721 1761285321.642389 write 0x027a87 thru 0x0336df (0xbc59 bytes) 000722 1761285321.646128 fallocate 0x000305 thru 0x008579 (0x8275 bytes) 000723 1761285321.659033 fallocate 0x03dac4 thru 0x03ffff (0x253c bytes) 000724 1761285321.677038 read 0x00fd2a thru 0x01ed82 (0xf059 bytes) 000725 1761285321.691908 read 0x00b844 thru 0x010601 (0x4dbe bytes) 000727 1761285321.692998 punch from 0x01bc88 to 0x01c1b2 (0x52b bytes) 000729 1761285321.703197 mapwrite 0x0183dc thru 0x01cc49 (0x486e bytes) 000730 1761285321.757124 fallocate 0x02318d thru 0x025f7a (0x2dee bytes) 000731 1761285321.773250 MIRROR_RESYNC 2 mirrors 000732 1761285322.559545 read 0x007b7b thru 0x00cd2c (0x51b2 bytes) 000733 1761285322.569114 read 0x01d45c thru 0x027032 (0x9bd7 bytes) 000734 1761285322.584852 write 0x03c75b thru 0x03ffff (0x38a5 bytes) 000735 1761285322.617320 punch from 0x038c27 to 0x03ffff (0x73d9 bytes) 000736 1761285322.634130 mapwrite 0x004c7f thru 0x00dd9f (0x9121 bytes) 000738 1761285322.715151 trunc from 0x00b2f3 to 0x00e06d (0x2d7b bytes) 000739 1761285322.722827 trunc from 0x040000 to 0x0383ed (0x7c12 bytes) 000740 1761285322.746451 mapwrite 0x03b596 thru 0x03ffff (0x4a6a bytes) 000741 1761285322.853670 MIRROR_RESYNC 2 mirrors 000742 1761285323.572444 mapwrite 0x0276d9 thru 0x02ca69 (0x5391 bytes) 000743 1761285323.653756 fallocate 0x028cc1 thru 0x031681 (0x89c1 bytes) 000744 1761285323.661772 mapread 0x0002c6 thru 0x003ff9 (0x3d34 bytes) 000745 1761285323.677362 write 0x0020ef thru 0x003cf5 (0x1c07 bytes) 000746 1761285323.678338 punch from 0x02d5fe to 0x03c15c (0xeb5f bytes) 000747 1761285323.685355 punch from 0x008fbd to 0x012ad4 (0x9b18 bytes) 000748 1761285323.696983 trunc from 0x040000 to 0x01f879 (0x20786 bytes) 000749 1761285323.732231 MIRROR_EXTEND to 3 mirrors 000750 1761285324.506461 fallocate 0x018caf thru 0x0264e9 (0xd83b bytes) 000751 1761285324.536451 read 0x010305 thru 0x01de56 (0xdb52 bytes) 000752 1761285324.554294 punch from 0x00e9df to 0x01232e (0x3950 bytes) 000753 1761285324.566270 trunc from 0x0264ea to 0x030f54 (0xaa6b bytes) 000754 1761285324.608556 mapread 0x00d15f thru 0x010e98 (0x3d3a bytes) 000755 1761285324.637673 read 0x0254a2 thru 0x02cc64 (0x77c3 bytes) 000756 1761285324.648874 trunc from 0x030f55 to 0x00494f (0x2c605 bytes) 000757 1761285324.679879 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000758 1761285324.909965 punch from 0x00052a to 0x00494f (0x4426 bytes) 000759 1761285324.917584 trunc from 0x004950 to 0x015b54 (0x11205 bytes) 000760 1761285324.941516 punch from 0x0013ce to 0x00fb3b (0xe76e bytes) 000761 1761285324.949806 trunc from 0x00a500 to 0x015b54 (0xb655 bytes) 000762 1761285324.955569 trunc from 0x015b55 to 0x00a9d9 (0xb17b bytes) 000763 1761285324.989236 mapwrite 0x014be4 thru 0x020359 (0xb776 bytes) 000764 1761285325.083282 mapwrite 0x019d65 thru 0x021fcc (0x8268 bytes) 000765 1761285325.150271 mapwrite 0x0022a3 thru 0x00fd24 (0xda82 bytes) 000766 1761285325.193328 trunc from 0x021fcd to 0x015a41 (0xc58b bytes) 000767 1761285325.218455 MIRROR_RESYNC 2 mirrors 000768 1761285325.953153 write 0x03e2a3 thru 0x03ffff (0x1d5d bytes) 000769 1761285325.998933 mapread 0x0233bb thru 0x031eca (0xeb10 bytes) 000770 1761285326.073860 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000771 1761285326.313672 mapwrite 0x01224c thru 0x01f17f (0xcf34 bytes) 000772 1761285326.385398 trunc from 0x02c4de to 0x036f2c (0xaa4f bytes) 000774 1761285326.392729 trunc from 0x040000 to 0x0091a6 (0x36e59 bytes) 000775 1761285326.423142 write 0x038c9a thru 0x03ffff (0x7366 bytes) 000776 1761285326.424542 fallocate 0x028b4d thru 0x0306d7 (0x7b8b bytes) 000777 1761285326.429707 punch from 0x0354db to 0x03e87a (0x93a0 bytes) 000778 1761285326.442703 trunc from 0x040000 to 0x0011e5 (0x3ee1a bytes) 000779 1761285326.460351 trunc from 0x000159 to 0x0011e5 (0x108d bytes) 000780 1761285326.469827 write 0x02c83e thru 0x03bedf (0xf6a2 bytes) 000781 1761285326.472664 fallocate 0x0188c5 thru 0x018b5a (0x296 bytes) 000782 1761285326.485950 punch from 0x00d5b6 to 0x016aff (0x954a bytes) 000783 1761285326.502229 MIRROR_EXTEND to 2 mirrors 000784 1761285327.133162 fallocate 0x032b93 thru 0x036f18 (0x4386 bytes) 000785 1761285327.166096 mapwrite 0x00064f thru 0x0070ff (0x6ab1 bytes) 000786 1761285327.229447 punch from 0x00b5c8 to 0x012b4b (0x7584 bytes) 000787 1761285327.243146 MIRROR_EXTEND to 3 mirrors 000788 1761285327.870557 fallocate 0x0294d8 thru 0x029db4 (0x8dd bytes) 000789 1761285327.906204 mapread 0x03b9b1 thru 0x03bedf (0x52f bytes) 000791 1761285327.919189 MIRROR_RESYNC 3 mirrors 000792 1761285328.621602 mapread 0x00cf2f thru 0x011e40 (0x4f12 bytes) 000793 1761285328.663740 mapwrite 0x02bc23 thru 0x02fed6 (0x42b4 bytes) 000794 1761285328.821019 mapwrite 0x026eb3 thru 0x03340e (0xc55c bytes) 000796 1761285328.849398 fallocate 0x031d0b thru 0x03fe26 (0xe11c bytes) 000797 1761285328.859643 trunc from 0x0384be to 0x03bedf (0x3a22 bytes) 000798 1761285328.865440 fallocate 0x020f52 thru 0x02e4d9 (0xd588 bytes) 000799 1761285328.878997 write 0x020f4f thru 0x027c09 (0x6cbb bytes) 000801 1761285328.896905 read 0x02f8d7 thru 0x037e8a (0x85b4 bytes) 000804 1761285328.908588 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000805 1761285329.170864 trunc from 0x020ebf to 0x026640 (0x5782 bytes) 000806 1761285329.182168 write 0x005f60 thru 0x00f0a9 (0x914a bytes) 000808 1761285329.198207 trunc from 0x03bee0 to 0x037dd1 (0x410e bytes) 000809 1761285329.213715 fallocate 0x03b451 thru 0x03c161 (0xd11 bytes) 000810 1761285329.222715 read 0x007576 thru 0x010d78 (0x9803 bytes) 000812 1761285329.228156 read 0x02cebb thru 0x037dd1 (0xaf17 bytes) 000814 1761285329.238970 punch from 0x025c77 to 0x0353c0 (0xf74a bytes) 000815 1761285329.245215 read 0x02c30d thru 0x037dd1 (0xbac5 bytes) 000816 1761285329.263027 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000817 1761285329.472094 fallocate 0x0069e2 thru 0x00f928 (0x8f47 bytes) 000818 1761285329.480382 fallocate 0x01eb91 thru 0x0294f1 (0xa961 bytes) 000819 1761285329.486957 mapwrite 0x0206d5 thru 0x02a043 (0x996f bytes) 000820 1761285329.534852 fallocate 0x01ac9b thru 0x01bae6 (0xe4c bytes) 000822 1761285329.541808 trunc from 0x037dd2 to 0x01a90e (0x1d4c3 bytes) 000823 1761285329.571367 trunc from 0x01a90f to 0x00999d (0x10f71 bytes) 000824 1761285329.590326 write 0x016e95 thru 0x0261da (0xf346 bytes) 000825 1761285329.593864 fallocate 0x03cc1a thru 0x03ffff (0x33e6 bytes) 000826 1761285329.604525 fallocate 0x015b44 thru 0x01d1b5 (0x7672 bytes) 000827 1761285329.609392 mapwrite 0x03ef9c thru 0x03ffff (0x1064 bytes) 000829 1761285329.674058 trunc from 0x02e2e1 to 0x03a996 (0xc6b6 bytes) 000830 1761285329.684873 read 0x019ef4 thru 0x0261af (0xc2bc bytes) 000831 1761285329.688735 trunc from 0x026b96 to 0x028dff (0x226a bytes) 000832 1761285329.702875 punch from 0x015562 to 0x016aaa (0x1549 bytes) 000833 1761285329.714696 fallocate 0x011d14 thru 0x019f4e (0x823b bytes) 000834 1761285329.724954 trunc from 0x040000 to 0x01b226 (0x24dd9 bytes) 000835 1761285329.756370 punch from 0x006a19 to 0x016072 (0xf65a bytes) 000838 1761285329.765695 punch from 0x010a3a to 0x01b226 (0xa7ed bytes) 000839 1761285329.779923 trunc from 0x01acca to 0x01b226 (0x55d bytes) 000840 1761285329.790409 fallocate 0x007ec7 thru 0x00ff79 (0x80b3 bytes) 000841 1761285329.797030 trunc from 0x01b227 to 0x026546 (0xb320 bytes) 000843 1761285329.817077 write 0x018d9f thru 0x02725d (0xe4bf bytes) 000845 1761285329.831107 fallocate 0x020289 thru 0x0283cb (0x8143 bytes) 000846 1761285329.835691 read 0x00a3b4 thru 0x0173cb (0xd018 bytes) 000847 1761285329.844835 trunc from 0x009e2e to 0x00b22d (0x1400 bytes) 000848 1761285329.849719 read 0x020d0c thru 0x024c42 (0x3f37 bytes) 000849 1761285329.860280 MIRROR_EXTEND to 2 mirrors 000850 1761285330.706873 trunc from 0x025dcd to 0x02725d (0x1491 bytes) 000851 1761285330.750406 mapwrite 0x014b0e thru 0x015984 (0xe77 bytes) 000852 1761285330.775906 mapwrite 0x02c92d thru 0x034d56 (0x842a bytes) 000853 1761285330.828585 trunc from 0x034d57 to 0x033083 (0x1cd3 bytes) 000854 1761285330.846671 write 0x033ec3 thru 0x037b53 (0x3c91 bytes) 000855 1761285330.847957 mapread 0x00eb96 thru 0x011b63 (0x2fce bytes) 000856 1761285330.860453 punch from 0x02f7be to 0x03471d (0x4f60 bytes) 000857 1761285330.873837 read 0x02f902 thru 0x037b53 (0x8252 bytes) 000858 1761285330.884458 mapwrite 0x03448a thru 0x03ffff (0xbb76 bytes) 000859 1761285330.939109 mapwrite 0x03a634 thru 0x03ffff (0x59cc bytes) 000860 1761285330.964142 trunc from 0x040000 to 0x02517b (0x1ae84 bytes) 000861 1761285330.991873 mapread 0x01df69 thru 0x02517b (0x7213 bytes) 000862 1761285331.008643 punch from 0x000e94 to 0x00400a (0x3177 bytes) 000863 1761285331.014793 write 0x009012 thru 0x010b37 (0x7b26 bytes) 000864 1761285331.019738 write 0x01aafc thru 0x01c121 (0x1626 bytes) 000865 1761285331.024998 mapread 0x00cef2 thru 0x00e167 (0x1276 bytes) 000866 1761285331.032040 mapread 0x01de41 thru 0x01dfd5 (0x195 bytes) 000867 1761285331.033968 fallocate 0x0323c9 thru 0x034741 (0x2379 bytes) 000868 1761285331.042094 fallocate 0x025eab thru 0x031252 (0xb3a8 bytes) 000869 1761285331.055850 punch from 0x022e91 to 0x0254c4 (0x2634 bytes) 000870 1761285331.073927 mapwrite 0x024ed5 thru 0x027d9c (0x2ec8 bytes) 000871 1761285331.107234 punch from 0x01eb85 to 0x02a16e (0xb5ea bytes) 000872 1761285331.118956 trunc from 0x034742 to 0x03d4ef (0x8dae bytes) 000873 1761285331.141643 mapwrite 0x0093ac thru 0x0160ea (0xcd3f bytes) 000874 1761285331.164905 mapread 0x00418d thru 0x004fcb (0xe3f bytes) 000875 1761285331.179365 fallocate 0x003961 thru 0x007076 (0x3716 bytes) 000876 1761285331.187945 trunc from 0x03d4f0 to 0x00d5f5 (0x2fefa bytes) 000877 1761285331.203210 mapwrite 0x00307d thru 0x006a00 (0x3984 bytes) 000878 1761285331.234765 mapwrite 0x013d9b thru 0x0219c9 (0xdc2f bytes) 000879 1761285331.306238 mapread 0x0193b6 thru 0x01a26e (0xeb9 bytes) 000880 1761285331.307017 read 0x007a8d thru 0x014879 (0xcded bytes) 000881 1761285331.307603 read 0x01266a thru 0x01d8d9 (0xb270 bytes) 000882 1761285331.308121 read 0x01cbc1 thru 0x0219c9 (0x4e09 bytes) 000883 1761285331.312182 fallocate 0x03d7ee thru 0x03fbb7 (0x23ca bytes) 000885 1761285331.321838 trunc from 0x00ab71 to 0x015ffa (0xb48a bytes) 000886 1761285331.334132 read 0x03dfae thru 0x03fbb7 (0x1c0a bytes) 000887 1761285331.353967 trunc from 0x03fbb8 to 0x015c66 (0x29f51 bytes) 000888 1761285331.387248 trunc from 0x006047 to 0x015407 (0xf3c1 bytes) 000890 1761285331.393320 read 0x014584 thru 0x015c66 (0x16e3 bytes) 000892 1761285331.431453 trunc from 0x015c67 to 0x00f926 (0x6340 bytes) 000893 1761285331.463408 mapread 0x00322d thru 0x00c860 (0x9634 bytes) 000895 1761285331.465429 write 0x02243d thru 0x02ce6f (0xaa33 bytes) 000896 1761285331.467027 trunc from 0x02ce70 to 0x024297 (0x8bd8 bytes) 000897 1761285331.487431 read 0x003643 thru 0x012604 (0xefc2 bytes) 000898 1761285331.506319 mapwrite 0x00e6e1 thru 0x010ca5 (0x25c5 bytes) 000899 1761285331.534287 read 0x01d821 thru 0x024297 (0x6a77 bytes) 000900 1761285331.535006 write 0x00b796 thru 0x00e252 (0x2abd bytes) 000902 1761285331.536361 trunc from 0x01ebde to 0x024297 (0x56ba bytes) 000903 1761285331.544987 fallocate 0x007923 thru 0x011bc3 (0xa2a1 bytes) 000904 1761285331.561832 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x51:0x0]: Invalid argument (22) 000905 1761285331.690886 fallocate 0x019a9d thru 0x01eb5c (0x50c0 bytes) 000906 1761285331.696114 mapread 0x005db7 thru 0x011321 (0xb56b bytes) 000907 1761285331.729102 fallocate 0x032722 thru 0x03dcf4 (0xb5d3 bytes) 000908 1761285331.733514 trunc from 0x0209d1 to 0x02a23f (0x986f bytes) 000909 1761285331.739218 read 0x02b8fa thru 0x037712 (0xbe19 bytes) 000910 1761285331.748421 mapread 0x02a6f7 thru 0x02b910 (0x121a bytes) 000911 1761285331.757241 mapread 0x0114ae thru 0x017c73 (0x67c6 bytes) 000914 1761285331.759211 fallocate 0x02be9c thru 0x02e3db (0x2540 bytes) 000917 1761285331.769154 trunc from 0x03dcf5 to 0x005174 (0x38b80 bytes) 000918 1761285331.792891 trunc from 0x005175 to 0x008275 (0x3101 bytes) 000919 1761285331.820884 punch from 0x004d50 to 0x008275 (0x3526 bytes) 000920 1761285331.827096 punch from 0x003991 to 0x007fff (0x466f bytes) 000921 1761285331.836461 mapwrite 0x010826 thru 0x0204dd (0xfcb8 bytes) 000922 1761285331.886967 read 0x0094e3 thru 0x0167c6 (0xd2e4 bytes) 000923 1761285331.887699 write 0x03ddfe thru 0x03ffff (0x2202 bytes) 000924 1761285331.888739 read 0x03472d thru 0x034967 (0x23b bytes) 000928 1761285331.905646 mapwrite 0x03f423 thru 0x03ffff (0xbdd bytes) 000929 1761285331.918407 mapread 0x03b2e9 thru 0x03d767 (0x247f bytes) 000931 1761285331.920287 mapwrite 0x017376 thru 0x021583 (0xa20e bytes) 000932 1761285331.952776 write 0x036550 thru 0x0376bb (0x116c bytes) 000933 1761285331.955058 trunc from 0x040000 to 0x0267d6 (0x19829 bytes) 000934 1761285331.974156 punch from 0x0036a1 to 0x0108f3 (0xd253 bytes) 000935 1761285331.983258 punch from 0x00cede to 0x0156a1 (0x87c4 bytes) 000936 1761285331.990146 trunc from 0x0267d7 to 0x016629 (0x101ad bytes) 000937 1761285332.006984 mapread 0x00a383 thru 0x016562 (0xc1e0 bytes) 000938 1761285332.067722 write 0x006aeb thru 0x00e38f (0x78a5 bytes) 000940 1761285332.078852 MIRROR_EXTEND to 2 mirrors 000941 1761285332.770965 punch from 0x00a810 to 0x011ec5 (0x76b6 bytes) 000942 1761285332.812374 mapread 0x008dd3 thru 0x00a27e (0x14ac bytes) 000943 1761285332.842280 mapread 0x00d656 thru 0x016629 (0x8fd4 bytes) 000944 1761285332.856794 MIRROR_RESYNC 2 mirrors 000945 1761285333.516520 mapread 0x00c86c thru 0x016629 (0x9dbe bytes) 000946 1761285333.533040 write 0x02e62a thru 0x03c12c (0xdb03 bytes) 000947 1761285333.556126 write 0x01f589 thru 0x02bb31 (0xc5a9 bytes) 000948 1761285333.567706 mapread 0x003355 thru 0x0100d2 (0xcd7e bytes) 000949 1761285333.590138 read 0x037e98 thru 0x038d64 (0xecd bytes) 000950 1761285333.597740 MIRROR_RESYNC 2 mirrors 000951 1761285334.470274 read 0x01bdee thru 0x02476f (0x8982 bytes) 000952 1761285334.489831 write 0x02d4d3 thru 0x03c492 (0xefc0 bytes) 000953 1761285334.537702 fallocate 0x013e11 thru 0x020956 (0xcb46 bytes) 000954 1761285334.548175 mapread 0x00d6aa thru 0x01b494 (0xddeb bytes) 000955 1761285334.563144 mapwrite 0x0202df thru 0x0261f4 (0x5f16 bytes) 000956 1761285334.587404 fallocate 0x011fe9 thru 0x01f4ab (0xd4c3 bytes) 000957 1761285334.593457 fallocate 0x022c7e thru 0x025a01 (0x2d84 bytes) 000958 1761285334.599343 mapread 0x01a04b thru 0x028ca2 (0xec58 bytes) 000959 1761285334.600438 write 0x0160c2 thru 0x01d226 (0x7165 bytes) 000960 1761285334.602283 mapread 0x017691 thru 0x023a0e (0xc37e bytes) 000961 1761285334.603161 trunc from 0x002612 to 0x00ed51 (0xc740 bytes) 000962 1761285334.608046 mapwrite 0x03be95 thru 0x03ffff (0x416b bytes) 000963 1761285334.684813 mapwrite 0x025cf1 thru 0x02d24b (0x755b bytes) 000964 1761285334.714052 read 0x0111b6 thru 0x014dfd (0x3c48 bytes) 000965 1761285334.714668 mapwrite 0x022a45 thru 0x02e27b (0xb837 bytes) 000966 1761285334.738276 read 0x025ca8 thru 0x02642c (0x785 bytes) 000967 1761285334.739785 fallocate 0x0094c5 thru 0x012f64 (0x9aa0 bytes) 000968 1761285334.748177 write 0x03f743 thru 0x03fa16 (0x2d4 bytes) 000969 1761285334.749037 read 0x02ab4a thru 0x0348da (0x9d91 bytes) 000970 1761285334.749627 read 0x034de1 thru 0x03d9a3 (0x8bc3 bytes) 000971 1761285334.750142 write 0x01031e thru 0x018ceb (0x89ce bytes) 000972 1761285334.751890 trunc from 0x040000 to 0x011bc8 (0x2e437 bytes) 000973 1761285334.773647 trunc from 0x001b13 to 0x006558 (0x4a46 bytes) 000974 1761285334.778900 mapwrite 0x0356ef thru 0x03ffff (0xa911 bytes) 000975 1761285334.831253 write 0x03e9f0 thru 0x03ffff (0x1610 bytes) 000976 1761285334.833702 mapread 0x01f26f thru 0x0202ac (0x103e bytes) 000977 1761285334.835446 read 0x030e38 thru 0x033996 (0x2b5f bytes) 000978 1761285334.841847 MIRROR_RESYNC 2 mirrors 000979 1761285335.429587 mapwrite 0x03b515 thru 0x03d6dd (0x21c9 bytes) 000980 1761285335.512304 trunc from 0x02a1ae to 0x02c69b (0x24ee bytes) 000981 1761285335.518849 trunc from 0x040000 to 0x016a4f (0x295b0 bytes) 000982 1761285335.546525 MIRROR_EXTEND to 3 mirrors 000983 1761285336.106590 mapread 0x00212b thru 0x00ad5b (0x8c31 bytes) 000984 1761285336.174907 trunc from 0x000479 to 0x003766 (0x32ee bytes) 000986 1761285336.250716 mapwrite 0x008626 thru 0x01782e (0xf209 bytes) 000987 1761285336.323969 punch from 0x007387 to 0x008188 (0xe02 bytes) 000988 1761285336.329355 trunc from 0x005405 to 0x008f2a (0x3b26 bytes) 000989 1761285336.336293 fallocate 0x036b28 thru 0x03aeee (0x43c7 bytes) 000990 1761285336.342262 punch from 0x0365cf to 0x03aeee (0x4920 bytes) 000991 1761285336.346921 read 0x02a4d0 thru 0x0337c5 (0x92f6 bytes) 000992 1761285336.358250 MIRROR_RESYNC 3 mirrors 000993 1761285337.022262 trunc from 0x03aeef to 0x03fcc5 (0x4dd7 bytes) 000994 1761285337.111340 trunc from 0x03fcc6 to 0x004cf4 (0x3afd1 bytes) 000995 1761285337.147726 MIRROR_RESYNC 3 mirrors 000996 1761285337.752206 punch from 0x004713 to 0x004cf4 (0x5e2 bytes) 000997 1761285337.785066 punch from 0x0019a3 to 0x004cf4 (0x3352 bytes) 000998 1761285337.802148 MIRROR_RESYNC 3 mirrors 000999 1761285338.523459 punch from 0x000db8 to 0x004cf4 (0x3f3d bytes) 001000 1761285338.569010 trunc from 0x004cf5 to 0x025c47 (0x20f53 bytes) All operations completed A-OK! PASS 70a (77s) == sanity-flr test 71: check mirror extend parent fid ==== 01:55:43 (1761285343) striped dir -i1 -c2 -H crush2 /mnt/lustre/d71.sanity-flr ** lfs fid2path /mnt/lustre [0x240000404:0x230:0x0] /mnt/lustre/d71.sanity-flr/f71.sanity-flr PASS 71 (9s) == sanity-flr test 200a: stress test ===================== 01:55:51 (1761285351) Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre2 Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre3 fail_loc=0x1A03 fail_loc=0x1A03 Extending file size to 2860320 .. lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..Extending file size to 4602144 .. Extending file size to 8990176 .. Extending file size to 7448768 .. Extending file size to 8510976 .. Extending file size to 8985728 .. failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed Extending file size to 9002880 .. resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..Extending file size to 9150752 .. done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..Waiting 59473 59474 59475 59478 59480 failed 192.168.201.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg110-client.virtnet /mnt/lustre2 (opts:) 192.168.201.110@tcp:/lustre /mnt/lustre3 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg110-client.virtnet /mnt/lustre3 (opts:) 3 PASS 200a (75s) == sanity-flr test 200b: racing IO, mirror extend and resync ========================================================== 01:57:06 (1761285426) Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre2 Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre3 fail_loc=0x1423 resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..Extending file size to 6185152 .. Extending file size to 1702784 .. failed Extending file size to 4279424 .. Extending file size to 7192384 .. resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8136096 .. Extending file size to 8383936 .. failed Extending file size to 5595168 .. resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8550848 .. done lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8702432 .. done lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done Extending file size to 9289920 .. resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed Extending file size to 8649760 .. resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8734848 .. done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 9385600 .. done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed Waiting 61798 61799 61800 61801 61804 192.168.201.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg110-client.virtnet /mnt/lustre2 (opts:) 192.168.201.110@tcp:/lustre /mnt/lustre3 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg110-client.virtnet /mnt/lustre3 (opts:) 3 PASS 200b (71s) == sanity-flr test 200c: layout change racing with open: LOVEA changes ========================================================== 01:58:17 (1761285497) Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2504 1268800 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 2160 1269144 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 23036 3583984 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 13976 3593044 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 37012 7177028 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2504 1268800 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1414116 2160 1269144 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3833116 23036 3583984 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3833116 13976 3593044 1% /mnt/lustre2[OST:1] filesystem_summary: 7666232 37012 7177028 1% /mnt/lustre2 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0705112 s, 29.7 MB/s fail_loc=0x80000175 fail_val=10 mirror extend mirror extend done fail_loc=0 fail_val=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.10787 s, 19.4 MB/s dd completed Chunks to be verified in /mnt/lustre/f200c.sanity-flr: [0, 0xffffffffffffffff) [1] 1 lfs verify: '/mnt/lustre/f200c.sanity-flr' chunk [0, 0xffffffffffffffff) is only valid in mirror 1: skipped 192.168.201.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg110-client.virtnet /mnt/lustre2 (opts:) PASS 200c (10s) SKIP: sanity-flr test_201 skipping ALWAYS excluded test 201 == sanity-flr test 202: lfs setstripe --add-component wide striping ========================================================== 01:58:28 (1761285508) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0836924 s, 25.1 MB/s PASS 202 (5s) == sanity-flr test 203: mirror file preserve mirror ID === 01:58:33 (1761285513) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x9a:0x0]: Invalid argument (22) /mnt/lustre/f203.sanity-flr lcm_layout_gen: 3 lcm_mirror_count: 1 lcm_entry_count: 1 lcme_id: 131074 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xae:0x0] } /mnt/lustre/f203.sanity-flr lcm_layout_gen: 4 lcm_mirror_count: 2 lcm_entry_count: 2 lcme_id: 131074 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xae:0x0] } lcme_id: 196609 lcme_mirror_id: 3 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xaf:0x0] } - 1: { l_ost_idx: 0, l_fid: [0x280000401:0xca:0x0] } PASS 203 (4s) == sanity-flr test 204a: FLR write/stale/resync tests with self-extending mirror ========================================================== 01:58:37 (1761285517) striped dir -i0 -c2 -H all_char /mnt/lustre/d204a.sanity-flr 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.046668 s, 44.9 MB/s /mnt/lustre/d204a.sanity-flr/f204a.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb0:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xcb:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 1048576 lcme_extent.e_end: 68157440 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xcc:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 68157440 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 PASS 204a (3s) == sanity-flr test 204b: FLR write/stale/resync tests with self-extending primary ========================================================== 01:58:40 (1761285520) striped dir -i0 -c2 -H crush2 /mnt/lustre/d204b.sanity-flr 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0691948 s, 30.3 MB/s /mnt/lustre/d204b.sanity-flr/f204b.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb1:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: 68157440 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb2:0x0] } lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 68157440 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xcd:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 /mnt/lustre/d204b.sanity-flr/f204b.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb1:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: 68157440 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb2:0x0] } lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 68157440 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xcd:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xce:0x0] } PASS 204b (4s) == sanity-flr test 204c: FLR write/stale/resync test with component removal ========================================================== 01:58:44 (1761285524) striped dir -i0 -c2 -H crush /mnt/lustre/d204c.sanity-flr 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.087169 s, 24.1 MB/s /mnt/lustre/d204c.sanity-flr/f204c.sanity-flr lcm_layout_gen: 7 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb3:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xcf:0x0] } lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 /mnt/lustre/d204c.sanity-flr/f204c.sanity-flr lcm_layout_gen: 9 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb3:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xcf:0x0] } lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xd0:0x0] } PASS 204c (29s) == sanity-flr test 204d: FLR write/stale/resync sel test with repeated comp ========================================================== 01:59:13 (1761285553) Waiting for MDT destroys to complete Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg110-server mds-ost sync done. striped dir -i0 -c2 -H all_char /mnt/lustre/d204d.sanity-flr 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0592376 s, 17.7 MB/s /mnt/lustre/d204d.sanity-flr/f204d.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2fb:0x0] } lcme_id: 65541 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x2fc:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x2fb:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 /mnt/lustre/d204d.sanity-flr/f204d.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2fb:0x0] } lcme_id: 65541 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x2fc:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x2fb:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2fc:0x0] } PASS 204d (45s) == sanity-flr test 204e: FLR write/stale/resync sel test with repeated comp ========================================================== 01:59:58 (1761285598) Waiting for MDT destroys to complete Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg110-server mds-ost sync done. striped dir -i0 -c2 -H all_char /mnt/lustre/d204e.sanity-flr /mnt/lustre/d204e.sanity-flr/f204e.sanity-flr lcm_layout_gen: 4 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xd1:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xd2:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 67108864 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0819549 s, 25.6 MB/s /mnt/lustre/d204e.sanity-flr/f204e.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xd1:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xd2:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb4:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 /mnt/lustre/d204e.sanity-flr/f204e.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xd1:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xd2:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb4:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 PASS 204e (46s) == sanity-flr test 204f: FLR write/stale/resync sel w/forced extension ========================================================== 02:00:44 (1761285644) Waiting for MDT destroys to complete Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg110-server mds-ost sync done. striped dir -i0 -c2 -H all_char /mnt/lustre/d204f.sanity-flr Creating new pool test_204f oleg110-server: Pool lustre.test_204f created Adding targets to pool oleg110-server: OST lustre-OST0000_UUID added to pool lustre.test_204f oleg110-server: OST lustre-OST0001_UUID added to pool lustre.test_204f 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0813935 s, 25.8 MB/s /mnt/lustre/d204f.sanity-flr/f204f.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x2fd:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_pool: test_204f lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2fd:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lmm_pool: test_204f /mnt/lustre/d204f.sanity-flr/f204f.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x2fd:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_pool: test_204f lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2fd:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lmm_pool: test_204f Destroy the created pools: test_204f lustre.test_204f oleg110-server: OST lustre-OST0000_UUID removed from pool lustre.test_204f oleg110-server: OST lustre-OST0001_UUID removed from pool lustre.test_204f oleg110-server: Pool lustre.test_204f destroyed PASS 204f (61s) == sanity-flr test 205a: lfs mirror extend to set prefer flag ========================================================== 02:01:45 (1761285705) PASS 205a (7s) == sanity-flr test 205b: lfs mirror extend to set nocompr flag ========================================================== 02:01:52 (1761285712) PASS 205b (6s) == sanity-flr test 206: lfs setstripe -pool .. --comp-flags=.. ========================================================== 02:01:58 (1761285718) oleg110-server: Pool lustre.test_206 created Adding targets to pool oleg110-server: OST lustre-OST0000_UUID added to pool lustre.test_206 oleg110-server: OST lustre-OST0001_UUID added to pool lustre.test_206 Destroy the created pools: test_206 lustre.test_206 oleg110-server: OST lustre-OST0000_UUID removed from pool lustre.test_206 oleg110-server: OST lustre-OST0001_UUID removed from pool lustre.test_206 oleg110-server: Pool lustre.test_206 destroyed PASS 206 (26s) == sanity-flr test 207: create another replica with existing out-of-sync one ========================================================== 02:02:24 (1761285744) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0828846 s, 12.7 MB/s 2 mirror IDs: 1 2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0437378 s, 24.0 MB/s 2 mirror IDs: 1 2 2 mirror IDs: 1 2 3 mirror IDs: 1 2 3 CRC-32 checksum value for chunk [0, 0x100000): Mirror 1: 0x13a3a385 Mirror 3: 0x13a3a385 PASS 207 (8s) == sanity-flr test 208a: mirror selection to prefer non-rotational devices for reads ========================================================== 02:02:32 (1761285752) SKIP: sanity-flr test_208a needs >= 4 OSTs SKIP 208a (3s) == sanity-flr test 208b: mirror selection to prefer non-rotational devices for writes ========================================================== 02:02:35 (1761285755) SKIP: sanity-flr test_208b needs >= 4 OSTs SKIP 208b (2s) == sanity-flr test 209a: skip fast reads after layout invalidation ========================================================== 02:02:38 (1761285758) Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre2 osd-ldiskfs.lustre-OST0000.nonrotational=0 osd-ldiskfs.lustre-OST0001.nonrotational=0 osd-ldiskfs.lustre-OST0001.nonrotational=1 replicated file created open(O_RDONLY) and first read from OST first read complete 4+0 records in 4+0 records out 4 bytes copied, 0.0471779 s, 0.1 kB/s BBBB written which made replica on ost1 stale fast read from pagecache in the original process read via /mnt/lustre2 new open(2) Buffer address : 0x22b0000 Read this (4 bytes): BBBB fast read from pagecache after 5s in the original process Buffer address : 0x1fe0000 Read this (4 bytes): AAAA PAUSING Buffer address : 0x1fe0000 Read this (4 bytes): BBBB PAUSING Buffer address : 0x1fe0000 Read this (4 bytes): BBBB read via new open(2) Buffer address : 0x1570000 Read this (4 bytes): BBBB 192.168.201.110@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg110-client.virtnet /mnt/lustre2 (opts:) PASS 209a (30s) == sanity-flr test 209b: pagecache can be used after LL cancellation ========================================================== 02:03:08 (1761285788) 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0157686 s, 520 kB/s the very first read read with warm cache PASS 209b (6s) == sanity-flr test 210a: handle broken mirrored lovea ==== 02:03:14 (1761285794) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408764 s, 25.7 MB/s fail_loc=0x1428 /mnt/lustre/f210a.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 240 lcm_flags: ro lcm_layout_gen: 1 lcm_mirror_count: 2 lcm_entry_count: 2 components: - lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lcme_offset: 128 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0xbb lmm_fid: [0x200000405:0xbb:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lmm_objects: - 0: { l_ost_idx: -1, l_fid: [0:0x0:0x0] } - lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lcme_offset: 184 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0xbb lmm_fid: [0x200000405:0xbb:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xdc:0x0] } File: /mnt/lustre/f210a.sanity-flr Size: 1048576 Blocks: 4096 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205339611323 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-24 02:03:15.000000000 -0400 Modify: 2025-10-24 02:03:15.000000000 -0400 Change: 2025-10-24 02:03:16.000000000 -0400 Birth: 2025-10-24 02:03:15.000000000 -0400 PASS 210a (6s) == sanity-flr test 210b: handle broken mirrored lovea (unlink) ========================================================== 02:03:21 (1761285801) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489282 s, 21.4 MB/s oleg110-server: debugfs 1.47.1-wc2 (08-Nov-2024) fail_loc=0x1428 Waiting for MDT destroys to complete oleg110-server: debugfs 1.47.1-wc2 (08-Nov-2024) oleg110-server: O/280000401/d29/dd: File not found by ext2_lookup PASS 210b (16s) == sanity-flr test 211: mirror delete should not cause bad size ========================================================== 02:03:38 (1761285818) 10+0 records in 10+0 records out 40960 bytes (41 kB, 40 KiB) copied, 0.145897 s, 281 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0138194 s, 296 kB/s size after second write -rw-r--r-- 1 root root 4096 Oct 24 02:03 /mnt/lustre/f211.sanity-flr lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0xc0:0x0]: Invalid argument (22) size after mirror delete -rw-r--r-- 1 root root 4096 Oct 24 02:03 /mnt/lustre/f211.sanity-flr PASS 211 (6s) == sanity-flr test complete, duration 1308 sec =========== 02:03:43 (1761285823) === sanity-flr: start cleanup 02:03:44 (1761285824) === === sanity-flr: finish cleanup 02:03:49 (1761285829) ===