-----============= acceptance-small: sanity-flr ============----- Thu Oct 23 14:47:49 EDT 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg315-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanity-flr.*ex': No such file or directory excepting tests: 6 201 44c === sanity-flr: start setup 14:48:07 (1761245287) === oleg315-client.virtnet: executing check_config_client /mnt/lustre oleg315-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg315-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff96748692b800.idle_timeout=debug osc.lustre-OST0001-osc-ffff96748692b800.idle_timeout=debug disable quota as required oleg315-server: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 === sanity-flr: finish setup 14:48:43 (1761245323) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f8220] == sanity-flr test 0a: lfs mirror create with -N option == 14:48:46 (1761245326) PASS 0a (18s) == sanity-flr test 0b: lfs mirror create plain layout mirrors ========================================================== 14:49:04 (1761245344) SKIP: sanity-flr test_0b need >= 4 OSTs SKIP 0b (3s) == sanity-flr test 0c: lfs mirror create composite layout mirrors ========================================================== 14:49:07 (1761245347) SKIP: sanity-flr test_0c need >= 4 OSTs SKIP 0c (4s) == sanity-flr test 0d: lfs mirror extend with -N option == 14:49:11 (1761245351) PASS 0d (22s) == sanity-flr test 0e: lfs mirror extend plain layout mirrors ========================================================== 14:49:33 (1761245373) SKIP: sanity-flr test_0e need >= 4 OSTs SKIP 0e (3s) == sanity-flr test 0f: lfs mirror extend composite layout mirrors ========================================================== 14:49:36 (1761245376) SKIP: sanity-flr test_0f need >= 4 OSTs SKIP 0f (3s) == sanity-flr test 0g: lfs mirror create flags support === 14:49:39 (1761245379) error: lfs create: --flags must be specified with --mirror-count|-N option Create a mirrored file. usage: lfs mirror create --mirror-count|-N[MIRROR_COUNT] [SETSTRIPE_OPTIONS] ... FILENAME|DIRECTORY ... lfs commands used to manage files with mirrored components: lfs mirror create - create a mirrored file or directory lfs mirror extend - add mirror(s) to an existing file lfs mirror split - split a mirror from an existing mirrored file lfs mirror resync - resynchronize out-of-sync mirrored file(s) lfs mirror read - read a mirror content of a mirrored file lfs mirror write - write to a mirror of a mirrored file lfs mirror copy - copy a mirror to other mirror(s) of a file lfs mirror verify - verify mirrored file(s) lfs create: component flag 'foo' not supported Create a mirrored file. usage: lfs mirror create --mirror-count|-N[MIRROR_COUNT] [SETSTRIPE_OPTIONS] ... FILENAME|DIRECTORY ... lfs commands used to manage files with mirrored components: lfs mirror create - create a mirrored file or directory lfs mirror extend - add mirror(s) to an existing file lfs mirror split - split a mirror from an existing mirrored file lfs mirror resync - resynchronize out-of-sync mirrored file(s) lfs mirror read - read a mirror content of a mirrored file lfs mirror write - write to a mirror of a mirrored file lfs mirror copy - copy a mirror to other mirror(s) of a file lfs mirror verify - verify mirrored file(s) lfs create: unsupported mirror flags: stale lfs create: unsupported mirror flags: prefer,init lfs create: inverted flags are not supported Create a mirrored file. usage: lfs mirror create --mirror-count|-N[MIRROR_COUNT] [SETSTRIPE_OPTIONS] ... FILENAME|DIRECTORY ... lfs commands used to manage files with mirrored components: lfs mirror create - create a mirrored file or directory lfs mirror extend - add mirror(s) to an existing file lfs mirror split - split a mirror from an existing mirrored file lfs mirror resync - resynchronize out-of-sync mirrored file(s) lfs mirror read - read a mirror content of a mirrored file lfs mirror write - write to a mirror of a mirrored file lfs mirror copy - copy a mirror to other mirror(s) of a file lfs mirror verify - verify mirrored file(s) osc.lustre-OST0000-osc-ffff96748692b800.stats=clear osc.lustre-OST0001-osc-ffff96748692b800.stats=clear PASS 0g (8s) == sanity-flr test 0h: set, clear and test flags for FLR files ========================================================== 14:49:48 (1761245388) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d0h.sanity-flr lfs setstripe: component flag 'foo' not supported Create a file with specified striping/composite layout, or set the default layout on an existing directory. Usage: setstripe [--component-add|--component-del|--delete|-d] [--comp-set --comp-id|-I COMP_ID|--comp-flags=COMP_FLAGS] [--component-end|-E END_OFFSET] [--copy=SOURCE_LAYOUT_FILE]|--yaml|-y YAML_TEMPLATE_FILE] [--extension-size|--ext-size|-z EXT_SIZE] [--help|-h] [--foreign=FOREIGN_TYPE --xattr|-x LAYOUT] [--layout|-L PATTERN] [--mode FILE_MODE] [--mirror-count|-N[MIRROR_COUNT]] [--ost|-o OST_INDEX[,OST_INDEX,...]] [--overstripe-count|-C STRIPE_COUNT] [--pool|-p POOL_NAME] [--stripe-count|-c STRIPE_COUNT] [--stripe-index|-i START_OST_IDX] [--stripe-size|-S STRIPE_SIZE] FILENAME|DIRECTORY lfs: component flag 'foo' not supported error: getstripe bad component flags 'prefer,foo'. List the layout pattern for a given file or files in a directory or recursively for all files in a directory tree. Usage: getstripe [--ost|-O OST_NAME] [--quiet|-q] [--verbose|-v] [--stripe-count|-c] [--stripe-index|-i] [--fid|-F] [--pool|-p] [--stripe-size|-S] [--directory|-d] [--mdt-index|-m] [--recursive|-r] [--raw|-R] [--layout|-L] [--generation|-g] [--yaml|-y] [--help|-h] [--hex-idx] [--component-id|-I[=COMP_ID]] [--component-flags[=COMP_FLAGS]] [--component-count] [--extension-size|--ext-size|-z] [--component-start[=[+-]START_OFFSET]] [--component-end|-E[[+-]END_OFFSET]] [[!] --mirror-index=[+-]MIRROR_INDEX | [!] --mirror-id=[+-]MIRROR_ID] [--mirror-count|-N] [--no-follow] FILENAME|DIRECTORY lfs setstripe: cannot clear 'stale' flags from component. Please use lfs-mirror-resync(1) instead PASS 0h (11s) == sanity-flr test 0j: test lfs mirror read/write commands ========================================================== 14:49:59 (1761245399) PASS 0j (9s) == sanity-flr test 0k: mirroring a file in directory with default FLR layout ========================================================== 14:50:08 (1761245408) SKIP: sanity-flr test_0k need >= 3 OSTs SKIP 0k (3s) == sanity-flr test 1: create components with setstripe options ========================================================== 14:50:11 (1761245411) lfs mirror mirror: cannot get UNLOCK lease, ext 4 for [0x200000402:0x1f:0x0]: Numerical result out of range (34) error: lfs extend: /mnt/lustre/f1.sanity-flr: cannot merge layout: Numerical result out of range PASS 1 (9s) == sanity-flr test 2: create components from existing files ========================================================== 14:50:20 (1761245420) PASS 2 (6s) == sanity-flr test 3: create components from files located on different MDTs ========================================================== 14:50:26 (1761245426) PASS 3 (7s) == sanity-flr test 4: Make sure mirror attributes can be inhertied from directory ========================================================== 14:50:33 (1761245433) striped dir -i0 -c2 -H crush2 /mnt/lustre/d4.sanity-flr striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d4.sanity-flr/td error: lfs extend: /mnt/lustre/d4.sanity-flr: cannot get the last init comp: Is a directory PASS 4 (8s) == sanity-flr test 5: Make sure init size work for mirrored layout ========================================================== 14:50:41 (1761245441) /mnt/lustre/f5.sanity-flr has type file OK /mnt/lustre/f5.sanity-flr has size 12345 OK /mnt/lustre/f5.sanity-flr has type file OK /mnt/lustre/f5.sanity-flr has size 12345 OK PASS 5 (7s) SKIP: sanity-flr test_6a skipping ALWAYS excluded test 6a (base 6) SKIP: sanity-flr test_6b skipping ALWAYS excluded test 6b (base 6) == sanity-flr test 7: Create mirror with DoM component === 14:50:52 (1761245452) PASS 7 (7s) == sanity-flr test 21: glimpse should report accurate i_blocks ========================================================== 14:50:58 (1761245458) 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.19823 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051715 s, 20.3 MB/s PASS 21 (10s) == sanity-flr test 22: no glimpse to OSTs for READ_ONLY files ========================================================== 14:51:08 (1761245468) 14+0 records in 14+0 records out 14680064 bytes (15 MB, 14 MiB) copied, 0.331869 s, 44.2 MB/s PASS 22 (7s) == sanity-flr test 31: make sure glimpse request can be retried ========================================================== 14:51:15 (1761245475) Stopping /mnt/lustre-ost1 (opts:) on oleg315-server oleg315-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff96748692b800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff96748692b800.ost_server_uuid in DISCONN state after 0 sec Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg315-server: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg315-client: oleg315-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg315-server: oleg315-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg315-server: *.lustre-OST0000.recovery_status status: COMPLETE Stopping /mnt/lustre-ost2 (opts:) on oleg315-server oleg315-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0001-osc-ffff96748692b800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff96748692b800.ost_server_uuid in DISCONN state after 0 sec Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg315-server: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg315-client: oleg315-server: ssh exited with exit code 1 Started lustre-OST0001 affected facets: ost2 oleg315-server: oleg315-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg315-server: *.lustre-OST0001.recovery_status status: COMPLETE PASS 31 (66s) == sanity-flr test 32: data should be mirrored to newly created mirror ========================================================== 14:52:21 (1761245541) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.109118 s, 19.2 MB/s Stopping /mnt/lustre-ost1 (opts:) on oleg315-server oleg315-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff96748692b800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff96748692b800.ost_server_uuid in DISCONN state after 0 sec /mnt/lustre/f32.sanity-flr has type file OK /mnt/lustre/f32.sanity-flr has size 2097152 OK reading file from the 2nd mirror and verify checksum Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg315-server: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg315-client: oleg315-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg315-server: oleg315-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg315-server: *.lustre-OST0000.recovery_status status: COMPLETE PASS 32 (36s) == sanity-flr test 33a: read can choose available mirror to read ========================================================== 14:52:57 (1761245577) error: lfs extend: /mnt/lustre/f33a.sanity-flr: file busy or contents don't match: Invalid argument /mnt/lustre/f33a.sanity-flr has type file OK /mnt/lustre/f33a.sanity-flr has size 500 OK reading file (data can be provided by any ost)... Stopping /mnt/lustre-ost1 (opts:) on oleg315-server oleg315-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff96748692b800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff96748692b800.ost_server_uuid in DISCONN state after 0 sec reading file (data should be provided by ost2)... Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg315-server: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg315-client: oleg315-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg315-server: oleg315-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg315-server: *.lustre-OST0000.recovery_status status: COMPLETE Stopping /mnt/lustre-ost2 (opts:) on oleg315-server oleg315-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0001-osc-ffff96748692b800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff96748692b800.ost_server_uuid in DISCONN state after 0 sec /mnt/lustre/f33a.sanity-flr has type file OK /mnt/lustre/f33a.sanity-flr has size 500 OK reading file (data should be provided by ost1)... Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg315-server: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg315-client: oleg315-server: ssh exited with exit code 1 Started lustre-OST0001 affected facets: ost2 oleg315-server: oleg315-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg315-server: *.lustre-OST0001.recovery_status status: COMPLETE PASS 33a (65s) == sanity-flr test 33b: avoid reading from unhealthy mirror ========================================================== 14:54:02 (1761245642) 25+0 records in 25+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 2.78696 s, 18.8 MB/s -rw-r--r-- 1 root root 52428800 Oct 23 14:54 /mnt/lustre/f33b.sanity-flr reading file (data can be provided by any ost)... real 0m0.750s user 0m0.000s sys 0m0.152s Stopping /mnt/lustre-ost1 (opts:) on oleg315-server oleg315-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff96748692b800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff96748692b800.ost_server_uuid in DISCONN state after 0 sec reading file (data should be provided by ost2)... real 0m1.058s user 0m0.001s sys 0m0.172s Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg315-server: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg315-client: oleg315-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg315-server: oleg315-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg315-server: *.lustre-OST0000.recovery_status status: COMPLETE Stopping /mnt/lustre-ost2 (opts:) on oleg315-server oleg315-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0001-osc-ffff96748692b800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff96748692b800.ost_server_uuid in DISCONN state after 0 sec reading file (data should be provided by ost1)... real 0m1.590s user 0m0.000s sys 0m0.175s Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg315-server: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg315-client: oleg315-server: ssh exited with exit code 1 Started lustre-OST0001 affected facets: ost2 oleg315-server: oleg315-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg315-server: *.lustre-OST0001.recovery_status status: COMPLETE oleg315-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff96748692b800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff96748692b800.ost_server_uuid in FULL state after 0 sec PASS 33b (79s) == sanity-flr test 33c: keep reading among unhealthy mirrors ========================================================== 14:55:21 (1761245721) SKIP: sanity-flr test_33c need >= 3 OSTs SKIP 33c (3s) == sanity-flr test 34a: read mirrored file with multiple stripes ========================================================== 14:55:25 (1761245725) SKIP: sanity-flr test_34a need >= 4 OSTs SKIP 34a (4s) == sanity-flr test 34b: read mirrored file with multiple components ========================================================== 14:55:28 (1761245728) SKIP: sanity-flr test_34b need >= 4 OSTs SKIP 34b (3s) == sanity-flr test 35: allow to write to mirrored files == 14:55:31 (1761245731) PASS 35 (6s) == sanity-flr test 36a: write to mirrored files ========== 14:55:37 (1761245737) 12+0 records in 12+0 records out 12582912 bytes (13 MB, 12 MiB) copied, 0.487982 s, 25.8 MB/s fail_val=10 fail_loc=0x1A01 fail_loc=0 PASS 36a (12s) == sanity-flr test 36b: write should not hung when prefered mirror is stale ========================================================== 14:55:49 (1761245749) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362562 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260247 s, 40.3 MB/s PASS 36b (8s) == sanity-flr test 36c: change prefer mirror during write shouldn't hung ========================================================== 14:55:57 (1761245757) PASS 36c (8s) == sanity-flr test 36d: write/punch FLR file update OST layout version ========================================================== 14:56:05 (1761245765) ** create /mnt/lustre/f36d.sanity-flr ** mirror extend /mnt/lustre/f36d.sanity-flr (1/2) flv=2 olv=0 ** mirror extend /mnt/lustre/f36d.sanity-flr (2/2) flv=3 olv=0 ** write /mnt/lustre/f36d.sanity-flr (1/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000975772 s, 1.0 MB/s flv=4 olv=4 ** write /mnt/lustre/f36d.sanity-flr (2/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00842136 s, 122 kB/s flv=4 olv=4 ** resync /mnt/lustre/f36d.sanity-flr flv=6 olv=6 ** truncate /mnt/lustre/f36d.sanity-flr (1/2) flv=7 olv=7 ** truncate /mnt/lustre/f36d.sanity-flr (2/2) flv=7 olv=7 ** resync /mnt/lustre/f36d.sanity-flr flv=9 olv=9 ** write /mnt/lustre/f36d.sanity-flr (1/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00218213 s, 469 kB/s flv=10 olv=10 ** write /mnt/lustre/f36d.sanity-flr (2/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00101282 s, 1.0 MB/s flv=10 olv=10 PASS 36d (10s) == sanity-flr test 37: mirror I/O API verification ======= 14:56:15 (1761245775) create test files with size 15743319 .. 271d12413e5a91a7a00f44c2ec850239 - b2d52a4a637396d418cc903d2d4104e0 - 9babc0b480e47919787ddde1f6e4c7ca - 3 Verifying mirror read .. Verifying mirror write .. Verifying mirror copy .. mirror copied successfully: 2 3 Stopping client oleg315-client.virtnet /mnt/lustre (opts:) Starting client: oleg315-client.virtnet: -o user_xattr,flock 192.168.203.115@tcp:/lustre /mnt/lustre PASS 37 (19s) == sanity-flr test 38: resync ============================ 14:56:35 (1761245795) 4 mirror IDs: 1 2 3 4 mirror_io: 391: hit by error injection mirror_io: 391: hit by error injection PASS 38 (12s) == sanity-flr test 39: check FLR+PFL (a.k.a. PFLR) creation ========================================================== 14:56:47 (1761245807) PASS 39 (5s) == sanity-flr test 40: PFLR rdonly state instantiation check ========================================================== 14:56:52 (1761245812) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.04773 s, 22.0 MB/s composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 760 lcm_flags: wp lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 7 components: - lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 0 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - l_ost_idx: 0 l_fid: 0x280000401:0x3f:0x0 - lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - l_ost_idx: 0 l_fid: 0x280000401:0x40:0x0 - lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: prefer lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - l_ost_idx: 1 l_fid: 0x2c0000401:0x3c:0x0 - lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: 0 lcme_extent.e_start: 1048576 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131079 lcme_mirror_id: 2 lcme_flags: 0 lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206218 s, 50.8 MB/s composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 760 lcm_flags: wp lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 7 components: - lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 0 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - l_ost_idx: 1 l_fid: 0x2c0000401:0x3d:0x0 - lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - l_ost_idx: 1 l_fid: 0x2c0000401:0x3e:0x0 - lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: prefer lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - l_ost_idx: 0 l_fid: 0x280000401:0x41:0x0 - lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: 0 lcme_extent.e_start: 1048576 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131079 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 PASS 40 (7s) == sanity-flr test 41: lfs mirror resync check =========== 14:56:59 (1761245819) **create two FLR files /mnt/lustre/f41.sanity-flr /mnt/lustre/f41.sanity-flr-1 **verify files be RDONLY 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.066455 s, 31.6 MB/s 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.302431 s, 13.9 MB/s **verify files be WRITE_PENDING **verify files have stale component **full resync **verify /mnt/lustre/f41.sanity-flr-1 data consistency in all mirrors **verify files be RDONLY **verify files do not contain stale component **write /mnt/lustre/f41.sanity-flr-1 for partial resync test 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.109805 s, 19.1 MB/s **only resync mirror 2 **resync mirror 2 again **verify /mnt/lustre/f41.sanity-flr-1 contains stale component **full resync /mnt/lustre/f41.sanity-flr-1 **full resync /mnt/lustre/f41.sanity-flr-1 again **verify /mnt/lustre/f41.sanity-flr-1 does not contain stale component PASS 41 (9s) == sanity-flr test 42: lfs mirror verify ================= 14:57:08 (1761245828) SKIP: sanity-flr test_42 need >= 4 OSTs SKIP 42 (2s) == sanity-flr test 43a: mirror pick on write ============= 14:57:10 (1761245830) SKIP: sanity-flr test_43a needs >= 3 OSTs SKIP 43a (2s) == sanity-flr test 43b: allow writing to multiple preferred mirror file ========================================================== 14:57:13 (1761245833) striped dir -i1 -c2 -H crush2 /mnt/lustre/d43b.sanity-flr ** write to /mnt/lustre/d43b.sanity-flr/f43b.sanity-flr 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047752 s, 22.0 MB/s ** resync /mnt/lustre/d43b.sanity-flr/f43b.sanity-flr PASS 43b (6s) == sanity-flr test 44a: lfs mirror split check =========== 14:57:19 (1761245839) 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.186225 s, 16.9 MB/s PASS 44a (8s) == sanity-flr test 44b: mirror split does not create stale file ========================================================== 14:57:27 (1761245847) /mnt/lustre/d44b.sanity-flr/f44b.sanity-flr lcm_layout_gen: 2 lcm_mirror_count: 2 lcm_entry_count: 2 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0x4a:0x0] } lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x48:0x0] } split mirror_id 1 id 65537 /mnt/lustre/d44b.sanity-flr/f44b.sanity-flr lcm_layout_gen: 5 lcm_mirror_count: 1 lcm_entry_count: 1 lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x48:0x0] } split mirror_id 2 id 131073 lfs split: '/mnt/lustre/d44b.sanity-flr/f44b.sanity-flr' is not a FLR file. Cannot resync file make sure there's no stale comp in the file PASS 44b (7s) SKIP: sanity-flr test_44c skipping ALWAYS excluded test 44c == sanity-flr test 44d: lfs mirror split does not break size ========================================================== 14:57:36 (1761245856) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.270856 s, 38.7 MB/s ** before mirror ops, file size=10485760 ** after mirror extend, file size=10485760 lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000403:0x7:0x0]: Invalid argument (22) ** after mirror split, file size=10485760 Stopping client oleg315-client.virtnet /mnt/lustre (opts:) Starting client: oleg315-client.virtnet: -o user_xattr,flock 192.168.203.115@tcp:/lustre /mnt/lustre ** after mirror split & remount, file size=10485760 PASS 44d (10s) == sanity-flr test 44e: basic FLR SOM tests + disable SOM ========================================================== 14:57:45 (1761245865) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d44e.sanity-flr llite.lustre-ffff9674987ee800.xattr_cache=0 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.278487 s, 37.7 MB/s ** before mirror resync, file size=10485760 file: /mnt/lustre/d44e.sanity-flr/f44e.sanity-flr size: 10485760 blocks: 40960 flags: 1 Stopping client oleg315-client.virtnet /mnt/lustre (opts:) Starting client: oleg315-client.virtnet: -o user_xattr,flock 192.168.203.115@tcp:/lustre /mnt/lustre mdt.lustre-MDT0000.enable_strict_som=0 mdt.lustre-MDT0001.enable_strict_som=0 file: /mnt/lustre/d44e.sanity-flr/f44e.sanity-flr size: 10485760 blocks: 40960 flags: 1 llite.lustre-ffff967491c4d800.xattr_cache=1 PASS 44e (16s) == sanity-flr test 45: Verify setstripe/getstripe with YAML with FLR file ========================================================== 14:58:01 (1761245881) striped dir -i1 -c2 -H all_char /mnt/lustre/d45.sanity-flr 1. FLR file: getstripe --yaml /mnt/lustre/d45.sanity-flr/f45.sanity-flr 1. FLR file: setstripe --yaml=/mnt/lustre/d45.sanity-flr/template /mnt/lustre/d45.sanity-flr/f45.sanity-flr.copy 1. FLR file: compare 2. FLR file with flags: getstripe --yaml /mnt/lustre/d45.sanity-flr/f45.sanity-flr 2. FLR file with flags: setstripe --yaml=/mnt/lustre/d45.sanity-flr/template /mnt/lustre/d45.sanity-flr/f45.sanity-flr.copy 2. FLR file with flags: compare PASS 45 (34s) == sanity-flr test 46: Verify setstripe --copy option ==== 14:58:36 (1761245916) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d46.sanity-flr ** 1. PFL file ** 2. plain file ** 3. FLR file ** 4. PFL dir striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d46.sanity-flr/dir striped dir -i0 -c2 -H all_char /mnt/lustre/d46.sanity-flr/dir.copy ** 5. plain dir ** 6. FLR dir PASS 46 (30s) == sanity-flr test 47: Verify mirror obj alloc =========== 14:59:05 (1761245945) SKIP: sanity-flr test_47 needs >= 3 OSTs SKIP 47 (2s) == sanity-flr test 48: Verify snapshot mirror ============ 14:59:07 (1761245947) ** create 2 mirrors FLR file /mnt/lustre/f48.sanity-flr ** write it 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.208791 s, 15.1 MB/s ** resync the file ** snapshot mirror 2 ** write it again 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.147351 s, 21.3 MB/s ** resync it again ** verify mirror 2 doesn't change original checksum: 385c42aa7d6b8ee735d1107585ff6bd8 - mirror 1 checksum: f7fbc1cb29e16177fbbfd6517ff02258 - mirror 2 checksum: 385c42aa7d6b8ee735d1107585ff6bd8 - ** mirror 2 stripe info /mnt/lustre/f48.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 448 lcm_flags: wp lcm_layout_gen: 10 lcm_mirror_count: 2 lcm_entry_count: 4 components: - lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale,nosync lcme_timestamp: 'Thu Oct 23 14:59:09 2025' lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lcme_offset: 336 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0xa lmm_fid: [0x200000405:0xa:0x0] lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x51:0x0] } - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,stale,nosync lcme_timestamp: 'Thu Oct 23 14:59:09 2025' lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lcme_offset: 392 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0xa lmm_fid: [0x200000405:0xa:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x52:0x0] } ** resync mirror 2 ** verify mirror 2 resync-ed original checksum: 385c42aa7d6b8ee735d1107585ff6bd8 - mirror 1 checksum: f7fbc1cb29e16177fbbfd6517ff02258 - mirror 2 checksum: f7fbc1cb29e16177fbbfd6517ff02258 - ** mirror 2 stripe info /mnt/lustre/f48.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 448 lcm_flags: ro lcm_layout_gen: 12 lcm_mirror_count: 2 lcm_entry_count: 4 components: - lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,nosync lcme_timestamp: 'Thu Oct 23 14:59:09 2025' lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lcme_offset: 336 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0xa lmm_fid: [0x200000405:0xa:0x0] lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x51:0x0] } - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,nosync lcme_timestamp: 'Thu Oct 23 14:59:09 2025' lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lcme_offset: 392 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0xa lmm_fid: [0x200000405:0xa:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x52:0x0] } PASS 48 (10s) == sanity-flr test 49a: FIEMAP upon FLR file ============= 14:59:17 (1761245957) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0526563 s, 19.9 MB/s Filesystem type is: bd00bd0 File size of /mnt/lustre/f49a.sanity-flr is 1048576 (1024 blocks of 1024 bytes) ext: device_logical: physical_offset: length: dev: flags: 0: 0.. 1023: 312320.. 313343: 1024: 0001: net,eof 1: 0.. 1023: 308224.. 309247: 1024: 0000: last,net,eof /mnt/lustre/f49a.sanity-flr: 2 extents found FIEMAP on /mnt/lustre/f49a.sanity-flr succeeded PASS 49a (7s) == sanity-flr test 50A: mirror split update layout generation ========================================================== 14:59:24 (1761245964) Failing mds1 on oleg315-server Stopping /mnt/lustre-mds1 (opts:) on oleg315-server 14:59:34 (1761245974) shut down facet: mds1 facet_host: oleg315-server facet_failover_host: oleg315-server Failover mds1 to oleg315-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg315-server: oleg315-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg315-client: oleg315-server: ssh exited with exit code 1 Started lustre-MDT0000 14:59:54 (1761245994) targets are mounted 14:59:54 (1761245994) facet_failover done oleg315-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x10:0x0]: Invalid argument (22) /mnt/lustre/d50A.sanity-flr/f50A.sanity-flr layout generation from 1 to 2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194934 s, 53.8 MB/s /mnt/lustre/d50A.sanity-flr/f50A.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 136 lcm_flags: 0 lcm_layout_gen: 2 lcm_mirror_count: 1 lcm_entry_count: 1 components: - lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lcme_offset: 80 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x10 lmm_fid: [0x200000405:0x10:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x54:0x0] } PASS 50A (41s) == sanity-flr test 50a: mirror extend/copy preserves sparseness ========================================================== 15:00:06 (1761246006) ** create striped file /mnt/lustre/d50a.sanity-flr/f50a.sanity-flr ** write 1st data chunk at 1M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0323806 s, 632 kB/s ** write 2nd data chunk at 2M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0202494 s, 1.0 MB/s ** create hole at the file end ** verify sparseness first data offset: 1044480 hole at the end: 3500000 ** extend the file with new mirror ** verify mirror #2 sparseness first data offset: 1044480 hole at the end: 3500000 ** copy mirror #2 to mirror #1 mirror copied successfully: 1 ** verify mirror #1 sparseness again first data offset: 1044480 hole at the end: 3500000 ** final consumed blocks: 104 PASS 50a (7s) == sanity-flr test 50b: mirror rsync handles sparseness == 15:00:13 (1761246013) ** create mirrored file /mnt/lustre/d50b.sanity-flr/f50b.sanity-flr ** write data chunk at 1M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0644435 s, 318 kB/s ** create hole at the file end ** verify sparseness first data offset: 1044480 hole at the end: 3500000 ** resync mirror #2 to mirror #1 ** consumed blocks: 96 ** truncate file down ** write data chunk at 2M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0121391 s, 1.7 MB/s ** resync mirror #2 to mirror #1 with nosync 1st component ** resync mirror #2 to mirror #1 again ** final consumed blocks: 96 PASS 50b (8s) == sanity-flr test 50c: punch_hole/mmap_write stale other mirrors ========================================================== 15:00:21 (1761246021) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d50c.sanity-flr 4+0 records in 4+0 records out 16384 bytes (16 kB, 16 KiB) copied, 0.00616214 s, 2.7 MB/s PASS 50c (7s) == sanity-flr test 50d: mirror rsync keep holes ========== 15:00:28 (1761246028) ** create mirrored file /mnt/lustre/d50d.sanity-flr/f50d.sanity-flr ** write data chunk at 1M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0403677 s, 507 kB/s ** create hole at the file start ** verify sparseness first data offset: 1048576 ** resync mirror #2 ** verify sparseness of mirror #2 first data offset: 1048576 PASS 50d (6s) == sanity-flr test 60a: mirror extend sets correct size on sparse file ========================================================== 15:00:34 (1761246034) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0056669 s, 723 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0293237 s, 140 kB/s PASS 60a (7s) == sanity-flr test 60b: mirror merge/split cancel client's in-memory layout gen ========================================================== 15:00:41 (1761246041) striped dir -i0 -c2 -H crush2 /mnt/lustre/d60b.sanity-flr lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x1c:0x0]: Invalid argument (22) PASS 60b (20s) == sanity-flr test 61a: mirror extend and migrate preserve timestamps ========================================================== 15:01:01 (1761246061) mdd.lustre-MDT0000.atime_diff=1 mdd.lustre-MDT0001.atime_diff=1 create /mnt/lustre/d61a.sanity-flr/f61a.sanity-flr create /mnt/lustre/d61a.sanity-flr/f61a.sanity-flr-2 sleep 5 seconds, then cat f61a.sanity-flr XXX sleep 5 seconds, then re-write f61a.sanity-flr sleep 5 seconds mirror merge f61a.sanity-flr-2 to f61a.sanity-flr and test timestamps mirror extend f61a.sanity-flr and test timestamps migrate f61a.sanity-flr and test timestamps normal user migrate f61a.sanity-flr and test timestamps running as uid/gid/euid/egid 500/500/500/500, groups: 0 [/home/green/git/lustre-release/lustre/utils/lfs] [migrate] [-n] [/mnt/lustre/d61a.sanity-flr/f61a.sanity-flr] mdd.lustre-MDT0000.atime_diff=60 mdd.lustre-MDT0001.atime_diff=60 PASS 61a (28s) == sanity-flr test 61b: mirror extend and split preserve timestamps ========================================================== 15:01:29 (1761246089) create /mnt/lustre/d61b.sanity-flr/f61b.sanity-flr sleep 5 seconds, then cat f61b.sanity-flr XXX sleep 5 seconds, then re-write f61b.sanity-flr sleep 5 seconds, then test timestamps mirror extend f61b.sanity-flr and test timestamps mirror split f61b.sanity-flr and test timestamps lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x16:0x0]: Invalid argument (22) normal user mirror extend f61b.sanity-flr and test timestamps running as uid/gid/euid/egid 500/500/500/500, groups: 0 [/home/green/git/lustre-release/lustre/utils/lfs] [mirror] [extend] [-N] [-c1] [-i1] [/mnt/lustre/d61b.sanity-flr/f61b.sanity-flr] PASS 61b (23s) == sanity-flr test 61c: mirror resync preserves timestamps ========================================================== 15:01:52 (1761246112) create /mnt/lustre/d61c.sanity-flr/f61c.sanity-flr sleep 5 seconds, then cat f61c.sanity-flr XXX sleep 5 seconds, then mirror extend f61c.sanity-flr and write it sleep 5 seconds, then resync f61c.sanity-flr and test timestamps normal user resync f61c.sanity-flr and test timestamps running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [mirror] [resync] [/mnt/lustre/d61c.sanity-flr/f61c.sanity-flr] PASS 61c (22s) == sanity-flr test 62: read/write with unknown type of mirror ========================================================== 15:02:15 (1761246135) create mirror file with unknown magic fail_loc=0x1426 fail_val=2 create mirror file with unknown pattern fail_loc=0x1427 fail_val=1 PASS 62 (7s) == sanity-flr test 70: mirror create and split race ====== 15:02:21 (1761246141) striped dir -i0 -c2 -H crush /mnt/lustre/d70.sanity-flr mirror create pid 55924 mirror split pid 55925 mirror create and split race for 60 seconds, should not crash /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 55924 Killed while true; do rm -f $tf; $LFS mirror create -N -E 1M -c -1 -E eof -N $tf; echo xxxx > $tf; done (wd: ~) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 55925 Killed while true; do $LFS mirror split -d --mirror-id=1 $tf &> /dev/null; done (wd: ~) PASS 70 (68s) == sanity-flr test 70a: flr mode fsx test ================ 15:03:29 (1761246209) striped dir -i0 -c2 -H crush /mnt/lustre/d70a.sanity-flr Seed set to 6210 skipping zero length zero range truncating to largest ever: 0x1bb5b 000002 1761246211.247732 trunc from 00000000 to 0x01bb5a (0x1bb5b bytes) 000003 1761246211.266994 write 0x01e70a thru 0x02c943 (0xe23a bytes) 000005 1761246211.270848 punch from 0x00921a to 0x00bed3 (0x2cba bytes) fallocating to largest ever: 0x3ea06 000006 1761246211.279777 fallocate 0x0373e7 thru 0x03ea05 (0x761f bytes) 000009 1761246211.295447 MIRROR_EXTEND to 2 mirrors fallocating to largest ever: 0x40000 000010 1761246212.259104 fallocate 0x03e5cc thru 0x03ffff (0x1a34 bytes) 000012 1761246212.297251 mapread 0x020d96 thru 0x02309a (0x2305 bytes) 000013 1761246212.320280 mapwrite 0x00a9c7 thru 0x00f279 (0x48b3 bytes) 000015 1761246212.356412 trunc from 0x040000 to 0x025dc2 (0x1a23d bytes) 000016 1761246212.389329 write 0x01fea6 thru 0x021e04 (0x1f5f bytes) 000017 1761246212.390530 mapread 0x005270 thru 0x010614 (0xb3a5 bytes) 000019 1761246212.393819 trunc from 0x025dc3 to 0x02366b (0x2757 bytes) 000020 1761246212.422752 read 0x011286 thru 0x0123a6 (0x1121 bytes) 000021 1761246212.423457 punch from 0x00b9bc to 0x00e8e2 (0x2f27 bytes) 000022 1761246212.430303 mapwrite 0x03cc57 thru 0x03ffff (0x33a9 bytes) 000023 1761246212.477153 trunc from 0x040000 to 0x03edfb (0x1204 bytes) 000024 1761246212.502887 read 0x018be1 thru 0x019f00 (0x1320 bytes) 000025 1761246212.522962 read 0x0299ea thru 0x030d5f (0x7376 bytes) 000026 1761246212.524260 trunc from 0x005fc9 to 0x00bb95 (0x5bcd bytes) 000027 1761246212.534247 mapread 0x035fd8 thru 0x03c442 (0x646b bytes) 000028 1761246212.535281 write 0x01b8bd thru 0x021fc8 (0x670c bytes) 000029 1761246212.536456 mapread 0x02747b thru 0x031372 (0x9ef8 bytes) 000030 1761246212.543488 MIRROR_EXTEND to 3 mirrors 000031 1761246213.418979 punch from 0x025842 to 0x035670 (0xfe2f bytes) 000032 1761246213.471655 punch from 0x0340d3 to 0x0399ca (0x58f8 bytes) 000033 1761246213.477747 trunc from 0x03edfc to 0x03d17d (0x1c7e bytes) 000034 1761246213.503471 read 0x0030e9 thru 0x00833f (0x5257 bytes) 000035 1761246213.511928 punch from 0x0393b9 to 0x03d17d (0x3dc5 bytes) 000036 1761246213.516937 fallocate 0x02e864 thru 0x031bd0 (0x336d bytes) 000037 1761246213.525466 trunc from 0x039aee to 0x03d17d (0x3690 bytes) 000038 1761246213.536287 MIRROR_RESYNC 3 mirrors 000039 1761246214.604933 mapread 0x02eeab thru 0x02f502 (0x658 bytes) 000040 1761246214.626695 read 0x0332d5 thru 0x03d17d (0x9ea9 bytes) 000041 1761246214.636892 read 0x036966 thru 0x03d17d (0x6818 bytes) 000042 1761246214.642486 mapwrite 0x010d91 thru 0x0208d2 (0xfb42 bytes) 000043 1761246214.900904 trunc from 0x03d17e to 0x014c8b (0x284f2 bytes) 000044 1761246214.940186 mapread 0x00b88d thru 0x014669 (0x8ddd bytes) 000045 1761246214.980252 read 0x00e34a thru 0x014c8b (0x6942 bytes) 000046 1761246214.980953 trunc from 0x0085f3 to 0x00bdfb (0x3809 bytes) 000047 1761246214.989726 trunc from 0x014556 to 0x014c8b (0x736 bytes) 000048 1761246214.997556 mapwrite 0x0359a6 thru 0x03ad02 (0x535d bytes) 000049 1761246215.052646 fallocate 0x002891 thru 0x0052e9 (0x2a59 bytes) 000050 1761246215.060616 punch from 0x01310b to 0x01c99a (0x9890 bytes) 000051 1761246215.068685 trunc from 0x01039f to 0x016fcc (0x6c2e bytes) 000054 1761246215.077623 read 0x02a65e thru 0x033116 (0x8ab9 bytes) 000055 1761246215.078383 mapwrite 0x00a75c thru 0x011151 (0x69f6 bytes) 000056 1761246215.138464 mapwrite 0x0383f6 thru 0x03ffff (0x7c0a bytes) 000057 1761246215.197053 mapwrite 0x02b593 thru 0x02b846 (0x2b4 bytes) 000058 1761246215.219468 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000059 1761246215.552912 trunc from 0x040000 to 0x02ae2c (0x151d3 bytes) 000060 1761246215.585130 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000061 1761246215.796019 read 0x026713 thru 0x02ae2c (0x471a bytes) 000062 1761246215.808266 read 0x00c88c thru 0x01822f (0xb9a4 bytes) 000063 1761246215.823304 write 0x002125 thru 0x00c6c9 (0xa5a5 bytes) 000064 1761246215.829687 read 0x000119 thru 0x006386 (0x626e bytes) 000065 1761246215.830536 read 0x02a55b thru 0x02ae2c (0x8d2 bytes) 000066 1761246215.831134 trunc from 0x02ae2d to 0x006a4e (0x243de bytes) 000068 1761246215.874259 fallocate 0x0365b8 thru 0x03d38c (0x6dd5 bytes) 000070 1761246215.882088 mapwrite 0x008f1f thru 0x00e44d (0x552f bytes) 000071 1761246215.913063 MIRROR_EXTEND to 2 mirrors 000072 1761246216.684083 MIRROR_RESYNC 2 mirrors 000073 1761246216.998560 fallocate 0x028f1b thru 0x02d359 (0x443f bytes) 000074 1761246217.052616 mapwrite 0x01af51 thru 0x026ebf (0xbf6f bytes) 000075 1761246217.113075 mapwrite 0x000676 thru 0x0069c5 (0x6350 bytes) 000076 1761246217.137618 mapread 0x02f887 thru 0x039160 (0x98da bytes) 000077 1761246217.159960 MIRROR_RESYNC 2 mirrors 000078 1761246217.830269 mapwrite 0x019589 thru 0x023fef (0xaa67 bytes) 000079 1761246217.968452 punch from 0x03492b to 0x03d38c (0x8a62 bytes) 000080 1761246217.983480 MIRROR_EXTEND to 3 mirrors 000081 1761246218.892473 trunc from 0x03d38d to 0x02d2cc (0x100c0 bytes) 000082 1761246218.951531 read 0x010b35 thru 0x0113a5 (0x871 bytes) 000083 1761246218.966930 mapread 0x01359c thru 0x013717 (0x17c bytes) 000084 1761246218.977677 trunc from 0x00bee2 to 0x013b57 (0x7c76 bytes) 000086 1761246218.987994 write 0x033da4 thru 0x037a9e (0x3cfb bytes) 000087 1761246218.989776 trunc from 0x037a9f to 0x00b4a3 (0x2c5fb bytes) 000089 1761246219.017079 write 0x008fba thru 0x0107fa (0x7841 bytes) 000090 1761246219.026883 trunc from 0x00a4bd to 0x0107fa (0x633e bytes) 000091 1761246219.059162 punch from 0x00635d to 0x006b71 (0x815 bytes) 000092 1761246219.067669 mapwrite 0x025026 thru 0x033d55 (0xed30 bytes) 000093 1761246219.172993 read 0x031223 thru 0x031a54 (0x832 bytes) 000094 1761246219.173679 read 0x0272a2 thru 0x033c93 (0xc9f2 bytes) 000095 1761246219.187624 MIRROR_EXTEND to 4 mirrors 000096 1761246219.913492 mapwrite 0x01643c thru 0x0181f0 (0x1db5 bytes) 000097 1761246220.041642 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000098 1761246220.316247 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000099 1761246220.553123 read 0x0169d2 thru 0x023f88 (0xd5b7 bytes) 000101 1761246220.578203 trunc from 0x00e7d7 to 0x0158d3 (0x70fd bytes) 000102 1761246220.589747 mapread 0x02842c thru 0x033d55 (0xb92a bytes) 000103 1761246220.633352 trunc from 0x0243d4 to 0x028703 (0x4330 bytes) 000104 1761246220.653579 mapwrite 0x027730 thru 0x035c60 (0xe531 bytes) 000105 1761246220.780899 trunc from 0x010429 to 0x010a7d (0x655 bytes) 000106 1761246220.786045 read 0x03398b thru 0x035c60 (0x22d6 bytes) 000107 1761246220.786701 read 0x023757 thru 0x0329d2 (0xf27c bytes) 000108 1761246220.787359 write 0x016f15 thru 0x01c1c1 (0x52ad bytes) 000111 1761246220.793824 mapwrite 0x01ae59 thru 0x024800 (0x99a8 bytes) 000112 1761246220.840109 write 0x005281 thru 0x01510c (0xfe8c bytes) 000113 1761246220.855869 punch from 0x00cf9f to 0x0166b9 (0x971b bytes) 000114 1761246220.904004 fallocate 0x0338eb thru 0x03a91d (0x7033 bytes) 000115 1761246220.918041 trunc from 0x00183f to 0x003d56 (0x2518 bytes) 000116 1761246220.956316 read 0x00bbf0 thru 0x015bcc (0x9fdd bytes) 000117 1761246220.974133 fallocate 0x03cb87 thru 0x03ffff (0x3479 bytes) 000118 1761246220.990596 mapwrite 0x019311 thru 0x02540e (0xc0fe bytes) 000119 1761246221.018951 mapwrite 0x02eebb thru 0x03c25a (0xd3a0 bytes) 000120 1761246221.115520 punch from 0x0159e2 to 0x025396 (0xf9b5 bytes) 000121 1761246221.127931 read 0x00f228 thru 0x0146f5 (0x54ce bytes) 000122 1761246221.129373 trunc from 0x034bab to 0x034ff2 (0x448 bytes) 000123 1761246221.135185 write 0x0244c1 thru 0x029645 (0x5185 bytes) 000124 1761246221.153236 mapread 0x03b7b8 thru 0x03c25a (0xaa3 bytes) 000125 1761246221.155052 mapwrite 0x036b7b thru 0x03ce3e (0x62c4 bytes) 000126 1761246221.218637 write 0x00f316 thru 0x019a20 (0xa70b bytes) 000127 1761246221.221995 trunc from 0x03ce3f to 0x038588 (0x48b6 bytes) 000128 1761246221.244767 read 0x0277a0 thru 0x0352f6 (0xdb57 bytes) 000129 1761246221.245529 read 0x0372aa thru 0x038588 (0x12df bytes) 000130 1761246221.263884 write 0x0318a7 thru 0x037f06 (0x6660 bytes) 000131 1761246221.266750 trunc from 0x02c09a to 0x02fa63 (0x39ca bytes) 000132 1761246221.272306 read 0x027c63 thru 0x02d41f (0x57bd bytes) 000133 1761246221.290561 trunc from 0x038589 to 0x03f5d2 (0x704a bytes) 000135 1761246221.313194 write 0x01248c thru 0x020ae9 (0xe65e bytes) 000136 1761246221.317676 mapread 0x00a0ad thru 0x011428 (0x737c bytes) 000138 1761246221.319974 mapwrite 0x00cb12 thru 0x012fdd (0x64cc bytes) 000139 1761246221.358082 trunc from 0x03f5d3 to 0x02bbb7 (0x13a1b bytes) 000140 1761246221.380124 trunc from 0x01f745 to 0x02bbb7 (0xc473 bytes) 000142 1761246221.386716 read 0x023d3b thru 0x02bbb7 (0x7e7d bytes) 000143 1761246221.408056 MIRROR_RESYNC 2 mirrors 000145 1761246222.103706 fallocate 0x0306f7 thru 0x03b4f9 (0xae03 bytes) 000146 1761246222.158638 trunc from 0x03b4fa to 0x030572 (0xaf87 bytes) 000147 1761246222.229369 trunc from 0x006d21 to 0x00de5c (0x713c bytes) 000148 1761246222.248647 MIRROR_EXTEND to 3 mirrors 000149 1761246223.090585 MIRROR_EXTEND to 4 mirrors 000150 1761246224.195876 MIRROR_RESYNC 4 mirrors 000151 1761246225.174499 write 0x0137e5 thru 0x020249 (0xca65 bytes) 000152 1761246225.236042 MIRROR_RESYNC 4 mirrors 000153 1761246226.562393 read 0x02b658 thru 0x02fefe (0x48a7 bytes) 000154 1761246226.572893 mapread 0x0108ac thru 0x014a74 (0x41c9 bytes) 000155 1761246226.595197 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000156 1761246226.986987 write 0x000e36 thru 0x00cf9b (0xc166 bytes) 000157 1761246227.056491 MIRROR_RESYNC 3 mirrors 000158 1761246228.028160 trunc from 0x01c225 to 0x01c2fc (0xd8 bytes) 000159 1761246228.075279 trunc from 0x030573 to 0x01e764 (0x11e0e bytes) 000160 1761246228.107356 mapwrite 0x012117 thru 0x021cec (0xfbd6 bytes) 000161 1761246228.228942 mapread 0x00b606 thru 0x01914d (0xdb48 bytes) 000162 1761246228.250946 punch from 0x00dc74 to 0x016dd1 (0x915e bytes) 000163 1761246228.262898 read 0x0160b6 thru 0x018f0a (0x2e55 bytes) 000164 1761246228.273678 read 0x02173d thru 0x021cec (0x5b0 bytes) 000165 1761246228.274396 trunc from 0x00ea5c to 0x01cd5c (0xe301 bytes) 000166 1761246228.292660 write 0x006995 thru 0x013a05 (0xd071 bytes) 000167 1761246228.304677 trunc from 0x008c68 to 0x00cf4b (0x42e4 bytes) 000168 1761246228.328063 trunc from 0x021ced to 0x016c31 (0xb0bb bytes) 000169 1761246228.357752 trunc from 0x0003f5 to 0x00168b (0x1297 bytes) 000170 1761246228.364747 punch from 0x004070 to 0x00ed97 (0xad28 bytes) 000171 1761246228.373607 punch from 0x00cfaf to 0x0163f9 (0x944b bytes) 000172 1761246228.380446 trunc from 0x016c32 to 0x01d672 (0x6a41 bytes) 000173 1761246228.398134 trunc from 0x0079b0 to 0x014822 (0xce73 bytes) 000174 1761246228.403753 trunc from 0x01d673 to 0x010a79 (0xcbf9 bytes) 000175 1761246228.437363 MIRROR_RESYNC 3 mirrors 000176 1761246229.019605 punch from 0x006a79 to 0x00caa5 (0x602d bytes) 000177 1761246229.079446 mapwrite 0x00d89b thru 0x01c693 (0xedf9 bytes) 000178 1761246229.217174 trunc from 0x01c694 to 0x02c8aa (0x10217 bytes) 000179 1761246229.244520 write 0x00a96a thru 0x01232a (0x79c1 bytes) 000180 1761246229.250512 trunc from 0x02506e to 0x02c8aa (0x783d bytes) 000181 1761246229.260459 trunc from 0x02c8ab to 0x0077dd (0x250cd bytes) 000182 1761246229.296737 punch from 0x00313d to 0x00562c (0x24f0 bytes) 000183 1761246229.302568 trunc from 0x0077de to 0x021fa3 (0x1a7c6 bytes) 000185 1761246229.317841 mapread 0x01bd7f thru 0x02182e (0x5ab0 bytes) 000186 1761246229.328296 trunc from 0x0074fe to 0x00cd49 (0x584c bytes) 000187 1761246229.335618 mapwrite 0x03cfdb thru 0x03ffff (0x3025 bytes) 000188 1761246229.382364 trunc from 0x00ed17 to 0x013505 (0x47ef bytes) 000189 1761246229.388467 write 0x01baa9 thru 0x021df9 (0x6351 bytes) 000190 1761246229.389893 mapread 0x0245f0 thru 0x030913 (0xc324 bytes) 000191 1761246229.418988 mapwrite 0x032372 thru 0x03ffff (0xdc8e bytes) 000192 1761246229.462590 MIRROR_EXTEND to 4 mirrors 000193 1761246230.239506 MIRROR_RESYNC 4 mirrors 000194 1761246231.060415 trunc from 0x02bd43 to 0x038a83 (0xcd41 bytes) 000195 1761246231.110664 MIRROR_RESYNC 4 mirrors 000196 1761246232.370401 MIRROR_EXTEND to 5 mirrors 000197 1761246233.142636 trunc from 0x040000 to 0x035460 (0xab9f bytes) 000198 1761246233.193370 write 0x037cda thru 0x03ffff (0x8326 bytes) 000199 1761246233.195670 fallocate 0x03b380 thru 0x03c909 (0x158a bytes) 000200 1761246233.202219 write 0x00a381 thru 0x019ff9 (0xfc79 bytes) 000201 1761246233.221291 read 0x00eeed thru 0x018d67 (0x9e7b bytes) 000202 1761246233.225267 trunc from 0x00f5e4 to 0x0192ab (0x9cc8 bytes) 000203 1761246233.251859 write 0x00c851 thru 0x016a18 (0xa1c8 bytes) 000204 1761246233.264931 mapwrite 0x002680 thru 0x00a3fa (0x7d7b bytes) 000205 1761246233.316505 trunc from 0x040000 to 0x02afc3 (0x1503c bytes) 000206 1761246233.361397 fallocate 0x033e81 thru 0x03438c (0x50c bytes) 000207 1761246233.378436 MIRROR_RESYNC 5 mirrors 000208 1761246234.578287 mapread 0x026747 thru 0x02afc3 (0x487d bytes) 000209 1761246234.598262 punch from 0x013f35 to 0x01a3dc (0x64a8 bytes) 000212 1761246234.635752 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000213 1761246234.853092 mapread 0x00f746 thru 0x01b265 (0xbb20 bytes) 000214 1761246234.879146 write 0x0092cc thru 0x0093e8 (0x11d bytes) 000215 1761246234.892900 punch from 0x017a6a to 0x0258b4 (0xde4b bytes) 000216 1761246234.906493 mapread 0x001dbb thru 0x00ab67 (0x8dad bytes) 000217 1761246234.914232 write 0x02cfd0 thru 0x02da62 (0xa93 bytes) 000218 1761246234.915486 trunc from 0x02da63 to 0x010e13 (0x1cc4f bytes) 000219 1761246234.938582 MIRROR_EXTEND to 5 mirrors 000220 1761246235.597584 MIRROR_EXTEND to 6 mirrors 000221 1761246236.224702 read 0x0060a6 thru 0x010e13 (0xad6e bytes) 000222 1761246236.232501 read 0x002db8 thru 0x00bbfe (0x8e47 bytes) 000223 1761246236.240431 trunc from 0x003fcc to 0x00eb0b (0xab40 bytes) 000224 1761246236.286350 trunc from 0x00fa16 to 0x010526 (0xb11 bytes) 000225 1761246236.293200 fallocate 0x022980 thru 0x02e493 (0xbb14 bytes) 000226 1761246236.298661 write 0x035a62 thru 0x03c121 (0x66c0 bytes) 000227 1761246236.308993 write 0x03ea75 thru 0x03ffff (0x158b bytes) 000228 1761246236.310119 trunc from 0x00dd2e to 0x010ad7 (0x2daa bytes) 000229 1761246236.321087 write 0x0318c8 thru 0x0357f8 (0x3f31 bytes) 000230 1761246236.341135 mapwrite 0x030977 thru 0x03df20 (0xd5aa bytes) 000231 1761246236.361831 trunc from 0x040000 to 0x01b2c8 (0x24d37 bytes) 000232 1761246236.389723 write 0x0068e8 thru 0x010957 (0xa070 bytes) 000233 1761246236.403916 mapread 0x0093d0 thru 0x017c8a (0xe8bb bytes) 000234 1761246236.405990 mapread 0x00a924 thru 0x0157c6 (0xaea3 bytes) 000235 1761246236.412988 MIRROR_RESYNC 6 mirrors 000236 1761246237.691456 trunc from 0x01b2c9 to 0x013f9b (0x732d bytes) 000237 1761246237.750174 mapread 0x010ad7 thru 0x013f9b (0x34c5 bytes) 000238 1761246237.780003 MIRROR_RESYNC 6 mirrors 000239 1761246239.116881 read 0x0032ab thru 0x00feb4 (0xcc0a bytes) 000240 1761246239.129021 trunc from 0x013f9c to 0x036288 (0x222ed bytes) 000241 1761246239.183994 trunc from 0x032f81 to 0x034b33 (0x1bb3 bytes) 000242 1761246239.190728 trunc from 0x004738 to 0x00ca59 (0x8322 bytes) 000243 1761246239.197328 mapwrite 0x01a8fb thru 0x022e18 (0x851e bytes) 000244 1761246239.231966 mapwrite 0x026475 thru 0x031272 (0xadfe bytes) 000245 1761246239.297974 trunc from 0x036289 to 0x03dafa (0x7872 bytes) 000246 1761246239.329841 read 0x02bb8a thru 0x037eb4 (0xc32b bytes) 000247 1761246239.341362 read 0x0290cf thru 0x031a1e (0x8950 bytes) 000248 1761246239.343039 write 0x01058e thru 0x010dcf (0x842 bytes) 000249 1761246239.344015 mapread 0x03c2e4 thru 0x03dafa (0x1817 bytes) 000250 1761246239.344880 mapread 0x006172 thru 0x006a44 (0x8d3 bytes) 000252 1761246239.346108 fallocate 0x03100f thru 0x03ffff (0xeff1 bytes) 000253 1761246239.352855 fallocate 0x007bf1 thru 0x0119c3 (0x9dd3 bytes) 000254 1761246239.364951 MIRROR_RESYNC 6 mirrors 000255 1761246241.355602 fallocate 0x018b81 thru 0x01aa17 (0x1e97 bytes) 000256 1761246241.396637 mapread 0x038cd0 thru 0x03dafa (0x4e2b bytes) 000257 1761246241.438741 fallocate 0x015ee8 thru 0x016005 (0x11e bytes) 000258 1761246241.450780 read 0x021177 thru 0x025a8c (0x4916 bytes) 000259 1761246241.465979 trunc from 0x03dafb to 0x018aae (0x2504c bytes) 000260 1761246241.491611 trunc from 0x00843a to 0x00b42e (0x2ff5 bytes) 000261 1761246241.499250 read 0x007b36 thru 0x0150c4 (0xd58f bytes) 000262 1761246241.508951 mapread 0x0171dd thru 0x018aae (0x18d2 bytes) 000263 1761246241.511009 trunc from 0x0168a5 to 0x018aae (0x220a bytes) 000264 1761246241.519929 fallocate 0x00d463 thru 0x01070e (0x32ac bytes) 000265 1761246241.524478 trunc from 0x018aaf to 0x007cc2 (0x10dec bytes) 000266 1761246241.552676 punch from 0x0007f7 to 0x0050b4 (0x48be bytes) 000267 1761246241.560379 trunc from 0x007cc3 to 0x010ab3 (0x8df1 bytes) 000268 1761246241.583181 mapread 0x00f94d thru 0x010ab3 (0x1167 bytes) 000269 1761246241.593411 write 0x002ec4 thru 0x003dd3 (0xf10 bytes) 000270 1761246241.602190 trunc from 0x00e480 to 0x00e601 (0x182 bytes) 000271 1761246241.608875 fallocate 0x005d7a thru 0x00ee2a (0x90b1 bytes) 000273 1761246241.622109 punch from 0x006375 to 0x007920 (0x15ac bytes) 000274 1761246241.630148 mapread 0x000104 thru 0x00d3e3 (0xd2e0 bytes) 000275 1761246241.694870 mapwrite 0x0025d5 thru 0x005dbf (0x37eb bytes) 000276 1761246241.708045 punch from 0x00b569 to 0x010ab3 (0x554b bytes) 000277 1761246241.716492 MIRROR_RESYNC 6 mirrors 000278 1761246243.123217 trunc from 0x010918 to 0x010ab3 (0x19c bytes) 000279 1761246243.159777 mapread 0x00fb91 thru 0x010ab3 (0xf23 bytes) 000280 1761246243.182712 fallocate 0x0045d5 thru 0x0065c2 (0x1fee bytes) 000281 1761246243.190645 write 0x02eea9 thru 0x032370 (0x34c8 bytes) 000282 1761246243.193100 mapread 0x016e04 thru 0x02043d (0x963a bytes) 000283 1761246243.211509 fallocate 0x0382a2 thru 0x03c7e4 (0x4543 bytes) 000284 1761246243.218381 write 0x03c036 thru 0x03ffff (0x3fca bytes) 000285 1761246243.228561 mapread 0x017ed4 thru 0x01d3aa (0x54d7 bytes) 000286 1761246243.229577 trunc from 0x040000 to 0x00f222 (0x30ddd bytes) 000287 1761246243.252206 trunc from 0x00f223 to 0x0208f4 (0x116d2 bytes) 000288 1761246243.272020 trunc from 0x0208f5 to 0x005ffd (0x1a8f7 bytes) 000289 1761246243.300622 MIRROR_RESYNC 6 mirrors 000290 1761246244.275223 fallocate 0x02863c thru 0x029b6e (0x1533 bytes) 000291 1761246244.311755 mapwrite 0x02a2f0 thru 0x0357c1 (0xb4d2 bytes) 000292 1761246244.390892 trunc from 0x0357c2 to 0x024005 (0x117bc bytes) 000293 1761246244.420197 mapwrite 0x03c234 thru 0x03ffff (0x3dcc bytes) 000294 1761246244.461278 MIRROR_EXTEND to 7 mirrors 000296 1761246245.193510 mapwrite 0x01b5af thru 0x025aad (0xa4ff bytes) 000297 1761246245.391476 fallocate 0x013c20 thru 0x016aae (0x2e8f bytes) 000298 1761246245.404926 fallocate 0x00b614 thru 0x0150ec (0x9ad9 bytes) 000300 1761246245.414712 mapread 0x00fd87 thru 0x0125e9 (0x2863 bytes) 000301 1761246245.431919 MIRROR_EXTEND to 8 mirrors 000302 1761246246.132217 fallocate 0x0151ee thru 0x0229d0 (0xd7e3 bytes) 000303 1761246246.179493 mapread 0x038eb4 thru 0x03ffff (0x714c bytes) 000304 1761246246.200225 trunc from 0x004be8 to 0x00d7e2 (0x8bfb bytes) 000305 1761246246.205825 fallocate 0x023a9c thru 0x028c58 (0x51bd bytes) 000306 1761246246.213586 mapwrite 0x001d07 thru 0x00e6f3 (0xc9ed bytes) 000307 1761246246.246493 write 0x01f2ea thru 0x02e266 (0xef7d bytes) 000308 1761246246.269875 fallocate 0x00c213 thru 0x01b6d9 (0xf4c7 bytes) 000309 1761246246.274888 punch from 0x004bc4 to 0x013609 (0xea46 bytes) 000310 1761246246.281012 punch from 0x03f911 to 0x03ffff (0x6ef bytes) 000311 1761246246.291398 MIRROR_EXTEND to 9 mirrors 000312 1761246246.967890 write 0x023a59 thru 0x027053 (0x35fb bytes) 000313 1761246247.044262 mapread 0x0085a6 thru 0x011bdc (0x9637 bytes) 000314 1761246247.065184 write 0x010940 thru 0x01a5f4 (0x9cb5 bytes) 000315 1761246247.071264 mapwrite 0x010e7a thru 0x0151e1 (0x4368 bytes) 000316 1761246247.109927 MIRROR_RESYNC 9 mirrors 000317 1761246249.010460 fallocate 0x004bec thru 0x01216b (0xd580 bytes) 000318 1761246249.054895 trunc from 0x0375fa to 0x03877e (0x1185 bytes) 000319 1761246249.059537 mapwrite 0x00736c thru 0x007dae (0xa43 bytes) 000320 1761246249.194539 MIRROR_SPLIT mirror 2 to 8 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000321 1761246249.447250 read 0x00c2e5 thru 0x014eea (0x8c06 bytes) 000324 1761246249.460750 mapread 0x00b280 thru 0x00eac4 (0x3845 bytes) 000325 1761246249.467384 trunc from 0x0392ac to 0x03ffff (0x6d54 bytes) 000326 1761246249.472559 write 0x00690b thru 0x0130c4 (0xc7ba bytes) 000327 1761246249.489550 trunc from 0x00d807 to 0x010b1b (0x3315 bytes) 000329 1761246249.504932 write 0x02ded9 thru 0x03c6ba (0xe7e2 bytes) 000330 1761246249.523882 fallocate 0x01270a thru 0x022352 (0xfc49 bytes) 000331 1761246249.529508 read 0x0244ae thru 0x03002b (0xbb7e bytes) 000332 1761246249.538659 punch from 0x035041 to 0x038741 (0x3701 bytes) 000333 1761246249.552486 punch from 0x03a814 to 0x03ffff (0x57ec bytes) 000334 1761246249.559491 mapwrite 0x00d3ce thru 0x0118cb (0x44fe bytes) 000335 1761246249.576596 mapwrite 0x00c756 thru 0x00e245 (0x1af0 bytes) 000336 1761246249.596656 write 0x0099c3 thru 0x0169f8 (0xd036 bytes) 000337 1761246249.605480 MIRROR_EXTEND to 9 mirrors 000338 1761246250.282380 punch from 0x038b80 to 0x03ffff (0x7480 bytes) 000339 1761246250.341647 read 0x00b6bd thru 0x013a6d (0x83b1 bytes) 000340 1761246250.355558 mapread 0x0035fe thru 0x006faf (0x39b2 bytes) 000341 1761246250.374660 mapread 0x03dedb thru 0x03ffff (0x2125 bytes) 000342 1761246250.385233 trunc from 0x040000 to 0x00290c (0x3d6f3 bytes) 000343 1761246250.436004 punch from 0x000da0 to 0x00290c (0x1b6d bytes) 000344 1761246250.441575 read 0x002775 thru 0x00290c (0x198 bytes) 000345 1761246250.448174 write 0x022c60 thru 0x02384f (0xbf0 bytes) 000346 1761246250.449667 trunc from 0x018ead to 0x02384f (0xa9a3 bytes) 000347 1761246250.473446 fallocate 0x02859d thru 0x02891e (0x382 bytes) 000348 1761246250.481296 mapwrite 0x012b06 thru 0x01eaad (0xbfa8 bytes) 000349 1761246250.516449 mapwrite 0x01d1e6 thru 0x026188 (0x8fa3 bytes) 000351 1761246250.558897 read 0x014671 thru 0x015d69 (0x16f9 bytes) 000352 1761246250.559507 mapread 0x00de92 thru 0x01ccd7 (0xee46 bytes) 000355 1761246250.562590 mapwrite 0x02db78 thru 0x030141 (0x25ca bytes) 000356 1761246250.609964 MIRROR_SPLIT mirror 4 to 8 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000357 1761246250.886972 read 0x0187e6 thru 0x0188b8 (0xd3 bytes) 000359 1761246250.892922 read 0x00c30e thru 0x018599 (0xc28c bytes) 000360 1761246250.904292 punch from 0x004ebe to 0x00e9eb (0x9b2e bytes) 000361 1761246250.913384 MIRROR_SPLIT mirror 1 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000362 1761246251.143407 read 0x028bc2 thru 0x030141 (0x7580 bytes) 000363 1761246251.159052 mapread 0x02263f thru 0x029b44 (0x7506 bytes) 000364 1761246251.208050 read 0x009250 thru 0x0102a3 (0x7054 bytes) 000365 1761246251.224696 mapwrite 0x032b65 thru 0x0343ed (0x1889 bytes) 000367 1761246251.275152 mapwrite 0x01c888 thru 0x026413 (0x9b8c bytes) 000368 1761246251.302161 fallocate 0x02fbcb thru 0x039a53 (0x9e89 bytes) 000370 1761246251.312124 mapwrite 0x005f19 thru 0x0145fa (0xe6e2 bytes) 000371 1761246251.349644 trunc from 0x0343ee to 0x00c86e (0x27b7f bytes) 000372 1761246251.372725 trunc from 0x00c86f to 0x0069ac (0x5ec2 bytes) 000373 1761246251.398532 read 0x003a3e thru 0x0069ac (0x2f6f bytes) 000375 1761246251.400962 fallocate 0x00febc thru 0x01794f (0x7a94 bytes) 000376 1761246251.407034 trunc from 0x017950 to 0x0337a1 (0x1be52 bytes) 000377 1761246251.425641 punch from 0x020c04 to 0x026196 (0x5593 bytes) 000378 1761246251.430890 fallocate 0x02eb14 thru 0x02ed89 (0x276 bytes) 000379 1761246251.438560 read 0x002d1f thru 0x011c71 (0xef53 bytes) 000380 1761246251.451544 punch from 0x00ac1f to 0x00fb50 (0x4f32 bytes) 000381 1761246251.458853 trunc from 0x0337a2 to 0x01943b (0x1a366 bytes) 000382 1761246251.478934 punch from 0x004d73 to 0x0087db (0x3a69 bytes) 000383 1761246251.483527 fallocate 0x02e558 thru 0x03d8e3 (0xf38c bytes) 000384 1761246251.491309 read 0x02c77e thru 0x039730 (0xcfb3 bytes) 000385 1761246251.501526 fallocate 0x02a10e thru 0x02daa7 (0x399a bytes) 000386 1761246251.506991 read 0x037071 thru 0x03d8e3 (0x6873 bytes) 000387 1761246251.507520 fallocate 0x00b09b thru 0x01291e (0x7884 bytes) 000388 1761246251.513778 trunc from 0x03d8e4 to 0x00cdf8 (0x30aeb bytes) 000390 1761246251.531638 trunc from 0x00cdf9 to 0x011d58 (0x4f60 bytes) 000391 1761246251.550275 mapwrite 0x02888c thru 0x0345fb (0xbd70 bytes) 000392 1761246251.622252 fallocate 0x03b177 thru 0x03ffff (0x4e89 bytes) 000394 1761246251.636853 fallocate 0x03c2ee thru 0x03ffff (0x3d12 bytes) 000395 1761246251.648125 trunc from 0x040000 to 0x03f860 (0x79f bytes) 000397 1761246251.671328 mapwrite 0x03d052 thru 0x03ffff (0x2fae bytes) 000398 1761246251.726921 trunc from 0x040000 to 0x023b8f (0x1c470 bytes) 000399 1761246251.750030 MIRROR_EXTEND to 8 mirrors 000400 1761246252.707516 trunc from 0x01221a to 0x01449d (0x2284 bytes) 000401 1761246252.757927 fallocate 0x00ae46 thru 0x011388 (0x6543 bytes) 000402 1761246252.774394 mapwrite 0x025c3a thru 0x03016d (0xa534 bytes) 000403 1761246252.868186 trunc from 0x03016e to 0x00d7f0 (0x2297d bytes) 000404 1761246252.887262 write 0x00dbe6 thru 0x0140d5 (0x64f0 bytes) 000405 1761246252.897551 punch from 0x002663 to 0x00d4f8 (0xae96 bytes) 000406 1761246252.910995 trunc from 0x0140d6 to 0x006349 (0xdd8c bytes) 000407 1761246252.930071 mapwrite 0x020f97 thru 0x02b35e (0xa3c8 bytes) 000408 1761246253.039138 write 0x02ef50 thru 0x03dc09 (0xecba bytes) 000409 1761246253.042722 write 0x0303a3 thru 0x03e7e2 (0xe440 bytes) 000410 1761246253.048483 trunc from 0x03e7e3 to 0x02f799 (0xf049 bytes) 000411 1761246253.080313 trunc from 0x02f79a to 0x01cf49 (0x12850 bytes) 000413 1761246253.124919 MIRROR_RESYNC 8 mirrors 000414 1761246254.851931 trunc from 0x002e40 to 0x011119 (0xe2da bytes) 000416 1761246254.885321 mapread 0x010070 thru 0x0110d1 (0x1062 bytes) 000418 1761246254.902009 mapwrite 0x03a57b thru 0x03ffff (0x5a85 bytes) 000419 1761246254.961641 fallocate 0x032721 thru 0x038d23 (0x6603 bytes) 000420 1761246254.969424 trunc from 0x03e324 to 0x03ffff (0x1cdc bytes) 000421 1761246254.974904 trunc from 0x03e35b to 0x03f29f (0xf45 bytes) 000422 1761246254.980121 fallocate 0x03ff16 thru 0x03ffff (0xea bytes) 000423 1761246254.993967 trunc from 0x03956b to 0x03e5cb (0x5061 bytes) 000424 1761246255.002480 punch from 0x0297ea to 0x0343d7 (0xabee bytes) 000425 1761246255.008766 fallocate 0x01e004 thru 0x02140d (0x340a bytes) 000427 1761246255.019934 trunc from 0x00894c to 0x009105 (0x7ba bytes) 000428 1761246255.027328 trunc from 0x040000 to 0x02ede1 (0x1121e bytes) 000429 1761246255.050442 mapwrite 0x008e08 thru 0x01337e (0xa577 bytes) 000430 1761246255.087997 write 0x03af51 thru 0x03c958 (0x1a08 bytes) 000431 1761246255.089261 read 0x00dc64 thru 0x00df70 (0x30d bytes) 000432 1761246255.089887 mapwrite 0x008d07 thru 0x00f542 (0x683c bytes) 000433 1761246255.115948 punch from 0x001aa4 to 0x00d1cc (0xb729 bytes) 000436 1761246255.127984 mapread 0x02b4eb thru 0x039bc2 (0xe6d8 bytes) 000437 1761246255.147880 trunc from 0x03c959 to 0x032dd5 (0x9b83 bytes) 000438 1761246255.166449 trunc from 0x009f47 to 0x018f65 (0xf01f bytes) 000439 1761246255.173515 trunc from 0x032dd6 to 0x038075 (0x52a0 bytes) 000440 1761246255.194558 read 0x007f3f thru 0x00dcff (0x5dc1 bytes) 000441 1761246255.204220 write 0x0002de thru 0x0037dd (0x3500 bytes) 000442 1761246255.205356 trunc from 0x038076 to 0x03ac69 (0x2bf4 bytes) 000444 1761246255.235952 MIRROR_SPLIT mirror 7 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000446 1761246255.528406 trunc from 0x017199 to 0x02649a (0xf302 bytes) 000447 1761246255.535379 read 0x035b52 thru 0x037cab (0x215a bytes) 000449 1761246255.546916 MIRROR_SPLIT mirror 2 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000451 1761246255.842249 mapread 0x028dbf thru 0x035cad (0xceef bytes) 000452 1761246255.877188 mapwrite 0x02aee4 thru 0x0362e2 (0xb3ff bytes) 000453 1761246255.914589 trunc from 0x03ac6a to 0x03e7b0 (0x3b47 bytes) 000454 1761246255.953339 write 0x0047b0 thru 0x00b78d (0x6fde bytes) 000455 1761246255.970723 mapwrite 0x0003a8 thru 0x010338 (0xff91 bytes) 000456 1761246256.000292 punch from 0x01041f to 0x0148fe (0x44e0 bytes) 000457 1761246256.006417 mapread 0x012ed7 thru 0x016619 (0x3743 bytes) 000458 1761246256.017130 read 0x02b7a0 thru 0x0331e1 (0x7a42 bytes) 000459 1761246256.022681 MIRROR_EXTEND to 7 mirrors 000462 1761246256.892092 mapread 0x022913 thru 0x02de1f (0xb50d bytes) 000463 1761246256.918448 mapread 0x01be48 thru 0x02aa10 (0xebc9 bytes) 000464 1761246256.930824 trunc from 0x03e7b1 to 0x0209e8 (0x1ddc8 bytes) 000465 1761246257.001937 punch from 0x009bbc to 0x012360 (0x87a5 bytes) 000466 1761246257.009237 trunc from 0x0209e9 to 0x009258 (0x17790 bytes) 000467 1761246257.046236 fallocate 0x005715 thru 0x00da05 (0x82f1 bytes) 000468 1761246257.056469 trunc from 0x00c675 to 0x00da05 (0x1391 bytes) 000469 1761246257.062334 mapread 0x000208 thru 0x003183 (0x2f7c bytes) 000470 1761246257.081773 punch from 0x00b7cc to 0x00da05 (0x223a bytes) 000471 1761246257.090930 read 0x008f03 thru 0x00da05 (0x4b03 bytes) 000472 1761246257.108284 MIRROR_SPLIT mirror 2 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000473 1761246257.305075 mapwrite 0x03ca6c thru 0x03ffff (0x3594 bytes) 000475 1761246257.364163 write 0x025f22 thru 0x029d5b (0x3e3a bytes) 000476 1761246257.372943 read 0x036c2d thru 0x03ffff (0x93d3 bytes) 000477 1761246257.379014 mapwrite 0x012b07 thru 0x01ecc0 (0xc1ba bytes) 000478 1761246257.413657 mapread 0x033443 thru 0x033df7 (0x9b5 bytes) 000479 1761246257.423766 mapread 0x01f14f thru 0x02946c (0xa31e bytes) 000480 1761246257.424583 mapwrite 0x02c12d thru 0x02e392 (0x2266 bytes) 000481 1761246257.441662 trunc from 0x001262 to 0x00a699 (0x9438 bytes) 000482 1761246257.446336 mapread 0x0300f4 thru 0x039327 (0x9234 bytes) 000484 1761246257.448858 mapwrite 0x01318b thru 0x01f71d (0xc593 bytes) 000485 1761246257.468192 mapwrite 0x03c47a thru 0x03ffff (0x3b86 bytes) 000486 1761246257.483071 read 0x02098d thru 0x02660c (0x5c80 bytes) 000487 1761246257.483621 trunc from 0x01e003 to 0x0295f7 (0xb5f5 bytes) 000488 1761246257.491069 mapread 0x030983 thru 0x036c20 (0x629e bytes) 000489 1761246257.492322 write 0x01ad51 thru 0x027b4e (0xcdfe bytes) 000490 1761246257.504199 trunc from 0x040000 to 0x0234e9 (0x1cb16 bytes) 000491 1761246257.529264 write 0x000407 thru 0x007ef7 (0x7af1 bytes) 000492 1761246257.530386 trunc from 0x0234ea to 0x025deb (0x2902 bytes) 000493 1761246257.553455 read 0x008d36 thru 0x00a7f9 (0x1ac4 bytes) 000494 1761246257.554517 fallocate 0x00c7f4 thru 0x00e767 (0x1f74 bytes) 000495 1761246257.561355 trunc from 0x015b7c to 0x0165d5 (0xa5a bytes) 000496 1761246257.566438 mapwrite 0x01c958 thru 0x022113 (0x57bc bytes) 000498 1761246257.584853 fallocate 0x0318d3 thru 0x03c74c (0xae7a bytes) 000499 1761246257.597609 MIRROR_RESYNC 6 mirrors 000500 1761246259.017743 trunc from 0x03c74d to 0x03ec33 (0x24e7 bytes) 000501 1761246259.070403 trunc from 0x03ec34 to 0x003873 (0x3b3c0 bytes) 000502 1761246259.102921 trunc from 0x001872 to 0x003873 (0x2002 bytes) 000503 1761246259.108292 mapwrite 0x00cec8 thru 0x013dc8 (0x6f01 bytes) 000504 1761246259.158747 fallocate 0x01ddee thru 0x025dfe (0x8011 bytes) 000506 1761246259.171716 trunc from 0x013dc9 to 0x011e73 (0x1f55 bytes) 000508 1761246259.208222 fallocate 0x0201eb thru 0x02ca46 (0xc85c bytes) 000510 1761246259.214352 punch from 0x0140e3 to 0x014619 (0x537 bytes) 000511 1761246259.223359 read 0x020344 thru 0x025855 (0x5512 bytes) 000513 1761246259.231492 punch from 0x01c622 to 0x025146 (0x8b25 bytes) 000514 1761246259.239970 mapwrite 0x01cc3a thru 0x01e9c8 (0x1d8f bytes) 000515 1761246259.272187 punch from 0x026030 to 0x02ca46 (0x6a17 bytes) 000516 1761246259.277948 mapread 0x008fb6 thru 0x00bd0f (0x2d5a bytes) 000517 1761246259.279423 trunc from 0x016218 to 0x020627 (0xa410 bytes) 000518 1761246259.284588 punch from 0x004ead to 0x00f667 (0xa7bb bytes) 000519 1761246259.292329 trunc from 0x005977 to 0x00eb9d (0x9227 bytes) 000521 1761246259.299030 mapread 0x013396 thru 0x021036 (0xdca1 bytes) 000522 1761246259.325003 trunc from 0x02ca47 to 0x031536 (0x4af0 bytes) 000523 1761246259.347591 read 0x017bf7 thru 0x0196bb (0x1ac5 bytes) 000524 1761246259.348283 fallocate 0x01fe43 thru 0x0292ed (0x94ab bytes) 000526 1761246259.357197 trunc from 0x0149e5 to 0x0224c9 (0xdae5 bytes) 000527 1761246259.364295 write 0x03b602 thru 0x03ffff (0x49fe bytes) 000528 1761246259.366322 trunc from 0x0198d3 to 0x0244a3 (0xabd1 bytes) 000529 1761246259.373215 write 0x029a51 thru 0x033f6e (0xa51e bytes) 000531 1761246259.391448 read 0x0132ab thru 0x01b8c6 (0x861c bytes) 000532 1761246259.399351 MIRROR_RESYNC 6 mirrors 000533 1761246261.138426 trunc from 0x01be8b to 0x024cbf (0x8e35 bytes) 000534 1761246261.177782 read 0x0018f0 thru 0x002c05 (0x1316 bytes) 000535 1761246261.187106 mapwrite 0x02420d thru 0x02c779 (0x856d bytes) 000536 1761246261.258444 mapwrite 0x00d2ff thru 0x0145cc (0x72ce bytes) 000537 1761246261.310448 trunc from 0x03fa0d to 0x03ffff (0x5f3 bytes) 000538 1761246261.319372 fallocate 0x02d1bc thru 0x02e03a (0xe7f bytes) 000539 1761246261.329711 trunc from 0x040000 to 0x034c8f (0xb370 bytes) 000540 1761246261.364613 fallocate 0x0035c5 thru 0x0100ec (0xcb28 bytes) 000541 1761246261.380198 fallocate 0x03ef67 thru 0x03ffff (0x1099 bytes) 000542 1761246261.388640 fallocate 0x00a26f thru 0x014701 (0xa493 bytes) 000543 1761246261.394830 read 0x014ce3 thru 0x021484 (0xc7a2 bytes) 000544 1761246261.408119 read 0x02f75d thru 0x038663 (0x8f07 bytes) 000545 1761246261.408750 mapread 0x02488c thru 0x032ae0 (0xe255 bytes) 000546 1761246261.409647 trunc from 0x013abf to 0x021789 (0xdccb bytes) 000547 1761246261.415348 mapread 0x018311 thru 0x023d5f (0xba4f bytes) 000548 1761246261.429079 fallocate 0x0258c1 thru 0x02604a (0x78a bytes) 000549 1761246261.434555 mapwrite 0x007c19 thru 0x00967f (0x1a67 bytes) 000550 1761246261.454676 mapread 0x00be5b thru 0x017f5f (0xc105 bytes) 000551 1761246261.455701 mapread 0x01029e thru 0x01e61b (0xe37e bytes) 000553 1761246261.458672 write 0x0033e7 thru 0x004844 (0x145e bytes) 000554 1761246261.460310 punch from 0x020ca0 to 0x027abc (0x6e1d bytes) 000555 1761246261.466240 write 0x0070b5 thru 0x00ba6e (0x49ba bytes) 000556 1761246261.467291 read 0x03d1ee thru 0x03ffff (0x2e12 bytes) 000557 1761246261.467853 mapwrite 0x005555 thru 0x006ba1 (0x164d bytes) 000558 1761246261.492697 read 0x019ff7 thru 0x0223e1 (0x83eb bytes) 000559 1761246261.511969 fallocate 0x007d5a thru 0x010c10 (0x8eb7 bytes) 000560 1761246261.523155 MIRROR_SPLIT mirror 3 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000561 1761246261.777251 mapwrite 0x000246 thru 0x00e1c6 (0xdf81 bytes) 000562 1761246261.838210 MIRROR_RESYNC 5 mirrors 000563 1761246262.975702 MIRROR_RESYNC 5 mirrors 000564 1761246263.200176 mapread 0x033030 thru 0x039d14 (0x6ce5 bytes) 000565 1761246263.231479 mapread 0x0038b9 thru 0x00689d (0x2fe5 bytes) 000566 1761246263.245584 fallocate 0x03e4de thru 0x03ffff (0x1b22 bytes) 000567 1761246263.275516 mapread 0x01e209 thru 0x01f973 (0x176b bytes) 000568 1761246263.283176 mapread 0x0040e1 thru 0x007b51 (0x3a71 bytes) 000569 1761246263.291892 punch from 0x03c707 to 0x03ffff (0x38f9 bytes) 000570 1761246263.301269 MIRROR_RESYNC 5 mirrors 000571 1761246264.372554 punch from 0x00802b to 0x013894 (0xb86a bytes) 000572 1761246264.434187 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000573 1761246264.714213 trunc from 0x00a616 to 0x01a23d (0xfc28 bytes) 000574 1761246264.722161 read 0x023608 thru 0x033118 (0xfb11 bytes) 000575 1761246264.734663 mapwrite 0x00efc3 thru 0x015507 (0x6545 bytes) 000576 1761246264.781134 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000577 1761246265.006083 trunc from 0x040000 to 0x03197b (0xe684 bytes) 000578 1761246265.029270 trunc from 0x00c22d to 0x011b16 (0x58ea bytes) 000579 1761246265.035955 write 0x007745 thru 0x0164cb (0xed87 bytes) 000580 1761246265.052276 MIRROR_RESYNC 3 mirrors 000581 1761246265.880641 fallocate 0x007e3d thru 0x00cad2 (0x4c96 bytes) 000582 1761246265.937705 trunc from 0x03197c to 0x017857 (0x1a124 bytes) 000583 1761246265.973629 fallocate 0x03bf04 thru 0x03ffff (0x40fc bytes) 000584 1761246265.981930 trunc from 0x040000 to 0x01fa27 (0x205d8 bytes) 000585 1761246265.996924 punch from 0x019818 to 0x01f561 (0x5d4a bytes) 000586 1761246266.002862 trunc from 0x011c2b to 0x015985 (0x3d5b bytes) 000587 1761246266.009336 punch from 0x009444 to 0x00cb11 (0x36ce bytes) 000588 1761246266.016122 mapwrite 0x03b2a9 thru 0x03ffff (0x4d57 bytes) 000590 1761246266.069494 mapread 0x02d511 thru 0x02ee80 (0x1970 bytes) 000591 1761246266.080387 write 0x02d997 thru 0x03a70b (0xcd75 bytes) 000592 1761246266.081901 mapwrite 0x004d6b thru 0x009a8f (0x4d25 bytes) 000593 1761246266.125560 mapwrite 0x0035b7 thru 0x006057 (0x2aa1 bytes) 000594 1761246266.142320 read 0x025a14 thru 0x03376c (0xdd59 bytes) 000595 1761246266.149018 mapread 0x01e958 thru 0x0209f9 (0x20a2 bytes) 000597 1761246266.157428 fallocate 0x0170c3 thru 0x01f544 (0x8482 bytes) 000599 1761246266.174997 MIRROR_EXTEND to 4 mirrors 000600 1761246266.929403 trunc from 0x013d76 to 0x015d44 (0x1fcf bytes) 000601 1761246266.979012 mapwrite 0x01a4ab thru 0x027de7 (0xd93d bytes) 000602 1761246267.074289 write 0x02b62d thru 0x02ed97 (0x376b bytes) 000604 1761246267.098640 punch from 0x0123cb to 0x020ac1 (0xe6f7 bytes) 000605 1761246267.113839 MIRROR_EXTEND to 5 mirrors 000608 1761246267.895313 read 0x0039cb thru 0x0083ee (0x4a24 bytes) 000609 1761246267.909660 punch from 0x0235ed to 0x02a01d (0x6a31 bytes) 000610 1761246267.949267 punch from 0x0325d5 to 0x03a60d (0x8039 bytes) 000611 1761246267.956032 trunc from 0x038488 to 0x03cd53 (0x48cc bytes) 000612 1761246267.967687 MIRROR_EXTEND to 6 mirrors 000613 1761246268.851369 punch from 0x01aa87 to 0x01d764 (0x2cde bytes) 000614 1761246268.888256 punch from 0x0099d1 to 0x013073 (0x96a3 bytes) 000615 1761246268.894467 fallocate 0x019380 thru 0x019810 (0x491 bytes) 000616 1761246268.907215 write 0x01b283 thru 0x01bbd9 (0x957 bytes) 000617 1761246268.920681 trunc from 0x040000 to 0x037f6e (0x8091 bytes) 000618 1761246268.954252 read 0x00168e thru 0x010b96 (0xf509 bytes) 000619 1761246268.967727 trunc from 0x00e68f to 0x01aaeb (0xc45d bytes) 000622 1761246268.980455 read 0x00b18d thru 0x010d2f (0x5ba3 bytes) 000623 1761246268.994978 fallocate 0x03d3e8 thru 0x03d5f1 (0x20a bytes) 000624 1761246269.002849 fallocate 0x006967 thru 0x012201 (0xb89b bytes) 000626 1761246269.009331 read 0x00d5ad thru 0x01a1e5 (0xcc39 bytes) 000627 1761246269.010016 trunc from 0x01b785 to 0x0240d4 (0x8950 bytes) 000628 1761246269.025273 fallocate 0x00a679 thru 0x0149f3 (0xa37b bytes) 000629 1761246269.030928 mapwrite 0x018e16 thru 0x01cc61 (0x3e4c bytes) 000630 1761246269.079980 trunc from 0x01d852 to 0x022452 (0x4c01 bytes) 000631 1761246269.105119 MIRROR_SPLIT mirror 1 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000632 1761246269.434567 mapread 0x030612 thru 0x03ba89 (0xb478 bytes) 000633 1761246269.465014 mapwrite 0x02c8d2 thru 0x03b7b7 (0xeee6 bytes) 000634 1761246269.507303 trunc from 0x003006 to 0x00c6ba (0x96b5 bytes) 000635 1761246269.514081 write 0x01516b thru 0x01cdd4 (0x7c6a bytes) 000636 1761246269.540809 mapwrite 0x01df52 thru 0x02b062 (0xd111 bytes) 000637 1761246269.591304 trunc from 0x03d5f2 to 0x015578 (0x28079 bytes) 000638 1761246269.609586 write 0x006be5 thru 0x010629 (0x9a45 bytes) 000639 1761246269.625091 trunc from 0x015579 to 0x03d2e7 (0x27d6f bytes) 000640 1761246269.653188 MIRROR_EXTEND to 6 mirrors 000641 1761246270.314130 fallocate 0x01e81f thru 0x0296fb (0xaedd bytes) 000643 1761246270.351401 mapread 0x0275aa thru 0x032c4c (0xb6a3 bytes) 000644 1761246270.380016 write 0x004c07 thru 0x0087cd (0x3bc7 bytes) 000645 1761246270.395445 write 0x010d5e thru 0x01cb0e (0xbdb1 bytes) 000646 1761246270.401608 mapread 0x0199aa thru 0x021b53 (0x81aa bytes) 000647 1761246270.412113 write 0x0029d0 thru 0x006a8c (0x40bd bytes) 000648 1761246270.413326 mapread 0x0191d7 thru 0x0291ad (0xffd7 bytes) 000649 1761246270.414880 mapread 0x006f64 thru 0x01364b (0xc6e8 bytes) 000650 1761246270.417008 fallocate 0x02909b thru 0x035725 (0xc68b bytes) 000651 1761246270.421773 trunc from 0x001097 to 0x010a54 (0xf9be bytes) 000653 1761246270.439302 read 0x03bb3d thru 0x03d2e7 (0x17ab bytes) 000655 1761246270.446172 write 0x008567 thru 0x00bbb2 (0x364c bytes) 000657 1761246270.456444 mapread 0x03b408 thru 0x03d2e7 (0x1ee0 bytes) 000658 1761246270.462074 MIRROR_SPLIT mirror 1 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000659 1761246270.729831 mapread 0x017247 thru 0x01e946 (0x7700 bytes) 000660 1761246270.746607 read 0x0236de thru 0x0239cc (0x2ef bytes) 000661 1761246270.753217 mapread 0x002d59 thru 0x003a74 (0xd1c bytes) 000663 1761246270.762568 fallocate 0x024a0c thru 0x028d2b (0x4320 bytes) 000664 1761246270.767889 punch from 0x01a099 to 0x022169 (0x80d1 bytes) 000665 1761246270.774186 fallocate 0x02f54d thru 0x03b752 (0xc206 bytes) 000666 1761246270.780582 write 0x02b753 thru 0x03ac59 (0xf507 bytes) 000667 1761246270.805682 mapread 0x0065b4 thru 0x00d3f2 (0x6e3f bytes) 000668 1761246270.818245 MIRROR_SPLIT mirror 2 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000669 1761246271.116187 mapread 0x028dc9 thru 0x036587 (0xd7bf bytes) 000670 1761246271.172266 fallocate 0x005b50 thru 0x0139cf (0xde80 bytes) 000671 1761246271.188234 mapread 0x000b32 thru 0x00a4c7 (0x9996 bytes) 000672 1761246271.211502 MIRROR_EXTEND to 5 mirrors 000673 1761246272.039025 MIRROR_SPLIT mirror 4 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000674 1761246272.369192 mapwrite 0x001eef thru 0x011543 (0xf655 bytes) 000675 1761246272.473633 fallocate 0x00a112 thru 0x00f04e (0x4f3d bytes) 000676 1761246272.479744 trunc from 0x03d2e8 to 0x034957 (0x8990 bytes) 000677 1761246272.498302 trunc from 0x02423f to 0x0288a2 (0x4664 bytes) 000678 1761246272.505658 write 0x003ffb thru 0x00e8c9 (0xa8cf bytes) 000679 1761246272.511825 mapread 0x02b244 thru 0x02e9ec (0x37a9 bytes) 000680 1761246272.530963 read 0x031cd5 thru 0x034957 (0x2c83 bytes) 000681 1761246272.541504 fallocate 0x03235b thru 0x039318 (0x6fbe bytes) 000682 1761246272.567557 mapwrite 0x0168ae thru 0x01bc29 (0x537c bytes) 000683 1761246272.641320 write 0x02965b thru 0x035898 (0xc23e bytes) 000684 1761246272.642634 read 0x0268a2 thru 0x02c83b (0x5f9a bytes) 000685 1761246272.643173 mapread 0x01ebaf thru 0x026205 (0x7657 bytes) 000686 1761246272.666077 MIRROR_EXTEND to 5 mirrors 000687 1761246273.347456 fallocate 0x0243da thru 0x032e84 (0xeaab bytes) 000688 1761246273.399003 read 0x01dc14 thru 0x0251d2 (0x75bf bytes) 000689 1761246273.412857 trunc from 0x00f852 to 0x0110cd (0x187c bytes) 000690 1761246273.421549 trunc from 0x035899 to 0x035966 (0xce bytes) 000691 1761246273.474700 fallocate 0x00e047 thru 0x014d0a (0x6cc4 bytes) 000692 1761246273.486356 mapwrite 0x013cea thru 0x01eaeb (0xae02 bytes) 000693 1761246273.547610 fallocate 0x0223c0 thru 0x025b3a (0x377b bytes) 000694 1761246273.553750 punch from 0x01e8c3 to 0x028e6a (0xa5a8 bytes) 000695 1761246273.562944 mapwrite 0x024157 thru 0x02e397 (0xa241 bytes) 000696 1761246273.618605 read 0x0008d8 thru 0x005644 (0x4d6d bytes) 000697 1761246273.628856 write 0x0310b0 thru 0x03a27b (0x91cc bytes) 000698 1761246273.635587 mapwrite 0x00466c thru 0x006b74 (0x2509 bytes) 000699 1761246273.672673 trunc from 0x03a27c to 0x02a21f (0x1005c bytes) 000700 1761246273.701399 trunc from 0x02a220 to 0x004920 (0x258ff bytes) 000701 1761246273.725365 fallocate 0x02d039 thru 0x031c07 (0x4bcf bytes) 000702 1761246273.735771 write 0x02b011 thru 0x02ffc8 (0x4fb8 bytes) 000703 1761246273.737006 mapread 0x02679d thru 0x02ffc8 (0x982c bytes) 000704 1761246273.786582 mapwrite 0x00084e thru 0x00c4ee (0xbca1 bytes) 000705 1761246273.863521 fallocate 0x02e9e4 thru 0x03cb1a (0xe137 bytes) 000706 1761246273.868404 fallocate 0x007bd8 thru 0x016ac0 (0xeee9 bytes) 000707 1761246273.873336 trunc from 0x0244fe to 0x02b8cd (0x73d0 bytes) 000708 1761246273.880265 trunc from 0x027d61 to 0x028324 (0x5c4 bytes) 000709 1761246273.886073 read 0x00a6b8 thru 0x0161bd (0xbb06 bytes) 000710 1761246273.901410 fallocate 0x01f55f thru 0x021818 (0x22ba bytes) 000711 1761246273.908441 trunc from 0x005c71 to 0x011011 (0xb3a1 bytes) 000712 1761246273.915044 trunc from 0x01e43c to 0x0253c4 (0x6f89 bytes) 000713 1761246273.922419 punch from 0x026e38 to 0x02e039 (0x7202 bytes) 000714 1761246273.931638 read 0x0241e8 thru 0x024df9 (0xc12 bytes) 000715 1761246273.953668 punch from 0x01df42 to 0x0243d8 (0x6497 bytes) 000716 1761246273.959203 trunc from 0x02bc17 to 0x02ffc8 (0x43b2 bytes) 000717 1761246273.964234 fallocate 0x0008e3 thru 0x00d90d (0xd02b bytes) 000718 1761246273.968534 mapread 0x01451f thru 0x017eda (0x39bc bytes) 000719 1761246273.970135 trunc from 0x02ffc9 to 0x006b9b (0x2942d bytes) 000720 1761246273.989689 write 0x007f0f thru 0x016768 (0xe85a bytes) 000721 1761246273.992608 fallocate 0x0312c4 thru 0x032623 (0x1360 bytes) 000722 1761246273.996716 trunc from 0x00e81e to 0x016768 (0x7f4b bytes) 000723 1761246274.011984 write 0x003fdc thru 0x00dc0f (0x9c34 bytes) 000724 1761246274.013290 mapread 0x0061ae thru 0x00addd (0x4c30 bytes) 000725 1761246274.014060 punch from 0x00fd62 to 0x016768 (0x6a07 bytes) 000726 1761246274.023434 punch from 0x007525 to 0x0139dd (0xc4b9 bytes) 000728 1761246274.051389 write 0x030710 thru 0x03e546 (0xde37 bytes) 000729 1761246274.053467 write 0x02100a thru 0x02759d (0x6594 bytes) 000730 1761246274.073678 punch from 0x02a163 to 0x033436 (0x92d4 bytes) 000733 1761246274.099854 fallocate 0x0240c5 thru 0x027026 (0x2f62 bytes) 000734 1761246274.112984 mapwrite 0x015f50 thru 0x01e10d (0x81be bytes) 000735 1761246274.245034 mapwrite 0x01f7df thru 0x024e75 (0x5697 bytes) 000736 1761246274.278036 mapwrite 0x00f0cb thru 0x019a18 (0xa94e bytes) 000737 1761246274.349439 punch from 0x005b03 to 0x00e07a (0x8578 bytes) 000738 1761246274.354035 mapread 0x01b268 thru 0x027a93 (0xc82c bytes) 000739 1761246274.355002 read 0x0208ce thru 0x02da20 (0xd153 bytes) 000740 1761246274.366062 fallocate 0x01778c thru 0x01f6bb (0x7f30 bytes) 000741 1761246274.377362 MIRROR_EXTEND to 6 mirrors 000742 1761246275.028214 MIRROR_EXTEND to 7 mirrors 000743 1761246275.756557 read 0x0003ba thru 0x00eabd (0xe704 bytes) 000744 1761246275.773695 fallocate 0x00d555 thru 0x011fcc (0x4a78 bytes) 000745 1761246275.828764 mapread 0x004213 thru 0x009465 (0x5253 bytes) 000746 1761246275.848378 write 0x0366a2 thru 0x03c1b3 (0x5b12 bytes) 000747 1761246275.872175 punch from 0x002390 to 0x00c52b (0xa19c bytes) 000748 1761246275.887627 MIRROR_EXTEND to 8 mirrors 000749 1761246276.789164 mapread 0x011491 thru 0x01567b (0x41eb bytes) 000750 1761246276.832026 MIRROR_RESYNC 8 mirrors 000751 1761246278.586801 fallocate 0x01a73b thru 0x01c214 (0x1ada bytes) 000752 1761246278.642203 fallocate 0x010d84 thru 0x020cff (0xff7c bytes) 000753 1761246278.655834 punch from 0x00f161 to 0x014d89 (0x5c29 bytes) 000754 1761246278.666771 write 0x00c868 thru 0x014f91 (0x872a bytes) 000755 1761246278.684610 mapread 0x034998 thru 0x03a600 (0x5c69 bytes) 000756 1761246278.712625 mapread 0x038b98 thru 0x03e546 (0x59af bytes) 000757 1761246278.733681 write 0x024b1d thru 0x02f9ee (0xaed2 bytes) 000758 1761246278.763447 mapwrite 0x02919e thru 0x0374d1 (0xe334 bytes) 000760 1761246278.812654 mapread 0x02ce33 thru 0x0389f9 (0xbbc7 bytes) 000762 1761246278.819710 MIRROR_RESYNC 8 mirrors 000763 1761246280.225416 MIRROR_EXTEND to 9 mirrors 000764 1761246281.174572 write 0x03da31 thru 0x03ffff (0x25cf bytes) 000766 1761246281.236444 MIRROR_RESYNC 9 mirrors 000768 1761246282.706554 write 0x034efb thru 0x03dc49 (0x8d4f bytes) 000769 1761246282.767055 fallocate 0x0154ee thru 0x0169a6 (0x14b9 bytes) 000770 1761246282.777673 mapread 0x02449b thru 0x02633a (0x1ea0 bytes) 000772 1761246282.802374 fallocate 0x0035c5 thru 0x00fcd1 (0xc70d bytes) 000773 1761246282.810667 fallocate 0x037870 thru 0x03ffff (0x8790 bytes) 000774 1761246282.819884 mapread 0x018fca thru 0x02406f (0xb0a6 bytes) 000775 1761246282.835564 mapwrite 0x03ab59 thru 0x03e199 (0x3641 bytes) 000776 1761246282.881542 trunc from 0x03c682 to 0x03ffff (0x397e bytes) 000777 1761246282.887424 write 0x03b6be thru 0x03be1e (0x761 bytes) 000778 1761246282.889523 trunc from 0x02ba41 to 0x033d30 (0x82f0 bytes) 000779 1761246282.896044 read 0x012f3e thru 0x01bb08 (0x8bcb bytes) 000780 1761246282.896659 write 0x0222ce thru 0x024469 (0x219c bytes) 000781 1761246282.897513 read 0x028f1c thru 0x0313e7 (0x84cc bytes) 000782 1761246282.913547 mapwrite 0x02c7c2 thru 0x0327e0 (0x601f bytes) 000783 1761246282.947175 trunc from 0x040000 to 0x025828 (0x1a7d7 bytes) 000784 1761246282.988120 mapwrite 0x03e6e0 thru 0x03ffff (0x1920 bytes) 000785 1761246283.065758 fallocate 0x0157bb thru 0x016c72 (0x14b8 bytes) 000786 1761246283.077190 fallocate 0x0361ce thru 0x03ffff (0x9e32 bytes) 000787 1761246283.092583 mapread 0x02eef0 thru 0x03b5ca (0xc6db bytes) 000788 1761246283.202882 trunc from 0x004565 to 0x011135 (0xcbd1 bytes) 000789 1761246283.226130 MIRROR_EXTEND to 10 mirrors 000790 1761246283.952931 mapread 0x00277f thru 0x00de62 (0xb6e4 bytes) 000792 1761246284.025346 punch from 0x03abd6 to 0x03af59 (0x384 bytes) 000793 1761246284.107170 read 0x01ff33 thru 0x021a1c (0x1aea bytes) 000794 1761246284.131621 write 0x02dc07 thru 0x02e1de (0x5d8 bytes) 000795 1761246284.156634 trunc from 0x040000 to 0x006a48 (0x395b7 bytes) 000796 1761246284.204182 write 0x0193aa thru 0x01e429 (0x5080 bytes) 000797 1761246284.206682 trunc from 0x00af0d to 0x00b561 (0x655 bytes) 000798 1761246284.211909 trunc from 0x01e42a to 0x00d513 (0x10f16 bytes) 000799 1761246284.230474 trunc from 0x00d514 to 0x0352f3 (0x27de0 bytes) 000800 1761246284.270024 MIRROR_RESYNC 10 mirrors 000801 1761246286.004180 trunc from 0x0352f4 to 0x02e058 (0x729b bytes) 000802 1761246286.058723 fallocate 0x027efd thru 0x0363d8 (0xe4dc bytes) 000803 1761246286.072500 MIRROR_SPLIT mirror 7 to 9 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000804 1761246286.315758 punch from 0x0147c6 to 0x02154b (0xcd86 bytes) 000805 1761246286.324760 read 0x0192fb thru 0x019b01 (0x807 bytes) 000806 1761246286.340266 trunc from 0x02e059 to 0x023496 (0xabc2 bytes) 000807 1761246286.383598 read 0x010744 thru 0x01c7d1 (0xc08e bytes) 000808 1761246286.401256 MIRROR_EXTEND to 10 mirrors 000809 1761246287.034503 write 0x014527 thru 0x01e667 (0xa141 bytes) 000810 1761246287.089384 fallocate 0x03324f thru 0x033737 (0x4e9 bytes) 000811 1761246287.095401 write 0x00e336 thru 0x01a1e7 (0xbeb2 bytes) 000812 1761246287.106574 write 0x00f953 thru 0x017d02 (0x83b0 bytes) 000813 1761246287.107848 trunc from 0x0328ca to 0x033737 (0xe6e bytes) 000814 1761246287.125131 MIRROR_SPLIT mirror 9 to 9 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000815 1761246287.429437 MIRROR_RESYNC 9 mirrors 000816 1761246289.542154 fallocate 0x03f664 thru 0x03ffff (0x99c bytes) 000817 1761246289.615222 fallocate 0x035bcc thru 0x03ffff (0xa434 bytes) 000818 1761246289.628230 mapwrite 0x031908 thru 0x03ffff (0xe6f8 bytes) 000819 1761246289.717432 mapwrite 0x033f98 thru 0x034b2f (0xb98 bytes) 000820 1761246289.737295 punch from 0x022769 to 0x02eff3 (0xc88b bytes) 000822 1761246289.748583 fallocate 0x03e0ac thru 0x03ffff (0x1f54 bytes) 000823 1761246289.770951 MIRROR_RESYNC 9 mirrors 000824 1761246291.682013 mapread 0x02d132 thru 0x034f0d (0x7ddc bytes) 000826 1761246291.706937 mapread 0x00b516 thru 0x010cec (0x57d7 bytes) 000827 1761246291.713423 trunc from 0x040000 to 0x01eac2 (0x2153d bytes) 000828 1761246291.758895 read 0x00dcf6 thru 0x01ccc2 (0xefcd bytes) 000829 1761246291.770612 mapread 0x00346e thru 0x012cfb (0xf88e bytes) 000830 1761246291.780679 mapread 0x00fd92 thru 0x01d174 (0xd3e3 bytes) 000831 1761246291.787694 mapwrite 0x00a3e4 thru 0x00ce47 (0x2a64 bytes) 000832 1761246291.801872 trunc from 0x01eac3 to 0x01e588 (0x53a bytes) 000833 1761246291.840446 mapread 0x01e4bf thru 0x01e588 (0xca bytes) 000835 1761246291.842043 mapread 0x017067 thru 0x01cb5e (0x5af8 bytes) 000836 1761246291.843948 write 0x00f464 thru 0x01316f (0x3d0c bytes) 000837 1761246291.846224 trunc from 0x016e0f to 0x01e588 (0x777a bytes) 000838 1761246291.858270 read 0x0095a8 thru 0x012e62 (0x98bb bytes) 000839 1761246291.864247 trunc from 0x01e589 to 0x01e2b2 (0x2d6 bytes) 000840 1761246291.896202 fallocate 0x0359a5 thru 0x03ffff (0xa65b bytes) 000841 1761246291.905868 mapread 0x00ee7c thru 0x01d688 (0xe80d bytes) 000842 1761246291.923404 mapread 0x009dba thru 0x00c685 (0x28cc bytes) 000843 1761246291.928302 fallocate 0x00f105 thru 0x01a788 (0xb684 bytes) 000844 1761246291.936080 read 0x00c927 thru 0x0194c1 (0xcb9b bytes) 000845 1761246291.937524 write 0x02291d thru 0x02f7e7 (0xcecb bytes) 000846 1761246291.939446 fallocate 0x031b83 thru 0x035d20 (0x419e bytes) 000847 1761246291.949666 trunc from 0x0112f0 to 0x012df2 (0x1b03 bytes) 000848 1761246291.969716 write 0x027e90 thru 0x03508e (0xd1ff bytes) 000849 1761246291.972603 mapwrite 0x02dfcd thru 0x03a64b (0xc67f bytes) 000850 1761246292.001038 trunc from 0x025f23 to 0x02754c (0x162a bytes) 000851 1761246292.010638 trunc from 0x02eef0 to 0x0343b1 (0x54c2 bytes) 000852 1761246292.023103 MIRROR_EXTEND to 10 mirrors 000853 1761246292.970145 read 0x0002e8 thru 0x008473 (0x818c bytes) 000855 1761246293.006759 trunc from 0x040000 to 0x0072b8 (0x38d47 bytes) 000856 1761246293.071065 read 0x006842 thru 0x00704a (0x809 bytes) 000857 1761246293.090852 trunc from 0x00169c to 0x004c02 (0x3567 bytes) 000858 1761246293.098649 read 0x004b09 thru 0x0072b8 (0x27b0 bytes) 000859 1761246293.105171 mapwrite 0x0122fd thru 0x01dfd5 (0xbcd9 bytes) 000860 1761246293.187105 mapwrite 0x03c50c thru 0x03ffff (0x3af4 bytes) 000861 1761246293.255613 mapwrite 0x0223ae thru 0x02e44f (0xc0a2 bytes) 000862 1761246293.281453 trunc from 0x01195d to 0x01f07f (0xd723 bytes) 000863 1761246293.287200 write 0x003395 thru 0x004bc0 (0x182c bytes) 000864 1761246293.288215 mapwrite 0x02c8cf thru 0x0356e3 (0x8e15 bytes) 000865 1761246293.306425 mapwrite 0x034820 thru 0x03ffff (0xb7e0 bytes) 000867 1761246293.324940 punch from 0x004f14 to 0x0062b9 (0x13a6 bytes) 000868 1761246293.339390 MIRROR_EXTEND to 11 mirrors 000869 1761246294.215219 trunc from 0x040000 to 0x0101b9 (0x2fe46 bytes) 000870 1761246294.325351 read 0x0016c2 thru 0x0042c5 (0x2c04 bytes) 000871 1761246294.333677 fallocate 0x01bc3c thru 0x0247d9 (0x8b9e bytes) 000872 1761246294.338514 trunc from 0x0247da to 0x01217f (0x1265a bytes) 000873 1761246294.360563 mapwrite 0x0397e7 thru 0x03e751 (0x4f6b bytes) 000874 1761246294.422489 fallocate 0x03487a thru 0x03edd5 (0xa55c bytes) 000875 1761246294.445468 MIRROR_RESYNC 11 mirrors 000876 1761246297.156835 punch from 0x03a39f to 0x03e751 (0x43b3 bytes) 000877 1761246297.202356 mapread 0x03c4ad thru 0x03e751 (0x22a5 bytes) 000878 1761246297.229435 mapread 0x016436 thru 0x025e43 (0xfa0e bytes) 000879 1761246297.254078 punch from 0x03b8e4 to 0x03bb5c (0x279 bytes) 000880 1761246297.260570 punch from 0x0102a6 to 0x010698 (0x3f3 bytes) 000881 1761246297.270676 trunc from 0x0373ea to 0x03e751 (0x7368 bytes) 000883 1761246297.287382 MIRROR_RESYNC 11 mirrors 000884 1761246299.971568 write 0x02a377 thru 0x0361c0 (0xbe4a bytes) 000885 1761246300.031667 write 0x0047d5 thru 0x00d859 (0x9085 bytes) 000886 1761246300.049365 mapwrite 0x03f57c thru 0x03ffff (0xa84 bytes) 000887 1761246300.095091 read 0x01fa0c thru 0x021566 (0x1b5b bytes) 000888 1761246300.102356 mapread 0x011682 thru 0x01294f (0x12ce bytes) 000889 1761246300.114178 fallocate 0x005449 thru 0x006742 (0x12fa bytes) 000890 1761246300.119768 read 0x033f8c thru 0x03ffff (0xc074 bytes) 000891 1761246300.130222 punch from 0x036159 to 0x038d73 (0x2c1b bytes) 000892 1761246300.137630 fallocate 0x03ba3f thru 0x03ffff (0x45c1 bytes) 000893 1761246300.143042 fallocate 0x00c433 thru 0x011238 (0x4e06 bytes) 000894 1761246300.147651 fallocate 0x0279bf thru 0x035ccf (0xe311 bytes) 000896 1761246300.154988 trunc from 0x040000 to 0x02a42c (0x15bd3 bytes) 000897 1761246300.174005 fallocate 0x000087 thru 0x00e51b (0xe495 bytes) 000898 1761246300.179200 trunc from 0x02a42d to 0x025708 (0x4d24 bytes) 000899 1761246300.196092 trunc from 0x025709 to 0x012ce6 (0x12a22 bytes) 000901 1761246300.216032 write 0x02f2d2 thru 0x03cb86 (0xd8b5 bytes) 000902 1761246300.218941 fallocate 0x021ddc thru 0x02a5b3 (0x87d8 bytes) 000903 1761246300.224533 trunc from 0x020ad2 to 0x020bf5 (0x124 bytes) 000904 1761246300.233595 mapread 0x002fc5 thru 0x00ad4a (0x7d86 bytes) 000905 1761246300.235084 trunc from 0x02056c to 0x02eb2d (0xe5c2 bytes) 000906 1761246300.241406 punch from 0x030ba8 to 0x0358aa (0x4d03 bytes) 000907 1761246300.257092 mapread 0x029ebc thru 0x02e680 (0x47c5 bytes) 000908 1761246300.302761 write 0x03d7c9 thru 0x03ffff (0x2837 bytes) 000909 1761246300.304057 trunc from 0x01c4a0 to 0x0208d7 (0x4438 bytes) 000910 1761246300.310279 trunc from 0x040000 to 0x023a17 (0x1c5e8 bytes) 000911 1761246300.327443 read 0x019e50 thru 0x01b9ee (0x1b9f bytes) 000912 1761246300.338484 trunc from 0x023a18 to 0x00c469 (0x175ae bytes) 000913 1761246300.355811 fallocate 0x02edc5 thru 0x03107b (0x22b7 bytes) 000914 1761246300.363524 fallocate 0x034d2c thru 0x03ffff (0xb2d4 bytes) 000915 1761246300.372645 write 0x0161d9 thru 0x01c0c5 (0x5eed bytes) 000916 1761246300.376037 mapwrite 0x00b826 thru 0x014dbe (0x9599 bytes) 000917 1761246300.433261 fallocate 0x01bb05 thru 0x0278b7 (0xbdb3 bytes) 000918 1761246300.439343 fallocate 0x014dd0 thru 0x023740 (0xe971 bytes) 000919 1761246300.446148 trunc from 0x0170a6 to 0x01751c (0x477 bytes) 000920 1761246300.453176 trunc from 0x023741 to 0x0135c4 (0x1017c bytes) 000922 1761246300.471631 read 0x009f0b thru 0x0135c4 (0x96ba bytes) 000923 1761246300.472395 trunc from 0x00e9bb to 0x0135c4 (0x4c0a bytes) 000924 1761246300.479125 fallocate 0x024aa0 thru 0x027e49 (0x33aa bytes) 000925 1761246300.485191 fallocate 0x003926 thru 0x011731 (0xde0c bytes) 000926 1761246300.494324 trunc from 0x027e4a to 0x0383f5 (0x105ac bytes) 000927 1761246300.513423 fallocate 0x01d4cb thru 0x02aa14 (0xd54a bytes) 000928 1761246300.518398 punch from 0x004a1f to 0x00d672 (0x8c54 bytes) 000929 1761246300.523743 fallocate 0x03131d thru 0x03966d (0x8351 bytes) 000930 1761246300.529534 trunc from 0x03966e to 0x00aa97 (0x2ebd6 bytes) 000931 1761246300.546334 write 0x00a0c4 thru 0x01306f (0x8fac bytes) 000932 1761246300.555548 fallocate 0x03efde thru 0x03ffff (0x1022 bytes) 000934 1761246300.564366 MIRROR_RESYNC 11 mirrors 000937 1761246303.157185 trunc from 0x0378f9 to 0x03dcbf (0x63c7 bytes) 000938 1761246303.207269 trunc from 0x040000 to 0x0183de (0x27c21 bytes) 000939 1761246303.237632 mapwrite 0x038980 thru 0x03ffff (0x7680 bytes) 000940 1761246303.295299 fallocate 0x00c9b2 thru 0x00ed7c (0x23cb bytes) 000941 1761246303.301697 write 0x00dba9 thru 0x01877f (0xabd7 bytes) 000942 1761246303.332336 trunc from 0x040000 to 0x00eb74 (0x3148b bytes) 000943 1761246303.354397 fallocate 0x030c78 thru 0x034f06 (0x428f bytes) 000944 1761246303.364778 write 0x03b5a4 thru 0x03ffff (0x4a5c bytes) 000945 1761246303.367733 read 0x02afbe thru 0x02b73f (0x782 bytes) 000947 1761246303.385959 MIRROR_EXTEND to 12 mirrors 000948 1761246304.522166 trunc from 0x007c44 to 0x01297a (0xad37 bytes) 000949 1761246304.565742 MIRROR_RESYNC 12 mirrors 000950 1761246306.858024 write 0x00823a thru 0x010db2 (0x8b79 bytes) 000951 1761246306.927623 trunc from 0x00e50f to 0x01dbdc (0xf6ce bytes) 000952 1761246306.953960 mapread 0x0092bc thru 0x00fb87 (0x68cc bytes) 000953 1761246306.976579 read 0x00f6b9 thru 0x0179f7 (0x833f bytes) 000954 1761246306.992915 trunc from 0x01b9cb to 0x028f61 (0xd597 bytes) 000955 1761246307.001054 trunc from 0x040000 to 0x0308ae (0xf751 bytes) 000956 1761246307.030384 mapwrite 0x0173fe thru 0x023fe9 (0xcbec bytes) 000957 1761246307.084549 MIRROR_RESYNC 12 mirrors 000958 1761246310.325617 trunc from 0x0308af to 0x034198 (0x38ea bytes) 000959 1761246310.397685 trunc from 0x034199 to 0x005b3f (0x2e659 bytes) 000960 1761246310.434984 fallocate 0x01f905 thru 0x02e790 (0xee8c bytes) 000961 1761246310.441391 trunc from 0x02e348 to 0x02e790 (0x449 bytes) 000962 1761246310.445711 fallocate 0x010dc7 thru 0x015a24 (0x4c5e bytes) 000963 1761246310.451055 read 0x009992 thru 0x00f8f9 (0x5f68 bytes) 000966 1761246310.462257 write 0x007cb3 thru 0x008079 (0x3c7 bytes) 000967 1761246310.476343 MIRROR_EXTEND to 13 mirrors 000969 1761246311.220693 fallocate 0x02ea5f thru 0x036988 (0x7f2a bytes) 000970 1761246311.282659 trunc from 0x036989 to 0x038b0b (0x2183 bytes) 000971 1761246311.343281 trunc from 0x016d6c to 0x02638d (0xf622 bytes) 000972 1761246311.356078 MIRROR_SPLIT mirror 10 to 12 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000973 1761246311.657940 mapread 0x02bf2e thru 0x02de58 (0x1f2b bytes) 000974 1761246311.687153 punch from 0x02fd63 to 0x03326a (0x3508 bytes) 000975 1761246311.693542 mapwrite 0x02d3a3 thru 0x036937 (0x9595 bytes) 000976 1761246311.729770 mapwrite 0x03e1a4 thru 0x03ffff (0x1e5c bytes) 000977 1761246311.801656 fallocate 0x003d03 thru 0x0062fd (0x25fb bytes) 000978 1761246311.823174 MIRROR_EXTEND to 13 mirrors 000979 1761246312.648290 mapread 0x01a907 thru 0x01ec72 (0x436c bytes) 000980 1761246312.704667 MIRROR_SPLIT mirror 2 to 12 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000405:0x1c9:0x0]: Invalid argument (22) 000981 1761246313.113685 trunc from 0x0192d0 to 0x02727e (0xdfaf bytes) 000982 1761246313.156658 trunc from 0x01b7b0 to 0x027024 (0xb875 bytes) 000983 1761246313.162086 fallocate 0x000906 thru 0x0054a6 (0x4ba1 bytes) 000984 1761246313.171586 MIRROR_EXTEND to 13 mirrors 000985 1761246314.142881 trunc from 0x040000 to 0x01ba7d (0x24582 bytes) 000987 1761246314.209534 trunc from 0x0055ce to 0x0107a8 (0xb1db bytes) 000988 1761246314.214995 trunc from 0x01ba7e to 0x03117e (0x15701 bytes) 000989 1761246314.248249 read 0x01e30b thru 0x01f820 (0x1516 bytes) 000990 1761246314.267242 MIRROR_RESYNC 13 mirrors 000991 1761246316.762458 read 0x012a5f thru 0x01dd00 (0xb2a2 bytes) 000992 1761246316.773947 fallocate 0x00ed2e thru 0x0144c0 (0x5793 bytes) 000993 1761246316.832572 write 0x02175d thru 0x02f472 (0xdd16 bytes) 000994 1761246316.859780 MIRROR_RESYNC 13 mirrors 000995 1761246320.085620 punch from 0x013998 to 0x01d6f3 (0x9d5c bytes) 000996 1761246320.132939 trunc from 0x03117f to 0x03ce22 (0xbca4 bytes) 000997 1761246320.163217 mapread 0x004293 thru 0x0086b3 (0x4421 bytes) 000998 1761246320.186395 mapwrite 0x013498 thru 0x01d4d1 (0xa03a bytes) 000999 1761246320.232321 fallocate 0x016e23 thru 0x01b581 (0x475f bytes) 001000 1761246320.238269 punch from 0x00e51d to 0x01cd05 (0xe7e9 bytes) All operations completed A-OK! PASS 70a (116s) == sanity-flr test 71: check mirror extend parent fid ==== 15:05:25 (1761246325) striped dir -i1 -c2 -H crush2 /mnt/lustre/d71.sanity-flr ** lfs fid2path /mnt/lustre [0x240000405:0x208:0x0] /mnt/lustre/d71.sanity-flr/f71.sanity-flr PASS 71 (9s) == sanity-flr test 200a: stress test ===================== 15:05:35 (1761246335) Starting client: oleg315-client.virtnet: -o user_xattr,flock 192.168.203.115@tcp:/lustre /mnt/lustre2 Starting client: oleg315-client.virtnet: -o user_xattr,flock 192.168.203.115@tcp:/lustre /mnt/lustre3 fail_loc=0x1A03 fail_loc=0x1A03 resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..Extending file size to 8000864 .. Extending file size to 1836160 .. failed Extending file size to 7973152 .. lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..Extending file size to 8117280 .. failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..Extending file size to 8255680 .. failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed Extending file size to 8503584 .. lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 9260192 .. done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8513536 .. done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8831712 .. failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Waiting 59205 59206 59207 59211 59212 done 192.168.203.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg315-client.virtnet /mnt/lustre2 (opts:) 192.168.203.115@tcp:/lustre /mnt/lustre3 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg315-client.virtnet /mnt/lustre3 (opts:) 3 PASS 200a (74s) == sanity-flr test 200b: racing IO, mirror extend and resync ========================================================== 15:06:50 (1761246410) Starting client: oleg315-client.virtnet: -o user_xattr,flock 192.168.203.115@tcp:/lustre /mnt/lustre2 Starting client: oleg315-client.virtnet: -o user_xattr,flock 192.168.203.115@tcp:/lustre /mnt/lustre3 fail_loc=0x1423 resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..Extending file size to 4246336 .. Extending file size to 3606304 .. failed Extending file size to 4802336 .. resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed Extending file size to 4905248 .. lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8825152 .. Extending file size to 7213152 .. done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed Extending file size to 7722784 .. resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed Extending file size to 8568992 .. resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed Extending file size to 8873920 .. resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..Waiting 61356 61358 61361 61362 61363 done 192.168.203.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg315-client.virtnet /mnt/lustre2 (opts:) 192.168.203.115@tcp:/lustre /mnt/lustre3 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg315-client.virtnet /mnt/lustre3 (opts:) 3 PASS 200b (73s) == sanity-flr test 200c: layout change racing with open: LOVEA changes ========================================================== 15:08:03 (1761246483) Starting client: oleg315-client.virtnet: -o user_xattr,flock 192.168.203.115@tcp:/lustre /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2324 1268980 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 2336 1268968 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 14468 3592552 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 22168 3584852 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 36636 7177404 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2324 1268980 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1414116 2336 1268968 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3833116 14468 3592552 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3833116 22168 3584852 1% /mnt/lustre2[OST:1] filesystem_summary: 7666232 36636 7177404 1% /mnt/lustre2 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.12909 s, 16.2 MB/s fail_loc=0x80000175 fail_val=10 mirror extend mirror extend done fail_loc=0 fail_val=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.189462 s, 11.1 MB/s dd completed Chunks to be verified in /mnt/lustre/f200c.sanity-flr: [0, 0xffffffffffffffff) [1] 1 lfs verify: '/mnt/lustre/f200c.sanity-flr' chunk [0, 0xffffffffffffffff) is only valid in mirror 1: skipped 192.168.203.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg315-client.virtnet /mnt/lustre2 (opts:) PASS 200c (14s) SKIP: sanity-flr test_201 skipping ALWAYS excluded test 201 == sanity-flr test 202: lfs setstripe --add-component wide striping ========================================================== 15:08:19 (1761246499) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0728657 s, 28.8 MB/s PASS 202 (6s) == sanity-flr test 203: mirror file preserve mirror ID === 15:08:25 (1761246505) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x5c:0x0]: Invalid argument (22) /mnt/lustre/f203.sanity-flr lcm_layout_gen: 3 lcm_mirror_count: 1 lcm_entry_count: 1 lcme_id: 131074 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xaa:0x0] } /mnt/lustre/f203.sanity-flr lcm_layout_gen: 4 lcm_mirror_count: 2 lcm_entry_count: 2 lcme_id: 131074 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xaa:0x0] } lcme_id: 196609 lcme_mirror_id: 3 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xab:0x0] } - 1: { l_ost_idx: 1, l_fid: [0x2c0000401:0xab:0x0] } PASS 203 (6s) == sanity-flr test 204a: FLR write/stale/resync tests with self-extending mirror ========================================================== 15:08:31 (1761246511) striped dir -i0 -c2 -H crush /mnt/lustre/d204a.sanity-flr 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0740062 s, 28.3 MB/s /mnt/lustre/d204a.sanity-flr/f204a.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x279:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x27a:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 1048576 lcme_extent.e_end: 68157440 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x27b:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 68157440 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 PASS 204a (7s) == sanity-flr test 204b: FLR write/stale/resync tests with self-extending primary ========================================================== 15:08:38 (1761246518) striped dir -i0 -c2 -H crush2 /mnt/lustre/d204b.sanity-flr 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.122067 s, 17.2 MB/s /mnt/lustre/d204b.sanity-flr/f204b.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xac:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: 68157440 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xad:0x0] } lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 68157440 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xac:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 /mnt/lustre/d204b.sanity-flr/f204b.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xac:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: 68157440 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xad:0x0] } lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 68157440 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xac:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xad:0x0] } PASS 204b (7s) == sanity-flr test 204c: FLR write/stale/resync test with component removal ========================================================== 15:08:45 (1761246525) striped dir -i0 -c2 -H crush2 /mnt/lustre/d204c.sanity-flr 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0909735 s, 23.1 MB/s /mnt/lustre/d204c.sanity-flr/f204c.sanity-flr lcm_layout_gen: 7 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xae:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xae:0x0] } lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 /mnt/lustre/d204c.sanity-flr/f204c.sanity-flr lcm_layout_gen: 9 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xae:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xae:0x0] } lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xaf:0x0] } PASS 204c (31s) == sanity-flr test 204d: FLR write/stale/resync sel test with repeated comp ========================================================== 15:09:16 (1761246556) Waiting for MDT destroys to complete Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg315-server mds-ost sync done. striped dir -i0 -c2 -H crush2 /mnt/lustre/d204d.sanity-flr 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0661152 s, 15.9 MB/s /mnt/lustre/d204d.sanity-flr/f204d.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x27a:0x0] } lcme_id: 65541 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x27d:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x27c:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 /mnt/lustre/d204d.sanity-flr/f204d.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x27a:0x0] } lcme_id: 65541 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x27d:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x27c:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x27b:0x0] } PASS 204d (53s) == sanity-flr test 204e: FLR write/stale/resync sel test with repeated comp ========================================================== 15:10:09 (1761246609) Waiting for MDT destroys to complete Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg315-server mds-ost sync done. striped dir -i0 -c2 -H all_char /mnt/lustre/d204e.sanity-flr /mnt/lustre/d204e.sanity-flr/f204e.sanity-flr lcm_layout_gen: 4 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xaf:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xb0:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 67108864 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0627419 s, 33.4 MB/s /mnt/lustre/d204e.sanity-flr/f204e.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xaf:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xb0:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb0:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 /mnt/lustre/d204e.sanity-flr/f204e.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xaf:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xb0:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb0:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 PASS 204e (48s) == sanity-flr test 204f: FLR write/stale/resync sel w/forced extension ========================================================== 15:10:57 (1761246657) Waiting for MDT destroys to complete Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg315-server mds-ost sync done. striped dir -i0 -c2 -H crush2 /mnt/lustre/d204f.sanity-flr Creating new pool test_204f oleg315-server: Pool lustre.test_204f created Adding targets to pool oleg315-server: OST lustre-OST0000_UUID added to pool lustre.test_204f oleg315-server: OST lustre-OST0001_UUID added to pool lustre.test_204f 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.1424 s, 14.7 MB/s /mnt/lustre/d204f.sanity-flr/f204f.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xb1:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_pool: test_204f lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb1:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lmm_pool: test_204f /mnt/lustre/d204f.sanity-flr/f204f.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xb1:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_pool: test_204f lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xb1:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lmm_pool: test_204f Destroy the created pools: test_204f lustre.test_204f oleg315-server: OST lustre-OST0000_UUID removed from pool lustre.test_204f oleg315-server: OST lustre-OST0001_UUID removed from pool lustre.test_204f oleg315-server: Pool lustre.test_204f destroyed PASS 204f (64s) == sanity-flr test 205a: lfs mirror extend to set prefer flag ========================================================== 15:12:01 (1761246721) PASS 205a (6s) == sanity-flr test 205b: lfs mirror extend to set nocompr flag ========================================================== 15:12:07 (1761246727) PASS 205b (7s) == sanity-flr test 206: lfs setstripe -pool .. --comp-flags=.. ========================================================== 15:12:14 (1761246734) oleg315-server: Pool lustre.test_206 created Adding targets to pool oleg315-server: OST lustre-OST0000_UUID added to pool lustre.test_206 oleg315-server: OST lustre-OST0001_UUID added to pool lustre.test_206 Destroy the created pools: test_206 lustre.test_206 oleg315-server: OST lustre-OST0000_UUID removed from pool lustre.test_206 oleg315-server: OST lustre-OST0001_UUID removed from pool lustre.test_206 oleg315-server: Pool lustre.test_206 destroyed PASS 206 (24s) == sanity-flr test 207: create another replica with existing out-of-sync one ========================================================== 15:12:38 (1761246758) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0961797 s, 10.9 MB/s 2 mirror IDs: 1 2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231604 s, 45.3 MB/s 2 mirror IDs: 1 2 2 mirror IDs: 1 2 3 mirror IDs: 1 2 3 CRC-32 checksum value for chunk [0, 0x100000): Mirror 1: 0xc6239526 Mirror 3: 0xc6239526 PASS 207 (9s) == sanity-flr test 208a: mirror selection to prefer non-rotational devices for reads ========================================================== 15:12:47 (1761246767) SKIP: sanity-flr test_208a needs >= 4 OSTs SKIP 208a (3s) == sanity-flr test 208b: mirror selection to prefer non-rotational devices for writes ========================================================== 15:12:50 (1761246770) SKIP: sanity-flr test_208b needs >= 4 OSTs SKIP 208b (3s) == sanity-flr test 209a: skip fast reads after layout invalidation ========================================================== 15:12:53 (1761246773) Starting client: oleg315-client.virtnet: -o user_xattr,flock 192.168.203.115@tcp:/lustre /mnt/lustre2 osd-ldiskfs.lustre-OST0000.nonrotational=0 osd-ldiskfs.lustre-OST0001.nonrotational=0 osd-ldiskfs.lustre-OST0001.nonrotational=1 replicated file created open(O_RDONLY) and first read from OST first read complete 4+0 records in 4+0 records out 4 bytes copied, 0.113058 s, 0.0 kB/s BBBB written which made replica on ost1 stale fast read from pagecache in the original process read via /mnt/lustre2 new open(2) Buffer address : 0xe00000 Read this (4 bytes): BBBB fast read from pagecache after 5s in the original process Buffer address : 0x11f0000 Read this (4 bytes): AAAA PAUSING Buffer address : 0x11f0000 Read this (4 bytes): BBBB PAUSING Buffer address : 0x11f0000 Read this (4 bytes): BBBB read via new open(2) Buffer address : 0x650000 Read this (4 bytes): BBBB 192.168.203.115@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg315-client.virtnet /mnt/lustre2 (opts:) PASS 209a (31s) == sanity-flr test 209b: pagecache can be used after LL cancellation ========================================================== 15:13:24 (1761246804) 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0280281 s, 292 kB/s the very first read read with warm cache PASS 209b (6s) == sanity-flr test 210a: handle broken mirrored lovea ==== 15:13:30 (1761246810) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0416285 s, 25.2 MB/s fail_loc=0x1428 /mnt/lustre/f210a.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 240 lcm_flags: ro lcm_layout_gen: 1 lcm_mirror_count: 2 lcm_entry_count: 2 components: - lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lcme_offset: 128 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x7d lmm_fid: [0x200000405:0x7d:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lmm_objects: - 0: { l_ost_idx: -1, l_fid: [0:0x0:0x0] } - lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lcme_offset: 184 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x7d lmm_fid: [0x200000405:0x7d:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xbb:0x0] } File: /mnt/lustre/f210a.sanity-flr Size: 1048576 Blocks: 4096 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205339611261 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-23 15:13:32.000000000 -0400 Modify: 2025-10-23 15:13:32.000000000 -0400 Change: 2025-10-23 15:13:33.000000000 -0400 Birth: 2025-10-23 15:13:31.000000000 -0400 PASS 210a (7s) == sanity-flr test 210b: handle broken mirrored lovea (unlink) ========================================================== 15:13:37 (1761246817) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.052865 s, 19.8 MB/s oleg315-server: debugfs 1.47.1-wc2 (08-Nov-2024) fail_loc=0x1428 Waiting for MDT destroys to complete oleg315-server: debugfs 1.47.1-wc2 (08-Nov-2024) oleg315-server: O/280000401/d28/bc: File not found by ext2_lookup PASS 210b (21s) == sanity-flr test 211: mirror delete should not cause bad size ========================================================== 15:13:58 (1761246838) 10+0 records in 10+0 records out 40960 bytes (41 kB, 40 KiB) copied, 0.173984 s, 235 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0198037 s, 207 kB/s size after second write -rw-r--r-- 1 root root 4096 Oct 23 15:14 /mnt/lustre/f211.sanity-flr lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x82:0x0]: Invalid argument (22) size after mirror delete -rw-r--r-- 1 root root 4096 Oct 23 15:14 /mnt/lustre/f211.sanity-flr PASS 211 (6s) == sanity-flr test complete, duration 1575 sec =========== 15:14:05 (1761246845) === sanity-flr: start cleanup 15:14:06 (1761246846) === === sanity-flr: finish cleanup 15:14:12 (1761246852) ===