-----============= acceptance-small: sanity-flr ============----- Mon Jul 28 18:27:47 EDT 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg417-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanity-flr.*ex': No such file or directory excepting tests: 6 201 44c === sanity-flr: start setup 18:28:04 (1753741684) === oleg417-client.virtnet: executing check_config_client /mnt/lustre oleg417-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg417-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8fd4d0636800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8fd4d0636800.idle_timeout=debug disable quota as required oleg417-server: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 === sanity-flr: finish setup 18:28:39 (1753741719) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f8193] == sanity-flr test 0a: lfs mirror create with -N option == 18:28:42 (1753741722) PASS 0a (14s) == sanity-flr test 0b: lfs mirror create plain layout mirrors ========================================================== 18:28:56 (1753741736) SKIP: sanity-flr test_0b need >= 4 OSTs SKIP 0b (3s) == sanity-flr test 0c: lfs mirror create composite layout mirrors ========================================================== 18:28:59 (1753741739) SKIP: sanity-flr test_0c need >= 4 OSTs SKIP 0c (3s) == sanity-flr test 0d: lfs mirror extend with -N option == 18:29:02 (1753741742) PASS 0d (18s) == sanity-flr test 0e: lfs mirror extend plain layout mirrors ========================================================== 18:29:20 (1753741760) SKIP: sanity-flr test_0e need >= 4 OSTs SKIP 0e (4s) == sanity-flr test 0f: lfs mirror extend composite layout mirrors ========================================================== 18:29:24 (1753741764) SKIP: sanity-flr test_0f need >= 4 OSTs SKIP 0f (3s) == sanity-flr test 0g: lfs mirror create flags support === 18:29:27 (1753741767) error: lfs create: --flags must be specified with --mirror-count|-N option Create a mirrored file. usage: lfs mirror create --mirror-count|-N[MIRROR_COUNT] [SETSTRIPE_OPTIONS] ... FILENAME|DIRECTORY ... lfs commands used to manage files with mirrored components: lfs mirror create - create a mirrored file or directory lfs mirror extend - add mirror(s) to an existing file lfs mirror split - split a mirror from an existing mirrored file lfs mirror resync - resynchronize out-of-sync mirrored file(s) lfs mirror read - read a mirror content of a mirrored file lfs mirror write - write to a mirror of a mirrored file lfs mirror copy - copy a mirror to other mirror(s) of a file lfs mirror verify - verify mirrored file(s) lfs create: component flag 'foo' not supported Create a mirrored file. usage: lfs mirror create --mirror-count|-N[MIRROR_COUNT] [SETSTRIPE_OPTIONS] ... FILENAME|DIRECTORY ... lfs commands used to manage files with mirrored components: lfs mirror create - create a mirrored file or directory lfs mirror extend - add mirror(s) to an existing file lfs mirror split - split a mirror from an existing mirrored file lfs mirror resync - resynchronize out-of-sync mirrored file(s) lfs mirror read - read a mirror content of a mirrored file lfs mirror write - write to a mirror of a mirrored file lfs mirror copy - copy a mirror to other mirror(s) of a file lfs mirror verify - verify mirrored file(s) lfs create: unsupported mirror flags: stale lfs create: unsupported mirror flags: prefer,init lfs create: inverted flags are not supported Create a mirrored file. usage: lfs mirror create --mirror-count|-N[MIRROR_COUNT] [SETSTRIPE_OPTIONS] ... FILENAME|DIRECTORY ... lfs commands used to manage files with mirrored components: lfs mirror create - create a mirrored file or directory lfs mirror extend - add mirror(s) to an existing file lfs mirror split - split a mirror from an existing mirrored file lfs mirror resync - resynchronize out-of-sync mirrored file(s) lfs mirror read - read a mirror content of a mirrored file lfs mirror write - write to a mirror of a mirrored file lfs mirror copy - copy a mirror to other mirror(s) of a file lfs mirror verify - verify mirrored file(s) osc.lustre-OST0000-osc-ffff8fd4d0636800.stats=clear osc.lustre-OST0001-osc-ffff8fd4d0636800.stats=clear PASS 0g (7s) == sanity-flr test 0h: set, clear and test flags for FLR files ========================================================== 18:29:35 (1753741775) striped dir -i0 -c2 -H crush /mnt/lustre/d0h.sanity-flr lfs setstripe: component flag 'foo' not supported Create a file with specified striping/composite layout, or set the default layout on an existing directory. Usage: setstripe [--component-add|--component-del|--delete|-d] [--comp-set --comp-id|-I COMP_ID|--comp-flags=COMP_FLAGS] [--component-end|-E END_OFFSET] [--copy=SOURCE_LAYOUT_FILE]|--yaml|-y YAML_TEMPLATE_FILE] [--extension-size|--ext-size|-z EXT_SIZE] [--help|-h] [--foreign=FOREIGN_TYPE --xattr|-x LAYOUT] [--layout|-L PATTERN] [--mode FILE_MODE] [--mirror-count|-N[MIRROR_COUNT]] [--ost|-o OST_INDEX[,OST_INDEX,...]] [--overstripe-count|-C STRIPE_COUNT] [--pool|-p POOL_NAME] [--stripe-count|-c STRIPE_COUNT] [--stripe-index|-i START_OST_IDX] [--stripe-size|-S STRIPE_SIZE] FILENAME|DIRECTORY lfs: component flag 'foo' not supported error: getstripe bad component flags 'prefer,foo'. List the layout pattern for a given file or files in a directory or recursively for all files in a directory tree. Usage: getstripe [--ost|-O OST_NAME] [--quiet|-q] [--verbose|-v] [--stripe-count|-c] [--stripe-index|-i] [--fid|-F] [--pool|-p] [--stripe-size|-S] [--directory|-d] [--mdt-index|-m] [--recursive|-r] [--raw|-R] [--layout|-L] [--generation|-g] [--yaml|-y] [--help|-h] [--hex-idx] [--component-id|-I[=COMP_ID]] [--component-flags[=COMP_FLAGS]] [--component-count] [--extension-size|--ext-size|-z] [--component-start[=[+-]START_OFFSET]] [--component-end|-E[[+-]END_OFFSET]] [[!] --mirror-index=[+-]MIRROR_INDEX | [!] --mirror-id=[+-]MIRROR_ID] [--mirror-count|-N] [--no-follow] FILENAME|DIRECTORY lfs setstripe: cannot clear 'stale' flags from component. Please use lfs-mirror-resync(1) instead PASS 0h (8s) == sanity-flr test 0j: test lfs mirror read/write commands ========================================================== 18:29:43 (1753741783) PASS 0j (7s) == sanity-flr test 0k: mirroring a file in directory with default FLR layout ========================================================== 18:29:50 (1753741790) SKIP: sanity-flr test_0k need >= 3 OSTs SKIP 0k (3s) == sanity-flr test 1: create components with setstripe options ========================================================== 18:29:54 (1753741794) lfs mirror mirror: cannot get UNLOCK lease, ext 4 for [0x200000402:0x1f:0x0]: Numerical result out of range (34) error: lfs extend: /mnt/lustre/f1.sanity-flr: cannot merge layout: Numerical result out of range PASS 1 (9s) == sanity-flr test 2: create components from existing files ========================================================== 18:30:02 (1753741802) PASS 2 (6s) == sanity-flr test 3: create components from files located on different MDTs ========================================================== 18:30:08 (1753741808) PASS 3 (7s) == sanity-flr test 4: Make sure mirror attributes can be inhertied from directory ========================================================== 18:30:15 (1753741815) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d4.sanity-flr striped dir -i0 -c2 -H crush2 /mnt/lustre/d4.sanity-flr/td error: lfs extend: /mnt/lustre/d4.sanity-flr: cannot get the last init comp: Is a directory PASS 4 (6s) == sanity-flr test 5: Make sure init size work for mirrored layout ========================================================== 18:30:21 (1753741821) /mnt/lustre/f5.sanity-flr has type file OK /mnt/lustre/f5.sanity-flr has size 12345 OK /mnt/lustre/f5.sanity-flr has type file OK /mnt/lustre/f5.sanity-flr has size 12345 OK PASS 5 (6s) SKIP: sanity-flr test_6a skipping ALWAYS excluded test 6a (base 6) SKIP: sanity-flr test_6b skipping ALWAYS excluded test 6b (base 6) == sanity-flr test 7: Create mirror with DoM component === 18:30:30 (1753741830) PASS 7 (6s) == sanity-flr test 21: glimpse should report accurate i_blocks ========================================================== 18:30:36 (1753741836) 19+0 records in 19+0 records out 19922944 bytes (20 MB, 19 MiB) copied, 1.52806 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768299 s, 13.6 MB/s PASS 21 (9s) == sanity-flr test 22: no glimpse to OSTs for READ_ONLY files ========================================================== 18:30:45 (1753741845) 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 0.45018 s, 46.6 MB/s PASS 22 (7s) == sanity-flr test 31: make sure glimpse request can be retried ========================================================== 18:30:52 (1753741852) Stopping /mnt/lustre-ost1 (opts:) on oleg417-server oleg417-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff8fd4d0636800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8fd4d0636800.ost_server_uuid in DISCONN state after 0 sec Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg417-server: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg417-client: oleg417-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg417-server: oleg417-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg417-server: *.lustre-OST0000.recovery_status status: COMPLETE Stopping /mnt/lustre-ost2 (opts:) on oleg417-server oleg417-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0001-osc-ffff8fd4d0636800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8fd4d0636800.ost_server_uuid in DISCONN state after 0 sec Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg417-server: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg417-client: oleg417-server: ssh exited with exit code 1 Started lustre-OST0001 affected facets: ost2 oleg417-server: oleg417-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg417-server: *.lustre-OST0001.recovery_status status: COMPLETE PASS 31 (63s) == sanity-flr test 32: data should be mirrored to newly created mirror ========================================================== 18:31:55 (1753741915) 6+0 records in 6+0 records out 6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.282281 s, 22.3 MB/s Stopping /mnt/lustre-ost1 (opts:) on oleg417-server oleg417-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff8fd4d0636800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8fd4d0636800.ost_server_uuid in DISCONN state after 0 sec /mnt/lustre/f32.sanity-flr has type file OK /mnt/lustre/f32.sanity-flr has size 6291456 OK reading file from the 2nd mirror and verify checksum Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg417-server: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg417-client: oleg417-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg417-server: oleg417-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg417-server: *.lustre-OST0000.recovery_status status: COMPLETE PASS 32 (30s) == sanity-flr test 33a: read can choose available mirror to read ========================================================== 18:32:25 (1753741945) error: lfs extend: /mnt/lustre/f33a.sanity-flr: file busy or contents don't match: Invalid argument /mnt/lustre/f33a.sanity-flr has type file OK /mnt/lustre/f33a.sanity-flr has size 500 OK reading file (data can be provided by any ost)... Stopping /mnt/lustre-ost1 (opts:) on oleg417-server oleg417-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff8fd4d0636800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8fd4d0636800.ost_server_uuid in DISCONN state after 0 sec reading file (data should be provided by ost2)... Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg417-server: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg417-client: oleg417-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg417-server: oleg417-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg417-server: *.lustre-OST0000.recovery_status status: COMPLETE Stopping /mnt/lustre-ost2 (opts:) on oleg417-server oleg417-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0001-osc-ffff8fd4d0636800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8fd4d0636800.ost_server_uuid in DISCONN state after 0 sec /mnt/lustre/f33a.sanity-flr has type file OK /mnt/lustre/f33a.sanity-flr has size 500 OK reading file (data should be provided by ost1)... Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg417-server: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg417-client: oleg417-server: ssh exited with exit code 1 Started lustre-OST0001 affected facets: ost2 oleg417-server: oleg417-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg417-server: *.lustre-OST0001.recovery_status status: COMPLETE PASS 33a (65s) == sanity-flr test 33b: avoid reading from unhealthy mirror ========================================================== 18:33:30 (1753742010) 25+0 records in 25+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 2.72815 s, 19.2 MB/s -rw-r--r-- 1 root root 52428800 Jul 28 18:33 /mnt/lustre/f33b.sanity-flr reading file (data can be provided by any ost)... real 0m1.078s user 0m0.000s sys 0m0.154s Stopping /mnt/lustre-ost1 (opts:) on oleg417-server oleg417-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff8fd4d0636800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8fd4d0636800.ost_server_uuid in DISCONN state after 0 sec reading file (data should be provided by ost2)... real 0m0.883s user 0m0.000s sys 0m0.161s Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg417-server: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg417-client: oleg417-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg417-server: oleg417-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg417-server: *.lustre-OST0000.recovery_status status: COMPLETE Stopping /mnt/lustre-ost2 (opts:) on oleg417-server oleg417-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0001-osc-ffff8fd4d0636800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8fd4d0636800.ost_server_uuid in DISCONN state after 0 sec reading file (data should be provided by ost1)... real 0m0.885s user 0m0.000s sys 0m0.163s Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg417-server: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg417-client: oleg417-server: ssh exited with exit code 1 Started lustre-OST0001 affected facets: ost2 oleg417-server: oleg417-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg417-server: *.lustre-OST0001.recovery_status status: COMPLETE oleg417-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8fd4d0636800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8fd4d0636800.ost_server_uuid in FULL state after 0 sec PASS 33b (78s) == sanity-flr test 33c: keep reading among unhealthy mirrors ========================================================== 18:34:48 (1753742088) SKIP: sanity-flr test_33c need >= 3 OSTs SKIP 33c (3s) == sanity-flr test 34a: read mirrored file with multiple stripes ========================================================== 18:34:51 (1753742091) SKIP: sanity-flr test_34a need >= 4 OSTs SKIP 34a (3s) == sanity-flr test 34b: read mirrored file with multiple components ========================================================== 18:34:54 (1753742094) SKIP: sanity-flr test_34b need >= 4 OSTs SKIP 34b (3s) == sanity-flr test 35: allow to write to mirrored files == 18:34:58 (1753742098) PASS 35 (6s) == sanity-flr test 36a: write to mirrored files ========== 18:35:04 (1753742104) 12+0 records in 12+0 records out 12582912 bytes (13 MB, 12 MiB) copied, 0.41466 s, 30.3 MB/s fail_val=10 fail_loc=0x1A01 fail_loc=0 PASS 36a (10s) == sanity-flr test 36b: write should not hung when prefered mirror is stale ========================================================== 18:35:15 (1753742115) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225553 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204982 s, 51.2 MB/s PASS 36b (7s) == sanity-flr test 36c: change prefer mirror during write shouldn't hung ========================================================== 18:35:21 (1753742121) PASS 36c (6s) == sanity-flr test 36d: write/punch FLR file update OST layout version ========================================================== 18:35:27 (1753742127) ** create /mnt/lustre/f36d.sanity-flr ** mirror extend /mnt/lustre/f36d.sanity-flr (1/2) flv=2 olv=0 ** mirror extend /mnt/lustre/f36d.sanity-flr (2/2) flv=3 olv=0 ** write /mnt/lustre/f36d.sanity-flr (1/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00075534 s, 1.4 MB/s flv=4 olv=4 ** write /mnt/lustre/f36d.sanity-flr (2/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000777632 s, 1.3 MB/s flv=4 olv=4 ** resync /mnt/lustre/f36d.sanity-flr flv=6 olv=6 ** truncate /mnt/lustre/f36d.sanity-flr (1/2) flv=7 olv=7 ** truncate /mnt/lustre/f36d.sanity-flr (2/2) flv=7 olv=7 ** resync /mnt/lustre/f36d.sanity-flr flv=9 olv=9 ** write /mnt/lustre/f36d.sanity-flr (1/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000693589 s, 1.5 MB/s flv=10 olv=10 ** write /mnt/lustre/f36d.sanity-flr (2/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00084676 s, 1.2 MB/s flv=10 olv=10 PASS 36d (8s) == sanity-flr test 37: mirror I/O API verification ======= 18:35:35 (1753742135) create test files with size 15745771 .. 4815da0bf34ba9a5d3346c74a6103107 - 458d31be1b66ee92356bb1f9c6ebaa0c - 8db9d77374a7cf1fca8397c643b63f14 - 3 Verifying mirror read .. Verifying mirror write .. Verifying mirror copy .. mirror copied successfully: 2 3 Stopping client oleg417-client.virtnet /mnt/lustre (opts:) Starting client: oleg417-client.virtnet: -o user_xattr,flock 192.168.204.117@tcp:/lustre /mnt/lustre PASS 37 (20s) == sanity-flr test 38: resync ============================ 18:35:55 (1753742155) 4 mirror IDs: 1 2 3 4 mirror_io: 391: hit by error injection mirror_io: 391: hit by error injection PASS 38 (11s) == sanity-flr test 39: check FLR+PFL (a.k.a. PFLR) creation ========================================================== 18:36:07 (1753742167) PASS 39 (6s) == sanity-flr test 40: PFLR rdonly state instantiation check ========================================================== 18:36:13 (1753742173) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927249 s, 11.3 MB/s composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 760 lcm_flags: wp lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 7 components: - lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 0 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - l_ost_idx: 1 l_fid: 0x2c0000401:0x3a:0x0 - lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - l_ost_idx: 1 l_fid: 0x2c0000401:0x3b:0x0 - lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: prefer lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - l_ost_idx: 0 l_fid: 0x280000401:0x3a:0x0 - lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: 0 lcme_extent.e_start: 1048576 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131079 lcme_mirror_id: 2 lcme_flags: 0 lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301365 s, 34.8 MB/s composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 760 lcm_flags: wp lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 7 components: - lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 0 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - l_ost_idx: 0 l_fid: 0x280000401:0x3b:0x0 - lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - l_ost_idx: 0 l_fid: 0x280000401:0x3c:0x0 - lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: prefer lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - l_ost_idx: 1 l_fid: 0x2c0000401:0x3c:0x0 - lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: 0 lcme_extent.e_start: 1048576 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131079 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 PASS 40 (9s) == sanity-flr test 41: lfs mirror resync check =========== 18:36:22 (1753742182) **create two FLR files /mnt/lustre/f41.sanity-flr /mnt/lustre/f41.sanity-flr-1 **verify files be RDONLY 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.154152 s, 13.6 MB/s 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.430594 s, 9.7 MB/s **verify files be WRITE_PENDING **verify files have stale component **full resync **verify /mnt/lustre/f41.sanity-flr-1 data consistency in all mirrors **verify files be RDONLY **verify files do not contain stale component **write /mnt/lustre/f41.sanity-flr-1 for partial resync test 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0988026 s, 21.2 MB/s **only resync mirror 2 **resync mirror 2 again **verify /mnt/lustre/f41.sanity-flr-1 contains stale component **full resync /mnt/lustre/f41.sanity-flr-1 **full resync /mnt/lustre/f41.sanity-flr-1 again **verify /mnt/lustre/f41.sanity-flr-1 does not contain stale component PASS 41 (10s) == sanity-flr test 42: lfs mirror verify ================= 18:36:33 (1753742193) SKIP: sanity-flr test_42 need >= 4 OSTs SKIP 42 (3s) == sanity-flr test 43a: mirror pick on write ============= 18:36:36 (1753742196) SKIP: sanity-flr test_43a needs >= 3 OSTs SKIP 43a (3s) == sanity-flr test 43b: allow writing to multiple preferred mirror file ========================================================== 18:36:39 (1753742199) striped dir -i1 -c2 -H crush2 /mnt/lustre/d43b.sanity-flr ** write to /mnt/lustre/d43b.sanity-flr/f43b.sanity-flr 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274453 s, 38.2 MB/s ** resync /mnt/lustre/d43b.sanity-flr/f43b.sanity-flr PASS 43b (6s) == sanity-flr test 44a: lfs mirror split check =========== 18:36:46 (1753742206) 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.340752 s, 9.2 MB/s PASS 44a (9s) == sanity-flr test 44b: mirror split does not create stale file ========================================================== 18:36:54 (1753742214) /mnt/lustre/d44b.sanity-flr/f44b.sanity-flr lcm_layout_gen: 2 lcm_mirror_count: 2 lcm_entry_count: 2 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x45:0x0] } lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0x46:0x0] } split mirror_id 1 id 65537 /mnt/lustre/d44b.sanity-flr/f44b.sanity-flr lcm_layout_gen: 5 lcm_mirror_count: 1 lcm_entry_count: 1 lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0x46:0x0] } split mirror_id 2 id 131073 lfs split: '/mnt/lustre/d44b.sanity-flr/f44b.sanity-flr' is not a FLR file. Cannot resync file make sure there's no stale comp in the file PASS 44b (7s) SKIP: sanity-flr test_44c skipping ALWAYS excluded test 44c == sanity-flr test 44d: lfs mirror split does not break size ========================================================== 18:37:02 (1753742222) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.191443 s, 54.8 MB/s ** before mirror ops, file size=10485760 ** after mirror extend, file size=10485760 lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000403:0x7:0x0]: Invalid argument (22) ** after mirror split, file size=10485760 Stopping client oleg417-client.virtnet /mnt/lustre (opts:) Starting client: oleg417-client.virtnet: -o user_xattr,flock 192.168.204.117@tcp:/lustre /mnt/lustre ** after mirror split & remount, file size=10485760 PASS 44d (9s) == sanity-flr test 44e: basic FLR SOM tests + disable SOM ========================================================== 18:37:11 (1753742231) striped dir -i0 -c2 -H crush2 /mnt/lustre/d44e.sanity-flr llite.lustre-ffff8fd4c65c7000.xattr_cache=0 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.306983 s, 34.2 MB/s ** before mirror resync, file size=10485760 file: /mnt/lustre/d44e.sanity-flr/f44e.sanity-flr size: 10485760 blocks: 40960 flags: 1 Stopping client oleg417-client.virtnet /mnt/lustre (opts:) Starting client: oleg417-client.virtnet: -o user_xattr,flock 192.168.204.117@tcp:/lustre /mnt/lustre mdt.lustre-MDT0000.enable_strict_som=0 mdt.lustre-MDT0001.enable_strict_som=0 file: /mnt/lustre/d44e.sanity-flr/f44e.sanity-flr size: 10485760 blocks: 40960 flags: 1 llite.lustre-ffff8fd4d8d07800.xattr_cache=1 PASS 44e (13s) == sanity-flr test 45: Verify setstripe/getstripe with YAML with FLR file ========================================================== 18:37:24 (1753742244) striped dir -i1 -c2 -H crush /mnt/lustre/d45.sanity-flr 1. FLR file: getstripe --yaml /mnt/lustre/d45.sanity-flr/f45.sanity-flr 1. FLR file: setstripe --yaml=/mnt/lustre/d45.sanity-flr/template /mnt/lustre/d45.sanity-flr/f45.sanity-flr.copy 1. FLR file: compare 2. FLR file with flags: getstripe --yaml /mnt/lustre/d45.sanity-flr/f45.sanity-flr 2. FLR file with flags: setstripe --yaml=/mnt/lustre/d45.sanity-flr/template /mnt/lustre/d45.sanity-flr/f45.sanity-flr.copy 2. FLR file with flags: compare PASS 45 (34s) == sanity-flr test 46: Verify setstripe --copy option ==== 18:37:58 (1753742278) striped dir -i0 -c2 -H all_char /mnt/lustre/d46.sanity-flr ** 1. PFL file ** 2. plain file ** 3. FLR file ** 4. PFL dir striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d46.sanity-flr/dir striped dir -i0 -c2 -H crush /mnt/lustre/d46.sanity-flr/dir.copy ** 5. plain dir ** 6. FLR dir PASS 46 (33s) == sanity-flr test 47: Verify mirror obj alloc =========== 18:38:31 (1753742311) SKIP: sanity-flr test_47 needs >= 3 OSTs SKIP 47 (3s) == sanity-flr test 48: Verify snapshot mirror ============ 18:38:34 (1753742314) ** create 2 mirrors FLR file /mnt/lustre/f48.sanity-flr ** write it 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.188235 s, 16.7 MB/s ** resync the file ** snapshot mirror 2 ** write it again 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.0891741 s, 35.3 MB/s ** resync it again ** verify mirror 2 doesn't change original checksum: 3b35781d428914faced0ad2dddf8aee0 - mirror 1 checksum: 3e7402acdb73cac19e2cac87b1d6f942 - mirror 2 checksum: 3b35781d428914faced0ad2dddf8aee0 - ** mirror 2 stripe info /mnt/lustre/f48.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 448 lcm_flags: wp lcm_layout_gen: 10 lcm_mirror_count: 2 lcm_entry_count: 4 components: - lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale,nosync lcme_timestamp: 'Mon Jul 28 18:38:36 2025' lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lcme_offset: 336 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x9 lmm_fid: [0x200000405:0x9:0x0] lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x50:0x0] } - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,stale,nosync lcme_timestamp: 'Mon Jul 28 18:38:36 2025' lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lcme_offset: 392 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x9 lmm_fid: [0x200000405:0x9:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x51:0x0] } ** resync mirror 2 ** verify mirror 2 resync-ed original checksum: 3b35781d428914faced0ad2dddf8aee0 - mirror 1 checksum: 3e7402acdb73cac19e2cac87b1d6f942 - mirror 2 checksum: 3e7402acdb73cac19e2cac87b1d6f942 - ** mirror 2 stripe info /mnt/lustre/f48.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 448 lcm_flags: ro lcm_layout_gen: 12 lcm_mirror_count: 2 lcm_entry_count: 4 components: - lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,nosync lcme_timestamp: 'Mon Jul 28 18:38:36 2025' lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lcme_offset: 336 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x9 lmm_fid: [0x200000405:0x9:0x0] lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x50:0x0] } - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,nosync lcme_timestamp: 'Mon Jul 28 18:38:36 2025' lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lcme_offset: 392 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x9 lmm_fid: [0x200000405:0x9:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x51:0x0] } PASS 48 (7s) == sanity-flr test 49a: FIEMAP upon FLR file ============= 18:38:41 (1753742321) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353342 s, 29.7 MB/s Filesystem type is: bd00bd0 File size of /mnt/lustre/f49a.sanity-flr is 1048576 (1024 blocks of 1024 bytes) ext: device_logical: physical_offset: length: dev: flags: 0: 0.. 1023: 308224.. 309247: 1024: 0001: net,eof 1: 0.. 1023: 310272.. 311295: 1024: 0000: last,net,eof /mnt/lustre/f49a.sanity-flr: 2 extents found FIEMAP on /mnt/lustre/f49a.sanity-flr succeeded PASS 49a (6s) == sanity-flr test 50A: mirror split update layout generation ========================================================== 18:38:47 (1753742327) Failing mds1 on oleg417-server Stopping /mnt/lustre-mds1 (opts:) on oleg417-server 18:38:56 (1753742336) shut down facet: mds1 facet_host: oleg417-server facet_failover_host: oleg417-server Failover mds1 to oleg417-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg417-server: oleg417-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg417-client: oleg417-server: ssh exited with exit code 1 Started lustre-MDT0000 18:39:15 (1753742355) targets are mounted 18:39:15 (1753742355) facet_failover done oleg417-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0xf:0x0]: Invalid argument (22) /mnt/lustre/d50A.sanity-flr/f50A.sanity-flr layout generation from 1 to 2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223314 s, 47.0 MB/s /mnt/lustre/d50A.sanity-flr/f50A.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 136 lcm_flags: 0 lcm_layout_gen: 2 lcm_mirror_count: 1 lcm_entry_count: 1 components: - lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lcme_offset: 80 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0xf lmm_fid: [0x200000405:0xf:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x53:0x0] } PASS 50A (41s) == sanity-flr test 50a: mirror extend/copy preserves sparseness ========================================================== 18:39:28 (1753742368) ** create striped file /mnt/lustre/d50a.sanity-flr/f50a.sanity-flr ** write 1st data chunk at 1M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0331587 s, 618 kB/s ** write 2nd data chunk at 2M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0104469 s, 2.0 MB/s ** create hole at the file end ** verify sparseness first data offset: 1044480 hole at the end: 3500000 ** extend the file with new mirror ** verify mirror #2 sparseness first data offset: 1044480 hole at the end: 3500000 ** copy mirror #2 to mirror #1 mirror copied successfully: 1 ** verify mirror #1 sparseness again first data offset: 1044480 hole at the end: 3500000 ** final consumed blocks: 104 PASS 50a (9s) == sanity-flr test 50b: mirror rsync handles sparseness == 18:39:37 (1753742377) ** create mirrored file /mnt/lustre/d50b.sanity-flr/f50b.sanity-flr ** write data chunk at 1M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0524344 s, 391 kB/s ** create hole at the file end ** verify sparseness first data offset: 1044480 hole at the end: 3500000 ** resync mirror #2 to mirror #1 ** consumed blocks: 96 ** truncate file down ** write data chunk at 2M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0111657 s, 1.8 MB/s ** resync mirror #2 to mirror #1 with nosync 1st component ** resync mirror #2 to mirror #1 again ** final consumed blocks: 96 PASS 50b (9s) == sanity-flr test 50c: punch_hole/mmap_write stale other mirrors ========================================================== 18:39:46 (1753742386) striped dir -i0 -c2 -H all_char /mnt/lustre/d50c.sanity-flr 4+0 records in 4+0 records out 16384 bytes (16 kB, 16 KiB) copied, 0.0103325 s, 1.6 MB/s PASS 50c (7s) == sanity-flr test 50d: mirror rsync keep holes ========== 18:39:53 (1753742393) ** create mirrored file /mnt/lustre/d50d.sanity-flr/f50d.sanity-flr ** write data chunk at 1M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.028293 s, 724 kB/s ** create hole at the file start ** verify sparseness first data offset: 1048576 ** resync mirror #2 ** verify sparseness of mirror #2 first data offset: 1048576 PASS 50d (7s) == sanity-flr test 60a: mirror extend sets correct size on sparse file ========================================================== 18:40:00 (1753742400) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00769141 s, 533 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0275918 s, 148 kB/s PASS 60a (6s) == sanity-flr test 60b: mirror merge/split cancel client's in-memory layout gen ========================================================== 18:40:06 (1753742406) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d60b.sanity-flr lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x18:0x0]: Invalid argument (22) PASS 60b (18s) == sanity-flr test 61a: mirror extend and migrate preserve timestamps ========================================================== 18:40:24 (1753742424) mdd.lustre-MDT0000.atime_diff=1 mdd.lustre-MDT0001.atime_diff=1 create /mnt/lustre/d61a.sanity-flr/f61a.sanity-flr create /mnt/lustre/d61a.sanity-flr/f61a.sanity-flr-2 sleep 5 seconds, then cat f61a.sanity-flr XXX sleep 5 seconds, then re-write f61a.sanity-flr sleep 5 seconds mirror merge f61a.sanity-flr-2 to f61a.sanity-flr and test timestamps mirror extend f61a.sanity-flr and test timestamps migrate f61a.sanity-flr and test timestamps normal user migrate f61a.sanity-flr and test timestamps running as uid/gid/euid/egid 500/500/500/500, groups: 0 [/home/green/git/lustre-release/lustre/utils/lfs] [migrate] [-n] [/mnt/lustre/d61a.sanity-flr/f61a.sanity-flr] mdd.lustre-MDT0000.atime_diff=60 mdd.lustre-MDT0001.atime_diff=60 PASS 61a (27s) == sanity-flr test 61b: mirror extend and split preserve timestamps ========================================================== 18:40:51 (1753742451) create /mnt/lustre/d61b.sanity-flr/f61b.sanity-flr sleep 5 seconds, then cat f61b.sanity-flr XXX sleep 5 seconds, then re-write f61b.sanity-flr sleep 5 seconds, then test timestamps mirror extend f61b.sanity-flr and test timestamps mirror split f61b.sanity-flr and test timestamps lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x19:0x0]: Invalid argument (22) normal user mirror extend f61b.sanity-flr and test timestamps running as uid/gid/euid/egid 500/500/500/500, groups: 0 [/home/green/git/lustre-release/lustre/utils/lfs] [mirror] [extend] [-N] [-c1] [-i1] [/mnt/lustre/d61b.sanity-flr/f61b.sanity-flr] PASS 61b (23s) == sanity-flr test 61c: mirror resync preserves timestamps ========================================================== 18:41:14 (1753742474) create /mnt/lustre/d61c.sanity-flr/f61c.sanity-flr sleep 5 seconds, then cat f61c.sanity-flr XXX sleep 5 seconds, then mirror extend f61c.sanity-flr and write it sleep 5 seconds, then resync f61c.sanity-flr and test timestamps normal user resync f61c.sanity-flr and test timestamps running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [mirror] [resync] [/mnt/lustre/d61c.sanity-flr/f61c.sanity-flr] PASS 61c (22s) == sanity-flr test 62: read/write with unknown type of mirror ========================================================== 18:41:36 (1753742496) create mirror file with unknown magic fail_loc=0x1426 fail_val=2 create mirror file with unknown pattern fail_loc=0x1427 fail_val=1 PASS 62 (7s) == sanity-flr test 70: mirror create and split race ====== 18:41:43 (1753742503) striped dir -i0 -c2 -H crush2 /mnt/lustre/d70.sanity-flr mirror create pid 55886 mirror split pid 55887 mirror create and split race for 60 seconds, should not crash /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4953: 55886 Killed while true; do rm -f $tf; $LFS mirror create -N -E 1M -c -1 -E eof -N $tf; echo xxxx > $tf; done (wd: ~) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4953: 55887 Killed while true; do $LFS mirror split -d --mirror-id=1 $tf &> /dev/null; done (wd: ~) PASS 70 (67s) == sanity-flr test 70a: flr mode fsx test ================ 18:42:50 (1753742570) striped dir -i0 -c2 -H crush2 /mnt/lustre/d70a.sanity-flr Seed set to 2572 000001 1753742572.612222 write 0x016da3 thru 0x01bd1d (0x4f7b bytes) 000002 1753742572.633894 mapwrite 0x012367 thru 0x01d8c9 (0xb563 bytes) zero_range to largest ever: 0xdf51 000003 1753742572.764511 trunc from 0x005671 to 0x00df50 (0x88e0 bytes) 000004 1753742572.770950 punch from 0x0050ad to 0x0109d7 (0xb92b bytes) 000005 1753742572.778929 read 0x0124d2 thru 0x017083 (0x4bb2 bytes) 000006 1753742572.781444 trunc from 0x0120bb to 0x017023 (0x4f69 bytes) 000007 1753742572.789921 trunc from 0x0044c2 to 0x00b657 (0x7196 bytes) 000008 1753742572.798720 MIRROR_EXTEND to 2 mirrors 000009 1753742573.343756 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) truncating to largest ever: 0x3f941 000010 1753742573.490624 trunc from 0x01d8ca to 0x03f940 (0x22077 bytes) 000011 1753742573.529690 punch from 0x03ab8b to 0x03f474 (0x48ea bytes) 000012 1753742573.535507 mapread 0x034a90 thru 0x03c327 (0x7898 bytes) 000013 1753742573.561801 trunc from 0x03f941 to 0x01dab4 (0x21e8c bytes) 000014 1753742573.594704 punch from 0x015f29 to 0x018168 (0x2240 bytes) fallocating to largest ever: 0x40000 000015 1753742573.610934 fallocate 0x03c406 thru 0x03ffff (0x3bfa bytes) 000017 1753742573.625735 mapread 0x0092ba thru 0x00c265 (0x2fac bytes) 000019 1753742573.653272 write 0x01c963 thru 0x020615 (0x3cb3 bytes) 000020 1753742573.666605 trunc from 0x01a4b6 to 0x020615 (0x6160 bytes) 000021 1753742573.686826 mapwrite 0x033ab3 thru 0x03ffff (0xc54d bytes) 000022 1753742573.769588 trunc from 0x009228 to 0x00aa7a (0x1853 bytes) 000023 1753742573.779035 trunc from 0x00ab1c to 0x018ca0 (0xe185 bytes) 000024 1753742573.796949 write 0x000356 thru 0x006983 (0x662e bytes) 000025 1753742573.809582 write 0x030b2e thru 0x0316f0 (0xbc3 bytes) 000026 1753742573.812510 fallocate 0x0331c3 thru 0x03882b (0x5669 bytes) 000027 1753742573.822378 read 0x02b3c9 thru 0x02bd02 (0x93a bytes) 000028 1753742573.836480 mapread 0x01e790 thru 0x023291 (0x4b02 bytes) 000029 1753742573.853435 write 0x02a912 thru 0x031cd6 (0x73c5 bytes) 000030 1753742573.868886 mapwrite 0x038c00 thru 0x03ffff (0x7400 bytes) 000031 1753742573.918619 mapwrite 0x00babe thru 0x01102c (0x556f bytes) 000032 1753742573.969699 mapread 0x023e68 thru 0x0249e1 (0xb7a bytes) 000034 1753742573.970903 mapread 0x0261e0 thru 0x02f30c (0x912d bytes) 000035 1753742573.971735 read 0x00eef1 thru 0x01e493 (0xf5a3 bytes) 000036 1753742573.972331 read 0x031ad0 thru 0x03ffff (0xe530 bytes) 000038 1753742573.973319 fallocate 0x03df7b thru 0x03ffff (0x2085 bytes) 000039 1753742574.010727 punch from 0x025522 to 0x03518d (0xfc6c bytes) 000040 1753742574.021446 write 0x0071bc thru 0x008693 (0x14d8 bytes) 000041 1753742574.026493 fallocate 0x029e61 thru 0x02f4e8 (0x5688 bytes) 000042 1753742574.039220 fallocate 0x01c78e thru 0x01f01a (0x288d bytes) 000043 1753742574.048928 trunc from 0x040000 to 0x025df6 (0x1a209 bytes) 000045 1753742574.091364 mapread 0x012959 thru 0x01c423 (0x9acb bytes) 000046 1753742574.092290 write 0x014260 thru 0x01bf95 (0x7d36 bytes) 000047 1753742574.109333 MIRROR_EXTEND to 2 mirrors 000048 1753742574.758738 trunc from 0x025df7 to 0x016ae9 (0xf30d bytes) 000049 1753742574.844054 mapwrite 0x012968 thru 0x0211a4 (0xe83d bytes) 000050 1753742574.922281 mapwrite 0x01de1d thru 0x021287 (0x346b bytes) 000051 1753742574.961741 read 0x01db02 thru 0x021287 (0x3786 bytes) 000052 1753742574.962748 mapwrite 0x01cf30 thru 0x01ed52 (0x1e23 bytes) 000053 1753742574.984554 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000055 1753742575.133700 mapread 0x008ceb thru 0x016fad (0xe2c3 bytes) 000056 1753742575.156236 punch from 0x009779 to 0x00aa2b (0x12b3 bytes) 000057 1753742575.160352 trunc from 0x021288 to 0x0057ff (0x1ba88 bytes) 000058 1753742575.186684 trunc from 0x005800 to 0x00b3a0 (0x5ba1 bytes) 000059 1753742575.208650 punch from 0x00354d to 0x009007 (0x5abb bytes) 000060 1753742575.218276 mapwrite 0x02dae5 thru 0x031ebf (0x43db bytes) 000061 1753742575.253233 mapread 0x01fb06 thru 0x027569 (0x7a64 bytes) 000062 1753742575.265160 read 0x01c612 thru 0x0279bc (0xb3ab bytes) 000063 1753742575.265726 punch from 0x028214 to 0x031ebf (0x9cac bytes) 000064 1753742575.271784 write 0x02f917 thru 0x0362a1 (0x698b bytes) 000065 1753742575.288677 punch from 0x011cd0 to 0x01a7d7 (0x8b08 bytes) 000066 1753742575.292514 punch from 0x002a48 to 0x00aaa7 (0x8060 bytes) 000067 1753742575.297617 mapwrite 0x0152a1 thru 0x01841e (0x317e bytes) 000070 1753742575.343673 trunc from 0x01e09a to 0x0256d7 (0x763e bytes) 000071 1753742575.351071 read 0x02c818 thru 0x033726 (0x6f0f bytes) 000073 1753742575.356831 trunc from 0x0362a2 to 0x00bd1e (0x2a583 bytes) 000074 1753742575.390323 punch from 0x0031ee to 0x006939 (0x374c bytes) 000075 1753742575.402559 trunc from 0x00bd1f to 0x016a46 (0xad28 bytes) 000076 1753742575.418126 mapread 0x01403d thru 0x016a46 (0x2a0a bytes) 000077 1753742575.431906 trunc from 0x00a424 to 0x00c9ff (0x25dc bytes) 000079 1753742575.441273 write 0x020bbf thru 0x029256 (0x8698 bytes) 000080 1753742575.455556 punch from 0x01a2ea to 0x01cfb3 (0x2cca bytes) 000081 1753742575.476125 punch from 0x00f25b to 0x015295 (0x603b bytes) 000082 1753742575.495999 read 0x024d5a thru 0x029256 (0x44fd bytes) 000083 1753742575.496596 trunc from 0x0195ca to 0x026832 (0xd269 bytes) 000084 1753742575.518681 fallocate 0x00d58a thru 0x017935 (0xa3ac bytes) 000085 1753742575.525499 trunc from 0x029257 to 0x03beb8 (0x12c62 bytes) 000086 1753742575.551166 mapread 0x039ed6 thru 0x03beb8 (0x1fe3 bytes) 000087 1753742575.564423 punch from 0x00bf24 to 0x014ac7 (0x8ba4 bytes) 000088 1753742575.569979 mapread 0x03ac73 thru 0x03beb8 (0x1246 bytes) 000089 1753742575.571818 mapread 0x003393 thru 0x00df61 (0xabcf bytes) 000090 1753742575.594379 mapread 0x0130bb thru 0x01dd1b (0xac61 bytes) 000091 1753742575.620237 punch from 0x016f49 to 0x019361 (0x2419 bytes) 000092 1753742575.651039 mapread 0x01fd09 thru 0x02a1d2 (0xa4ca bytes) 000094 1753742575.652660 trunc from 0x017d28 to 0x018fcb (0x12a4 bytes) 000096 1753742575.658373 write 0x03cb8d thru 0x03ffff (0x3473 bytes) 000097 1753742575.659695 punch from 0x00de5b to 0x01650d (0x86b3 bytes) 000098 1753742575.664536 mapwrite 0x003876 thru 0x00afeb (0x7776 bytes) 000099 1753742575.682522 fallocate 0x02e926 thru 0x03cb1b (0xe1f6 bytes) 000100 1753742575.694330 trunc from 0x02136d to 0x024535 (0x31c9 bytes) 000101 1753742575.708397 mapwrite 0x014f5c thru 0x01a558 (0x55fd bytes) 000102 1753742575.753507 trunc from 0x040000 to 0x036e55 (0x91aa bytes) 000104 1753742575.771952 trunc from 0x013ca8 to 0x022361 (0xe6ba bytes) 000106 1753742575.777019 fallocate 0x02493c thru 0x027e23 (0x34e8 bytes) 000107 1753742575.783696 punch from 0x003d87 to 0x008c16 (0x4e90 bytes) 000108 1753742575.790327 mapread 0x028460 thru 0x02e742 (0x62e3 bytes) 000109 1753742575.791670 mapread 0x023528 thru 0x02bef0 (0x89c9 bytes) 000110 1753742575.799956 trunc from 0x036e56 to 0x02398f (0x134c6 bytes) 000111 1753742575.819364 mapread 0x017b33 thru 0x017b66 (0x34 bytes) 000112 1753742575.826350 trunc from 0x015ec6 to 0x01e37e (0x84b9 bytes) 000113 1753742575.831658 trunc from 0x023990 to 0x0232e8 (0x6a7 bytes) 000114 1753742575.848414 trunc from 0x00c213 to 0x00cd88 (0xb76 bytes) 000115 1753742575.853018 trunc from 0x0126df to 0x016b53 (0x4475 bytes) 000116 1753742575.857652 trunc from 0x0232e9 to 0x00f35d (0x13f8b bytes) 000117 1753742575.875232 read 0x00bea8 thru 0x00f35d (0x34b6 bytes) 000119 1753742575.883129 fallocate 0x01aaac thru 0x01c24a (0x179f bytes) 000121 1753742575.887991 mapread 0x000b50 thru 0x0014ac (0x95d bytes) 000122 1753742575.888765 read 0x00129c thru 0x003a84 (0x27e9 bytes) 000123 1753742575.889306 mapwrite 0x02fb85 thru 0x03516a (0x55e6 bytes) 000124 1753742575.936605 trunc from 0x03516b to 0x03bcbb (0x6b51 bytes) 000125 1753742575.950532 read 0x013da1 thru 0x01e2ef (0xa54f bytes) 000126 1753742575.951183 mapread 0x00519c thru 0x0136f2 (0xe557 bytes) 000127 1753742575.951896 mapread 0x01cea8 thru 0x028515 (0xb66e bytes) 000128 1753742575.963930 trunc from 0x03bcbc to 0x02ddfb (0xdec0 bytes) 000129 1753742575.981111 trunc from 0x027b6f to 0x02ddfb (0x628d bytes) 000130 1753742575.987418 trunc from 0x02ddfc to 0x029eb4 (0x3f47 bytes) 000131 1753742576.004418 trunc from 0x01cdb4 to 0x029eb4 (0xd101 bytes) 000132 1753742576.013704 mapwrite 0x03daea thru 0x03ffff (0x2516 bytes) 000133 1753742576.066484 read 0x0146c9 thru 0x01f35b (0xac93 bytes) 000134 1753742576.071721 MIRROR_EXTEND to 2 mirrors 000135 1753742576.733584 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000136 1753742576.931156 fallocate 0x030cd9 thru 0x0354cb (0x47f3 bytes) 000137 1753742576.965258 mapwrite 0x02ceaa thru 0x038464 (0xb5bb bytes) 000138 1753742577.039032 mapwrite 0x029243 thru 0x037a42 (0xe800 bytes) 000139 1753742577.097217 mapwrite 0x015b79 thru 0x01ac7a (0x5102 bytes) 000140 1753742577.135481 mapwrite 0x0347aa thru 0x03c945 (0x819c bytes) 000142 1753742577.160978 read 0x02cfdc thru 0x03844d (0xb472 bytes) 000143 1753742577.165593 trunc from 0x020b40 to 0x02a815 (0x9cd6 bytes) 000144 1753742577.173526 mapwrite 0x01f934 thru 0x0202c7 (0x994 bytes) 000146 1753742577.210312 trunc from 0x040000 to 0x03d166 (0x2e99 bytes) 000149 1753742577.284464 read 0x0295f5 thru 0x02a148 (0xb54 bytes) 000150 1753742577.295782 trunc from 0x03d167 to 0x0221a6 (0x1afc0 bytes) 000151 1753742577.327332 fallocate 0x011f0d thru 0x019fb4 (0x80a8 bytes) 000154 1753742577.336443 write 0x00a9fc thru 0x00df23 (0x3528 bytes) 000155 1753742577.338063 fallocate 0x00f36a thru 0x013f2e (0x4bc5 bytes) 000156 1753742577.346141 mapread 0x02141f thru 0x0221a6 (0xd88 bytes) 000157 1753742577.346875 read 0x00cdeb thru 0x016157 (0x936d bytes) 000158 1753742577.347449 fallocate 0x00d651 thru 0x010f02 (0x38b2 bytes) 000159 1753742577.354744 fallocate 0x02c6e5 thru 0x037bad (0xb4c9 bytes) 000164 1753742577.362518 punch from 0x003ef5 to 0x012729 (0xe835 bytes) 000165 1753742577.372541 mapwrite 0x022b6a thru 0x02edf1 (0xc288 bytes) 000166 1753742577.480474 MIRROR_EXTEND to 2 mirrors 000167 1753742578.032498 mapread 0x00a4ce thru 0x00bbba (0x16ed bytes) 000168 1753742578.052786 punch from 0x01135a to 0x019e4e (0x8af5 bytes) 000169 1753742578.117998 MIRROR_EXTEND to 3 mirrors 000170 1753742578.685551 mapwrite 0x002976 thru 0x00b8ae (0x8f39 bytes) 000173 1753742578.803227 read 0x013232 thru 0x01f79f (0xc56e bytes) 000174 1753742578.819582 trunc from 0x020183 to 0x02d3e5 (0xd263 bytes) 000175 1753742578.827451 read 0x004914 thru 0x00f7d3 (0xaec0 bytes) 000176 1753742578.831256 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000177 1753742579.119829 fallocate 0x00d420 thru 0x018782 (0xb363 bytes) 000178 1753742579.127114 punch from 0x02ed37 to 0x02edf1 (0xbb bytes) 000179 1753742579.135586 mapread 0x014783 thru 0x01c6c2 (0x7f40 bytes) 000180 1753742579.155882 trunc from 0x00a2e7 to 0x018bdf (0xe8f9 bytes) 000181 1753742579.166295 MIRROR_RESYNC 2 mirrors 000185 1753742579.907857 mapread 0x00a38f thru 0x0164db (0xc14d bytes) 000187 1753742579.929810 fallocate 0x0268d1 thru 0x035548 (0xec78 bytes) 000188 1753742579.956258 read 0x023b91 thru 0x024d6f (0x11df bytes) 000189 1753742579.965913 mapread 0x0098d1 thru 0x014961 (0xb091 bytes) 000190 1753742579.980825 mapread 0x00865c thru 0x00a70f (0x20b4 bytes) 000191 1753742579.981714 write 0x02d9da thru 0x032cf2 (0x5319 bytes) 000192 1753742579.994893 trunc from 0x032cf3 to 0x00c934 (0x263be bytes) 000193 1753742580.027831 mapread 0x000524 thru 0x004547 (0x4024 bytes) 000194 1753742580.032988 trunc from 0x00c935 to 0x02b772 (0x1ee3e bytes) 000196 1753742580.058318 mapread 0x01fe8c thru 0x02b772 (0xb8e7 bytes) 000197 1753742580.081420 fallocate 0x02f82e thru 0x035ac2 (0x6295 bytes) 000199 1753742580.122372 MIRROR_EXTEND to 3 mirrors 000200 1753742580.667876 read 0x003995 thru 0x00524b (0x18b7 bytes) 000201 1753742580.678312 mapwrite 0x03d743 thru 0x03ffff (0x28bd bytes) 000202 1753742580.809070 mapwrite 0x02a70b thru 0x037db5 (0xd6ab bytes) 000203 1753742580.879712 mapread 0x00b34f thru 0x00dadf (0x2791 bytes) 000204 1753742580.891190 trunc from 0x040000 to 0x026fec (0x19013 bytes) 000205 1753742580.914854 write 0x01be14 thru 0x01c9ca (0xbb7 bytes) 000206 1753742580.939461 fallocate 0x03361a thru 0x03ffff (0xc9e6 bytes) 000207 1753742580.944419 punch from 0x022e78 to 0x02678c (0x3915 bytes) 000208 1753742580.948050 trunc from 0x02be4e to 0x034145 (0x82f8 bytes) 000209 1753742580.954860 write 0x02eef5 thru 0x036af0 (0x7bfc bytes) 000211 1753742580.972269 mapwrite 0x0271da thru 0x02a715 (0x353c bytes) 000213 1753742581.035284 punch from 0x03ef70 to 0x03ffff (0x1090 bytes) 000215 1753742581.045517 write 0x0168a9 thru 0x017b28 (0x1280 bytes) 000216 1753742581.052038 trunc from 0x040000 to 0x03fc66 (0x399 bytes) 000217 1753742581.073265 trunc from 0x011e81 to 0x01c6af (0xa82f bytes) 000218 1753742581.089889 trunc from 0x03fc67 to 0x03ae38 (0x4e2e bytes) 000219 1753742581.135028 MIRROR_RESYNC 3 mirrors 000220 1753742582.009872 punch from 0x00bd63 to 0x00cec7 (0x1165 bytes) 000221 1753742582.043989 trunc from 0x03ae39 to 0x01f60b (0x1b82d bytes) 000222 1753742582.067310 trunc from 0x003c99 to 0x012b17 (0xee7f bytes) 000223 1753742582.073510 trunc from 0x01f60c to 0x00d930 (0x11cdb bytes) 000224 1753742582.088582 write 0x00dc95 thru 0x018e34 (0xb1a0 bytes) 000225 1753742582.095543 trunc from 0x012159 to 0x018e34 (0x6cdc bytes) 000226 1753742582.105667 write 0x00a241 thru 0x00eea8 (0x4c68 bytes) 000227 1753742582.112205 fallocate 0x013efc thru 0x01f21d (0xb322 bytes) 000228 1753742582.124944 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000229 1753742582.513446 mapread 0x015cb1 thru 0x018e34 (0x3184 bytes) 000230 1753742582.542667 punch from 0x0033f4 to 0x00e95f (0xb56c bytes) 000231 1753742582.562261 trunc from 0x0082db to 0x00d71b (0x5441 bytes) 000232 1753742582.571843 punch from 0x010f7b to 0x018a08 (0x7a8e bytes) 000233 1753742582.579865 trunc from 0x018e35 to 0x020b8d (0x7d59 bytes) 000234 1753742582.611557 fallocate 0x00595f thru 0x00892f (0x2fd1 bytes) 000235 1753742582.618792 trunc from 0x017111 to 0x017501 (0x3f1 bytes) 000236 1753742582.623768 mapread 0x009bf7 thru 0x00ecd1 (0x50db bytes) 000237 1753742582.637015 trunc from 0x00e3b5 to 0x013c79 (0x58c5 bytes) 000238 1753742582.645270 read 0x0194f9 thru 0x01e70e (0x5216 bytes) 000239 1753742582.657678 mapread 0x00961f thru 0x009842 (0x224 bytes) 000240 1753742582.661184 fallocate 0x03fc3a thru 0x03ffff (0x3c6 bytes) 000241 1753742582.671947 mapwrite 0x037f1e thru 0x03ffff (0x80e2 bytes) 000242 1753742582.737890 trunc from 0x030987 to 0x0396b7 (0x8d31 bytes) 000243 1753742582.750270 MIRROR_EXTEND to 3 mirrors 000244 1753742583.466425 trunc from 0x006f04 to 0x00f6fe (0x87fb bytes) 000245 1753742583.531413 punch from 0x03e5c1 to 0x03e9de (0x41e bytes) 000246 1753742583.545285 trunc from 0x040000 to 0x01045f (0x2fba0 bytes) 000247 1753742583.587020 mapread 0x0100a4 thru 0x01045f (0x3bc bytes) 000249 1753742583.594511 read 0x00bba1 thru 0x00c7e4 (0xc44 bytes) 000251 1753742583.602238 mapwrite 0x029c5b thru 0x032af4 (0x8e9a bytes) 000252 1753742583.654993 mapread 0x023f81 thru 0x029dba (0x5e3a bytes) 000253 1753742583.656873 read 0x000f21 thru 0x00f31d (0xe3fd bytes) 000254 1753742583.664881 trunc from 0x032af5 to 0x030f89 (0x1b6b bytes) 000255 1753742583.682283 fallocate 0x0122fe thru 0x02062d (0xe330 bytes) 000256 1753742583.688828 trunc from 0x011623 to 0x01296a (0x1348 bytes) 000257 1753742583.695457 trunc from 0x0043ab to 0x009f02 (0x5b58 bytes) 000258 1753742583.705117 fallocate 0x000598 thru 0x000cd8 (0x741 bytes) 000259 1753742583.712044 mapwrite 0x0031a7 thru 0x012e0b (0xfc65 bytes) 000260 1753742583.822677 mapread 0x0213c3 thru 0x0306f9 (0xf337 bytes) 000261 1753742583.825338 mapread 0x022571 thru 0x02829a (0x5d2a bytes) 000262 1753742583.826886 write 0x0132ff thru 0x01b020 (0x7d22 bytes) 000263 1753742583.831286 fallocate 0x00e1c9 thru 0x015a51 (0x7889 bytes) 000264 1753742583.850267 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000265 1753742584.091426 write 0x013b86 thru 0x022abe (0xef39 bytes) 000267 1753742584.111512 punch from 0x00c392 to 0x00c6d5 (0x344 bytes) 000268 1753742584.131451 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000269 1753742584.261375 trunc from 0x005031 to 0x007f6b (0x2f3b bytes) 000270 1753742584.270579 mapwrite 0x00f1f0 thru 0x0104d0 (0x12e1 bytes) 000271 1753742584.297923 mapread 0x004034 thru 0x00642f (0x23fc bytes) 000272 1753742584.310171 fallocate 0x000a29 thru 0x004ac5 (0x409d bytes) 000273 1753742584.314543 trunc from 0x02480f to 0x02c7ee (0x7fe0 bytes) 000274 1753742584.321251 write 0x010fd6 thru 0x01383d (0x2868 bytes) 000275 1753742584.337626 mapread 0x016fb0 thru 0x0257d0 (0xe821 bytes) 000276 1753742584.356544 mapread 0x007ab8 thru 0x01783b (0xfd84 bytes) 000277 1753742584.357466 trunc from 0x030f8a to 0x016f15 (0x1a074 bytes) 000278 1753742584.374828 read 0x00466b thru 0x01362f (0xefc5 bytes) 000279 1753742584.375771 trunc from 0x00d563 to 0x016f15 (0x99b3 bytes) 000280 1753742584.397592 read 0x002833 thru 0x0103cd (0xdb9b bytes) 000281 1753742584.405739 trunc from 0x012e4a to 0x016f15 (0x40cc bytes) 000282 1753742584.419740 trunc from 0x016f16 to 0x03bc36 (0x24d21 bytes) 000283 1753742584.438073 mapwrite 0x01669b thru 0x01dc30 (0x7596 bytes) 000284 1753742584.515962 write 0x001097 thru 0x00c8f0 (0xb85a bytes) 000285 1753742584.525473 MIRROR_EXTEND to 2 mirrors 000286 1753742585.124928 punch from 0x02da8b to 0x02dab1 (0x27 bytes) 000287 1753742585.171034 punch from 0x00c872 to 0x0106ba (0x3e49 bytes) 000289 1753742585.178498 trunc from 0x0036e6 to 0x008a1d (0x5338 bytes) 000291 1753742585.188065 punch from 0x01f64c to 0x0225f2 (0x2fa7 bytes) 000292 1753742585.196976 read 0x00ff3a thru 0x0198a7 (0x996e bytes) 000293 1753742585.211767 MIRROR_RESYNC 2 mirrors 000294 1753742585.761861 trunc from 0x03bc37 to 0x026cef (0x14f47 bytes) 000295 1753742585.791113 write 0x02c79d thru 0x03263d (0x5ea1 bytes) 000296 1753742585.792497 read 0x02c630 thru 0x03263d (0x600e bytes) 000297 1753742585.793034 mapwrite 0x0394f9 thru 0x03ffff (0x6b07 bytes) 000298 1753742585.850928 trunc from 0x007329 to 0x00e09a (0x6d72 bytes) 000299 1753742585.856462 punch from 0x00f566 to 0x01a6a6 (0xb141 bytes) 000300 1753742585.862037 read 0x0172ca thru 0x02127a (0x9fb1 bytes) 000301 1753742585.873998 mapread 0x0122aa thru 0x015c94 (0x39eb bytes) 000302 1753742585.874884 write 0x015bb0 thru 0x01dc98 (0x80e9 bytes) 000303 1753742585.875961 read 0x0352d3 thru 0x03aeaa (0x5bd8 bytes) 000305 1753742585.876991 mapwrite 0x00e7cc thru 0x013d54 (0x5589 bytes) 000306 1753742585.893039 trunc from 0x0175eb to 0x017792 (0x1a8 bytes) 000307 1753742585.898791 read 0x03a6ae thru 0x03ffff (0x5952 bytes) 000308 1753742585.899410 write 0x01d926 thru 0x0241d4 (0x68af bytes) 000309 1753742585.900462 write 0x0338b9 thru 0x0355f2 (0x1d3a bytes) 000310 1753742585.901248 fallocate 0x03eafc thru 0x03ffff (0x1504 bytes) 000311 1753742585.907807 punch from 0x025e5c to 0x027766 (0x190b bytes) 000312 1753742585.912026 trunc from 0x040000 to 0x010e3c (0x2f1c3 bytes) 000313 1753742585.923047 write 0x003ceb thru 0x009fcb (0x62e1 bytes) 000314 1753742585.925536 punch from 0x007613 to 0x00f75f (0x814d bytes) 000315 1753742585.955563 write 0x005494 thru 0x01320d (0xdd7a bytes) 000316 1753742585.957019 read 0x010ce8 thru 0x012f92 (0x22ab bytes) 000317 1753742585.962501 trunc from 0x01320e to 0x000287 (0x12f86 bytes) 000318 1753742585.993840 MIRROR_EXTEND to 3 mirrors 000319 1753742586.362376 MIRROR_RESYNC 3 mirrors 000320 1753742586.741574 mapread 0x000206 thru 0x000287 (0x82 bytes) 000321 1753742586.746440 trunc from 0x000176 to 0x000287 (0x112 bytes) 000322 1753742586.776928 write 0x01d308 thru 0x02642b (0x9124 bytes) 000323 1753742586.778369 mapwrite 0x0382ec thru 0x03ffff (0x7d14 bytes) 000324 1753742586.862575 trunc from 0x02c68a to 0x038300 (0xbc77 bytes) 000325 1753742586.868549 read 0x039ba3 thru 0x03ffff (0x645d bytes) 000326 1753742586.869170 punch from 0x032a5e to 0x03ffff (0xd5a2 bytes) 000327 1753742586.885393 MIRROR_EXTEND to 4 mirrors 000328 1753742587.538761 mapwrite 0x02e33a thru 0x033a3d (0x5704 bytes) 000330 1753742587.669077 write 0x03f733 thru 0x03ffff (0x8cd bytes) 000332 1753742587.693896 write 0x028642 thru 0x03329f (0xac5e bytes) 000333 1753742587.699806 trunc from 0x040000 to 0x01bbb2 (0x2444d bytes) 000334 1753742587.748440 MIRROR_EXTEND to 5 mirrors 000335 1753742588.285509 punch from 0x00087e to 0x006190 (0x5913 bytes) 000336 1753742588.314158 trunc from 0x01bbb3 to 0x01aaa5 (0x110d bytes) 000337 1753742588.339833 mapwrite 0x029762 thru 0x03286c (0x910b bytes) 000338 1753742588.388245 trunc from 0x03286d to 0x0293c9 (0x94a3 bytes) 000339 1753742588.405106 mapread 0x020406 thru 0x021005 (0xc00 bytes) 000340 1753742588.413862 punch from 0x00946b to 0x016102 (0xcc98 bytes) 000341 1753742588.419486 mapwrite 0x028cbf thru 0x030887 (0x7bc9 bytes) 000342 1753742588.457071 mapwrite 0x009613 thru 0x0139f9 (0xa3e7 bytes) 000343 1753742588.500451 mapread 0x00dc7c thru 0x01a896 (0xcc1b bytes) 000344 1753742588.501430 mapwrite 0x00381b thru 0x00dd2e (0xa514 bytes) 000345 1753742588.524668 trunc from 0x030888 to 0x003a34 (0x2ce53 bytes) 000346 1753742588.561917 punch from 0x00075b to 0x003a34 (0x32da bytes) 000347 1753742588.586125 mapread 0x002268 thru 0x003a34 (0x17cd bytes) 000348 1753742588.591255 fallocate 0x013917 thru 0x015246 (0x1930 bytes) 000349 1753742588.606678 trunc from 0x00924c to 0x00b9de (0x2793 bytes) 000350 1753742588.624812 MIRROR_SPLIT mirror 2 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000352 1753742588.830062 fallocate 0x03f74e thru 0x03fbf9 (0x4ac bytes) 000353 1753742588.834559 write 0x039b24 thru 0x03ffff (0x64dc bytes) 000354 1753742588.844603 mapwrite 0x032551 thru 0x03ffff (0xdaaf bytes) 000355 1753742588.887357 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000356 1753742589.220116 trunc from 0x0145a9 to 0x023f3a (0xf992 bytes) 000357 1753742589.238147 mapwrite 0x0210dd thru 0x02a133 (0x9057 bytes) 000358 1753742589.351602 MIRROR_RESYNC 3 mirrors 000359 1753742590.184740 write 0x0168a3 thru 0x017ce6 (0x1444 bytes) 000360 1753742590.268314 read 0x01f91f thru 0x02ea5c (0xf13e bytes) 000363 1753742590.291732 trunc from 0x040000 to 0x03d512 (0x2aed bytes) 000364 1753742590.361455 mapwrite 0x0227df thru 0x02a90c (0x812e bytes) 000365 1753742590.408535 trunc from 0x03d513 to 0x016d8c (0x26786 bytes) 000366 1753742590.429462 trunc from 0x016d8d to 0x000973 (0x16419 bytes) 000367 1753742590.448375 read 0x000304 thru 0x000973 (0x670 bytes) 000368 1753742590.455152 punch from 0x00071b to 0x000973 (0x259 bytes) 000369 1753742590.462521 fallocate 0x03f711 thru 0x03ffff (0x8ef bytes) 000370 1753742590.466886 read 0x021726 thru 0x024066 (0x2941 bytes) 000371 1753742590.477143 trunc from 0x040000 to 0x03af62 (0x509d bytes) 000373 1753742590.500408 MIRROR_RESYNC 3 mirrors 000374 1753742590.912724 write 0x007da8 thru 0x00c32b (0x4584 bytes) 000375 1753742590.956005 mapwrite 0x026a5c thru 0x02ba46 (0x4feb bytes) 000377 1753742590.989939 punch from 0x023ad2 to 0x032f4e (0xf47d bytes) 000378 1753742590.994067 fallocate 0x037396 thru 0x03ffff (0x8c6a bytes) 000379 1753742591.000838 trunc from 0x03af63 to 0x01118c (0x29dd6 bytes) 000380 1753742591.034497 trunc from 0x01118d to 0x037923 (0x26797 bytes) 000381 1753742591.059020 fallocate 0x010000 thru 0x010706 (0x707 bytes) 000382 1753742591.065279 read 0x0366ba thru 0x0377d9 (0x1120 bytes) 000384 1753742591.071675 mapwrite 0x003bd3 thru 0x009e88 (0x62b6 bytes) 000385 1753742591.106264 MIRROR_RESYNC 3 mirrors 000386 1753742591.651847 read 0x000e20 thru 0x006385 (0x5566 bytes) 000387 1753742591.660569 MIRROR_RESYNC 3 mirrors 000388 1753742591.830555 trunc from 0x037924 to 0x02820e (0xf715 bytes) 000389 1753742591.911687 trunc from 0x003c47 to 0x013a2b (0xfde5 bytes) 000390 1753742591.917607 trunc from 0x02820f to 0x02c7eb (0x45dd bytes) 000391 1753742591.948634 MIRROR_RESYNC 3 mirrors 000392 1753742592.511519 fallocate 0x0081e2 thru 0x0087fc (0x61b bytes) 000393 1753742592.550621 write 0x02e62b thru 0x03af10 (0xc8e6 bytes) 000394 1753742592.554316 trunc from 0x0261a6 to 0x0316cc (0xb527 bytes) 000396 1753742592.575058 fallocate 0x018361 thru 0x01abac (0x284c bytes) 000397 1753742592.588544 write 0x00561c thru 0x00c922 (0x7307 bytes) 000398 1753742592.608863 trunc from 0x039dcb to 0x03af10 (0x1146 bytes) 000399 1753742592.623196 mapwrite 0x0299d9 thru 0x02a0ff (0x727 bytes) 000400 1753742592.664754 MIRROR_RESYNC 3 mirrors 000401 1753742593.578029 punch from 0x0280f9 to 0x037ca3 (0xfbab bytes) 000402 1753742593.617165 punch from 0x02c919 to 0x03597e (0x9066 bytes) 000403 1753742593.626765 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000404 1753742593.810441 trunc from 0x02b93e to 0x02ea9c (0x315f bytes) 000405 1753742593.821913 read 0x020598 thru 0x02b424 (0xae8d bytes) 000406 1753742593.833123 fallocate 0x03beb6 thru 0x03ffff (0x414a bytes) 000407 1753742593.840450 punch from 0x00c267 to 0x012b79 (0x6913 bytes) 000408 1753742593.846499 read 0x000757 thru 0x00e092 (0xd93c bytes) 000409 1753742593.860918 write 0x0298ab thru 0x0302f2 (0x6a48 bytes) 000410 1753742593.873021 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000411 1753742594.087268 read 0x023a15 thru 0x02d249 (0x9835 bytes) 000412 1753742594.100936 mapread 0x006caa thru 0x00d022 (0x6379 bytes) 000414 1753742594.140544 punch from 0x02e50d to 0x033267 (0x4d5b bytes) 000415 1753742594.155554 punch from 0x01178f to 0x014f23 (0x3795 bytes) 000416 1753742594.168744 read 0x0131a0 thru 0x0185c6 (0x5427 bytes) 000417 1753742594.196139 mapwrite 0x00582a thru 0x0092b7 (0x3a8e bytes) 000418 1753742594.239780 mapread 0x02ed0c thru 0x03af10 (0xc205 bytes) 000419 1753742594.253439 MIRROR_EXTEND to 2 mirrors 000420 1753742595.190267 mapwrite 0x007354 thru 0x0092c2 (0x1f6f bytes) 000422 1753742595.342461 mapread 0x028bab thru 0x036405 (0xd85b bytes) 000424 1753742595.356065 punch from 0x004a30 to 0x00f76a (0xad3b bytes) 000425 1753742595.360488 mapwrite 0x00632f thru 0x00c54e (0x6220 bytes) 000426 1753742595.388368 MIRROR_EXTEND to 3 mirrors 000427 1753742596.119414 write 0x02730d thru 0x02740e (0x102 bytes) 000428 1753742596.191487 trunc from 0x03af11 to 0x01f5eb (0x1b925 bytes) 000429 1753742596.240854 mapwrite 0x0267a6 thru 0x02b691 (0x4eec bytes) 000430 1753742596.315483 trunc from 0x02b692 to 0x02c557 (0xec6 bytes) 000431 1753742596.346150 trunc from 0x02c558 to 0x030ecf (0x4978 bytes) 000432 1753742596.375600 trunc from 0x030ed0 to 0x02832f (0x8ba0 bytes) 000434 1753742596.400645 mapread 0x0122d1 thru 0x0194b5 (0x71e5 bytes) 000435 1753742596.428819 MIRROR_RESYNC 3 mirrors 000436 1753742597.268823 read 0x01ebe6 thru 0x02133c (0x2757 bytes) 000437 1753742597.276277 read 0x00ae38 thru 0x00c43a (0x1603 bytes) 000438 1753742597.295642 write 0x00c196 thru 0x0148e9 (0x8754 bytes) 000439 1753742597.343058 mapread 0x001eea thru 0x004c21 (0x2d38 bytes) 000440 1753742597.369984 fallocate 0x00fce3 thru 0x016058 (0x6376 bytes) 000441 1753742597.386933 MIRROR_RESYNC 3 mirrors 000442 1753742598.092052 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000443 1753742598.346377 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000445 1753742598.465839 read 0x009400 thru 0x00d70d (0x430e bytes) 000446 1753742598.477252 trunc from 0x02744b to 0x02832f (0xee5 bytes) 000447 1753742598.494175 trunc from 0x025b1c to 0x02832f (0x2814 bytes) 000448 1753742598.501246 read 0x001a48 thru 0x00665e (0x4c17 bytes) 000449 1753742598.514217 punch from 0x027181 to 0x02832f (0x11af bytes) 000450 1753742598.521251 mapread 0x015d03 thru 0x01a00b (0x4309 bytes) 000451 1753742598.522592 write 0x0325fb thru 0x0384b0 (0x5eb6 bytes) 000453 1753742598.529464 mapwrite 0x01ade4 thru 0x0227ce (0x79eb bytes) 000454 1753742598.552375 trunc from 0x0304b4 to 0x0362e5 (0x5e32 bytes) 000455 1753742598.557583 punch from 0x032cb0 to 0x036919 (0x3c6a bytes) 000456 1753742598.563573 mapwrite 0x033355 thru 0x03ece8 (0xb994 bytes) 000457 1753742598.631592 trunc from 0x00592a to 0x012bd0 (0xd2a7 bytes) 000458 1753742598.637005 read 0x004a32 thru 0x00fedb (0xb4aa bytes) 000459 1753742598.650249 mapread 0x0085fd thru 0x01330f (0xad13 bytes) 000460 1753742598.651571 punch from 0x025c01 to 0x033c18 (0xe018 bytes) 000461 1753742598.660933 mapwrite 0x0147e9 thru 0x018e2b (0x4643 bytes) 000463 1753742598.680880 trunc from 0x03ece9 to 0x034d20 (0x9fc8 bytes) 000464 1753742598.695070 mapwrite 0x00d96f thru 0x00f88c (0x1f1e bytes) 000465 1753742598.718723 trunc from 0x034d21 to 0x02a03a (0xace6 bytes) 000466 1753742598.736668 write 0x0225a9 thru 0x02b8df (0x9337 bytes) 000467 1753742598.737945 write 0x011c71 thru 0x01da9f (0xbe2f bytes) 000468 1753742598.748062 MIRROR_EXTEND to 2 mirrors 000469 1753742599.352720 MIRROR_EXTEND to 3 mirrors 000470 1753742599.945822 trunc from 0x02b8e0 to 0x01920c (0x126d3 bytes) 000471 1753742600.008496 read 0x0031f9 thru 0x0049be (0x17c6 bytes) 000472 1753742600.021316 trunc from 0x018895 to 0x01920c (0x978 bytes) 000473 1753742600.025464 mapread 0x007711 thru 0x00dfa3 (0x6893 bytes) 000474 1753742600.054661 trunc from 0x003e58 to 0x0135b5 (0xf75e bytes) 000475 1753742600.059311 mapwrite 0x02d953 thru 0x0398d9 (0xbf87 bytes) 000476 1753742600.113011 read 0x02336e thru 0x032b5c (0xf7ef bytes) 000477 1753742600.122111 trunc from 0x031623 to 0x035e15 (0x47f3 bytes) 000478 1753742600.127649 punch from 0x00495a to 0x008523 (0x3bca bytes) 000479 1753742600.131813 mapwrite 0x004d62 thru 0x00bdae (0x704d bytes) 000480 1753742600.182860 mapwrite 0x03fd0f thru 0x03ffff (0x2f1 bytes) 000481 1753742600.224950 mapread 0x0014fe thru 0x009af0 (0x85f3 bytes) 000482 1753742600.225930 read 0x03e8f9 thru 0x03ffff (0x1707 bytes) 000483 1753742600.227460 mapread 0x01d289 thru 0x025f2c (0x8ca4 bytes) 000484 1753742600.229779 write 0x03b4a3 thru 0x03ffff (0x4b5d bytes) 000485 1753742600.241587 MIRROR_RESYNC 3 mirrors 000486 1753742601.248398 mapwrite 0x0086fb thru 0x0093fe (0xd04 bytes) 000488 1753742601.343913 trunc from 0x002945 to 0x0051f0 (0x28ac bytes) 000489 1753742601.349962 read 0x03a2f2 thru 0x03d664 (0x3373 bytes) 000490 1753742601.358438 mapread 0x03ca98 thru 0x03ffff (0x3568 bytes) 000491 1753742601.366649 read 0x004129 thru 0x00a630 (0x6508 bytes) 000492 1753742601.375225 write 0x016021 thru 0x01c4c4 (0x64a4 bytes) 000493 1753742601.376533 write 0x003649 thru 0x009ed2 (0x688a bytes) 000494 1753742601.378220 write 0x02cdf7 thru 0x036950 (0x9b5a bytes) 000495 1753742601.380417 trunc from 0x040000 to 0x034430 (0xbbcf bytes) 000496 1753742601.401963 mapwrite 0x01de7d thru 0x01fb76 (0x1cfa bytes) 000497 1753742601.429737 fallocate 0x006b99 thru 0x0142cb (0xd733 bytes) 000498 1753742601.440782 MIRROR_EXTEND to 4 mirrors 000499 1753742601.940301 fallocate 0x015caf thru 0x025104 (0xf456 bytes) 000500 1753742601.977124 trunc from 0x034431 to 0x011fee (0x22442 bytes) 000502 1753742602.013533 trunc from 0x0119f9 to 0x011fee (0x5f6 bytes) 000503 1753742602.017969 punch from 0x00fedf to 0x011457 (0x1579 bytes) 000504 1753742602.022657 read 0x00d49f thru 0x011fee (0x4b50 bytes) 000505 1753742602.029227 punch from 0x00bec3 to 0x011fee (0x612c bytes) 000506 1753742602.033824 fallocate 0x03ef9d thru 0x03fec0 (0xf24 bytes) 000507 1753742602.038526 mapwrite 0x00c4f5 thru 0x0112be (0x4dca bytes) 000508 1753742602.077333 fallocate 0x03d376 thru 0x03ffff (0x2c8a bytes) 000509 1753742602.083870 read 0x035624 thru 0x03b38d (0x5d6a bytes) 000510 1753742602.093823 write 0x036e72 thru 0x03e415 (0x75a4 bytes) 000511 1753742602.094937 trunc from 0x040000 to 0x01e178 (0x21e87 bytes) 000512 1753742602.116976 write 0x025c0e thru 0x02b632 (0x5a25 bytes) 000513 1753742602.118360 write 0x02fe9e thru 0x032cf8 (0x2e5b bytes) 000514 1753742602.119253 read 0x02b258 thru 0x02bb14 (0x8bd bytes) 000516 1753742602.120438 read 0x0178e3 thru 0x01df3d (0x665b bytes) 000517 1753742602.122348 trunc from 0x032cf9 to 0x000a98 (0x32260 bytes) 000518 1753742602.141133 punch from 0x00096a to 0x000a98 (0x12f bytes) 000519 1753742602.147321 trunc from 0x000a99 to 0x035f8b (0x354f3 bytes) 000521 1753742602.165461 fallocate 0x020d67 thru 0x02db04 (0xcd9e bytes) 000522 1753742602.171121 mapwrite 0x03f81f thru 0x03ffff (0x7e1 bytes) 000523 1753742602.223944 MIRROR_RESYNC 4 mirrors 000524 1753742602.850226 trunc from 0x000a50 to 0x00469b (0x3c4c bytes) 000525 1753742602.954439 punch from 0x01a30b to 0x01d379 (0x306f bytes) 000526 1753742602.976756 punch from 0x00d9a5 to 0x01c7a5 (0xee01 bytes) 000527 1753742602.982598 punch from 0x01e614 to 0x025e6a (0x7857 bytes) 000528 1753742602.992722 write 0x013e75 thru 0x019803 (0x598f bytes) 000529 1753742603.019997 mapread 0x017ad0 thru 0x01b791 (0x3cc2 bytes) 000530 1753742603.041323 punch from 0x03d079 to 0x03ffff (0x2f87 bytes) 000531 1753742603.059727 mapwrite 0x0362ea thru 0x03ffff (0x9d16 bytes) 000532 1753742603.130999 read 0x0176c8 thru 0x01e6a5 (0x6fde bytes) 000535 1753742603.133940 trunc from 0x0022cf to 0x004515 (0x2247 bytes) 000536 1753742603.149809 MIRROR_EXTEND to 5 mirrors 000537 1753742603.773765 mapwrite 0x039d9c thru 0x03ffff (0x6264 bytes) 000539 1753742603.892708 fallocate 0x00a046 thru 0x0124f5 (0x84b0 bytes) 000540 1753742603.897767 mapwrite 0x02dc38 thru 0x036502 (0x88cb bytes) 000541 1753742603.943482 trunc from 0x006189 to 0x0076c0 (0x1538 bytes) 000542 1753742603.948867 mapread 0x00f14e thru 0x01bc0a (0xcabd bytes) 000544 1753742603.974298 punch from 0x01035a to 0x014700 (0x43a7 bytes) 000545 1753742603.987016 read 0x006c83 thru 0x00bdf8 (0x5176 bytes) 000546 1753742603.995199 trunc from 0x040000 to 0x027bf7 (0x18408 bytes) 000547 1753742604.039547 trunc from 0x01660a to 0x0188e1 (0x22d8 bytes) 000548 1753742604.044007 punch from 0x00c6b9 to 0x016300 (0x9c48 bytes) 000549 1753742604.049898 punch from 0x020de9 to 0x02529a (0x44b2 bytes) 000550 1753742604.066603 MIRROR_RESYNC 5 mirrors 000551 1753742605.105123 mapwrite 0x009e05 thru 0x00e102 (0x42fe bytes) 000552 1753742605.228959 trunc from 0x027bf8 to 0x0338a2 (0xbcab bytes) 000553 1753742605.265886 trunc from 0x0338a3 to 0x008aaa (0x2adf8 bytes) 000554 1753742605.284475 trunc from 0x008aab to 0x0328f3 (0x29e49 bytes) 000555 1753742605.305033 read 0x00ee2f thru 0x01e157 (0xf329 bytes) 000556 1753742605.318998 fallocate 0x03db5d thru 0x03ffff (0x24a3 bytes) 000557 1753742605.335472 MIRROR_EXTEND to 6 mirrors 000558 1753742605.964751 trunc from 0x0328f4 to 0x013bf3 (0x1ed00 bytes) 000559 1753742606.074673 MIRROR_RESYNC 6 mirrors 000560 1753742606.875682 punch from 0x0037b8 to 0x013430 (0xfc79 bytes) 000561 1753742606.917351 read 0x0020e9 thru 0x00c98d (0xa8a5 bytes) 000562 1753742606.930742 fallocate 0x02776e thru 0x02e48e (0x6d21 bytes) 000563 1753742606.936278 read 0x005453 thru 0x00e004 (0x8bb2 bytes) 000564 1753742606.957265 MIRROR_RESYNC 6 mirrors 000565 1753742607.708496 trunc from 0x02e48f to 0x01d61f (0x10e6f bytes) 000566 1753742607.742976 read 0x009bee thru 0x016aba (0xcecd bytes) 000567 1753742607.750985 fallocate 0x02124a thru 0x0276de (0x6495 bytes) 000568 1753742607.757342 mapwrite 0x00374c thru 0x00c878 (0x912d bytes) 000569 1753742607.799347 MIRROR_EXTEND to 7 mirrors 000570 1753742608.345347 fallocate 0x015de9 thru 0x01655c (0x774 bytes) 000571 1753742608.396416 MIRROR_EXTEND to 8 mirrors 000572 1753742608.995974 punch from 0x023cf8 to 0x0276de (0x39e7 bytes) 000573 1753742609.022058 trunc from 0x01cabe to 0x025d58 (0x929b bytes) 000574 1753742609.028755 mapread 0x0230e8 thru 0x0276de (0x45f7 bytes) 000575 1753742609.045866 mapwrite 0x02284a thru 0x02b205 (0x89bc bytes) 000576 1753742609.098232 read 0x01d8f6 thru 0x0240f2 (0x67fd bytes) 000577 1753742609.104329 mapwrite 0x012204 thru 0x02130d (0xf10a bytes) 000578 1753742609.140943 mapread 0x001917 thru 0x008136 (0x6820 bytes) 000579 1753742609.155922 MIRROR_EXTEND to 9 mirrors 000580 1753742609.613068 fallocate 0x020322 thru 0x0301b3 (0xfe92 bytes) 000581 1753742609.677807 MIRROR_RESYNC 9 mirrors 000583 1753742611.235747 mapwrite 0x02da7a thru 0x03ce2e (0xf3b5 bytes) 000584 1753742611.327292 read 0x00777f thru 0x0155e3 (0xde65 bytes) 000585 1753742611.334725 trunc from 0x00ffe2 to 0x01bf30 (0xbf4f bytes) 000586 1753742611.339902 punch from 0x035677 to 0x03a26b (0x4bf5 bytes) 000587 1753742611.345308 trunc from 0x03ce2f to 0x03d47a (0x64c bytes) 000588 1753742611.364684 MIRROR_RESYNC 9 mirrors 000589 1753742614.185217 trunc from 0x03a15d to 0x03b3fe (0x12a2 bytes) 000590 1753742614.230761 trunc from 0x02edf2 to 0x037786 (0x8995 bytes) 000591 1753742614.252795 MIRROR_EXTEND to 10 mirrors 000592 1753742614.866857 trunc from 0x039159 to 0x03d47a (0x4322 bytes) 000593 1753742614.898521 write 0x036799 thru 0x036cc5 (0x52d bytes) 000595 1753742614.916850 mapread 0x00bb54 thru 0x019b99 (0xe046 bytes) 000596 1753742614.936953 trunc from 0x03d47b to 0x016326 (0x27154 bytes) 000598 1753742614.958991 write 0x033cfa thru 0x0388d5 (0x4bdc bytes) 000599 1753742614.960258 trunc from 0x00792b to 0x013bff (0xc2d5 bytes) 000600 1753742614.966629 fallocate 0x01a8a5 thru 0x01df4f (0x36ab bytes) 000601 1753742614.972709 punch from 0x00a848 to 0x016586 (0xbd3f bytes) 000602 1753742614.977417 read 0x01345c thru 0x020c5f (0xd804 bytes) 000603 1753742614.984680 mapread 0x006b38 thru 0x01671b (0xfbe4 bytes) 000604 1753742614.998623 read 0x02203f thru 0x02efaf (0xcf71 bytes) 000605 1753742615.007466 fallocate 0x036c18 thru 0x03ffff (0x93e8 bytes) 000606 1753742615.013196 write 0x00f0a9 thru 0x01ac41 (0xbb99 bytes) 000607 1753742615.014467 read 0x032e83 thru 0x035b5d (0x2cdb bytes) 000608 1753742615.015009 read 0x0273f7 thru 0x03198f (0xa599 bytes) 000609 1753742615.024752 MIRROR_RESYNC 10 mirrors 000610 1753742617.357115 write 0x02083c thru 0x026c18 (0x63dd bytes) 000611 1753742617.439792 fallocate 0x01d36a thru 0x026638 (0x92cf bytes) 000612 1753742617.443603 fallocate 0x03ff77 thru 0x03ffff (0x89 bytes) 000613 1753742617.448451 trunc from 0x03b105 to 0x03ffff (0x4efb bytes) 000614 1753742617.453482 read 0x03c5b0 thru 0x03ffff (0x3a50 bytes) 000615 1753742617.461322 write 0x019812 thru 0x0285af (0xed9e bytes) 000616 1753742617.476173 mapread 0x01482f thru 0x023a9c (0xf26e bytes) 000617 1753742617.478117 fallocate 0x03e457 thru 0x03fb3b (0x16e5 bytes) 000618 1753742617.492134 MIRROR_SPLIT mirror 1 to 9 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000619 1753742617.782549 mapread 0x00e6c5 thru 0x018e00 (0xa73c bytes) 000620 1753742617.830119 trunc from 0x02b58f to 0x03114a (0x5bbc bytes) 000621 1753742617.836852 read 0x021113 thru 0x023166 (0x2054 bytes) 000623 1753742617.853921 MIRROR_SPLIT mirror 7 to 8 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000624 1753742618.035633 trunc from 0x040000 to 0x0295fa (0x16a05 bytes) 000625 1753742618.052772 mapread 0x023158 thru 0x0295fa (0x64a3 bytes) 000626 1753742618.075438 mapwrite 0x024e1b thru 0x030b44 (0xbd2a bytes) 000627 1753742618.119711 punch from 0x0253a5 to 0x02f903 (0xa55f bytes) 000628 1753742618.124811 trunc from 0x01b2f7 to 0x024108 (0x8e12 bytes) 000629 1753742618.129377 fallocate 0x00f2eb thru 0x015c7d (0x6993 bytes) 000630 1753742618.133744 read 0x00d135 thru 0x012e83 (0x5d4f bytes) 000632 1753742618.163468 punch from 0x023e84 to 0x0307a6 (0xc923 bytes) 000633 1753742618.171645 trunc from 0x030b45 to 0x027f3b (0x8c09 bytes) 000634 1753742618.188138 trunc from 0x017446 to 0x026c13 (0xf7ce bytes) 000635 1753742618.198815 read 0x02031b thru 0x025072 (0x4d58 bytes) 000636 1753742618.211938 trunc from 0x027f3c to 0x0336ab (0xb770 bytes) 000637 1753742618.224620 mapread 0x01c7f9 thru 0x0284ee (0xbcf6 bytes) 000638 1753742618.234995 MIRROR_SPLIT mirror 1 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000639 1753742618.489465 write 0x001dff thru 0x005549 (0x374b bytes) 000641 1753742618.508824 fallocate 0x02c46f thru 0x0363fb (0x9f8d bytes) 000642 1753742618.512463 write 0x0349d9 thru 0x03dd14 (0x933c bytes) 000643 1753742618.513932 write 0x027cda thru 0x031198 (0x94bf bytes) 000644 1753742618.526754 read 0x016058 thru 0x01f827 (0x97d0 bytes) 000645 1753742618.537972 MIRROR_RESYNC 7 mirrors 000646 1753742620.176750 mapwrite 0x013164 thru 0x01f2c5 (0xc162 bytes) 000647 1753742620.274436 fallocate 0x03bbb6 thru 0x03ffff (0x444a bytes) 000648 1753742620.285541 MIRROR_SPLIT mirror 3 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000649 1753742620.557178 read 0x00dcb0 thru 0x01be15 (0xe166 bytes) 000651 1753742620.582428 MIRROR_SPLIT mirror 1 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000652 1753742620.796468 MIRROR_EXTEND to 6 mirrors 000653 1753742621.439058 trunc from 0x040000 to 0x0329c8 (0xd637 bytes) 000654 1753742621.490863 fallocate 0x00753e thru 0x007a35 (0x4f8 bytes) 000655 1753742621.501388 MIRROR_EXTEND to 7 mirrors 000656 1753742622.075826 punch from 0x01b935 to 0x01f2c5 (0x3991 bytes) 000657 1753742622.119849 trunc from 0x024831 to 0x02ce4e (0x861e bytes) 000658 1753742622.125105 mapwrite 0x003594 thru 0x013275 (0xfce2 bytes) 000659 1753742622.167777 punch from 0x0168f9 to 0x01fcdd (0x93e5 bytes) 000660 1753742622.177642 mapread 0x00abcd thru 0x018e91 (0xe2c5 bytes) 000661 1753742622.196548 trunc from 0x0329c9 to 0x032971 (0x57 bytes) 000662 1753742622.231991 write 0x030096 thru 0x0301a7 (0x112 bytes) 000663 1753742622.258060 fallocate 0x028ddc thru 0x032791 (0x99b6 bytes) 000664 1753742622.264997 write 0x00a154 thru 0x019580 (0xf42d bytes) 000665 1753742622.266425 trunc from 0x011a33 to 0x01874c (0x6d1a bytes) 000666 1753742622.297688 read 0x018872 thru 0x021ac3 (0x9252 bytes) 000667 1753742622.310891 trunc from 0x032972 to 0x005a01 (0x2cf70 bytes) 000668 1753742622.344574 MIRROR_EXTEND to 8 mirrors 000669 1753742622.748314 MIRROR_RESYNC 8 mirrors 000670 1753742623.493786 trunc from 0x005a02 to 0x02140f (0x1ba0e bytes) 000671 1753742623.569141 trunc from 0x021410 to 0x03b698 (0x1a289 bytes) 000672 1753742623.588045 read 0x01f28e thru 0x01f520 (0x293 bytes) 000673 1753742623.593051 read 0x00f720 thru 0x01d8c1 (0xe1a2 bytes) 000674 1753742623.603923 fallocate 0x01faf4 thru 0x025087 (0x5594 bytes) 000675 1753742623.610266 read 0x00d15f thru 0x00ede8 (0x1c8a bytes) 000676 1753742623.611407 trunc from 0x03b699 to 0x001dea (0x398ae bytes) 000677 1753742623.625382 read 0x000a19 thru 0x001dea (0x13d2 bytes) 000678 1753742623.629864 MIRROR_RESYNC 8 mirrors 000679 1753742624.319398 trunc from 0x001deb to 0x03fdc8 (0x3dfde bytes) 000680 1753742624.359893 trunc from 0x00507b to 0x00e818 (0x979e bytes) 000681 1753742624.364434 trunc from 0x03fdc9 to 0x0050ec (0x3acdc bytes) 000682 1753742624.386517 read 0x000c24 thru 0x0050ec (0x44c9 bytes) 000683 1753742624.396889 punch from 0x0049a3 to 0x0050ec (0x74a bytes) 000684 1753742624.402539 read 0x004747 thru 0x0050ec (0x9a6 bytes) 000685 1753742624.406802 trunc from 0x0050ed to 0x02f771 (0x2a685 bytes) 000686 1753742624.422527 mapread 0x029498 thru 0x02bac0 (0x2629 bytes) 000687 1753742624.432874 trunc from 0x02e4c8 to 0x02f771 (0x12aa bytes) 000688 1753742624.438131 fallocate 0x001e49 thru 0x0043e4 (0x259c bytes) 000689 1753742624.448898 fallocate 0x000bb6 thru 0x0051cd (0x4618 bytes) 000690 1753742624.457377 read 0x016b5c thru 0x01e8cc (0x7d71 bytes) 000691 1753742624.457930 read 0x01a0f7 thru 0x01f18b (0x5095 bytes) 000692 1753742624.458471 mapread 0x00a625 thru 0x00aebe (0x89a bytes) 000693 1753742624.459244 mapread 0x02ee81 thru 0x02f771 (0x8f1 bytes) 000694 1753742624.467774 trunc from 0x02f772 to 0x02fba3 (0x432 bytes) 000695 1753742624.492683 read 0x017377 thru 0x01804e (0xcd8 bytes) 000696 1753742624.493281 trunc from 0x02fba4 to 0x029fed (0x5bb6 bytes) 000697 1753742624.531702 MIRROR_EXTEND to 9 mirrors 000698 1753742625.238614 fallocate 0x0133e5 thru 0x01d81e (0xa43a bytes) 000699 1753742625.284018 trunc from 0x029fee to 0x03bd81 (0x11d94 bytes) 000700 1753742625.330638 MIRROR_RESYNC 9 mirrors 000701 1753742626.411302 mapwrite 0x023166 thru 0x027b0e (0x49a9 bytes) 000702 1753742626.549308 fallocate 0x0262be thru 0x02a7ae (0x44f1 bytes) 000703 1753742626.555874 trunc from 0x03bd82 to 0x018326 (0x23a5b bytes) 000704 1753742626.588603 trunc from 0x005597 to 0x013deb (0xe855 bytes) 000705 1753742626.609465 mapwrite 0x01b6e8 thru 0x023eac (0x87c5 bytes) 000706 1753742626.698010 punch from 0x017759 to 0x01fbd9 (0x8481 bytes) 000707 1753742626.704171 mapread 0x0093d8 thru 0x00ef97 (0x5bc0 bytes) 000708 1753742626.714511 trunc from 0x023ead to 0x01e8d8 (0x55d4 bytes) 000709 1753742626.730343 read 0x01b3a3 thru 0x01e8d8 (0x3536 bytes) 000711 1753742626.736349 mapread 0x00075a thru 0x00672c (0x5fd3 bytes) 000712 1753742626.746809 read 0x0103e3 thru 0x01e8d8 (0xe4f6 bytes) 000713 1753742626.758045 MIRROR_SPLIT mirror 4 to 8 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000714 1753742626.963742 mapread 0x003855 thru 0x010928 (0xd0d4 bytes) 000715 1753742626.993937 fallocate 0x00bd7c thru 0x00f6a0 (0x3925 bytes) 000716 1753742627.006002 punch from 0x006bdb to 0x0086c6 (0x1aec bytes) 000717 1753742627.014321 MIRROR_RESYNC 8 mirrors 000718 1753742628.036367 trunc from 0x01e8d9 to 0x00cfa3 (0x11935 bytes) 000719 1753742628.080641 mapread 0x00b1a2 thru 0x00c615 (0x1474 bytes) 000721 1753742628.091573 trunc from 0x00cfa4 to 0x020154 (0x131b1 bytes) 000722 1753742628.114890 mapread 0x0197f2 thru 0x020154 (0x6963 bytes) 000723 1753742628.126498 trunc from 0x01518b to 0x01614d (0xfc3 bytes) 000724 1753742628.131986 punch from 0x011248 to 0x018d6c (0x7b25 bytes) 000725 1753742628.136968 trunc from 0x009a8b to 0x0144d6 (0xaa4c bytes) 000726 1753742628.141353 mapread 0x01b455 thru 0x020154 (0x4d00 bytes) 000727 1753742628.147636 MIRROR_RESYNC 8 mirrors 000728 1753742629.080276 trunc from 0x0041b8 to 0x004226 (0x6f bytes) 000729 1753742629.119311 read 0x015f19 thru 0x018c53 (0x2d3b bytes) 000730 1753742629.124770 read 0x00675b thru 0x012c31 (0xc4d7 bytes) 000731 1753742629.134744 mapread 0x00a20b thru 0x00b8a9 (0x169f bytes) 000732 1753742629.135587 trunc from 0x01de29 to 0x020154 (0x232c bytes) 000733 1753742629.139572 mapwrite 0x007f9d thru 0x00905d (0x10c1 bytes) 000734 1753742629.155788 write 0x038b45 thru 0x039715 (0xbd1 bytes) 000735 1753742629.156809 read 0x02a284 thru 0x0372a6 (0xd023 bytes) 000736 1753742629.165371 mapread 0x016964 thru 0x01e076 (0x7713 bytes) 000737 1753742629.167455 trunc from 0x037965 to 0x039715 (0x1db1 bytes) 000738 1753742629.191559 MIRROR_EXTEND to 9 mirrors 000739 1753742629.833976 mapread 0x020212 thru 0x02710b (0x6efa bytes) 000740 1753742629.913421 MIRROR_EXTEND to 10 mirrors 000741 1753742630.453485 read 0x010a75 thru 0x019b99 (0x9125 bytes) 000742 1753742630.461881 trunc from 0x00067b to 0x00bb8b (0xb511 bytes) 000743 1753742630.509338 fallocate 0x019994 thru 0x0236af (0x9d1c bytes) 000744 1753742630.514393 fallocate 0x03998c thru 0x03ffff (0x6674 bytes) 000746 1753742630.522165 trunc from 0x00fc3d to 0x01961f (0x99e3 bytes) 000748 1753742630.527369 mapread 0x011b76 thru 0x0151f6 (0x3681 bytes) 000749 1753742630.546063 mapread 0x02d980 thru 0x03106a (0x36eb bytes) 000750 1753742630.555485 write 0x033eb8 thru 0x038eb2 (0x4ffb bytes) 000751 1753742630.566715 mapread 0x012911 thru 0x01538d (0x2a7d bytes) 000752 1753742630.567615 mapwrite 0x00fa38 thru 0x01b25e (0xb827 bytes) 000753 1753742630.592062 punch from 0x01aa31 to 0x01db56 (0x3126 bytes) 000754 1753742630.597366 trunc from 0x039716 to 0x0223c1 (0x17354 bytes) 000756 1753742630.635784 read 0x01159c thru 0x016c74 (0x56d9 bytes) 000757 1753742630.636474 write 0x01c487 thru 0x02932e (0xcea8 bytes) 000758 1753742630.653111 MIRROR_EXTEND to 11 mirrors 000759 1753742631.147067 MIRROR_RESYNC 11 mirrors 000760 1753742633.180263 write 0x0333c0 thru 0x0372a0 (0x3ee1 bytes) 000761 1753742633.219560 trunc from 0x0372a1 to 0x00fbe2 (0x276be bytes) 000763 1753742633.252929 read 0x00c393 thru 0x00fbe2 (0x3850 bytes) 000764 1753742633.266211 read 0x001c84 thru 0x006351 (0x46ce bytes) 000765 1753742633.286077 write 0x01b71d thru 0x02a0bc (0xe9a0 bytes) 000766 1753742633.291933 punch from 0x008b8c to 0x00fbe6 (0x705b bytes) 000767 1753742633.300411 read 0x021373 thru 0x02a0bc (0x8d4a bytes) 000768 1753742633.310583 MIRROR_EXTEND to 12 mirrors 000769 1753742633.824492 write 0x031fb6 thru 0x03ffff (0xe04a bytes) 000770 1753742633.868062 punch from 0x0298ce to 0x03770e (0xde41 bytes) 000771 1753742633.893557 trunc from 0x040000 to 0x030752 (0xf8ad bytes) 000772 1753742633.978060 mapwrite 0x01eace thru 0x01facb (0xffe bytes) 000773 1753742634.000405 mapread 0x02510a thru 0x0258ae (0x7a5 bytes) 000774 1753742634.010750 trunc from 0x030753 to 0x033f06 (0x37b4 bytes) 000775 1753742634.027137 read 0x03248e thru 0x033f06 (0x1a79 bytes) 000776 1753742634.032667 mapwrite 0x0391b8 thru 0x03a254 (0x109d bytes) 000777 1753742634.066622 trunc from 0x00c468 to 0x01a0c3 (0xdc5c bytes) 000778 1753742634.071316 fallocate 0x017bfa thru 0x01b315 (0x371c bytes) 000779 1753742634.079932 fallocate 0x009ada thru 0x018728 (0xec4f bytes) 000780 1753742634.085077 fallocate 0x0191c6 thru 0x01c184 (0x2fbf bytes) 000781 1753742634.089897 write 0x012e30 thru 0x021a23 (0xebf4 bytes) 000782 1753742634.097391 trunc from 0x02fa4a to 0x034e16 (0x53cd bytes) 000783 1753742634.101076 trunc from 0x005478 to 0x012f66 (0xdaef bytes) 000785 1753742634.125972 write 0x02a96a thru 0x0353b6 (0xaa4d bytes) 000786 1753742634.127400 read 0x02fcf5 thru 0x03a254 (0xa560 bytes) 000787 1753742634.127986 trunc from 0x02528d to 0x02a324 (0x5098 bytes) 000788 1753742634.147765 fallocate 0x01ba2f thru 0x0255d4 (0x9ba6 bytes) 000789 1753742634.154353 trunc from 0x03a255 to 0x035aed (0x4767 bytes) 000790 1753742634.172777 read 0x02c14d thru 0x030c16 (0x4aca bytes) 000791 1753742634.180709 MIRROR_RESYNC 12 mirrors 000792 1753742636.259973 trunc from 0x035aee to 0x00761b (0x2e4d2 bytes) 000793 1753742636.318446 mapwrite 0x03f45e thru 0x03ffff (0xba2 bytes) 000794 1753742636.374189 fallocate 0x02a0c8 thru 0x02d1fb (0x3134 bytes) 000795 1753742636.385959 mapwrite 0x00f96a thru 0x011943 (0x1fda bytes) 000796 1753742636.421385 mapwrite 0x00ff96 thru 0x01e89f (0xe90a bytes) 000797 1753742636.486467 MIRROR_RESYNC 12 mirrors 000799 1753742638.431966 trunc from 0x01734b to 0x01850f (0x11c5 bytes) 000800 1753742638.491728 write 0x012c20 thru 0x01710c (0x44ed bytes) 000801 1753742638.500410 read 0x01cafd thru 0x01f4ba (0x29be bytes) 000802 1753742638.506874 fallocate 0x022e03 thru 0x02ca3e (0x9c3c bytes) 000803 1753742638.512394 trunc from 0x0234f4 to 0x0320b6 (0xebc3 bytes) 000804 1753742638.516949 read 0x0015e1 thru 0x003ccd (0x26ed bytes) 000805 1753742638.527904 trunc from 0x040000 to 0x006289 (0x39d76 bytes) 000806 1753742638.548295 fallocate 0x02023b thru 0x026646 (0x640c bytes) 000807 1753742638.554048 mapwrite 0x0338f6 thru 0x03b31b (0x7a26 bytes) 000808 1753742638.688839 mapread 0x02e219 thru 0x03631d (0x8105 bytes) 000809 1753742638.751325 read 0x00cb9a thru 0x013e57 (0x72be bytes) 000810 1753742638.768466 write 0x03776e thru 0x03ffff (0x8892 bytes) 000811 1753742638.773851 mapread 0x02d97a thru 0x03bb0c (0xe193 bytes) 000814 1753742638.775663 read 0x005b65 thru 0x011c57 (0xc0f3 bytes) 000815 1753742638.776302 trunc from 0x03c2c1 to 0x03ffff (0x3d3f bytes) 000816 1753742638.794609 fallocate 0x03ea30 thru 0x03f561 (0xb32 bytes) 000817 1753742638.810125 MIRROR_RESYNC 12 mirrors 000818 1753742641.080903 write 0x02da55 thru 0x0346d1 (0x6c7d bytes) 000819 1753742641.127412 write 0x026687 thru 0x026897 (0x211 bytes) 000820 1753742641.133955 trunc from 0x040000 to 0x021ff3 (0x1e00c bytes) 000821 1753742641.182128 mapwrite 0x005661 thru 0x0117d3 (0xc173 bytes) 000822 1753742641.247790 trunc from 0x021ff4 to 0x03f9bb (0x1d9c8 bytes) 000823 1753742641.283130 write 0x01ab44 thru 0x026c73 (0xc130 bytes) 000824 1753742641.324265 MIRROR_EXTEND to 13 mirrors 000826 1753742642.294785 write 0x00b825 thru 0x00ec02 (0x33de bytes) 000827 1753742642.341267 read 0x039ad3 thru 0x03d61f (0x3b4d bytes) 000828 1753742642.349289 fallocate 0x0120c3 thru 0x01d5ca (0xb508 bytes) 000830 1753742642.355401 trunc from 0x03f9bc to 0x0007fd (0x3f1be bytes) 000831 1753742642.384770 trunc from 0x0007fe to 0x020e10 (0x20613 bytes) 000832 1753742642.399910 mapwrite 0x01cb70 thru 0x021eb5 (0x5346 bytes) 000833 1753742642.432879 trunc from 0x00a162 to 0x00d892 (0x3731 bytes) 000834 1753742642.440290 fallocate 0x03911f thru 0x03ffff (0x6ee1 bytes) 000835 1753742642.443790 mapread 0x01009c thru 0x019239 (0x919e bytes) 000836 1753742642.450566 trunc from 0x020212 to 0x021eb5 (0x1ca4 bytes) 000837 1753742642.462405 MIRROR_RESYNC 13 mirrors 000839 1753742644.178435 mapread 0x00f231 thru 0x01ed95 (0xfb65 bytes) 000840 1753742644.215304 read 0x02100e thru 0x021851 (0x844 bytes) 000841 1753742644.215655 read 0x0123cb thru 0x01f67e (0xd2b4 bytes) 000842 1753742644.215960 read 0x01ee8d thru 0x021eb5 (0x3029 bytes) 000843 1753742644.216676 trunc from 0x020dfc to 0x021eb5 (0x10ba bytes) 000844 1753742644.257553 MIRROR_RESYNC 13 mirrors 000845 1753742645.866479 trunc from 0x00d867 to 0x0178e2 (0xa07c bytes) 000846 1753742645.915342 write 0x013d1c thru 0x020855 (0xcb3a bytes) 000848 1753742645.938398 mapread 0x015caf thru 0x019508 (0x385a bytes) 000849 1753742645.943008 mapwrite 0x02e9a2 thru 0x033352 (0x49b1 bytes) 000850 1753742646.039380 read 0x00459d thru 0x00ec7d (0xa6e1 bytes) 000851 1753742646.053673 trunc from 0x00e14e to 0x0158f3 (0x77a6 bytes) 000852 1753742646.062989 MIRROR_RESYNC 13 mirrors 000855 1753742648.426279 write 0x011de9 thru 0x014b01 (0x2d19 bytes) 000856 1753742648.486414 mapwrite 0x03d392 thru 0x03f89f (0x250e bytes) 000857 1753742648.536933 trunc from 0x03f8a0 to 0x02f85f (0x10040 bytes) 000858 1753742648.555990 punch from 0x021479 to 0x02f85f (0xe3e7 bytes) 000859 1753742648.565460 trunc from 0x00618b to 0x015340 (0xf1b6 bytes) 000861 1753742648.572895 mapwrite 0x024370 thru 0x026fb5 (0x2c46 bytes) 000862 1753742648.596198 write 0x039fa4 thru 0x03e9ac (0x4a09 bytes) 000863 1753742648.597924 punch from 0x008316 to 0x00fbd0 (0x78bb bytes) 000864 1753742648.604557 punch from 0x03151d to 0x0344c2 (0x2fa6 bytes) 000865 1753742648.610719 trunc from 0x03e9ad to 0x030b6f (0xde3d bytes) 000866 1753742648.634311 read 0x025ec2 thru 0x02784a (0x1989 bytes) 000867 1753742648.634934 trunc from 0x030b70 to 0x033e23 (0x32b4 bytes) 000868 1753742648.652830 trunc from 0x033e24 to 0x035f5a (0x2137 bytes) 000869 1753742648.685956 MIRROR_SPLIT mirror 4 to 12 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000870 1753742648.903303 mapread 0x01129e thru 0x013340 (0x20a3 bytes) 000871 1753742648.921856 trunc from 0x019bbc to 0x01a4b0 (0x8f5 bytes) 000872 1753742648.927249 fallocate 0x016bc7 thru 0x025297 (0xe6d1 bytes) 000873 1753742648.932752 trunc from 0x035f5b to 0x015920 (0x2063a bytes) 000874 1753742648.952769 trunc from 0x00f755 to 0x00fbad (0x459 bytes) 000875 1753742648.957672 mapread 0x00d3f5 thru 0x010572 (0x317e bytes) 000876 1753742648.967172 fallocate 0x00c1c4 thru 0x00f0b0 (0x2eed bytes) 000877 1753742648.971652 fallocate 0x00f6f4 thru 0x0165ac (0x6eb9 bytes) 000878 1753742648.979312 mapread 0x00ed78 thru 0x0165ac (0x7835 bytes) 000880 1753742648.985774 fallocate 0x00d34b thru 0x015848 (0x84fe bytes) 000881 1753742648.998971 trunc from 0x0165ad to 0x009ae9 (0xcac3 bytes) 000882 1753742649.035012 trunc from 0x004981 to 0x009ae9 (0x5169 bytes) 000883 1753742649.056413 MIRROR_EXTEND to 13 mirrors 000884 1753742649.807110 trunc from 0x009aea to 0x037988 (0x2de9f bytes) 000885 1753742649.862243 read 0x0102d8 thru 0x014030 (0x3d59 bytes) 000886 1753742649.868922 punch from 0x002005 to 0x00cba8 (0xaba4 bytes) 000890 1753742649.874679 trunc from 0x01ffb0 to 0x02b688 (0xb6d9 bytes) 000891 1753742649.880439 punch from 0x030aad to 0x030dfe (0x352 bytes) 000892 1753742649.886334 trunc from 0x037989 to 0x008bd9 (0x2edaf bytes) 000893 1753742649.912165 trunc from 0x0043a0 to 0x008bd9 (0x483a bytes) 000895 1753742649.919061 read 0x000946 thru 0x008bd9 (0x8294 bytes) 000896 1753742649.926372 trunc from 0x004866 to 0x008bd9 (0x4374 bytes) 000897 1753742649.930774 trunc from 0x00683b to 0x0072fc (0xac2 bytes) 000898 1753742649.935189 trunc from 0x004a2f to 0x008bd9 (0x41ab bytes) 000899 1753742649.941431 punch from 0x00455e to 0x0088c4 (0x4367 bytes) 000900 1753742649.946179 trunc from 0x008bda to 0x03a248 (0x3166f bytes) 000902 1753742649.961330 fallocate 0x029bd3 thru 0x035db3 (0xc1e1 bytes) 000903 1753742649.967555 trunc from 0x03a249 to 0x01c44d (0x1ddfb bytes) 000904 1753742649.988931 mapwrite 0x019cb7 thru 0x01d143 (0x348d bytes) 000905 1753742650.024195 read 0x00af7f thru 0x00be29 (0xeab bytes) 000906 1753742650.024788 trunc from 0x01d144 to 0x027d1e (0xabdb bytes) 000907 1753742650.041667 mapwrite 0x00d28b thru 0x00e7f4 (0x156a bytes) 000908 1753742650.054403 trunc from 0x027d1f to 0x038764 (0x10a46 bytes) 000909 1753742650.069863 fallocate 0x010d32 thru 0x01a556 (0x9825 bytes) 000910 1753742650.086611 MIRROR_SPLIT mirror 8 to 12 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000911 1753742650.357722 read 0x028769 thru 0x02940c (0xca4 bytes) 000914 1753742650.368475 write 0x010648 thru 0x01424d (0x3c06 bytes) 000915 1753742650.396928 mapwrite 0x015bc7 thru 0x02021a (0xa654 bytes) 000916 1753742650.461954 fallocate 0x03383c thru 0x03ffff (0xc7c4 bytes) 000917 1753742650.481561 read 0x00c63c thru 0x00eac9 (0x248e bytes) 000918 1753742650.502195 fallocate 0x018ad2 thru 0x026b32 (0xe061 bytes) 000919 1753742650.518369 trunc from 0x01426e to 0x018b52 (0x48e5 bytes) 000920 1753742650.526926 write 0x028a57 thru 0x02dd95 (0x533f bytes) 000921 1753742650.535707 write 0x03251a thru 0x032720 (0x207 bytes) 000922 1753742650.536647 mapread 0x03cb87 thru 0x03ffff (0x3479 bytes) 000923 1753742650.541178 mapread 0x0184bf thru 0x019168 (0xcaa bytes) 000924 1753742650.551677 trunc from 0x040000 to 0x01fdad (0x20252 bytes) 000925 1753742650.599157 punch from 0x007bed to 0x013b59 (0xbf6d bytes) 000927 1753742650.607990 mapwrite 0x00c404 thru 0x01a9bf (0xe5bc bytes) 000928 1753742650.806798 fallocate 0x01184d thru 0x01d80a (0xbfbe bytes) 000929 1753742650.820866 mapwrite 0x0293d2 thru 0x02dc54 (0x4883 bytes) 000930 1753742650.927776 punch from 0x01e8c6 to 0x02d177 (0xe8b2 bytes) 000931 1753742650.945152 read 0x01cca0 thru 0x0210e6 (0x4447 bytes) 000932 1753742650.963429 mapread 0x02a1d1 thru 0x02d2e7 (0x3117 bytes) 000933 1753742650.968582 write 0x014f28 thru 0x024ec2 (0xff9b bytes) 000934 1753742650.969850 trunc from 0x001035 to 0x0092c6 (0x8292 bytes) 000935 1753742650.990482 MIRROR_RESYNC 12 mirrors 000936 1753742653.134981 read 0x0171e1 thru 0x025281 (0xe0a1 bytes) 000937 1753742653.144886 mapwrite 0x0301ce thru 0x03eea9 (0xecdc bytes) 000938 1753742653.255412 mapread 0x002759 thru 0x0081c3 (0x5a6b bytes) 000939 1753742653.272988 read 0x015924 thru 0x015d3f (0x41c bytes) 000940 1753742653.289016 trunc from 0x03eeaa to 0x03b54a (0x395f bytes) 000941 1753742653.354579 MIRROR_RESYNC 12 mirrors 000942 1753742656.325703 mapread 0x02311b thru 0x02dc6e (0xab54 bytes) 000943 1753742656.346355 write 0x00ce62 thru 0x010720 (0x38bf bytes) 000945 1753742656.399492 punch from 0x028c0f to 0x0294cb (0x8bd bytes) 000946 1753742656.404346 mapread 0x02fa32 thru 0x033b63 (0x4132 bytes) 000947 1753742656.416168 trunc from 0x03b54b to 0x01ff26 (0x1b624 bytes) 000948 1753742656.443235 MIRROR_SPLIT mirror 10 to 11 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000949 1753742656.674625 trunc from 0x01e30c to 0x01ff26 (0x1c1b bytes) 000950 1753742656.693522 fallocate 0x031f83 thru 0x032379 (0x3f7 bytes) 000951 1753742656.701551 mapwrite 0x029db1 thru 0x03669d (0xc8ed bytes) 000952 1753742656.785609 fallocate 0x03325f thru 0x035334 (0x20d6 bytes) 000953 1753742656.791368 punch from 0x02b40c to 0x034f22 (0x9b17 bytes) 000954 1753742656.797816 punch from 0x018bf8 to 0x0213a9 (0x87b2 bytes) 000955 1753742656.809005 write 0x0105cc thru 0x01ac16 (0xa64b bytes) 000956 1753742656.837550 mapwrite 0x00838f thru 0x01259a (0xa20c bytes) 000959 1753742656.879411 write 0x0282f6 thru 0x02e2f3 (0x5ffe bytes) 000960 1753742656.883785 trunc from 0x03669e to 0x028854 (0xde49 bytes) 000962 1753742656.907912 MIRROR_EXTEND to 12 mirrors 000964 1753742657.571547 mapread 0x006573 thru 0x013150 (0xcbde bytes) 000965 1753742657.603934 trunc from 0x028855 to 0x005646 (0x2320e bytes) 000966 1753742657.656393 MIRROR_EXTEND to 13 mirrors 000967 1753742658.121886 read 0x001c99 thru 0x005646 (0x39ae bytes) 000968 1753742658.134847 trunc from 0x005647 to 0x00d4b7 (0x7e71 bytes) 000969 1753742658.184023 write 0x004c9e thru 0x0074ed (0x2850 bytes) 000970 1753742658.193205 punch from 0x007422 to 0x00d4b7 (0x6096 bytes) 000971 1753742658.202679 punch from 0x007d60 to 0x00d4b7 (0x5758 bytes) 000972 1753742658.208468 trunc from 0x00d4b8 to 0x00e3c4 (0xf0d bytes) 000974 1753742658.228618 read 0x009840 thru 0x00e3c4 (0x4b85 bytes) 000975 1753742658.235528 read 0x00c6df thru 0x00e3c4 (0x1ce6 bytes) 000976 1753742658.236102 read 0x003825 thru 0x00af63 (0x773f bytes) 000977 1753742658.236607 trunc from 0x00e3c5 to 0x0216bd (0x132f9 bytes) 000978 1753742658.254423 read 0x01f0f7 thru 0x0216bd (0x25c7 bytes) 000979 1753742658.264425 fallocate 0x035ae6 thru 0x03ffff (0xa51a bytes) 000980 1753742658.269405 trunc from 0x0216be to 0x02e097 (0xc9da bytes) 000981 1753742658.287195 punch from 0x00d55f to 0x011a27 (0x44c9 bytes) 000982 1753742658.292870 mapwrite 0x003813 thru 0x007153 (0x3941 bytes) 000983 1753742658.317973 mapwrite 0x009994 thru 0x01772a (0xdd97 bytes) 000984 1753742658.401519 trunc from 0x02e098 to 0x00008b (0x2e00c bytes) 000985 1753742658.417353 mapread 0x00008a thru 0x00008b (0x2 bytes) 000986 1753742658.418172 punch from 0x000006 to 0x00008b (0x86 bytes) 000987 1753742658.435581 MIRROR_SPLIT mirror 1 to 12 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000988 1753742658.757978 trunc from 0x000078 to 0x00008b (0x14 bytes) 000989 1753742658.770017 punch from 0x00000e to 0x00008b (0x7e bytes) 000990 1753742658.777538 trunc from 0x000053 to 0x00008b (0x39 bytes) 000991 1753742658.786810 mapread 0x000052 thru 0x00008b (0x3a bytes) 000992 1753742658.796476 read 0x000017 thru 0x00008b (0x75 bytes) 000993 1753742658.806677 MIRROR_SPLIT mirror 1 to 11 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x1d9:0x0]: Invalid argument (22) 000994 1753742659.138172 trunc from 0x00005e to 0x00008b (0x2e bytes) 000995 1753742659.147492 trunc from 0x00008c to 0x000720 (0x695 bytes) 000996 1753742659.179834 MIRROR_RESYNC 11 mirrors 000997 1753742659.884784 read 0x0006e1 thru 0x000720 (0x40 bytes) 000998 1753742659.889645 mapread 0x0002ff thru 0x000720 (0x422 bytes) 000999 1753742659.901130 MIRROR_EXTEND to 12 mirrors 001000 1753742660.344078 punch from 0x0004f8 to 0x000720 (0x229 bytes) All operations completed A-OK! PASS 70a (95s) == sanity-flr test 71: check mirror extend parent fid ==== 18:44:25 (1753742665) striped dir -i1 -c2 -H crush2 /mnt/lustre/d71.sanity-flr ** lfs fid2path /mnt/lustre [0x240000404:0x219:0x0] /mnt/lustre/d71.sanity-flr/f71.sanity-flr PASS 71 (10s) == sanity-flr test 200a: stress test ===================== 18:44:35 (1753742675) Starting client: oleg417-client.virtnet: -o user_xattr,flock 192.168.204.117@tcp:/lustre /mnt/lustre2 Starting client: oleg417-client.virtnet: -o user_xattr,flock 192.168.204.117@tcp:/lustre /mnt/lustre3 fail_loc=0x1A03 fail_loc=0x1A03 resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..Extending file size to 4422304 .. Extending file size to 7610336 .. failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8928000 .. Extending file size to 7787328 .. done lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..Extending file size to 8061024 .. failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done Extending file size to 8675104 .. resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed Extending file size to 8957024 .. resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed Extending file size to 9322272 .. lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Waiting 59353 59354 59355 59359 59360 done 192.168.204.117@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg417-client.virtnet /mnt/lustre2 (opts:) 192.168.204.117@tcp:/lustre /mnt/lustre3 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg417-client.virtnet /mnt/lustre3 (opts:) 3 PASS 200a (76s) == sanity-flr test 200b: racing IO, mirror extend and resync ========================================================== 18:45:51 (1753742751) Starting client: oleg417-client.virtnet: -o user_xattr,flock 192.168.204.117@tcp:/lustre /mnt/lustre2 Starting client: oleg417-client.virtnet: -o user_xattr,flock 192.168.204.117@tcp:/lustre /mnt/lustre3 fail_loc=0x1423 resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 1431712 .. Extending file size to 7886816 .. done Extending file size to 6989920 .. lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 7268128 .. Extending file size to 7917984 .. Extending file size to 8154528 .. Extending file size to 9069760 .. Extending file size to 8856512 .. done lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..Extending file size to 9020768 .. done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..Waiting 61412 61413 61414 61418 61419 failed 192.168.204.117@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg417-client.virtnet /mnt/lustre2 (opts:) 192.168.204.117@tcp:/lustre /mnt/lustre3 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg417-client.virtnet /mnt/lustre3 (opts:) 3 PASS 200b (72s) == sanity-flr test 200c: layout change racing with open: LOVEA changes ========================================================== 18:47:03 (1753742823) Starting client: oleg417-client.virtnet: -o user_xattr,flock 192.168.204.117@tcp:/lustre /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2420 1285268 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 2424 1285264 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 15596 3591424 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 21228 3585792 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 36824 7177216 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2420 1285268 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1414116 2424 1285264 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3833116 15596 3591424 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3833116 21228 3585792 1% /mnt/lustre2[OST:1] filesystem_summary: 7666232 36824 7177216 1% /mnt/lustre2 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.12412 s, 16.9 MB/s fail_loc=0x80000175 fail_val=10 mirror extend mirror extend done fail_loc=0 fail_val=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.161132 s, 13.0 MB/s dd completed Chunks to be verified in /mnt/lustre/f200c.sanity-flr: [0, 0xffffffffffffffff) [1] 1 lfs verify: '/mnt/lustre/f200c.sanity-flr' chunk [0, 0xffffffffffffffff) is only valid in mirror 1: skipped 192.168.204.117@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg417-client.virtnet /mnt/lustre2 (opts:) PASS 200c (14s) SKIP: sanity-flr test_201 skipping ALWAYS excluded test 201 == sanity-flr test 202: lfs setstripe --add-component wide striping ========================================================== 18:47:18 (1753742838) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.137696 s, 15.2 MB/s PASS 202 (7s) == sanity-flr test 203: mirror file preserve mirror ID === 18:47:25 (1753742845) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x58:0x0]: Invalid argument (22) /mnt/lustre/f203.sanity-flr lcm_layout_gen: 3 lcm_mirror_count: 1 lcm_entry_count: 1 lcme_id: 131074 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xa8:0x0] } /mnt/lustre/f203.sanity-flr lcm_layout_gen: 4 lcm_mirror_count: 2 lcm_entry_count: 2 lcme_id: 131074 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xa8:0x0] } lcme_id: 196609 lcme_mirror_id: 3 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xa9:0x0] } - 1: { l_ost_idx: 1, l_fid: [0x2c0000401:0xa9:0x0] } PASS 203 (6s) == sanity-flr test 204a: FLR write/stale/resync tests with self-extending mirror ========================================================== 18:47:31 (1753742851) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d204a.sanity-flr 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0773455 s, 27.1 MB/s /mnt/lustre/d204a.sanity-flr/f204a.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x2b3:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2b4:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 1048576 lcme_extent.e_end: 68157440 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2b5:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 68157440 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 PASS 204a (7s) == sanity-flr test 204b: FLR write/stale/resync tests with self-extending primary ========================================================== 18:47:38 (1753742858) striped dir -i0 -c2 -H crush2 /mnt/lustre/d204b.sanity-flr 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0917304 s, 22.9 MB/s /mnt/lustre/d204b.sanity-flr/f204b.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xaa:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: 68157440 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xab:0x0] } lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 68157440 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xaa:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 /mnt/lustre/d204b.sanity-flr/f204b.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xaa:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: 68157440 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xab:0x0] } lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 68157440 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xaa:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xab:0x0] } PASS 204b (8s) == sanity-flr test 204c: FLR write/stale/resync test with component removal ========================================================== 18:47:46 (1753742866) striped dir -i0 -c2 -H crush2 /mnt/lustre/d204c.sanity-flr 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.127334 s, 16.5 MB/s /mnt/lustre/d204c.sanity-flr/f204c.sanity-flr lcm_layout_gen: 7 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xac:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xac:0x0] } lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 /mnt/lustre/d204c.sanity-flr/f204c.sanity-flr lcm_layout_gen: 9 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xac:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xac:0x0] } lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xad:0x0] } PASS 204c (32s) == sanity-flr test 204d: FLR write/stale/resync sel test with repeated comp ========================================================== 18:48:18 (1753742898) Waiting for MDT destroys to complete Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg417-server mds-ost sync done. striped dir -i0 -c2 -H crush /mnt/lustre/d204d.sanity-flr 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100035 s, 10.5 MB/s /mnt/lustre/d204d.sanity-flr/f204d.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x2b4:0x0] } lcme_id: 65541 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2b7:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2b6:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 /mnt/lustre/d204d.sanity-flr/f204d.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x2b4:0x0] } lcme_id: 65541 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2b7:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2b6:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x2b5:0x0] } PASS 204d (49s) == sanity-flr test 204e: FLR write/stale/resync sel test with repeated comp ========================================================== 18:49:07 (1753742947) Waiting for MDT destroys to complete Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg417-server mds-ost sync done. striped dir -i0 -c2 -H crush /mnt/lustre/d204e.sanity-flr /mnt/lustre/d204e.sanity-flr/f204e.sanity-flr lcm_layout_gen: 4 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xad:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xae:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 67108864 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0817899 s, 25.6 MB/s /mnt/lustre/d204e.sanity-flr/f204e.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xad:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xae:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xae:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 /mnt/lustre/d204e.sanity-flr/f204e.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xad:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xae:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0xae:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 PASS 204e (51s) == sanity-flr test 204f: FLR write/stale/resync sel w/forced extension ========================================================== 18:49:58 (1753742998) Waiting for MDT destroys to complete Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg417-server mds-ost sync done. striped dir -i0 -c2 -H all_char /mnt/lustre/d204f.sanity-flr Creating new pool test_204f oleg417-server: Pool lustre.test_204f created Adding targets to pool oleg417-server: OST lustre-OST0000_UUID added to pool lustre.test_204f oleg417-server: OST lustre-OST0001_UUID added to pool lustre.test_204f 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.104972 s, 20.0 MB/s /mnt/lustre/d204f.sanity-flr/f204f.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2b8:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_pool: test_204f lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x2b6:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lmm_pool: test_204f /mnt/lustre/d204f.sanity-flr/f204f.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x2b8:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_pool: test_204f lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x2b6:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lmm_pool: test_204f Destroy the created pools: test_204f lustre.test_204f oleg417-server: OST lustre-OST0000_UUID removed from pool lustre.test_204f oleg417-server: OST lustre-OST0001_UUID removed from pool lustre.test_204f oleg417-server: Pool lustre.test_204f destroyed PASS 204f (72s) == sanity-flr test 205a: lfs mirror extend to set prefer flag ========================================================== 18:51:10 (1753743070) PASS 205a (9s) == sanity-flr test 205b: lfs mirror extend to set nocompr flag ========================================================== 18:51:19 (1753743079) PASS 205b (7s) == sanity-flr test 206: lfs setstripe -pool .. --comp-flags=.. ========================================================== 18:51:27 (1753743087) oleg417-server: Pool lustre.test_206 created Adding targets to pool oleg417-server: OST lustre-OST0000_UUID added to pool lustre.test_206 oleg417-server: OST lustre-OST0001_UUID added to pool lustre.test_206 Destroy the created pools: test_206 lustre.test_206 oleg417-server: OST lustre-OST0000_UUID removed from pool lustre.test_206 oleg417-server: OST lustre-OST0001_UUID removed from pool lustre.test_206 oleg417-server: Pool lustre.test_206 destroyed PASS 206 (27s) == sanity-flr test 207: create another replica with existing out-of-sync one ========================================================== 18:51:53 (1753743113) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119701 s, 8.8 MB/s 2 mirror IDs: 1 2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222411 s, 47.1 MB/s 2 mirror IDs: 1 2 2 mirror IDs: 1 2 3 mirror IDs: 1 2 3 CRC-32 checksum value for chunk [0, 0x400000): Mirror 1: 0xc886942 Mirror 3: 0xc886942 PASS 207 (10s) == sanity-flr test 208a: mirror selection to prefer non-rotational devices for reads ========================================================== 18:52:03 (1753743123) SKIP: sanity-flr test_208a needs >= 4 OSTs SKIP 208a (4s) == sanity-flr test 208b: mirror selection to prefer non-rotational devices for writes ========================================================== 18:52:07 (1753743127) SKIP: sanity-flr test_208b needs >= 4 OSTs SKIP 208b (3s) == sanity-flr test 209a: skip fast reads after layout invalidation ========================================================== 18:52:10 (1753743130) Starting client: oleg417-client.virtnet: -o user_xattr,flock 192.168.204.117@tcp:/lustre /mnt/lustre2 osd-ldiskfs.lustre-OST0000.nonrotational=0 osd-ldiskfs.lustre-OST0001.nonrotational=0 osd-ldiskfs.lustre-OST0001.nonrotational=1 replicated file created open(O_RDONLY) and first read from OST first read complete 4+0 records in 4+0 records out 4 bytes copied, 0.0773158 s, 0.1 kB/s BBBB written which made replica on ost1 stale fast read from pagecache in the original process read via /mnt/lustre2 new open(2) Buffer address : 0x1890000 Read this (4 bytes): BBBB fast read from pagecache after 5s in the original process Buffer address : 0x8b0000 Read this (4 bytes): AAAA PAUSING Buffer address : 0x8b0000 Read this (4 bytes): BBBB PAUSING Buffer address : 0x8b0000 Read this (4 bytes): BBBB read via new open(2) Buffer address : 0x1b50000 Read this (4 bytes): BBBB 192.168.204.117@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg417-client.virtnet /mnt/lustre2 (opts:) PASS 209a (33s) == sanity-flr test 209b: pagecache can be used after LL cancellation ========================================================== 18:52:44 (1753743164) 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0130104 s, 630 kB/s the very first read read with warm cache PASS 209b (6s) == sanity-flr test 210a: handle broken mirrored lovea ==== 18:52:50 (1753743170) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0369673 s, 28.4 MB/s fail_loc=0x1428 /mnt/lustre/f210a.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 240 lcm_flags: ro lcm_layout_gen: 1 lcm_mirror_count: 2 lcm_entry_count: 2 components: - lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lcme_offset: 128 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x77 lmm_fid: [0x200000405:0x77:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lmm_objects: - 0: { l_ost_idx: -1, l_fid: [0:0x0:0x0] } - lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lcme_offset: 184 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x77 lmm_fid: [0x200000405:0x77:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xb8:0x0] } File: /mnt/lustre/f210a.sanity-flr Size: 1048576 Blocks: 4096 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205339611255 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-07-28 18:52:52.000000000 -0400 Modify: 2025-07-28 18:52:52.000000000 -0400 Change: 2025-07-28 18:52:53.000000000 -0400 Birth: 2025-07-28 18:52:52.000000000 -0400 PASS 210a (8s) == sanity-flr test 210b: handle broken mirrored lovea (unlink) ========================================================== 18:52:58 (1753743178) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022402 s, 46.8 MB/s oleg417-server: debugfs 1.47.1-wc2 (08-Nov-2024) fail_loc=0x1428 Waiting for MDT destroys to complete oleg417-server: debugfs 1.47.1-wc2 (08-Nov-2024) oleg417-server: O/280000401/d25/b9: File not found by ext2_lookup PASS 210b (20s) == sanity-flr test 211: mirror delete should not cause bad size ========================================================== 18:53:18 (1753743198) 10+0 records in 10+0 records out 40960 bytes (41 kB, 40 KiB) copied, 0.15998 s, 256 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0151068 s, 271 kB/s size after second write -rw-r--r-- 1 root root 4096 Jul 28 18:53 /mnt/lustre/f211.sanity-flr lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x7c:0x0]: Invalid argument (22) size after mirror delete -rw-r--r-- 1 root root 4096 Jul 28 18:53 /mnt/lustre/f211.sanity-flr PASS 211 (7s) == sanity-flr test complete, duration 1537 sec =========== 18:53:25 (1753743205) === sanity-flr: start cleanup 18:53:27 (1753743207) === === sanity-flr: finish cleanup 18:53:34 (1753743214) ===