-----============= acceptance-small: sanity-flr ============----- Wed Mar 11 09:17:48 EDT 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg338-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanity-flr.*ex': No such file or directory excepting tests: 6 201 44c === sanity-flr: start setup 09:17:59 (1773235079) === oleg338-client.virtnet: executing check_config_client /mnt/lustre oleg338-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg338-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff897046ad9000.idle_timeout=debug osc.lustre-OST0001-osc-ffff897046ad9000.idle_timeout=debug disable quota as required oleg338-server: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all osd-ldiskfs.track_declares_assert=1 === sanity-flr: finish setup 09:18:20 (1773235100) === running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f8201] == sanity-flr test 0a: lfs mirror create with -N option == 09:18:22 (1773235102) PASS 0a (10s) == sanity-flr test 0b: lfs mirror create plain layout mirrors ========================================================== 09:18:32 (1773235112) SKIP: sanity-flr test_0b need >= 4 OSTs SKIP 0b (2s) == sanity-flr test 0c: lfs mirror create composite layout mirrors ========================================================== 09:18:34 (1773235114) SKIP: sanity-flr test_0c need >= 4 OSTs SKIP 0c (2s) == sanity-flr test 0d: lfs mirror extend with -N option == 09:18:36 (1773235116) PASS 0d (16s) == sanity-flr test 0e: lfs mirror extend plain layout mirrors ========================================================== 09:18:52 (1773235132) SKIP: sanity-flr test_0e need >= 4 OSTs SKIP 0e (2s) == sanity-flr test 0f: lfs mirror extend composite layout mirrors ========================================================== 09:18:54 (1773235134) SKIP: sanity-flr test_0f need >= 4 OSTs SKIP 0f (2s) == sanity-flr test 0g: lfs mirror create flags support === 09:18:56 (1773235136) error: lfs create: --flags must be specified with --mirror-count|-N option Create a mirrored file. usage: lfs mirror create --mirror-count|-N[MIRROR_COUNT] [SETSTRIPE_OPTIONS] ... FILENAME|DIRECTORY ... lfs commands used to manage files with mirrored components: lfs mirror create - create a mirrored file or directory lfs mirror extend - add mirror(s) to an existing file lfs mirror split - split a mirror from an existing mirrored file lfs mirror resync - resynchronize out-of-sync mirrored file(s) lfs mirror read - read a mirror content of a mirrored file lfs mirror write - write to a mirror of a mirrored file lfs mirror copy - copy a mirror to other mirror(s) of a file lfs mirror verify - verify mirrored file(s) lfs create: component flag 'foo' not supported Create a mirrored file. usage: lfs mirror create --mirror-count|-N[MIRROR_COUNT] [SETSTRIPE_OPTIONS] ... FILENAME|DIRECTORY ... lfs commands used to manage files with mirrored components: lfs mirror create - create a mirrored file or directory lfs mirror extend - add mirror(s) to an existing file lfs mirror split - split a mirror from an existing mirrored file lfs mirror resync - resynchronize out-of-sync mirrored file(s) lfs mirror read - read a mirror content of a mirrored file lfs mirror write - write to a mirror of a mirrored file lfs mirror copy - copy a mirror to other mirror(s) of a file lfs mirror verify - verify mirrored file(s) lfs create: unsupported mirror flags: stale lfs create: unsupported mirror flags: prefer,init lfs create: inverted flags are not supported Create a mirrored file. usage: lfs mirror create --mirror-count|-N[MIRROR_COUNT] [SETSTRIPE_OPTIONS] ... FILENAME|DIRECTORY ... lfs commands used to manage files with mirrored components: lfs mirror create - create a mirrored file or directory lfs mirror extend - add mirror(s) to an existing file lfs mirror split - split a mirror from an existing mirrored file lfs mirror resync - resynchronize out-of-sync mirrored file(s) lfs mirror read - read a mirror content of a mirrored file lfs mirror write - write to a mirror of a mirrored file lfs mirror copy - copy a mirror to other mirror(s) of a file lfs mirror verify - verify mirrored file(s) osc.lustre-OST0000-osc-ffff897046ad9000.stats=clear osc.lustre-OST0001-osc-ffff897046ad9000.stats=clear PASS 0g (5s) == sanity-flr test 0h: set, clear and test flags for FLR files ========================================================== 09:19:02 (1773235142) striped dir -i0 -c2 -H crush2 /mnt/lustre/d0h.sanity-flr lfs setstripe: component flag 'foo' not supported Create a file with specified striping/composite layout, or set the default layout on an existing directory. Usage: setstripe [--component-add|--component-del|--delete|-d] [--comp-set --comp-id|-I COMP_ID|--comp-flags=COMP_FLAGS] [--component-end|-E END_OFFSET] [--copy=SOURCE_LAYOUT_FILE]|--yaml|-y YAML_TEMPLATE_FILE] [--extension-size|--ext-size|-z EXT_SIZE] [--help|-h] [--foreign=FOREIGN_TYPE --xattr|-x LAYOUT] [--layout|-L PATTERN] [--mode FILE_MODE] [--mirror-count|-N[MIRROR_COUNT]] [--ost|-o OST_INDEX[,OST_INDEX,...]] [--overstripe-count|-C STRIPE_COUNT] [--pool|-p POOL_NAME] [--stripe-count|-c STRIPE_COUNT] [--stripe-index|-i START_OST_IDX] [--stripe-size|-S STRIPE_SIZE] FILENAME|DIRECTORY lfs: component flag 'foo' not supported error: getstripe bad component flags 'prefer,foo'. List the layout pattern for a given file or files in a directory or recursively for all files in a directory tree. Usage: getstripe [--ost|-O OST_NAME] [--quiet|-q] [--verbose|-v] [--stripe-count|-c] [--stripe-index|-i] [--fid|-F] [--pool|-p] [--stripe-size|-S] [--directory|-d] [--mdt-index|-m] [--recursive|-r] [--raw|-R] [--layout|-L] [--generation|-g] [--yaml|-y] [--help|-h] [--hex-idx] [--component-id|-I[=COMP_ID]] [--component-flags[=COMP_FLAGS]] [--component-count] [--extension-size|--ext-size|-z] [--component-start[=[+-]START_OFFSET]] [--component-end|-E[[+-]END_OFFSET]] [[!] --mirror-index=[+-]MIRROR_INDEX | [!] --mirror-id=[+-]MIRROR_ID] [--mirror-count|-N] [--no-follow] FILENAME|DIRECTORY lfs setstripe: cannot clear 'stale' flags from component. Please use lfs-mirror-resync(1) instead PASS 0h (7s) == sanity-flr test 0j: test lfs mirror read/write commands ========================================================== 09:19:08 (1773235148) PASS 0j (6s) == sanity-flr test 0k: mirroring a file in directory with default FLR layout ========================================================== 09:19:14 (1773235154) SKIP: sanity-flr test_0k need >= 3 OSTs SKIP 0k (2s) == sanity-flr test 1: create components with setstripe options ========================================================== 09:19:16 (1773235156) lfs mirror mirror: cannot get UNLOCK lease, ext 4 for [0x200000402:0x1f:0x0]: Numerical result out of range (34) error: lfs extend: /mnt/lustre/f1.sanity-flr: cannot merge layout: Numerical result out of range PASS 1 (6s) == sanity-flr test 2: create components from existing files ========================================================== 09:19:22 (1773235162) PASS 2 (4s) == sanity-flr test 3: create components from files located on different MDTs ========================================================== 09:19:26 (1773235166) PASS 3 (6s) == sanity-flr test 4: Make sure mirror attributes can be inhertied from directory ========================================================== 09:19:32 (1773235172) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d4.sanity-flr striped dir -i0 -c2 -H crush2 /mnt/lustre/d4.sanity-flr/td error: lfs extend: /mnt/lustre/d4.sanity-flr: cannot get the last init comp: Is a directory PASS 4 (5s) == sanity-flr test 5: Make sure init size work for mirrored layout ========================================================== 09:19:37 (1773235177) /mnt/lustre/f5.sanity-flr has type file OK /mnt/lustre/f5.sanity-flr has size 12345 OK /mnt/lustre/f5.sanity-flr has type file OK /mnt/lustre/f5.sanity-flr has size 12345 OK PASS 5 (5s) SKIP: sanity-flr test_6a skipping ALWAYS excluded test 6a (base 6) SKIP: sanity-flr test_6b skipping ALWAYS excluded test 6b (base 6) == sanity-flr test 7: Create mirror with DoM component === 09:19:44 (1773235184) PASS 7 (3s) == sanity-flr test 21: glimpse should report accurate i_blocks ========================================================== 09:19:47 (1773235187) 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.159482 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0733685 s, 14.3 MB/s PASS 21 (5s) == sanity-flr test 22: no glimpse to OSTs for READ_ONLY files ========================================================== 09:19:52 (1773235192) 9+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 0.171967 s, 54.9 MB/s PASS 22 (5s) == sanity-flr test 31: make sure glimpse request can be retried ========================================================== 09:19:57 (1773235197) Stopping /mnt/lustre-ost1 (opts:) on oleg338-server oleg338-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff897046ad9000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff897046ad9000.ost_server_uuid in DISCONN state after 2 sec Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg338-server: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg338-client: oleg338-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg338-server: oleg338-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg338-server: *.lustre-OST0000.recovery_status status: COMPLETE Stopping /mnt/lustre-ost2 (opts:) on oleg338-server oleg338-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0001-osc-ffff897046ad9000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff897046ad9000.ost_server_uuid in DISCONN state after 0 sec Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg338-server: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg338-client: oleg338-server: ssh exited with exit code 1 Started lustre-OST0001 affected facets: ost2 oleg338-server: oleg338-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg338-server: *.lustre-OST0001.recovery_status status: COMPLETE PASS 31 (42s) == sanity-flr test 32: data should be mirrored to newly created mirror ========================================================== 09:20:39 (1773235239) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.366293 s, 28.6 MB/s Stopping /mnt/lustre-ost1 (opts:) on oleg338-server oleg338-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff897046ad9000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff897046ad9000.ost_server_uuid in DISCONN state after 0 sec /mnt/lustre/f32.sanity-flr has type file OK /mnt/lustre/f32.sanity-flr has size 10485760 OK reading file from the 2nd mirror and verify checksum Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg338-server: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg338-client: oleg338-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg338-server: oleg338-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg338-server: *.lustre-OST0000.recovery_status status: COMPLETE PASS 32 (23s) == sanity-flr test 33a: read can choose available mirror to read ========================================================== 09:21:02 (1773235262) error: lfs extend: /mnt/lustre/f33a.sanity-flr: file busy or contents don't match: Invalid argument /mnt/lustre/f33a.sanity-flr has type file OK /mnt/lustre/f33a.sanity-flr has size 500 OK reading file (data can be provided by any ost)... Stopping /mnt/lustre-ost1 (opts:) on oleg338-server oleg338-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff897046ad9000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff897046ad9000.ost_server_uuid in DISCONN state after 0 sec reading file (data should be provided by ost2)... Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg338-server: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg338-client: oleg338-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg338-server: oleg338-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg338-server: *.lustre-OST0000.recovery_status status: COMPLETE Stopping /mnt/lustre-ost2 (opts:) on oleg338-server oleg338-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0001-osc-ffff897046ad9000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff897046ad9000.ost_server_uuid in DISCONN state after 0 sec /mnt/lustre/f33a.sanity-flr has type file OK /mnt/lustre/f33a.sanity-flr has size 500 OK reading file (data should be provided by ost1)... Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg338-server: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg338-client: oleg338-server: ssh exited with exit code 1 Started lustre-OST0001 affected facets: ost2 oleg338-server: oleg338-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg338-server: *.lustre-OST0001.recovery_status status: COMPLETE PASS 33a (46s) == sanity-flr test 33b: avoid reading from unhealthy mirror ========================================================== 09:21:48 (1773235308) 25+0 records in 25+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.75968 s, 29.8 MB/s -rw-r--r-- 1 root root 52428800 Mar 11 09:21 /mnt/lustre/f33b.sanity-flr reading file (data can be provided by any ost)... real 0m0.584s user 0m0.000s sys 0m0.163s Stopping /mnt/lustre-ost1 (opts:) on oleg338-server oleg338-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0000-osc-ffff897046ad9000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff897046ad9000.ost_server_uuid in DISCONN state after 0 sec reading file (data should be provided by ost2)... real 0m0.655s user 0m0.001s sys 0m0.150s Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg338-server: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg338-client: oleg338-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg338-server: oleg338-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg338-server: *.lustre-OST0000.recovery_status status: COMPLETE Stopping /mnt/lustre-ost2 (opts:) on oleg338-server oleg338-client.virtnet: executing wait_import_state (DISCONN|IDLE) osc.lustre-OST0001-osc-ffff897046ad9000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff897046ad9000.ost_server_uuid in DISCONN state after 0 sec reading file (data should be provided by ost1)... real 0m0.755s user 0m0.000s sys 0m0.180s Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg338-server: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg338-client: oleg338-server: ssh exited with exit code 1 Started lustre-OST0001 affected facets: ost2 oleg338-server: oleg338-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg338-server: *.lustre-OST0001.recovery_status status: COMPLETE oleg338-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff897046ad9000.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff897046ad9000.ost_server_uuid in FULL state after 0 sec PASS 33b (51s) == sanity-flr test 33c: keep reading among unhealthy mirrors ========================================================== 09:22:39 (1773235359) SKIP: sanity-flr test_33c need >= 3 OSTs SKIP 33c (2s) == sanity-flr test 34a: read mirrored file with multiple stripes ========================================================== 09:22:41 (1773235361) SKIP: sanity-flr test_34a need >= 4 OSTs SKIP 34a (2s) == sanity-flr test 34b: read mirrored file with multiple components ========================================================== 09:22:43 (1773235363) SKIP: sanity-flr test_34b need >= 4 OSTs SKIP 34b (2s) == sanity-flr test 35: allow to write to mirrored files == 09:22:45 (1773235365) PASS 35 (4s) == sanity-flr test 36a: write to mirrored files ========== 09:22:49 (1773235369) 12+0 records in 12+0 records out 12582912 bytes (13 MB, 12 MiB) copied, 0.305121 s, 41.2 MB/s fail_val=10 fail_loc=0x1A01 fail_loc=0 PASS 36a (8s) == sanity-flr test 36b: write should not hung when prefered mirror is stale ========================================================== 09:22:57 (1773235377) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187251 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156455 s, 67.0 MB/s PASS 36b (4s) == sanity-flr test 36c: change prefer mirror during write shouldn't hung ========================================================== 09:23:01 (1773235381) PASS 36c (5s) == sanity-flr test 36d: write/punch FLR file update OST layout version ========================================================== 09:23:06 (1773235386) ** create /mnt/lustre/f36d.sanity-flr ** mirror extend /mnt/lustre/f36d.sanity-flr (1/2) flv=2 olv=0 ** mirror extend /mnt/lustre/f36d.sanity-flr (2/2) flv=3 olv=0 ** write /mnt/lustre/f36d.sanity-flr (1/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000933234 s, 1.1 MB/s flv=4 olv=4 ** write /mnt/lustre/f36d.sanity-flr (2/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00209924 s, 488 kB/s flv=4 olv=4 ** resync /mnt/lustre/f36d.sanity-flr flv=6 olv=6 ** truncate /mnt/lustre/f36d.sanity-flr (1/2) flv=7 olv=7 ** truncate /mnt/lustre/f36d.sanity-flr (2/2) flv=7 olv=7 ** resync /mnt/lustre/f36d.sanity-flr flv=9 olv=9 ** write /mnt/lustre/f36d.sanity-flr (1/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000998718 s, 1.0 MB/s flv=10 olv=10 ** write /mnt/lustre/f36d.sanity-flr (2/2) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0007611 s, 1.3 MB/s flv=10 olv=10 PASS 36d (5s) == sanity-flr test 37: mirror I/O API verification ======= 09:23:11 (1773235391) create test files with size 15751677 .. c34eca8b8cf132489304482b4ca07841 - d9f1bb3e9701871dfe0a10a9467be65f - e5b2e7442cdb022835fbf818ea953def - 3 Verifying mirror read .. Verifying mirror write .. Verifying mirror copy .. mirror copied successfully: 2 3 Stopping client oleg338-client.virtnet /mnt/lustre (opts:) Starting client: oleg338-client.virtnet: -o user_xattr,flock 192.168.203.138@tcp:/lustre /mnt/lustre PASS 37 (12s) == sanity-flr test 38: resync ============================ 09:23:23 (1773235403) 4 mirror IDs: 1 2 3 4 mirror_io: 391: hit by error injection mirror_io: 391: hit by error injection PASS 38 (8s) == sanity-flr test 39: check FLR+PFL (a.k.a. PFLR) creation ========================================================== 09:23:31 (1773235411) PASS 39 (4s) == sanity-flr test 40: PFLR rdonly state instantiation check ========================================================== 09:23:35 (1773235415) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0572973 s, 18.3 MB/s composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 760 lcm_flags: wp lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 7 components: - lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 0 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - l_ost_idx: 0 l_fid: 0x280000401:0x3a:0x0 - lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - l_ost_idx: 0 l_fid: 0x280000401:0x3b:0x0 - lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: prefer lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - l_ost_idx: 1 l_fid: 0x2c0000401:0x36:0x0 - lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: 0 lcme_extent.e_start: 1048576 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131079 lcme_mirror_id: 2 lcme_flags: 0 lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x7 lmm_fid: 0x200000403:0x7:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307527 s, 34.1 MB/s composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 760 lcm_flags: wp lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 7 components: - lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 0 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - l_ost_idx: 1 l_fid: 0x2c0000401:0x37:0x0 - lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init,prefer lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - l_ost_idx: 1 l_fid: 0x2c0000401:0x38:0x0 - lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: prefer lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - l_ost_idx: 0 l_fid: 0x280000401:0x3c:0x0 - lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: 0 lcme_extent.e_start: 1048576 lcme_extent.e_end: 2097152 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 2097152 lcme_extent.e_end: 4194304 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 - lcme_id: 131079 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 4194304 lcme_extent.e_end: EOF sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000403 lmm_object_id: 0x9 lmm_fid: 0x200000403:0x9:0x0 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 PASS 40 (5s) == sanity-flr test 41: lfs mirror resync check =========== 09:23:41 (1773235421) **create two FLR files /mnt/lustre/f41.sanity-flr /mnt/lustre/f41.sanity-flr-1 **verify files be RDONLY 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0595932 s, 35.2 MB/s 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.179022 s, 23.4 MB/s **verify files be WRITE_PENDING **verify files have stale component **full resync **verify /mnt/lustre/f41.sanity-flr-1 data consistency in all mirrors **verify files be RDONLY **verify files do not contain stale component **write /mnt/lustre/f41.sanity-flr-1 for partial resync test 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0825299 s, 25.4 MB/s **only resync mirror 2 **resync mirror 2 again **verify /mnt/lustre/f41.sanity-flr-1 contains stale component **full resync /mnt/lustre/f41.sanity-flr-1 **full resync /mnt/lustre/f41.sanity-flr-1 again **verify /mnt/lustre/f41.sanity-flr-1 does not contain stale component PASS 41 (6s) == sanity-flr test 42: lfs mirror verify ================= 09:23:47 (1773235427) SKIP: sanity-flr test_42 need >= 4 OSTs SKIP 42 (1s) == sanity-flr test 43a: mirror pick on write ============= 09:23:48 (1773235428) SKIP: sanity-flr test_43a needs >= 3 OSTs SKIP 43a (2s) == sanity-flr test 43b: allow writing to multiple preferred mirror file ========================================================== 09:23:50 (1773235430) striped dir -i1 -c2 -H crush /mnt/lustre/d43b.sanity-flr ** write to /mnt/lustre/d43b.sanity-flr/f43b.sanity-flr 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157046 s, 66.8 MB/s ** resync /mnt/lustre/d43b.sanity-flr/f43b.sanity-flr PASS 43b (4s) == sanity-flr test 44a: lfs mirror split check =========== 09:23:54 (1773235434) 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.15637 s, 20.1 MB/s PASS 44a (5s) == sanity-flr test 44b: mirror split does not create stale file ========================================================== 09:23:59 (1773235439) /mnt/lustre/d44b.sanity-flr/f44b.sanity-flr lcm_layout_gen: 2 lcm_mirror_count: 2 lcm_entry_count: 2 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0x45:0x0] } lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x42:0x0] } split mirror_id 1 id 65537 /mnt/lustre/d44b.sanity-flr/f44b.sanity-flr lcm_layout_gen: 5 lcm_mirror_count: 1 lcm_entry_count: 1 lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x42:0x0] } split mirror_id 2 id 131073 lfs split: '/mnt/lustre/d44b.sanity-flr/f44b.sanity-flr' is not a FLR file. Cannot resync file make sure there's no stale comp in the file PASS 44b (5s) SKIP: sanity-flr test_44c skipping ALWAYS excluded test 44c == sanity-flr test 44d: lfs mirror split does not break size ========================================================== 09:24:05 (1773235445) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.185773 s, 56.4 MB/s ** before mirror ops, file size=10485760 ** after mirror extend, file size=10485760 lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000403:0x7:0x0]: Invalid argument (22) ** after mirror split, file size=10485760 Stopping client oleg338-client.virtnet /mnt/lustre (opts:) Starting client: oleg338-client.virtnet: -o user_xattr,flock 192.168.203.138@tcp:/lustre /mnt/lustre ** after mirror split & remount, file size=10485760 PASS 44d (12s) == sanity-flr test 44e: basic FLR SOM tests + disable SOM ========================================================== 09:24:17 (1773235457) striped dir -i0 -c2 -H crush2 /mnt/lustre/d44e.sanity-flr llite.lustre-ffff89704a50b800.xattr_cache=0 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.191801 s, 54.7 MB/s ** before mirror resync, file size=10485760 file: /mnt/lustre/d44e.sanity-flr/f44e.sanity-flr size: 10485760 blocks: 40960 flags: 1 Stopping client oleg338-client.virtnet /mnt/lustre (opts:) Starting client: oleg338-client.virtnet: -o user_xattr,flock 192.168.203.138@tcp:/lustre /mnt/lustre mdt.lustre-MDT0000.enable_strict_som=0 mdt.lustre-MDT0001.enable_strict_som=0 file: /mnt/lustre/d44e.sanity-flr/f44e.sanity-flr size: 10485760 blocks: 40960 flags: 1 llite.lustre-ffff897048409800.xattr_cache=1 PASS 44e (8s) == sanity-flr test 45: Verify setstripe/getstripe with YAML with FLR file ========================================================== 09:24:25 (1773235465) striped dir -i1 -c2 -H crush /mnt/lustre/d45.sanity-flr 1. FLR file: getstripe --yaml /mnt/lustre/d45.sanity-flr/f45.sanity-flr 1. FLR file: setstripe --yaml=/mnt/lustre/d45.sanity-flr/template /mnt/lustre/d45.sanity-flr/f45.sanity-flr.copy 1. FLR file: compare 2. FLR file with flags: getstripe --yaml /mnt/lustre/d45.sanity-flr/f45.sanity-flr 2. FLR file with flags: setstripe --yaml=/mnt/lustre/d45.sanity-flr/template /mnt/lustre/d45.sanity-flr/f45.sanity-flr.copy 2. FLR file with flags: compare PASS 45 (20s) == sanity-flr test 46: Verify setstripe --copy option ==== 09:24:45 (1773235485) striped dir -i0 -c2 -H all_char /mnt/lustre/d46.sanity-flr ** 1. PFL file ** 2. plain file ** 3. FLR file ** 4. PFL dir striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d46.sanity-flr/dir striped dir -i0 -c2 -H all_char /mnt/lustre/d46.sanity-flr/dir.copy ** 5. plain dir ** 6. FLR dir PASS 46 (23s) == sanity-flr test 47: Verify mirror obj alloc =========== 09:25:08 (1773235508) SKIP: sanity-flr test_47 needs >= 3 OSTs SKIP 47 (1s) == sanity-flr test 48: Verify snapshot mirror ============ 09:25:10 (1773235510) ** create 2 mirrors FLR file /mnt/lustre/f48.sanity-flr ** write it 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.146997 s, 21.4 MB/s ** resync the file ** snapshot mirror 2 ** write it again 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.116932 s, 26.9 MB/s ** resync it again ** verify mirror 2 doesn't change original checksum: 031a194d18b0f652edf4064c673e3919 - mirror 1 checksum: 976eeb569b18fbc44fd645074826d454 - mirror 2 checksum: 031a194d18b0f652edf4064c673e3919 - ** mirror 2 stripe info /mnt/lustre/f48.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 448 lcm_flags: wp lcm_layout_gen: 10 lcm_mirror_count: 2 lcm_entry_count: 4 components: - lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale,nosync lcme_timestamp: 'Wed Mar 11 09:25:11 2026' lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lcme_offset: 336 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x9 lmm_fid: [0x200000405:0x9:0x0] lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x4d:0x0] } - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,stale,nosync lcme_timestamp: 'Wed Mar 11 09:25:11 2026' lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lcme_offset: 392 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x9 lmm_fid: [0x200000405:0x9:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x4e:0x0] } ** resync mirror 2 ** verify mirror 2 resync-ed original checksum: 031a194d18b0f652edf4064c673e3919 - mirror 1 checksum: 976eeb569b18fbc44fd645074826d454 - mirror 2 checksum: 976eeb569b18fbc44fd645074826d454 - ** mirror 2 stripe info /mnt/lustre/f48.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 448 lcm_flags: ro lcm_layout_gen: 12 lcm_mirror_count: 2 lcm_entry_count: 4 components: - lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,nosync lcme_timestamp: 'Wed Mar 11 09:25:11 2026' lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lcme_offset: 336 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x9 lmm_fid: [0x200000405:0x9:0x0] lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x4d:0x0] } - lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,nosync lcme_timestamp: 'Wed Mar 11 09:25:11 2026' lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lcme_offset: 392 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x9 lmm_fid: [0x200000405:0x9:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x4e:0x0] } PASS 48 (5s) == sanity-flr test 49a: FIEMAP upon FLR file ============= 09:25:15 (1773235515) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222501 s, 47.1 MB/s Filesystem type is: bd00bd0 File size of /mnt/lustre/f49a.sanity-flr is 1048576 (1024 blocks of 1024 bytes) ext: device_logical: physical_offset: length: dev: flags: 0: 0.. 1023: 401408.. 402431: 1024: 0001: net,eof 1: 0.. 1023: 308224.. 309247: 1024: 0000: last,net,eof /mnt/lustre/f49a.sanity-flr: 2 extents found FIEMAP on /mnt/lustre/f49a.sanity-flr succeeded PASS 49a (4s) == sanity-flr test 50A: mirror split update layout generation ========================================================== 09:25:19 (1773235519) Failing mds1 on oleg338-server Stopping /mnt/lustre-mds1 (opts:) on oleg338-server 09:25:29 (1773235529) shut down facet: mds1 facet_host: oleg338-server facet_failover_host: oleg338-server Failover mds1 to oleg338-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg338-server: oleg338-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg338-client: oleg338-server: ssh exited with exit code 1 Started lustre-MDT0000 09:25:46 (1773235546) targets are mounted 09:25:46 (1773235546) facet_failover done oleg338-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0xf:0x0]: Invalid argument (22) /mnt/lustre/d50A.sanity-flr/f50A.sanity-flr layout generation from 1 to 2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251969 s, 41.6 MB/s /mnt/lustre/d50A.sanity-flr/f50A.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 136 lcm_flags: 0 lcm_layout_gen: 2 lcm_mirror_count: 1 lcm_entry_count: 1 components: - lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lcme_offset: 80 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0xf lmm_fid: [0x200000405:0xf:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x50:0x0] } PASS 50A (37s) == sanity-flr test 50a: mirror extend/copy preserves sparseness ========================================================== 09:25:56 (1773235556) ** create striped file /mnt/lustre/d50a.sanity-flr/f50a.sanity-flr ** write 1st data chunk at 1M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0243966 s, 839 kB/s ** write 2nd data chunk at 2M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0151477 s, 1.4 MB/s ** create hole at the file end ** verify sparseness first data offset: 1044480 hole at the end: 3500000 ** extend the file with new mirror ** verify mirror #2 sparseness first data offset: 1044480 hole at the end: 3500000 ** copy mirror #2 to mirror #1 mirror copied successfully: 1 ** verify mirror #1 sparseness again first data offset: 1044480 hole at the end: 3500000 ** final consumed blocks: 104 PASS 50a (5s) == sanity-flr test 50b: mirror rsync handles sparseness == 09:26:01 (1773235561) ** create mirrored file /mnt/lustre/d50b.sanity-flr/f50b.sanity-flr ** write data chunk at 1M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0365364 s, 561 kB/s ** create hole at the file end ** verify sparseness first data offset: 1044480 hole at the end: 3500000 ** resync mirror #2 to mirror #1 ** consumed blocks: 96 ** truncate file down ** write data chunk at 2M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.00591966 s, 3.5 MB/s ** resync mirror #2 to mirror #1 with nosync 1st component ** resync mirror #2 to mirror #1 again ** final consumed blocks: 96 PASS 50b (5s) == sanity-flr test 50c: punch_hole/mmap_write stale other mirrors ========================================================== 09:26:06 (1773235566) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d50c.sanity-flr 4+0 records in 4+0 records out 16384 bytes (16 kB, 16 KiB) copied, 0.00413142 s, 4.0 MB/s PASS 50c (4s) == sanity-flr test 50d: mirror rsync keep holes ========== 09:26:10 (1773235570) ** create mirrored file /mnt/lustre/d50d.sanity-flr/f50d.sanity-flr ** write data chunk at 1M boundary 20+0 records in 20+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0255874 s, 800 kB/s ** create hole at the file start ** verify sparseness first data offset: 1048576 ** resync mirror #2 ** verify sparseness of mirror #2 first data offset: 1048576 PASS 50d (5s) == sanity-flr test 60a: mirror extend sets correct size on sparse file ========================================================== 09:26:15 (1773235575) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00310019 s, 1.3 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0258851 s, 158 kB/s PASS 60a (4s) == sanity-flr test 60b: mirror merge/split cancel client's in-memory layout gen ========================================================== 09:26:19 (1773235579) striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d60b.sanity-flr lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x20:0x0]: Invalid argument (22) PASS 60b (11s) == sanity-flr test 61a: mirror extend and migrate preserve timestamps ========================================================== 09:26:30 (1773235590) mdd.lustre-MDT0000.atime_diff=1 mdd.lustre-MDT0001.atime_diff=1 create /mnt/lustre/d61a.sanity-flr/f61a.sanity-flr create /mnt/lustre/d61a.sanity-flr/f61a.sanity-flr-2 sleep 5 seconds, then cat f61a.sanity-flr XXX sleep 5 seconds, then re-write f61a.sanity-flr sleep 5 seconds mirror merge f61a.sanity-flr-2 to f61a.sanity-flr and test timestamps mirror extend f61a.sanity-flr and test timestamps migrate f61a.sanity-flr and test timestamps normal user migrate f61a.sanity-flr and test timestamps running as uid/gid/euid/egid 500/500/500/500, groups: 0 [/home/green/git/lustre-release/lustre/utils/lfs] [migrate] [-n] [/mnt/lustre/d61a.sanity-flr/f61a.sanity-flr] mdd.lustre-MDT0000.atime_diff=60 mdd.lustre-MDT0001.atime_diff=60 PASS 61a (24s) == sanity-flr test 61b: mirror extend and split preserve timestamps ========================================================== 09:26:54 (1773235614) create /mnt/lustre/d61b.sanity-flr/f61b.sanity-flr sleep 5 seconds, then cat f61b.sanity-flr XXX sleep 5 seconds, then re-write f61b.sanity-flr sleep 5 seconds, then test timestamps mirror extend f61b.sanity-flr and test timestamps mirror split f61b.sanity-flr and test timestamps lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x4a:0x0]: Invalid argument (22) normal user mirror extend f61b.sanity-flr and test timestamps running as uid/gid/euid/egid 500/500/500/500, groups: 0 [/home/green/git/lustre-release/lustre/utils/lfs] [mirror] [extend] [-N] [-c1] [-i1] [/mnt/lustre/d61b.sanity-flr/f61b.sanity-flr] PASS 61b (19s) == sanity-flr test 61c: mirror resync preserves timestamps ========================================================== 09:27:13 (1773235633) create /mnt/lustre/d61c.sanity-flr/f61c.sanity-flr sleep 5 seconds, then cat f61c.sanity-flr XXX sleep 5 seconds, then mirror extend f61c.sanity-flr and write it sleep 5 seconds, then resync f61c.sanity-flr and test timestamps normal user resync f61c.sanity-flr and test timestamps running as uid/gid/euid/egid 500/500/500/500, groups: 500 [/home/green/git/lustre-release/lustre/utils/lfs] [mirror] [resync] [/mnt/lustre/d61c.sanity-flr/f61c.sanity-flr] PASS 61c (21s) == sanity-flr test 62: read/write with unknown type of mirror ========================================================== 09:27:34 (1773235654) create mirror file with unknown magic fail_loc=0x1426 fail_val=2 create mirror file with unknown pattern fail_loc=0x1427 fail_val=1 PASS 62 (4s) == sanity-flr test 70: mirror create and split race ====== 09:27:38 (1773235658) striped dir -i0 -c2 -H crush2 /mnt/lustre/d70.sanity-flr mirror create pid 55854 mirror split pid 55855 mirror create and split race for 60 seconds, should not crash /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4919: 55854 Killed while true; do rm -f $tf; $LFS mirror create -N -E 1M -c -1 -E eof -N $tf; echo xxxx > $tf; done (wd: ~) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4919: 55855 Killed while true; do $LFS mirror split -d --mirror-id=1 $tf &> /dev/null; done (wd: ~) PASS 70 (64s) == sanity-flr test 70a: flr mode fsx test ================ 09:28:42 (1773235722) striped dir -i0 -c2 -H all_char /mnt/lustre/d70a.sanity-flr Seed set to 5723 000001 1773235724.012277 mapwrite 0x03e0a8 thru 0x03ffff (0x1f58 bytes) 000002 1773235724.047410 MIRROR_EXTEND to 2 mirrors 000003 1773235724.551349 write 0x0339c5 thru 0x033f72 (0x5ae bytes) 000004 1773235724.598891 punch from 0x02d47d to 0x030a2d (0x35b1 bytes) 000005 1773235724.603239 read 0x005025 thru 0x014384 (0xf360 bytes) 000006 1773235724.614723 punch from 0x0395ba to 0x03ffff (0x6a46 bytes) 000007 1773235724.624308 MIRROR_EXTEND to 3 mirrors 000009 1773235725.043140 write 0x003783 thru 0x01246f (0xeced bytes) 000010 1773235725.114827 MIRROR_EXTEND to 4 mirrors truncating to largest ever: 0x39186 000012 1773235725.543259 trunc from 0x040000 to 0x039185 (0x6e7a bytes) 000013 1773235725.588283 MIRROR_RESYNC 4 mirrors 000014 1773235726.217837 mapread 0x00d319 thru 0x011fc9 (0x4cb1 bytes) 000015 1773235726.233780 fallocate 0x00d558 thru 0x0186a6 (0xb14f bytes) 000016 1773235726.252794 mapwrite 0x025278 thru 0x032a5d (0xd7e6 bytes) 000017 1773235726.285228 mapread 0x002f42 thru 0x00e5e3 (0xb6a2 bytes) 000018 1773235726.290959 mapwrite 0x01f03e thru 0x0254a7 (0x646a bytes) 000019 1773235726.307221 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000020 1773235726.452368 trunc from 0x01c504 to 0x01c9d3 (0x4d0 bytes) 000021 1773235726.457227 write 0x0310bc thru 0x03e7a7 (0xd6ec bytes) 000022 1773235726.465730 trunc from 0x03e7a8 to 0x038742 (0x6065 bytes) 000023 1773235726.474889 read 0x01f648 thru 0x02f02c (0xf9e5 bytes) 000024 1773235726.481450 mapwrite 0x024a92 thru 0x030fee (0xc55d bytes) 000025 1773235726.502509 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000027 1773235726.686672 mapwrite 0x006bf1 thru 0x00cf44 (0x6354 bytes) 000028 1773235726.714570 mapread 0x02727f thru 0x02b6d3 (0x4455 bytes) 000029 1773235726.719669 mapwrite 0x020672 thru 0x02d328 (0xccb7 bytes) 000030 1773235726.736219 mapread 0x0120f6 thru 0x01f00f (0xcf1a bytes) 000032 1773235726.754068 mapwrite 0x00b51d thru 0x0147ce (0x92b2 bytes) 000033 1773235726.772651 trunc from 0x038743 to 0x01adbf (0x1d983 bytes) 000034 1773235726.787818 read 0x003613 thru 0x005d46 (0x2734 bytes) 000035 1773235726.788318 punch from 0x002d23 to 0x011a67 (0xed45 bytes) 000036 1773235726.796304 mapwrite 0x01ba22 thru 0x02a95d (0xef3c bytes) 000037 1773235726.845335 mapwrite 0x00a9be thru 0x012d3c (0x837f bytes) 000038 1773235726.873786 mapread 0x00ca29 thru 0x019ae5 (0xd0bd bytes) 000039 1773235726.874714 read 0x01ae09 thru 0x022eb0 (0x80a8 bytes) 000040 1773235726.875547 fallocate 0x008f15 thru 0x011872 (0x895e bytes) 000041 1773235726.879500 punch from 0x01a89c to 0x025a24 (0xb189 bytes) 000042 1773235726.883462 punch from 0x00c279 to 0x018691 (0xc419 bytes) 000043 1773235726.888301 punch from 0x0053ea to 0x00b69c (0x62b3 bytes) 000044 1773235726.900088 fallocate 0x000e43 thru 0x009c9b (0x8e59 bytes) 000045 1773235726.904812 trunc from 0x02a95e to 0x033cf2 (0x9395 bytes) truncating to largest ever: 0x3b360 000046 1773235726.919036 trunc from 0x033cf3 to 0x03b35f (0x766d bytes) 000048 1773235726.939220 mapread 0x0320e7 thru 0x03b35f (0x9279 bytes) 000049 1773235726.953142 MIRROR_RESYNC 2 mirrors 000050 1773235727.408656 trunc from 0x03b360 to 0x034e9f (0x64c0 bytes) 000051 1773235727.452107 MIRROR_EXTEND to 3 mirrors 000052 1773235727.916678 read 0x02c434 thru 0x034e9f (0x8a6c bytes) 000053 1773235727.932215 mapwrite 0x022893 thru 0x02e861 (0xbfcf bytes) 000054 1773235728.034281 trunc from 0x034ea0 to 0x01322f (0x21c70 bytes) fallocating to largest ever: 0x40000 000055 1773235728.058290 fallocate 0x03aa65 thru 0x03ffff (0x559b bytes) 000057 1773235728.063324 trunc from 0x040000 to 0x010956 (0x2f6a9 bytes) 000058 1773235728.083005 trunc from 0x010957 to 0x00d695 (0x32c1 bytes) 000059 1773235728.098510 mapwrite 0x019a13 thru 0x020d70 (0x735e bytes) 000060 1773235728.139764 trunc from 0x001f59 to 0x004084 (0x212c bytes) 000061 1773235728.145242 read 0x008712 thru 0x014a3c (0xc32b bytes) 000062 1773235728.158437 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000064 1773235728.302888 fallocate 0x007d7b thru 0x00dc3a (0x5ec0 bytes) 000065 1773235728.306677 punch from 0x00473d to 0x004fd1 (0x895 bytes) 000066 1773235728.309993 trunc from 0x01d1fa to 0x01f3a0 (0x21a7 bytes) 000067 1773235728.314701 mapread 0x011f68 thru 0x018fe7 (0x7080 bytes) 000068 1773235728.326998 mapwrite 0x01a925 thru 0x021eb6 (0x7592 bytes) 000070 1773235728.372257 fallocate 0x0069ef thru 0x011a05 (0xb017 bytes) 000071 1773235728.384166 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000072 1773235728.469383 read 0x00833c thru 0x0176a1 (0xf366 bytes) 000073 1773235728.476028 punch from 0x01b90b to 0x021a40 (0x6136 bytes) 000074 1773235728.481141 trunc from 0x021eb7 to 0x015c26 (0xc290 bytes) 000075 1773235728.506510 MIRROR_EXTEND to 2 mirrors 000076 1773235728.867439 trunc from 0x015c27 to 0x01c044 (0x641e bytes) 000078 1773235728.923717 trunc from 0x011da1 to 0x01c044 (0xa2a4 bytes) 000079 1773235728.927479 trunc from 0x017b4d to 0x01c044 (0x44f8 bytes) 000080 1773235728.930712 trunc from 0x01c045 to 0x0283a1 (0xc35d bytes) 000081 1773235728.947374 mapwrite 0x00d06a thru 0x01aff7 (0xdf8e bytes) 000082 1773235728.992222 trunc from 0x0283a2 to 0x004642 (0x23d5f bytes) 000083 1773235729.005071 trunc from 0x0039ea to 0x004642 (0xc59 bytes) 000084 1773235729.009158 write 0x01b8d7 thru 0x01f39a (0x3ac4 bytes) 000085 1773235729.010298 punch from 0x0153f0 to 0x01f39a (0x9fab bytes) 000086 1773235729.026222 MIRROR_EXTEND to 3 mirrors 000087 1773235729.411218 trunc from 0x01f39b to 0x026e2a (0x7a90 bytes) 000088 1773235729.435246 mapwrite 0x034307 thru 0x03e4ce (0xa1c8 bytes) 000089 1773235729.490448 write 0x038133 thru 0x03de04 (0x5cd2 bytes) 000090 1773235729.492047 trunc from 0x03e4cf to 0x01b9de (0x22af0 bytes) 000091 1773235729.517037 mapwrite 0x0047d2 thru 0x007d96 (0x35c5 bytes) 000092 1773235729.551178 write 0x024886 thru 0x027d2e (0x34a9 bytes) 000093 1773235729.552256 read 0x011984 thru 0x013c35 (0x22b2 bytes) 000094 1773235729.564908 MIRROR_EXTEND to 4 mirrors 000095 1773235729.951051 trunc from 0x027d2f to 0x03326a (0xb53c bytes) 000096 1773235730.016515 MIRROR_EXTEND to 5 mirrors 000097 1773235730.465638 trunc from 0x03326b to 0x03915e (0x5ef4 bytes) 000098 1773235730.491403 trunc from 0x03915f to 0x0184b7 (0x20ca7 bytes) 000099 1773235730.501935 mapread 0x012b08 thru 0x017eb0 (0x53a9 bytes) 000100 1773235730.514300 MIRROR_EXTEND to 6 mirrors 000102 1773235730.848165 trunc from 0x017556 to 0x0184b7 (0xf62 bytes) 000103 1773235730.879525 read 0x0121bc thru 0x0184b7 (0x62fc bytes) 000104 1773235730.886122 mapwrite 0x015386 thru 0x022d8f (0xda0a bytes) 000105 1773235730.943601 mapwrite 0x030aa6 thru 0x03a492 (0x99ed bytes) 000106 1773235730.997192 trunc from 0x00b3bf to 0x010d92 (0x59d4 bytes) 000107 1773235731.000438 write 0x01cd6a thru 0x01f68f (0x2926 bytes) 000109 1773235731.001547 fallocate 0x01c37b thru 0x02885e (0xc4e4 bytes) 000110 1773235731.004471 write 0x0088b7 thru 0x00ec97 (0x63e1 bytes) 000111 1773235731.015191 fallocate 0x005c78 thru 0x00e4b9 (0x8842 bytes) 000112 1773235731.020218 fallocate 0x0029c2 thru 0x00c350 (0x998f bytes) 000113 1773235731.030420 MIRROR_SPLIT mirror 4 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000114 1773235731.181599 mapread 0x0270f0 thru 0x0306cb (0x95dc bytes) 000115 1773235731.192306 trunc from 0x033130 to 0x03a492 (0x7363 bytes) 000116 1773235731.196103 read 0x033e29 thru 0x03a492 (0x666a bytes) 000117 1773235731.202010 fallocate 0x01d16e thru 0x022a46 (0x58d9 bytes) 000118 1773235731.206231 fallocate 0x01ad49 thru 0x01b37a (0x632 bytes) 000119 1773235731.210624 read 0x002ecb thru 0x012322 (0xf458 bytes) 000120 1773235731.220035 read 0x00b72d thru 0x016fee (0xb8c2 bytes) 000121 1773235731.220496 punch from 0x01802a to 0x0247f7 (0xc7ce bytes) 000122 1773235731.225174 trunc from 0x03a493 to 0x0323ff (0x8093 bytes) 000123 1773235731.240534 mapwrite 0x00e9b2 thru 0x018f59 (0xa5a8 bytes) 000124 1773235731.263316 mapwrite 0x025169 thru 0x0345ea (0xf482 bytes) 000125 1773235731.293829 write 0x013064 thru 0x019cb8 (0x6c55 bytes) 000126 1773235731.304717 trunc from 0x0316f8 to 0x0345ea (0x2ef3 bytes) 000127 1773235731.316086 MIRROR_EXTEND to 6 mirrors 000128 1773235731.704401 mapread 0x0011c2 thru 0x00a9bf (0x97fe bytes) 000130 1773235731.724211 mapwrite 0x003d13 thru 0x00f7d9 (0xbac7 bytes) 000131 1773235731.793218 write 0x01c6c4 thru 0x023f6e (0x78ab bytes) 000132 1773235731.802574 MIRROR_SPLIT mirror 3 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000133 1773235731.952964 mapwrite 0x02da74 thru 0x02e59b (0xb28 bytes) 000134 1773235731.966070 mapwrite 0x034b8f thru 0x03ffff (0xb471 bytes) 000136 1773235732.024414 trunc from 0x040000 to 0x032c8e (0xd371 bytes) 000137 1773235732.038214 trunc from 0x02625f to 0x02fcd2 (0x9a74 bytes) 000138 1773235732.042105 mapread 0x0021c9 thru 0x006186 (0x3fbe bytes) 000139 1773235732.048150 trunc from 0x032c8f to 0x003b60 (0x2f12e bytes) 000140 1773235732.062347 write 0x02616b thru 0x02e600 (0x8496 bytes) 000141 1773235732.063775 fallocate 0x007bd8 thru 0x00b9dc (0x3e05 bytes) 000142 1773235732.073469 trunc from 0x013e04 to 0x01bb42 (0x7d3f bytes) 000143 1773235732.080040 trunc from 0x016cb3 to 0x0204d1 (0x981f bytes) 000144 1773235732.093122 fallocate 0x02be14 thru 0x030144 (0x4331 bytes) 000145 1773235732.097960 write 0x0119a3 thru 0x0126c6 (0xd24 bytes) 000146 1773235732.106474 punch from 0x019890 to 0x0261ea (0xc95b bytes) 000147 1773235732.122388 write 0x0037d2 thru 0x00f404 (0xbc33 bytes) 000148 1773235732.123576 fallocate 0x003de6 thru 0x01121a (0xd435 bytes) 000149 1773235732.127393 write 0x017820 thru 0x01c45c (0x4c3d bytes) 000150 1773235732.132344 mapwrite 0x010fdc thru 0x01e105 (0xd12a bytes) 000151 1773235732.157032 trunc from 0x030145 to 0x0157bb (0x1a989 bytes) 000152 1773235732.169356 mapwrite 0x02137e thru 0x0236ba (0x233d bytes) 000153 1773235732.191603 trunc from 0x01f396 to 0x01f3d1 (0x3c bytes) 000154 1773235732.196714 trunc from 0x0236bb to 0x031a22 (0xe368 bytes) 000155 1773235732.208261 fallocate 0x032cfc thru 0x03c35f (0x9664 bytes) 000156 1773235732.214243 write 0x03be41 thru 0x03ffff (0x41bf bytes) 000158 1773235732.219677 mapwrite 0x001926 thru 0x00cf51 (0xb62c bytes) 000159 1773235732.230427 read 0x029d40 thru 0x03904e (0xf30f bytes) 000161 1773235732.236308 mapwrite 0x00e3e1 thru 0x0157c4 (0x73e4 bytes) 000162 1773235732.244343 trunc from 0x040000 to 0x0384e4 (0x7b1b bytes) 000163 1773235732.252701 mapwrite 0x02eaba thru 0x03c96a (0xdeb1 bytes) 000165 1773235732.272516 write 0x03f559 thru 0x03ffff (0xaa7 bytes) 000166 1773235732.273230 mapwrite 0x034c59 thru 0x03ffff (0xb3a7 bytes) 000167 1773235732.284490 mapread 0x037895 thru 0x03ffff (0x876b bytes) 000168 1773235732.285146 trunc from 0x015aca to 0x01f0e2 (0x9619 bytes) 000169 1773235732.288097 punch from 0x018ea2 to 0x01eaaf (0x5c0e bytes) 000170 1773235732.290473 trunc from 0x00a37a to 0x011d79 (0x7a00 bytes) 000171 1773235732.293425 fallocate 0x03d1bb thru 0x03ffff (0x2e45 bytes) 000172 1773235732.296087 fallocate 0x012610 thru 0x01ad2e (0x871f bytes) 000173 1773235732.302693 MIRROR_EXTEND to 6 mirrors 000174 1773235732.779085 MIRROR_EXTEND to 7 mirrors 000175 1773235733.242250 trunc from 0x00ebb5 to 0x015bbb (0x7007 bytes) 000176 1773235733.269499 mapwrite 0x010536 thru 0x0112e0 (0xdab bytes) 000177 1773235733.293994 fallocate 0x000955 thru 0x0060e7 (0x5793 bytes) 000178 1773235733.298497 trunc from 0x040000 to 0x034c10 (0xb3ef bytes) 000179 1773235733.336114 fallocate 0x0016ba thru 0x0100a0 (0xe9e7 bytes) 000180 1773235733.340032 read 0x00acf9 thru 0x018c03 (0xdf0b bytes) 000181 1773235733.351400 fallocate 0x016ab6 thru 0x0206e1 (0x9c2c bytes) 000182 1773235733.356116 trunc from 0x034c11 to 0x00c326 (0x288ea bytes) 000183 1773235733.372429 read 0x0087ad thru 0x00b0c9 (0x291d bytes) 000185 1773235733.377320 fallocate 0x02b488 thru 0x030e39 (0x59b2 bytes) 000186 1773235733.380540 write 0x02c3f7 thru 0x032de0 (0x69ea bytes) 000187 1773235733.381480 trunc from 0x00e9b0 to 0x01cd34 (0xe385 bytes) 000188 1773235733.384326 write 0x01e423 thru 0x028115 (0x9cf3 bytes) 000190 1773235733.394416 MIRROR_RESYNC 7 mirrors 000191 1773235734.488868 punch from 0x011cf7 to 0x020a2b (0xed35 bytes) 000192 1773235734.517004 MIRROR_SPLIT mirror 1 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000193 1773235734.669998 punch from 0x0076a1 to 0x0146d9 (0xd039 bytes) 000194 1773235734.675547 mapread 0x016c73 thru 0x026a43 (0xfdd1 bytes) 000195 1773235734.705186 fallocate 0x00b328 thru 0x00f5b3 (0x428c bytes) 000196 1773235734.710470 mapread 0x019440 thru 0x0265e8 (0xd1a9 bytes) 000197 1773235734.711164 write 0x0253eb thru 0x02b009 (0x5c1f bytes) 000198 1773235734.727197 punch from 0x017509 to 0x018b4f (0x1647 bytes) 000199 1773235734.732622 mapwrite 0x028fa9 thru 0x03205c (0x90b4 bytes) 000200 1773235734.761504 trunc from 0x032de1 to 0x00aea1 (0x27f3f bytes) 000201 1773235734.773243 mapwrite 0x005a3f thru 0x012320 (0xc8e2 bytes) 000202 1773235734.801992 mapwrite 0x01b3d9 thru 0x01f048 (0x3c70 bytes) 000203 1773235734.835058 mapread 0x012486 thru 0x015a13 (0x358e bytes) 000205 1773235734.839664 read 0x008378 thru 0x010d16 (0x899f bytes) 000206 1773235734.840130 trunc from 0x0149b1 to 0x01f048 (0xa698 bytes) 000207 1773235734.845225 mapread 0x001647 thru 0x010ca6 (0xf660 bytes) 000208 1773235734.845977 fallocate 0x014dc7 thru 0x01aca8 (0x5ee2 bytes) 000209 1773235734.852126 read 0x016ded thru 0x01bb12 (0x4d26 bytes) 000210 1773235734.861453 mapread 0x012ebb thru 0x01e65d (0xb7a3 bytes) 000211 1773235734.862548 read 0x01e6de thru 0x01f048 (0x96b bytes) 000212 1773235734.870815 MIRROR_SPLIT mirror 2 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000214 1773235735.031330 read 0x01cb7a thru 0x01f048 (0x24cf bytes) 000215 1773235735.038208 fallocate 0x009d05 thru 0x00f9f7 (0x5cf3 bytes) 000216 1773235735.042855 mapwrite 0x01fc8c thru 0x02217c (0x24f1 bytes) 000217 1773235735.092978 read 0x01520b thru 0x01be6f (0x6c65 bytes) 000218 1773235735.102088 trunc from 0x003f32 to 0x004545 (0x614 bytes) 000219 1773235735.104975 write 0x03a767 thru 0x03fd03 (0x559d bytes) 000220 1773235735.106117 punch from 0x01305f to 0x0169df (0x3981 bytes) 000222 1773235735.110621 mapread 0x034429 thru 0x03b5e3 (0x71bb bytes) 000223 1773235735.118065 fallocate 0x008c96 thru 0x01592e (0xcc99 bytes) 000224 1773235735.121581 trunc from 0x03fd04 to 0x01c54f (0x237b4 bytes) 000225 1773235735.137991 mapread 0x00465f thru 0x0124bf (0xde61 bytes) 000226 1773235735.146326 punch from 0x00fd37 to 0x01bfee (0xc2b8 bytes) 000227 1773235735.152814 fallocate 0x00ffcc thru 0x01e647 (0xe67c bytes) 000228 1773235735.160623 mapread 0x001902 thru 0x002a30 (0x112f bytes) 000229 1773235735.163263 trunc from 0x01e648 to 0x03f5c0 (0x20f79 bytes) 000231 1773235735.176584 write 0x00e398 thru 0x0187bf (0xa428 bytes) 000232 1773235735.186904 MIRROR_RESYNC 5 mirrors 000233 1773235735.868666 trunc from 0x01fe32 to 0x02e929 (0xeaf8 bytes) 000235 1773235735.900353 mapwrite 0x00e5c6 thru 0x0199c0 (0xb3fb bytes) 000236 1773235735.950434 fallocate 0x03e6a9 thru 0x03ffff (0x1957 bytes) 000237 1773235735.955053 write 0x01b9c5 thru 0x0237af (0x7deb bytes) 000238 1773235735.959887 fallocate 0x03a316 thru 0x03deac (0x3b97 bytes) 000239 1773235735.963989 mapwrite 0x002025 thru 0x00b710 (0x96ec bytes) 000240 1773235735.993111 mapwrite 0x039fe0 thru 0x03ffff (0x6020 bytes) 000241 1773235736.016230 mapread 0x007b8d thru 0x015ddf (0xe253 bytes) 000242 1773235736.017170 fallocate 0x0384ce thru 0x03904c (0xb7f bytes) 000243 1773235736.021606 trunc from 0x000716 to 0x00f35e (0xec49 bytes) 000244 1773235736.025965 read 0x031074 thru 0x03ad5c (0x9ce9 bytes) 000245 1773235736.031984 trunc from 0x040000 to 0x0352ae (0xad51 bytes) 000246 1773235736.058513 write 0x03b10f thru 0x03c6de (0x15d0 bytes) 000247 1773235736.059491 trunc from 0x02a5b0 to 0x02bb8c (0x15dd bytes) 000248 1773235736.061952 mapwrite 0x01a605 thru 0x01be7b (0x1877 bytes) 000249 1773235736.070999 mapwrite 0x03e7c4 thru 0x03ffff (0x183c bytes) 000250 1773235736.110538 MIRROR_EXTEND to 6 mirrors 000251 1773235736.515612 trunc from 0x0091c4 to 0x0173e9 (0xe226 bytes) 000252 1773235736.542020 read 0x02a8ce thru 0x03a25b (0xf98e bytes) 000253 1773235736.548681 trunc from 0x0034a4 to 0x00cf29 (0x9a86 bytes) 000254 1773235736.552522 trunc from 0x040000 to 0x017d5c (0x282a3 bytes) 000255 1773235736.587945 trunc from 0x008cd9 to 0x00d9e2 (0x4d0a bytes) 000256 1773235736.598033 trunc from 0x016664 to 0x017d5c (0x16f9 bytes) 000257 1773235736.602276 mapread 0x0041b1 thru 0x006617 (0x2467 bytes) 000258 1773235736.615412 read 0x000454 thru 0x00363f (0x31ec bytes) 000259 1773235736.623257 read 0x0150ed thru 0x017d5c (0x2c70 bytes) 000260 1773235736.624451 write 0x02eca5 thru 0x03007f (0x13db bytes) 000261 1773235736.626093 trunc from 0x030080 to 0x00f641 (0x20a3e bytes) 000262 1773235736.638390 fallocate 0x018c81 thru 0x021039 (0x83b9 bytes) 000263 1773235736.641336 read 0x00df7e thru 0x00f641 (0x16c4 bytes) 000264 1773235736.641987 read 0x0013f9 thru 0x009c3f (0x8847 bytes) 000265 1773235736.642398 read 0x00c07c thru 0x00f641 (0x35c6 bytes) 000266 1773235736.642924 mapwrite 0x001112 thru 0x00665f (0x554e bytes) 000267 1773235736.656375 write 0x004080 thru 0x011e9c (0xde1d bytes) 000268 1773235736.658083 mapwrite 0x009e10 thru 0x018bd4 (0xedc5 bytes) 000269 1773235736.690450 trunc from 0x018bd5 to 0x03239f (0x197cb bytes) 000270 1773235736.697999 trunc from 0x0090ef to 0x013036 (0x9f48 bytes) 000271 1773235736.703525 write 0x01bf48 thru 0x02ab1a (0xebd3 bytes) 000272 1773235736.715920 mapwrite 0x003e2c thru 0x00eb9b (0xad70 bytes) 000273 1773235736.764954 mapread 0x00113b thru 0x0029e3 (0x18a9 bytes) 000274 1773235736.765949 mapread 0x023120 thru 0x02ae75 (0x7d56 bytes) 000275 1773235736.770954 MIRROR_RESYNC 6 mirrors 000276 1773235737.922233 mapwrite 0x01f847 thru 0x026302 (0x6abc bytes) 000277 1773235738.011109 MIRROR_EXTEND to 7 mirrors 000278 1773235738.600438 MIRROR_RESYNC 7 mirrors 000279 1773235739.733296 MIRROR_EXTEND to 8 mirrors 000280 1773235740.263894 read 0x00c6a5 thru 0x01adff (0xe75b bytes) 000281 1773235740.277588 mapwrite 0x023590 thru 0x02eaca (0xb53b bytes) 000282 1773235740.370696 trunc from 0x0323a0 to 0x0139a8 (0x1e9f7 bytes) 000283 1773235740.402970 mapread 0x00de86 thru 0x011ba4 (0x3d1f bytes) 000286 1773235740.410450 trunc from 0x0138fb to 0x0139a8 (0xae bytes) 000287 1773235740.414668 mapwrite 0x0380d1 thru 0x03ed88 (0x6cb8 bytes) 000288 1773235740.447267 trunc from 0x03ed89 to 0x03e65a (0x72e bytes) 000289 1773235740.465506 mapwrite 0x024b94 thru 0x030866 (0xbcd3 bytes) 000290 1773235740.487590 trunc from 0x03e65b to 0x033a75 (0xabe5 bytes) 000291 1773235740.502134 read 0x009ada thru 0x014ef6 (0xb41d bytes) 000292 1773235740.511780 trunc from 0x01b1f9 to 0x021036 (0x5e3e bytes) 000293 1773235740.517101 trunc from 0x033a76 to 0x007cb9 (0x2bdbc bytes) 000294 1773235740.529026 trunc from 0x007cba to 0x02428a (0x1c5d1 bytes) 000295 1773235740.540392 read 0x01a4ea thru 0x02428a (0x9da1 bytes) 000296 1773235740.545505 trunc from 0x00f299 to 0x017dd6 (0x8b3e bytes) 000297 1773235740.549452 fallocate 0x01beb5 thru 0x02a966 (0xeab2 bytes) 000299 1773235740.553186 write 0x01ceff thru 0x01d953 (0xa55 bytes) 000300 1773235740.554067 write 0x0150d1 thru 0x01eedd (0x9e0d bytes) 000301 1773235740.565169 mapwrite 0x00aedb thru 0x01736f (0xc495 bytes) 000302 1773235740.611436 punch from 0x023597 to 0x025d0a (0x2774 bytes) 000303 1773235740.620172 mapread 0x01ec3a thru 0x02a966 (0xbd2d bytes) 000304 1773235740.636928 read 0x003600 thru 0x010f8f (0xd990 bytes) 000305 1773235740.641597 MIRROR_RESYNC 8 mirrors 000306 1773235741.810002 mapwrite 0x0243a3 thru 0x028f1a (0x4b78 bytes) 000308 1773235741.898439 trunc from 0x01f1a0 to 0x02a966 (0xb7c7 bytes) 000310 1773235741.904375 trunc from 0x01fb07 to 0x027495 (0x798f bytes) 000311 1773235741.910236 trunc from 0x02a967 to 0x03a744 (0xfdde bytes) 000312 1773235741.925951 fallocate 0x011075 thru 0x01f25e (0xe1ea bytes) 000313 1773235741.930272 fallocate 0x03b5f6 thru 0x03ffff (0x4a0a bytes) 000314 1773235741.934596 trunc from 0x040000 to 0x0341fa (0xbe05 bytes) 000315 1773235741.949837 trunc from 0x02e0f6 to 0x0341fa (0x6105 bytes) 000316 1773235741.955481 punch from 0x00d714 to 0x01978c (0xc079 bytes) 000317 1773235741.962532 trunc from 0x012d50 to 0x017b26 (0x4dd7 bytes) 000318 1773235741.972154 mapread 0x00b0cd thru 0x0181e9 (0xd11d bytes) 000319 1773235741.996517 read 0x009c25 thru 0x0199b8 (0xfd94 bytes) 000320 1773235741.997156 trunc from 0x00223b to 0x0039a9 (0x176f bytes) 000321 1773235742.003373 read 0x01db76 thru 0x028711 (0xab9c bytes) 000322 1773235742.018202 trunc from 0x0341fb to 0x01efa0 (0x1525a bytes) 000323 1773235742.048900 mapread 0x0134a4 thru 0x01c5c4 (0x9121 bytes) 000324 1773235742.056239 MIRROR_EXTEND to 9 mirrors 000325 1773235742.562899 trunc from 0x0053b3 to 0x00835b (0x2fa9 bytes) 000326 1773235742.592378 MIRROR_SPLIT mirror 1 to 8 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000328 1773235742.748956 punch from 0x0048f3 to 0x00e066 (0x9774 bytes) 000329 1773235742.752064 trunc from 0x010f55 to 0x01efa0 (0xe04c bytes) 000330 1773235742.755295 read 0x0001f5 thru 0x00df5e (0xdd6a bytes) 000331 1773235742.760355 write 0x0216fb thru 0x0290c0 (0x79c6 bytes) 000333 1773235742.766039 mapwrite 0x039db9 thru 0x03ffff (0x6247 bytes) 000334 1773235742.821967 mapwrite 0x01b675 thru 0x028bd4 (0xd560 bytes) 000335 1773235742.846851 trunc from 0x0289c3 to 0x028ddb (0x419 bytes) 000337 1773235742.852040 punch from 0x033f6c to 0x03ffff (0xc094 bytes) 000338 1773235742.860129 MIRROR_RESYNC 8 mirrors 000339 1773235744.159035 mapwrite 0x00df45 thru 0x01d397 (0xf453 bytes) 000340 1773235744.250586 MIRROR_SPLIT mirror 3 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000341 1773235744.397061 write 0x03ffbd thru 0x03ffff (0x43 bytes) 000342 1773235744.404648 punch from 0x02306f to 0x029475 (0x6407 bytes) 000343 1773235744.410298 punch from 0x03ff2f to 0x03ffff (0xd1 bytes) 000344 1773235744.419989 mapread 0x01ebe7 thru 0x01ee3e (0x258 bytes) 000345 1773235744.425629 mapwrite 0x03978b thru 0x03ffff (0x6875 bytes) 000346 1773235744.464831 write 0x002adf thru 0x007b9a (0x50bc bytes) 000347 1773235744.480088 MIRROR_EXTEND to 8 mirrors 000348 1773235745.138707 mapread 0x0249a2 thru 0x02c6e2 (0x7d41 bytes) 000349 1773235745.183445 write 0x005683 thru 0x00d378 (0x7cf6 bytes) 000350 1773235745.244466 fallocate 0x006c5c thru 0x00a3c1 (0x3766 bytes) 000351 1773235745.248894 mapread 0x00aa43 thru 0x010f7a (0x6538 bytes) 000352 1773235745.250503 mapwrite 0x019150 thru 0x01ea60 (0x5911 bytes) 000353 1773235745.271891 read 0x03abdf thru 0x03ffff (0x5421 bytes) 000354 1773235745.278525 punch from 0x02af15 to 0x0333b1 (0x849d bytes) 000355 1773235745.283184 read 0x02eee5 thru 0x039909 (0xaa25 bytes) 000356 1773235745.292928 write 0x014c35 thru 0x01b07e (0x644a bytes) 000357 1773235745.294067 punch from 0x0365a7 to 0x037f1f (0x1979 bytes) 000359 1773235745.299568 write 0x0385d4 thru 0x03af6d (0x299a bytes) 000360 1773235745.300495 punch from 0x0025df to 0x00487e (0x22a0 bytes) 000361 1773235745.304809 read 0x005435 thru 0x0060fa (0xcc6 bytes) 000362 1773235745.305184 trunc from 0x03c9eb to 0x03ffff (0x3615 bytes) 000363 1773235745.308454 trunc from 0x02f0a8 to 0x033916 (0x486f bytes) 000364 1773235745.311384 write 0x0141c1 thru 0x0222a9 (0xe0e9 bytes) 000366 1773235745.312871 trunc from 0x02b6a8 to 0x0312d3 (0x5c2c bytes) 000367 1773235745.315868 trunc from 0x040000 to 0x00ed73 (0x3128c bytes) 000368 1773235745.336829 read 0x001a01 thru 0x00ed73 (0xd373 bytes) 000372 1773235745.350874 trunc from 0x00bda1 to 0x00ed73 (0x2fd3 bytes) 000374 1773235745.361600 write 0x02d16f thru 0x0353e1 (0x8273 bytes) 000375 1773235745.364081 punch from 0x001d7b to 0x008520 (0x67a6 bytes) 000376 1773235745.372996 punch from 0x0082ea to 0x00cfc1 (0x4cd8 bytes) 000377 1773235745.378322 mapread 0x023a21 thru 0x026c11 (0x31f1 bytes) 000378 1773235745.388078 read 0x00a3bf thru 0x00c7e3 (0x2425 bytes) 000379 1773235745.399934 trunc from 0x0353e2 to 0x01fef7 (0x154ea bytes) 000380 1773235745.417110 mapread 0x00fd81 thru 0x017a42 (0x7cc2 bytes) 000381 1773235745.417966 trunc from 0x0141d3 to 0x01cf08 (0x8d36 bytes) 000382 1773235745.424405 fallocate 0x03accb thru 0x03ffff (0x5335 bytes) 000383 1773235745.429374 mapwrite 0x021bb1 thru 0x0304c0 (0xe910 bytes) 000385 1773235745.525049 read 0x01a609 thru 0x01ed23 (0x471b bytes) 000386 1773235745.530361 trunc from 0x01047b to 0x015dc7 (0x594d bytes) 000387 1773235745.538478 MIRROR_SPLIT mirror 4 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000388 1773235745.713052 fallocate 0x019bdf thru 0x020ad5 (0x6ef7 bytes) 000389 1773235745.719300 mapread 0x031e37 thru 0x03b812 (0x99dc bytes) 000390 1773235745.733023 mapread 0x00ad3f thru 0x018752 (0xda14 bytes) 000391 1773235745.745194 punch from 0x023b67 to 0x025fdf (0x2479 bytes) 000392 1773235745.749281 mapwrite 0x036e6f thru 0x038aef (0x1c81 bytes) 000393 1773235745.760565 read 0x03e86c thru 0x03ffff (0x1794 bytes) 000394 1773235745.765932 trunc from 0x028df6 to 0x02c22c (0x3437 bytes) 000395 1773235745.771277 read 0x030303 thru 0x035910 (0x560e bytes) 000396 1773235745.772001 write 0x01b3ed thru 0x01db81 (0x2795 bytes) 000397 1773235745.781352 read 0x0288a7 thru 0x029f72 (0x16cc bytes) 000398 1773235745.784294 trunc from 0x01f85f to 0x01ffea (0x78c bytes) 000399 1773235745.790106 write 0x03c1b9 thru 0x03ffff (0x3e47 bytes) 000400 1773235745.792053 write 0x039443 thru 0x03ffff (0x6bbd bytes) 000401 1773235745.793236 punch from 0x013b35 to 0x014996 (0xe62 bytes) 000402 1773235745.798139 MIRROR_SPLIT mirror 6 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000403 1773235745.929236 MIRROR_SPLIT mirror 1 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000404 1773235746.061624 read 0x01ffb0 thru 0x022eea (0x2f3b bytes) 000405 1773235746.069194 trunc from 0x040000 to 0x010ef1 (0x2f10e bytes) 000406 1773235746.088897 punch from 0x00f599 to 0x010ef1 (0x1959 bytes) 000408 1773235746.100046 trunc from 0x0064ba to 0x010ef1 (0xaa38 bytes) 000409 1773235746.105050 punch from 0x003858 to 0x004c93 (0x143c bytes) 000411 1773235746.109366 fallocate 0x0115df thru 0x01e5fe (0xd020 bytes) 000412 1773235746.114119 read 0x0086db thru 0x014171 (0xba97 bytes) 000413 1773235746.121917 fallocate 0x02d6a4 thru 0x03794d (0xa2aa bytes) 000414 1773235746.126497 mapread 0x01c16f thru 0x01e5fe (0x2490 bytes) 000415 1773235746.129908 fallocate 0x01663c thru 0x01979e (0x3163 bytes) 000416 1773235746.135831 read 0x00d786 thru 0x012fbb (0x5836 bytes) 000417 1773235746.140864 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000418 1773235746.324161 write 0x0215bf thru 0x022418 (0xe5a bytes) 000419 1773235746.331300 punch from 0x005da1 to 0x010abd (0xad1d bytes) 000420 1773235746.339378 MIRROR_RESYNC 4 mirrors 000421 1773235746.861516 mapwrite 0x037ad4 thru 0x03ffff (0x852c bytes) 000422 1773235746.923394 fallocate 0x03cd47 thru 0x03ffff (0x32b9 bytes) 000423 1773235746.933032 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000424 1773235747.109812 read 0x03340f thru 0x03abd6 (0x77c8 bytes) 000425 1773235747.119969 write 0x01cfe1 thru 0x021b92 (0x4bb2 bytes) 000426 1773235747.129503 read 0x027ce2 thru 0x03517e (0xd49d bytes) 000427 1773235747.139510 mapread 0x028ac8 thru 0x03134d (0x8886 bytes) 000428 1773235747.140322 trunc from 0x01a87a to 0x01e191 (0x3918 bytes) 000429 1773235747.153295 trunc from 0x040000 to 0x019606 (0x269f9 bytes) 000430 1773235747.171118 fallocate 0x03adfb thru 0x03fc18 (0x4e1e bytes) 000431 1773235747.175393 trunc from 0x019607 to 0x02831f (0xed19 bytes) 000432 1773235747.185333 trunc from 0x020ff2 to 0x02831f (0x732e bytes) 000434 1773235747.190945 write 0x018e04 thru 0x01a531 (0x172e bytes) 000435 1773235747.194549 mapread 0x025e00 thru 0x02831f (0x2520 bytes) 000436 1773235747.217601 trunc from 0x028320 to 0x00b798 (0x1cb87 bytes) 000437 1773235747.243442 MIRROR_EXTEND to 4 mirrors 000439 1773235747.701029 MIRROR_RESYNC 4 mirrors 000440 1773235748.210027 write 0x00d0a8 thru 0x010762 (0x36bb bytes) 000441 1773235748.244362 mapwrite 0x01cbfe thru 0x02bb38 (0xef3b bytes) 000442 1773235748.299338 mapread 0x01353e thru 0x02329e (0xfd61 bytes) 000444 1773235748.307297 trunc from 0x02bb39 to 0x0153f2 (0x16746 bytes) 000445 1773235748.318186 write 0x0351a9 thru 0x036593 (0x13eb bytes) 000446 1773235748.319227 mapwrite 0x0257f7 thru 0x02aabc (0x52c6 bytes) 000447 1773235748.335057 punch from 0x032616 to 0x036593 (0x3f7e bytes) 000448 1773235748.338463 trunc from 0x036594 to 0x0247c4 (0x11dcf bytes) 000449 1773235748.351792 punch from 0x00a59a to 0x011515 (0x6f7c bytes) 000450 1773235748.357294 read 0x00f0e5 thru 0x01127f (0x219b bytes) 000451 1773235748.363716 write 0x03ed3d thru 0x03ffff (0x12c3 bytes) 000452 1773235748.364934 trunc from 0x040000 to 0x028eb9 (0x17146 bytes) 000453 1773235748.380661 read 0x026e0a thru 0x028eb9 (0x20b0 bytes) 000454 1773235748.389948 fallocate 0x02629d thru 0x031146 (0xaeaa bytes) 000455 1773235748.398236 fallocate 0x028156 thru 0x0378be (0xf769 bytes) 000456 1773235748.402369 mapwrite 0x02a1ab thru 0x02dad7 (0x392d bytes) 000457 1773235748.427029 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000458 1773235748.597450 punch from 0x002ffa to 0x011b9d (0xeba4 bytes) 000460 1773235748.603298 mapwrite 0x010de9 thru 0x011fe0 (0x11f8 bytes) 000461 1773235748.623316 trunc from 0x0378bf to 0x01a5f6 (0x1d2c8 bytes) 000462 1773235748.641392 write 0x01597f thru 0x0221da (0xc85c bytes) 000463 1773235748.649272 trunc from 0x0221db to 0x02563e (0x3464 bytes) 000464 1773235748.666097 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000465 1773235748.774902 trunc from 0x01b0f9 to 0x01fe97 (0x4d9f bytes) 000466 1773235748.777920 mapwrite 0x0125d5 thru 0x01a261 (0x7c8d bytes) 000467 1773235748.808099 mapwrite 0x023c02 thru 0x027fb8 (0x43b7 bytes) 000468 1773235748.840161 read 0x00f931 thru 0x01571a (0x5dea bytes) 000469 1773235748.840801 trunc from 0x027fb9 to 0x039965 (0x119ad bytes) 000470 1773235748.860849 fallocate 0x03641f thru 0x039ba7 (0x3789 bytes) 000471 1773235748.867939 fallocate 0x01fa96 thru 0x0241c5 (0x4730 bytes) 000472 1773235748.873780 write 0x03695e thru 0x03ffff (0x96a2 bytes) 000473 1773235748.885124 punch from 0x03e188 to 0x03ffff (0x1e78 bytes) 000474 1773235748.897630 punch from 0x023dbb to 0x0327c6 (0xea0c bytes) 000475 1773235748.907247 trunc from 0x0237fa to 0x023a77 (0x27e bytes) 000476 1773235748.911853 mapwrite 0x021d06 thru 0x0220ad (0x3a8 bytes) 000477 1773235748.927354 mapwrite 0x001d3b thru 0x00faff (0xddc5 bytes) 000478 1773235748.954623 MIRROR_EXTEND to 3 mirrors 000479 1773235749.437673 fallocate 0x00520b thru 0x01156b (0xc361 bytes) 000480 1773235749.457864 punch from 0x035e24 to 0x03ffff (0xa1dc bytes) 000482 1773235749.463278 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000483 1773235749.671815 write 0x01660e thru 0x023ff9 (0xd9ec bytes) 000485 1773235749.692827 read 0x005dbf thru 0x00f249 (0x948b bytes) 000486 1773235749.703126 punch from 0x00f355 to 0x010385 (0x1031 bytes) 000487 1773235749.707425 fallocate 0x019d63 thru 0x0277f8 (0xda96 bytes) 000488 1773235749.713219 mapwrite 0x0195be thru 0x0232af (0x9cf2 bytes) 000489 1773235749.731703 write 0x01061f thru 0x01665f (0x6041 bytes) 000490 1773235749.741314 write 0x01c3e3 thru 0x01da70 (0x168e bytes) 000491 1773235749.742254 write 0x0180ab thru 0x01a1b7 (0x210d bytes) 000493 1773235749.744416 punch from 0x03c126 to 0x03ffff (0x3eda bytes) 000494 1773235749.754976 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000495 1773235749.865134 mapwrite 0x03badc thru 0x03e97e (0x2ea3 bytes) 000496 1773235749.891358 trunc from 0x01a917 to 0x026c19 (0xc303 bytes) 000497 1773235749.894828 write 0x0128b7 thru 0x01a63b (0x7d85 bytes) 000498 1773235749.904720 punch from 0x02d272 to 0x03490d (0x769c bytes) 000499 1773235749.909951 trunc from 0x040000 to 0x029a82 (0x1657d bytes) 000500 1773235749.923010 write 0x000527 thru 0x004d48 (0x4822 bytes) 000501 1773235749.930003 trunc from 0x00fc40 to 0x012dc6 (0x3187 bytes) 000502 1773235749.947517 fallocate 0x00471d thru 0x01445e (0xfd42 bytes) 000503 1773235749.959332 MIRROR_EXTEND to 2 mirrors 000504 1773235750.444188 mapread 0x016f85 thru 0x018c12 (0x1c8e bytes) 000505 1773235750.467795 trunc from 0x01ea36 to 0x0212e1 (0x28ac bytes) 000506 1773235750.498182 read 0x028780 thru 0x029a82 (0x1303 bytes) 000507 1773235750.503038 read 0x0248ad thru 0x029616 (0x4d6a bytes) 000508 1773235750.511561 mapwrite 0x005ffc thru 0x0107e1 (0xa7e6 bytes) 000509 1773235750.533794 write 0x015508 thru 0x01b87c (0x6375 bytes) 000510 1773235750.542898 fallocate 0x02d017 thru 0x03a9ac (0xd996 bytes) 000511 1773235750.549032 trunc from 0x03a9ad to 0x026953 (0x14059 bytes) 000512 1773235750.563437 trunc from 0x026954 to 0x01da30 (0x8f23 bytes) 000513 1773235750.574685 trunc from 0x01da31 to 0x00800d (0x15a23 bytes) 000514 1773235750.589179 MIRROR_EXTEND to 3 mirrors 000515 1773235750.900348 write 0x01385a thru 0x022ca2 (0xf449 bytes) 000516 1773235750.929358 punch from 0x0091b4 to 0x0172f3 (0xe140 bytes) 000517 1773235750.949061 mapwrite 0x0229e8 thru 0x027522 (0x4b3b bytes) 000518 1773235751.015714 write 0x00ee48 thru 0x012058 (0x3211 bytes) 000519 1773235751.024238 read 0x01c179 thru 0x027522 (0xb3aa bytes) 000520 1773235751.029611 MIRROR_EXTEND to 4 mirrors 000522 1773235751.510457 mapread 0x00a445 thru 0x013adf (0x969b bytes) 000523 1773235751.541967 mapread 0x0174fc thru 0x01ce04 (0x5909 bytes) 000524 1773235751.559924 trunc from 0x027523 to 0x02b746 (0x4224 bytes) 000525 1773235751.636083 mapwrite 0x0106a9 thru 0x01f0b6 (0xea0e bytes) 000526 1773235751.690143 read 0x0011b7 thru 0x00b667 (0xa4b1 bytes) 000527 1773235751.697854 trunc from 0x006698 to 0x00f8d1 (0x923a bytes) 000528 1773235751.704439 mapwrite 0x032e15 thru 0x03ffff (0xd1eb bytes) 000529 1773235751.747889 mapread 0x02289a thru 0x02bbc4 (0x932b bytes) 000530 1773235751.752654 fallocate 0x03bc88 thru 0x03ffff (0x4378 bytes) 000531 1773235751.755715 trunc from 0x02cdcf to 0x03429a (0x74cc bytes) 000533 1773235751.761467 trunc from 0x040000 to 0x01a106 (0x25ef9 bytes) 000534 1773235751.775244 read 0x011e3f thru 0x014b51 (0x2d13 bytes) 000535 1773235751.775920 write 0x005a81 thru 0x009174 (0x36f4 bytes) 000536 1773235751.782138 mapread 0x00b294 thru 0x0130f4 (0x7e61 bytes) 000537 1773235751.789112 fallocate 0x017d41 thru 0x01efba (0x727a bytes) 000538 1773235751.796590 MIRROR_EXTEND to 5 mirrors 000539 1773235752.204964 trunc from 0x0039e4 to 0x00c0d4 (0x86f1 bytes) 000540 1773235752.224511 mapread 0x0186a8 thru 0x01a106 (0x1a5f bytes) 000541 1773235752.240473 MIRROR_SPLIT mirror 3 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000542 1773235752.367125 MIRROR_EXTEND to 5 mirrors 000543 1773235752.760887 mapread 0x015a2e thru 0x01a106 (0x46d9 bytes) 000544 1773235752.789550 write 0x02c05c thru 0x03a5e9 (0xe58e bytes) 000545 1773235752.816895 mapread 0x02ec6e thru 0x035b55 (0x6ee8 bytes) 000546 1773235752.817825 mapread 0x00c22d thru 0x013a39 (0x780d bytes) 000547 1773235752.825910 punch from 0x01723e to 0x01a3a8 (0x316b bytes) 000548 1773235752.831316 write 0x031655 thru 0x03fc4a (0xe5f6 bytes) 000549 1773235752.832874 read 0x03af22 thru 0x03fc4a (0x4d29 bytes) 000550 1773235752.833348 write 0x02ea08 thru 0x032102 (0x36fb bytes) 000551 1773235752.834397 write 0x03f092 thru 0x03ffff (0xf6e bytes) 000552 1773235752.835089 read 0x014364 thru 0x020994 (0xc631 bytes) 000553 1773235752.842455 trunc from 0x040000 to 0x00dc3b (0x323c4 bytes) 000554 1773235752.863695 write 0x03d940 thru 0x03ffff (0x26c0 bytes) 000556 1773235752.868561 trunc from 0x00fb58 to 0x011180 (0x1629 bytes) 000557 1773235752.873848 trunc from 0x037a37 to 0x03ffff (0x85c9 bytes) 000558 1773235752.890809 MIRROR_EXTEND to 6 mirrors 000559 1773235753.309728 write 0x03d78f thru 0x03ffff (0x2871 bytes) 000560 1773235753.336220 write 0x01b803 thru 0x028a44 (0xd242 bytes) 000561 1773235753.345109 trunc from 0x040000 to 0x01dc24 (0x223db bytes) 000562 1773235753.362004 punch from 0x0174bd to 0x01a2a1 (0x2de5 bytes) 000563 1773235753.365447 trunc from 0x010cdb to 0x01a762 (0x9a88 bytes) 000565 1773235753.368933 write 0x0395fe thru 0x03ffff (0x6a02 bytes) 000566 1773235753.370151 mapwrite 0x020576 thru 0x028070 (0x7afb bytes) 000568 1773235753.392183 punch from 0x005a3c to 0x00cea9 (0x746e bytes) 000569 1773235753.397257 trunc from 0x040000 to 0x01f9ac (0x20653 bytes) 000570 1773235753.411089 mapwrite 0x006816 thru 0x00b87a (0x5065 bytes) 000571 1773235753.440035 read 0x014931 thru 0x01f9ac (0xb07c bytes) 000574 1773235753.457326 mapwrite 0x010fbb thru 0x019d6f (0x8db5 bytes) 000575 1773235753.475294 mapread 0x015e09 thru 0x01e8eb (0x8ae3 bytes) 000576 1773235753.476117 read 0x00130a thru 0x00270b (0x1402 bytes) 000577 1773235753.476851 trunc from 0x01f9ad to 0x015174 (0xa838 bytes) 000578 1773235753.486189 trunc from 0x0146cc to 0x015174 (0xaa9 bytes) 000579 1773235753.490553 mapwrite 0x013f88 thru 0x01f0b1 (0xb12a bytes) 000580 1773235753.515464 trunc from 0x015ead to 0x01f0b1 (0x9205 bytes) 000581 1773235753.519501 trunc from 0x014c16 to 0x01f0b1 (0xa49c bytes) 000582 1773235753.525904 MIRROR_EXTEND to 7 mirrors 000583 1773235753.972058 punch from 0x01efdd to 0x01f0b1 (0xd5 bytes) 000584 1773235753.995042 mapwrite 0x011bb5 thru 0x0132b7 (0x1703 bytes) 000585 1773235754.017495 trunc from 0x01f0b2 to 0x01e533 (0xb7e bytes) 000586 1773235754.048173 punch from 0x00dc52 to 0x0190c6 (0xb475 bytes) 000587 1773235754.054315 trunc from 0x01e1a2 to 0x01e533 (0x392 bytes) 000588 1773235754.058532 write 0x001aee thru 0x011721 (0xfc34 bytes) 000589 1773235754.075908 MIRROR_SPLIT mirror 2 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000590 1773235754.245478 mapread 0x01de15 thru 0x01e533 (0x71f bytes) 000591 1773235754.254872 mapwrite 0x022797 thru 0x02b904 (0x916e bytes) 000592 1773235754.292051 write 0x03b488 thru 0x03ffff (0x4b78 bytes) 000593 1773235754.293273 fallocate 0x011c9d thru 0x02079c (0xeb00 bytes) 000594 1773235754.298168 punch from 0x00681e to 0x00cc57 (0x643a bytes) 000595 1773235754.302611 write 0x030c08 thru 0x0332fc (0x26f5 bytes) 000596 1773235754.309090 mapwrite 0x017865 thru 0x01de99 (0x6635 bytes) 000598 1773235754.327823 mapread 0x037fb3 thru 0x03ffff (0x804d bytes) 000599 1773235754.336077 mapwrite 0x004977 thru 0x011617 (0xcca1 bytes) 000600 1773235754.375630 mapwrite 0x01dee1 thru 0x024a05 (0x6b25 bytes) 000601 1773235754.393024 mapread 0x034e7d thru 0x03ad49 (0x5ecd bytes) 000602 1773235754.402295 MIRROR_EXTEND to 7 mirrors 000603 1773235754.850758 trunc from 0x00ddc0 to 0x01b357 (0xd598 bytes) 000604 1773235754.882255 MIRROR_EXTEND to 8 mirrors 000605 1773235755.291062 write 0x01be40 thru 0x02b5b6 (0xf777 bytes) 000606 1773235755.325805 mapwrite 0x010faf thru 0x01aea9 (0x9efb bytes) 000608 1773235755.354398 write 0x01714a thru 0x01b45f (0x4316 bytes) 000609 1773235755.356572 write 0x024f47 thru 0x02a363 (0x541d bytes) 000610 1773235755.357862 mapwrite 0x02c350 thru 0x036be2 (0xa893 bytes) 000611 1773235755.385364 mapread 0x01cada thru 0x020cb8 (0x41df bytes) 000612 1773235755.386167 mapread 0x008f75 thru 0x00f5d5 (0x6661 bytes) 000613 1773235755.392427 mapread 0x0018eb thru 0x01188c (0xffa2 bytes) 000614 1773235755.393323 punch from 0x03bd46 to 0x03e0e9 (0x23a4 bytes) 000615 1773235755.397947 mapread 0x02f45a thru 0x031182 (0x1d29 bytes) 000616 1773235755.399222 trunc from 0x02946a to 0x0294f6 (0x8d bytes) 000617 1773235755.404127 punch from 0x00f150 to 0x01290c (0x37bd bytes) 000618 1773235755.408630 mapwrite 0x02451c thru 0x02c988 (0x846d bytes) 000619 1773235755.431058 write 0x00fda8 thru 0x01972e (0x9987 bytes) 000620 1773235755.437416 fallocate 0x0143f8 thru 0x016ac9 (0x26d2 bytes) 000621 1773235755.446086 MIRROR_RESYNC 8 mirrors 000622 1773235756.636975 mapread 0x014848 thru 0x016955 (0x210e bytes) 000623 1773235756.655788 trunc from 0x01c580 to 0x01f7da (0x325b bytes) 000624 1773235756.689008 trunc from 0x040000 to 0x020d5b (0x1f2a4 bytes) 000625 1773235756.711214 mapwrite 0x0139f8 thru 0x017684 (0x3c8d bytes) 000626 1773235756.741278 punch from 0x016ce6 to 0x020bf5 (0x9f10 bytes) 000628 1773235756.746263 trunc from 0x00f4ac to 0x01c82c (0xd381 bytes) 000629 1773235756.750649 trunc from 0x013765 to 0x020839 (0xd0d5 bytes) 000630 1773235756.755129 write 0x017971 thru 0x02124c (0x98dc bytes) 000631 1773235756.763361 trunc from 0x02124d to 0x010e14 (0x10438 bytes) 000632 1773235756.778497 read 0x00516a thru 0x00982b (0x46c2 bytes) 000633 1773235756.785105 trunc from 0x00725a to 0x00ab3d (0x38e4 bytes) 000634 1773235756.790896 mapwrite 0x018ab1 thru 0x024af9 (0xc049 bytes) 000635 1773235756.846177 mapread 0x012914 thru 0x01589b (0x2f88 bytes) 000636 1773235756.847060 mapread 0x0019eb thru 0x00faea (0xe100 bytes) 000637 1773235756.852923 punch from 0x01afe1 to 0x024af9 (0x9b19 bytes) 000638 1773235756.858974 write 0x012b7c thru 0x02207f (0xf504 bytes) 000639 1773235756.867691 punch from 0x01906e to 0x01ba96 (0x2a29 bytes) 000640 1773235756.882471 mapwrite 0x00b2cf thru 0x016f62 (0xbc94 bytes) 000641 1773235756.909820 write 0x02cb3b thru 0x036396 (0x985c bytes) 000642 1773235756.911284 mapwrite 0x0304aa thru 0x035493 (0x4fea bytes) 000643 1773235756.930634 write 0x02d822 thru 0x03bbdb (0xe3ba bytes) 000644 1773235756.937052 MIRROR_EXTEND to 9 mirrors 000645 1773235757.439936 mapwrite 0x00638e thru 0x00b84f (0x54c2 bytes) 000646 1773235757.522460 mapwrite 0x013f71 thru 0x017f02 (0x3f92 bytes) 000647 1773235757.540326 mapwrite 0x00a44d thru 0x00c53b (0x20ef bytes) 000648 1773235757.552619 trunc from 0x001519 to 0x00dd49 (0xc831 bytes) 000649 1773235757.565595 mapwrite 0x038866 thru 0x03ffff (0x779a bytes) 000650 1773235757.621237 mapread 0x03a941 thru 0x03ffff (0x56bf bytes) 000651 1773235757.623000 mapwrite 0x03ce53 thru 0x03ffff (0x31ad bytes) 000652 1773235757.632651 trunc from 0x00dbf5 to 0x00f70e (0x1b1a bytes) 000653 1773235757.636716 fallocate 0x020bcd thru 0x02e2ec (0xd720 bytes) 000655 1773235757.640368 mapwrite 0x01bdb7 thru 0x0202c0 (0x450a bytes) 000656 1773235757.666687 mapwrite 0x03a6b5 thru 0x03df55 (0x38a1 bytes) 000657 1773235757.685350 trunc from 0x040000 to 0x033957 (0xc6a8 bytes) 000658 1773235757.702317 trunc from 0x01c62f to 0x01d78b (0x115d bytes) 000659 1773235757.706356 fallocate 0x03828e thru 0x038cf3 (0xa66 bytes) 000660 1773235757.709911 trunc from 0x038cf4 to 0x01028a (0x28a69 bytes) 000661 1773235757.725314 fallocate 0x037fea thru 0x038f97 (0xfae bytes) 000663 1773235757.734360 MIRROR_SPLIT mirror 8 to 8 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000664 1773235757.881227 write 0x0038e1 thru 0x007b0e (0x422e bytes) 000665 1773235757.891188 mapread 0x00bd0e thru 0x01028a (0x457d bytes) 000667 1773235757.901428 write 0x035f65 thru 0x03ffff (0xa09b bytes) 000668 1773235757.902838 mapwrite 0x01a676 thru 0x01bea8 (0x1833 bytes) 000669 1773235757.921902 punch from 0x02244b to 0x02a8f8 (0x84ae bytes) 000670 1773235757.926208 fallocate 0x010e45 thru 0x014f66 (0x4122 bytes) 000671 1773235757.930789 mapwrite 0x02d060 thru 0x033765 (0x6706 bytes) 000672 1773235757.946661 mapread 0x01f9ce thru 0x02161e (0x1c51 bytes) 000673 1773235757.947953 fallocate 0x01e317 thru 0x02371d (0x5407 bytes) 000674 1773235757.954455 trunc from 0x018976 to 0x022093 (0x971e bytes) 000675 1773235757.963557 mapread 0x010cb0 thru 0x014457 (0x37a8 bytes) 000676 1773235757.972137 punch from 0x013d45 to 0x01a5fd (0x68b9 bytes) 000677 1773235757.980944 MIRROR_EXTEND to 9 mirrors 000678 1773235758.585169 trunc from 0x030bb9 to 0x0395c6 (0x8a0e bytes) 000680 1773235758.610006 mapread 0x01f106 thru 0x020c54 (0x1b4f bytes) 000681 1773235758.620247 trunc from 0x040000 to 0x029e51 (0x161ae bytes) 000682 1773235758.644901 fallocate 0x0258dc thru 0x02b4be (0x5be3 bytes) 000683 1773235758.648107 read 0x01917b thru 0x01ba05 (0x288b bytes) 000684 1773235758.652308 mapread 0x008887 thru 0x01401b (0xb795 bytes) 000685 1773235758.668939 write 0x00b5d7 thru 0x0123b4 (0x6dde bytes) 000686 1773235758.669964 trunc from 0x0029b9 to 0x010478 (0xdac0 bytes) 000687 1773235758.685627 read 0x0047f8 thru 0x008f05 (0x470e bytes) 000688 1773235758.698375 punch from 0x01fd4b to 0x0240bd (0x4373 bytes) 000689 1773235758.707237 read 0x0173be thru 0x024dc7 (0xda0a bytes) 000690 1773235758.715359 write 0x009845 thru 0x014cc2 (0xb47e bytes) 000691 1773235758.717048 mapread 0x022ef7 thru 0x02b4be (0x85c8 bytes) 000692 1773235758.719171 trunc from 0x02b4bf to 0x02e8ff (0x3441 bytes) 000693 1773235758.742621 mapread 0x000145 thru 0x00d457 (0xd313 bytes) 000695 1773235758.747266 fallocate 0x0192f8 thru 0x01cbb7 (0x38c0 bytes) 000696 1773235758.752189 write 0x00aa77 thru 0x011530 (0x6aba bytes) 000697 1773235758.753153 write 0x013083 thru 0x01c522 (0x94a0 bytes) 000698 1773235758.763693 MIRROR_SPLIT mirror 7 to 8 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000699 1773235758.946711 punch from 0x00c7a4 to 0x01800c (0xb869 bytes) 000700 1773235758.954013 trunc from 0x02e900 to 0x015609 (0x192f6 bytes) 000701 1773235758.966876 trunc from 0x01560a to 0x0302f9 (0x1acf0 bytes) 000702 1773235758.978326 trunc from 0x02a5a0 to 0x02aafc (0x55d bytes) 000704 1773235758.983177 mapwrite 0x02bfc9 thru 0x034933 (0x896b bytes) 000705 1773235759.014286 trunc from 0x034934 to 0x004ca4 (0x2fc8f bytes) 000706 1773235759.025003 trunc from 0x0031bb to 0x004ca4 (0x1aea bytes) 000707 1773235759.031343 trunc from 0x002725 to 0x004ca4 (0x2580 bytes) 000708 1773235759.037272 trunc from 0x003252 to 0x004ca4 (0x1a53 bytes) 000709 1773235759.042536 read 0x001876 thru 0x004ca4 (0x342f bytes) 000711 1773235759.052396 fallocate 0x0028da thru 0x011e0b (0xf532 bytes) 000712 1773235759.059959 mapread 0x001709 thru 0x004ca4 (0x359c bytes) 000713 1773235759.063914 write 0x00a9ca thru 0x00ee20 (0x4457 bytes) 000714 1773235759.066765 trunc from 0x00ee21 to 0x017dc0 (0x8fa0 bytes) 000715 1773235759.094434 write 0x013bfd thru 0x01db5d (0x9f61 bytes) 000717 1773235759.103879 read 0x00c782 thru 0x0157b1 (0x9030 bytes) 000718 1773235759.104241 write 0x003fba thru 0x004cdc (0xd23 bytes) 000719 1773235759.106856 write 0x014a0f thru 0x01a79e (0x5d90 bytes) 000720 1773235759.108287 fallocate 0x01363f thru 0x019668 (0x602a bytes) 000722 1773235759.114586 trunc from 0x01db5e to 0x01ed34 (0x11d7 bytes) 000723 1773235759.131307 mapread 0x0061de thru 0x014abb (0xe8de bytes) 000724 1773235759.144906 mapwrite 0x00428f thru 0x013ff1 (0xfd63 bytes) 000725 1773235759.167843 trunc from 0x005aca to 0x00f9d8 (0x9f0f bytes) 000726 1773235759.173121 trunc from 0x01ed35 to 0x0305b7 (0x11883 bytes) 000727 1773235759.187067 trunc from 0x0305b8 to 0x014294 (0x1c323 bytes) 000728 1773235759.199052 mapread 0x0076cb thru 0x00923c (0x1b72 bytes) 000731 1773235759.212522 trunc from 0x014295 to 0x012337 (0x1f5d bytes) 000732 1773235759.229370 read 0x00f6c4 thru 0x012337 (0x2c74 bytes) 000733 1773235759.234501 trunc from 0x00cb57 to 0x012337 (0x57e1 bytes) 000734 1773235759.237824 trunc from 0x012338 to 0x0168e5 (0x45ae bytes) 000735 1773235759.246844 trunc from 0x0168e6 to 0x037c3c (0x21357 bytes) 000736 1773235759.261536 mapread 0x00391e thru 0x00d5a4 (0x9c87 bytes) 000737 1773235759.279014 mapread 0x00bb82 thru 0x00c849 (0xcc8 bytes) 000738 1773235759.279776 read 0x0054ba thru 0x013560 (0xe0a7 bytes) 000739 1773235759.286438 write 0x02d0ad thru 0x0391f8 (0xc14c bytes) 000740 1773235759.287349 mapread 0x033127 thru 0x0391f8 (0x60d2 bytes) 000741 1773235759.288094 trunc from 0x0391f9 to 0x008291 (0x30f67 bytes) 000742 1773235759.298791 punch from 0x005d44 to 0x008291 (0x254e bytes) 000743 1773235759.302772 trunc from 0x00262a to 0x008291 (0x5c68 bytes) 000744 1773235759.306074 mapwrite 0x020ea5 thru 0x022b5a (0x1cb6 bytes) 000745 1773235759.337604 write 0x032015 thru 0x039083 (0x706f bytes) 000746 1773235759.338900 fallocate 0x023d64 thru 0x0285ee (0x488b bytes) 000747 1773235759.343253 trunc from 0x01e45d to 0x02667d (0x8221 bytes) 000748 1773235759.350004 MIRROR_RESYNC 8 mirrors 000749 1773235760.158667 MIRROR_SPLIT mirror 3 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000750 1773235760.476863 write 0x0121bb thru 0x01f758 (0xd59e bytes) 000751 1773235760.508839 fallocate 0x00a69b thru 0x0154d1 (0xae37 bytes) 000752 1773235760.513965 mapwrite 0x01f5db thru 0x0225ec (0x3012 bytes) 000753 1773235760.534787 trunc from 0x01db93 to 0x029d2d (0xc19b bytes) 000754 1773235760.540970 fallocate 0x0149c4 thru 0x01f011 (0xa64e bytes) 000755 1773235760.545797 punch from 0x02ba27 to 0x039083 (0xd65d bytes) 000756 1773235760.551733 trunc from 0x039084 to 0x017754 (0x2192f bytes) 000757 1773235760.579723 fallocate 0x03ce6a thru 0x03ffff (0x3196 bytes) 000758 1773235760.583315 read 0x03236f thru 0x03c371 (0xa003 bytes) 000759 1773235760.588690 punch from 0x00a540 to 0x01539c (0xae5d bytes) 000760 1773235760.592859 mapwrite 0x019426 thru 0x01fe3c (0x6a17 bytes) 000762 1773235760.612060 MIRROR_EXTEND to 8 mirrors 000763 1773235760.985670 mapread 0x0292f9 thru 0x02ead6 (0x57de bytes) 000764 1773235761.023154 read 0x029020 thru 0x02e16e (0x514f bytes) 000765 1773235761.031128 write 0x008de3 thru 0x010e30 (0x804e bytes) 000766 1773235761.077818 fallocate 0x023bb7 thru 0x02c060 (0x84aa bytes) 000767 1773235761.082103 read 0x01393e thru 0x013a65 (0x128 bytes) 000768 1773235761.089341 trunc from 0x03c07a to 0x03f3f1 (0x3378 bytes) 000769 1773235761.095139 fallocate 0x025add thru 0x03078a (0xacae bytes) 000770 1773235761.098369 mapwrite 0x03e2e6 thru 0x03ffff (0x1d1a bytes) 000771 1773235761.126392 trunc from 0x040000 to 0x02b2c8 (0x14d37 bytes) 000773 1773235761.155865 MIRROR_EXTEND to 9 mirrors 000774 1773235761.538703 MIRROR_SPLIT mirror 7 to 8 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000775 1773235761.735517 write 0x00da7f thru 0x01c161 (0xe6e3 bytes) 000776 1773235761.780522 read 0x01e429 thru 0x0273f9 (0x8fd1 bytes) 000777 1773235761.789675 write 0x026afb thru 0x031157 (0xa65d bytes) 000778 1773235761.791109 mapread 0x015f21 thru 0x019b12 (0x3bf2 bytes) 000779 1773235761.791708 punch from 0x005558 to 0x012699 (0xd142 bytes) 000780 1773235761.805954 punch from 0x01945b to 0x0246b4 (0xb25a bytes) 000781 1773235761.809605 mapwrite 0x0257ca thru 0x02bab1 (0x62e8 bytes) 000782 1773235761.832556 trunc from 0x031158 to 0x02c513 (0x4c44 bytes) 000783 1773235761.874602 write 0x0288f0 thru 0x037d30 (0xf441 bytes) 000784 1773235761.876098 fallocate 0x03fe03 thru 0x03ffff (0x1fd bytes) 000786 1773235761.881448 punch from 0x021c2c to 0x02d71f (0xbaf4 bytes) 000787 1773235761.903708 mapread 0x02d5af thru 0x037d30 (0xa782 bytes) 000788 1773235761.911421 write 0x0135e8 thru 0x01e272 (0xac8b bytes) 000789 1773235761.919417 trunc from 0x037d31 to 0x01a626 (0x1d70a bytes) 000790 1773235761.935434 trunc from 0x01a627 to 0x03e09f (0x23a79 bytes) 000791 1773235761.946480 punch from 0x018384 to 0x02333e (0xafbb bytes) 000792 1773235761.958469 write 0x01e353 thru 0x0229d8 (0x4686 bytes) 000793 1773235761.963937 fallocate 0x022dd6 thru 0x02a86e (0x7a99 bytes) 000794 1773235761.967406 write 0x03a3a7 thru 0x03a5f4 (0x24e bytes) 000795 1773235761.973598 trunc from 0x03e0a0 to 0x021aa7 (0x1c5f8 bytes) 000796 1773235761.986636 write 0x03cb76 thru 0x03ffff (0x348a bytes) 000797 1773235761.990716 MIRROR_EXTEND to 9 mirrors 000799 1773235762.487803 write 0x0381a2 thru 0x039219 (0x1078 bytes) 000800 1773235762.524229 fallocate 0x0061d8 thru 0x00cb3a (0x6963 bytes) 000801 1773235762.528142 read 0x0254ba thru 0x02f7a5 (0xa2ec bytes) 000803 1773235762.535678 read 0x03a1d6 thru 0x03ffff (0x5e2a bytes) 000804 1773235762.543316 read 0x002a2d thru 0x0110ae (0xe682 bytes) 000805 1773235762.543996 trunc from 0x010bff to 0x01c4f8 (0xb8fa bytes) 000806 1773235762.548496 read 0x029111 thru 0x02dd3c (0x4c2c bytes) 000807 1773235762.549062 mapread 0x00ebfa thru 0x016506 (0x790d bytes) 000808 1773235762.562605 write 0x02a382 thru 0x02b5db (0x125a bytes) 000809 1773235762.565130 mapread 0x02b07d thru 0x02fe54 (0x4dd8 bytes) 000810 1773235762.566859 trunc from 0x010784 to 0x01994a (0x91c7 bytes) 000811 1773235762.570711 punch from 0x0353ed to 0x03ffff (0xac13 bytes) 000812 1773235762.585371 fallocate 0x022e7e thru 0x02c642 (0x97c5 bytes) 000813 1773235762.589693 punch from 0x011121 to 0x014016 (0x2ef6 bytes) 000815 1773235762.597038 fallocate 0x012058 thru 0x01f231 (0xd1da bytes) 000816 1773235762.601536 trunc from 0x00d530 to 0x010458 (0x2f29 bytes) 000817 1773235762.605253 mapread 0x02a2c9 thru 0x02d620 (0x3358 bytes) 000818 1773235762.606126 trunc from 0x03ba0f to 0x03cef9 (0x14eb bytes) 000819 1773235762.609487 mapread 0x00fb5a thru 0x01d55e (0xda05 bytes) 000820 1773235762.655199 mapread 0x023a6c thru 0x02f6a5 (0xbc3a bytes) 000821 1773235762.656025 trunc from 0x01e55b to 0x01fe83 (0x1929 bytes) 000823 1773235762.659504 trunc from 0x009d0b to 0x00f43a (0x5730 bytes) 000824 1773235762.663855 mapwrite 0x03ad90 thru 0x03bd68 (0xfd9 bytes) 000825 1773235762.677993 punch from 0x01bba0 to 0x025ca9 (0xa10a bytes) 000826 1773235762.681327 trunc from 0x0041fb to 0x008ffc (0x4e02 bytes) 000827 1773235762.685464 fallocate 0x01f9b5 thru 0x02832e (0x897a bytes) 000828 1773235762.689021 write 0x008e7d thru 0x016af0 (0xdc74 bytes) 000829 1773235762.693308 read 0x0173e2 thru 0x01e230 (0x6e4f bytes) 000830 1773235762.700575 mapwrite 0x03dd71 thru 0x03ffff (0x228f bytes) 000831 1773235762.723710 trunc from 0x040000 to 0x004922 (0x3b6dd bytes) 000832 1773235762.747551 MIRROR_EXTEND to 10 mirrors 000833 1773235763.075074 mapwrite 0x00c1cd thru 0x01052e (0x4362 bytes) 000834 1773235763.170333 MIRROR_RESYNC 10 mirrors 000835 1773235764.076824 MIRROR_RESYNC 10 mirrors 000836 1773235764.307149 write 0x03e124 thru 0x03ffff (0x1edc bytes) 000837 1773235764.336176 MIRROR_EXTEND to 11 mirrors 000838 1773235764.786799 fallocate 0x033e31 thru 0x03856b (0x473b bytes) 000839 1773235764.823680 mapwrite 0x02f303 thru 0x034c3e (0x593c bytes) 000840 1773235764.851219 fallocate 0x02e93f thru 0x03c590 (0xdc52 bytes) 000841 1773235764.855817 punch from 0x0068ea to 0x014dc6 (0xe4dd bytes) 000842 1773235764.859608 mapwrite 0x031b5f thru 0x0353d7 (0x3879 bytes) 000843 1773235764.872400 mapread 0x00322f thru 0x00c5f3 (0x93c5 bytes) 000844 1773235764.879463 mapwrite 0x00ce36 thru 0x00ffd5 (0x31a0 bytes) 000845 1773235764.893052 trunc from 0x040000 to 0x006756 (0x398a9 bytes) 000846 1773235764.910679 mapread 0x004f6f thru 0x006756 (0x17e8 bytes) 000847 1773235764.911473 read 0x0031a8 thru 0x006756 (0x35af bytes) 000848 1773235764.911920 fallocate 0x01119a thru 0x018fc4 (0x7e2b bytes) 000849 1773235764.916616 trunc from 0x018fc5 to 0x033b48 (0x1ab84 bytes) 000850 1773235764.927251 mapread 0x0254f0 thru 0x029302 (0x3e13 bytes) 000851 1773235764.934732 mapwrite 0x00e1be thru 0x011000 (0x2e43 bytes) 000852 1773235764.952444 write 0x00c642 thru 0x019d38 (0xd6f7 bytes) 000853 1773235764.957361 mapread 0x006277 thru 0x00a114 (0x3e9e bytes) 000854 1773235764.958127 mapwrite 0x038a54 thru 0x03cb08 (0x40b5 bytes) 000856 1773235764.985615 punch from 0x01f1dc to 0x0211ac (0x1fd1 bytes) 000857 1773235764.989404 write 0x01a328 thru 0x02730c (0xcfe5 bytes) 000858 1773235764.995869 punch from 0x00aca5 to 0x00fafe (0x4e5a bytes) 000859 1773235765.000863 punch from 0x038dd1 to 0x03cb08 (0x3d38 bytes) 000861 1773235765.006665 punch from 0x01eeb3 to 0x029096 (0xa1e4 bytes) 000862 1773235765.015894 mapwrite 0x014acd thru 0x01c0d5 (0x7609 bytes) 000863 1773235765.027027 read 0x002344 thru 0x010f50 (0xec0d bytes) 000864 1773235765.034241 read 0x0213f9 thru 0x02946c (0x8074 bytes) 000865 1773235765.034990 fallocate 0x001f8a thru 0x00b05d (0x90d4 bytes) 000866 1773235765.039124 read 0x00000b thru 0x004163 (0x4159 bytes) 000867 1773235765.040992 mapread 0x02fdd1 thru 0x038881 (0x8ab1 bytes) 000868 1773235765.042331 mapwrite 0x0231b0 thru 0x02e990 (0xb7e1 bytes) 000869 1773235765.057897 read 0x012272 thru 0x019f55 (0x7ce4 bytes) 000870 1773235765.058423 fallocate 0x01f2e2 thru 0x02b5c5 (0xc2e4 bytes) 000871 1773235765.069641 MIRROR_RESYNC 11 mirrors 000872 1773235766.915007 mapread 0x034fbc thru 0x03cb08 (0x7b4d bytes) 000873 1773235766.935345 read 0x011e8a thru 0x0146d0 (0x2847 bytes) 000874 1773235766.944547 mapwrite 0x025ac9 thru 0x032ac1 (0xcff9 bytes) 000875 1773235767.049627 read 0x02790c thru 0x02d948 (0x603d bytes) 000876 1773235767.050267 trunc from 0x03cb09 to 0x0360db (0x6a2d bytes) 000877 1773235767.067657 trunc from 0x0360dc to 0x0003a5 (0x35d36 bytes) 000878 1773235767.083465 write 0x021a43 thru 0x030919 (0xeed7 bytes) 000879 1773235767.085968 mapread 0x01d240 thru 0x0267ff (0x95c0 bytes) 000880 1773235767.096466 mapread 0x012188 thru 0x01969b (0x7514 bytes) 000881 1773235767.097261 trunc from 0x01497e to 0x01b2f4 (0x6977 bytes) 000882 1773235767.100669 fallocate 0x003974 thru 0x00c2d3 (0x8960 bytes) 000883 1773235767.108124 MIRROR_SPLIT mirror 1 to 10 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000884 1773235767.285212 punch from 0x00886e to 0x00fecf (0x7662 bytes) 000885 1773235767.290984 trunc from 0x020809 to 0x02ae9d (0xa695 bytes) 000886 1773235767.296870 read 0x0116d2 thru 0x0129ec (0x131b bytes) 000887 1773235767.301313 mapwrite 0x02a616 thru 0x02ff9c (0x5987 bytes) 000888 1773235767.329278 trunc from 0x03091a to 0x00457c (0x2c39d bytes) 000889 1773235767.346587 write 0x0094ed thru 0x00a376 (0xe8a bytes) 000890 1773235767.347419 mapwrite 0x0029fe thru 0x008afd (0x6100 bytes) 000891 1773235767.366029 read 0x001b5d thru 0x008538 (0x69dc bytes) 000892 1773235767.366940 mapread 0x004ed9 thru 0x00a376 (0x549e bytes) 000893 1773235767.367708 trunc from 0x0089a5 to 0x00a376 (0x19d2 bytes) 000894 1773235767.373670 mapread 0x001307 thru 0x00a376 (0x9070 bytes) 000895 1773235767.383156 mapwrite 0x02628f thru 0x028ccb (0x2a3d bytes) 000896 1773235767.416314 mapwrite 0x0060fb thru 0x00f0e1 (0x8fe7 bytes) 000897 1773235767.439292 MIRROR_SPLIT mirror 1 to 9 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000898 1773235767.575998 mapread 0x0006da thru 0x003be0 (0x3507 bytes) 000899 1773235767.592792 read 0x00374e thru 0x0118aa (0xe15d bytes) 000900 1773235767.603909 MIRROR_RESYNC 9 mirrors 000902 1773235768.776913 mapwrite 0x013a16 thru 0x01f112 (0xb6fd bytes) 000904 1773235768.868147 trunc from 0x028ccc to 0x03b951 (0x12c86 bytes) 000905 1773235768.885807 read 0x032a38 thru 0x03b951 (0x8f1a bytes) 000906 1773235768.890941 trunc from 0x01c1b7 to 0x01c75c (0x5a6 bytes) 000907 1773235768.893978 trunc from 0x01305c to 0x022ae6 (0xfa8b bytes) 000908 1773235768.897401 punch from 0x018120 to 0x023b2d (0xba0e bytes) 000909 1773235768.900183 trunc from 0x03b952 to 0x027bbd (0x13d94 bytes) 000910 1773235768.914654 punch from 0x00acd0 to 0x01579d (0xaace bytes) 000911 1773235768.918435 trunc from 0x027bbe to 0x00ceec (0x1acd1 bytes) 000912 1773235768.927105 trunc from 0x009486 to 0x00ceec (0x3a67 bytes) 000914 1773235768.933036 punch from 0x00ba7b to 0x00ceec (0x1472 bytes) 000915 1773235768.936388 trunc from 0x00ceed to 0x03eb33 (0x31c47 bytes) 000916 1773235768.946655 trunc from 0x03eb34 to 0x022af3 (0x1c040 bytes) 000917 1773235768.956012 trunc from 0x022af4 to 0x00a081 (0x18a72 bytes) 000919 1773235768.966623 write 0x00b47e thru 0x00f29a (0x3e1d bytes) 000920 1773235768.967841 read 0x008718 thru 0x00f29a (0x6b83 bytes) 000921 1773235768.972056 trunc from 0x00f29b to 0x00bf7a (0x3320 bytes) 000922 1773235768.983652 write 0x01f7b8 thru 0x027a00 (0x8249 bytes) 000923 1773235768.985228 punch from 0x023773 to 0x027a00 (0x428e bytes) 000924 1773235768.998281 mapwrite 0x0300f8 thru 0x03206e (0x1f77 bytes) 000925 1773235769.038976 write 0x01ac7d thru 0x021859 (0x6bdd bytes) 000927 1773235769.041054 write 0x02a4bd thru 0x02a811 (0x355 bytes) 000928 1773235769.043429 trunc from 0x03206f to 0x00a5b7 (0x27ab7 bytes) 000929 1773235769.054811 trunc from 0x00a5b8 to 0x000823 (0x9d94 bytes) 000930 1773235769.071879 fallocate 0x03cb10 thru 0x03ffff (0x34f0 bytes) 000931 1773235769.078862 fallocate 0x0145cc thru 0x015553 (0xf88 bytes) 000932 1773235769.083324 trunc from 0x000824 to 0x01365c (0x12e39 bytes) 000933 1773235769.098384 write 0x0021c9 thru 0x00b1b9 (0x8ff1 bytes) 000934 1773235769.111612 read 0x00a80e thru 0x00c813 (0x2006 bytes) 000935 1773235769.125003 MIRROR_EXTEND to 10 mirrors 000936 1773235769.548701 mapread 0x00f84e thru 0x01365c (0x3e0f bytes) 000937 1773235769.582916 trunc from 0x01365d to 0x002068 (0x115f4 bytes) 000938 1773235769.632244 read 0x001514 thru 0x002068 (0xb55 bytes) 000939 1773235769.636444 read 0x0011c8 thru 0x002068 (0xea1 bytes) 000940 1773235769.637036 mapread 0x00182e thru 0x002068 (0x83b bytes) 000941 1773235769.638012 trunc from 0x001642 to 0x002068 (0xa27 bytes) 000942 1773235769.644890 mapread 0x000d1b thru 0x002068 (0x134e bytes) 000944 1773235769.649867 write 0x02375a thru 0x02b92c (0x81d3 bytes) 000945 1773235769.651267 mapread 0x009e7a thru 0x00b69c (0x1823 bytes) 000947 1773235769.658049 trunc from 0x01f9f8 to 0x0251f4 (0x57fd bytes) 000948 1773235769.667948 mapwrite 0x00032e thru 0x00f3d4 (0xf0a7 bytes) 000949 1773235769.683118 read 0x02344f thru 0x0246b3 (0x1265 bytes) 000950 1773235769.691164 fallocate 0x003a6f thru 0x007c95 (0x4227 bytes) 000951 1773235769.693993 trunc from 0x023fa8 to 0x025a4e (0x1aa7 bytes) 000954 1773235769.700149 write 0x02cc17 thru 0x034f73 (0x835d bytes) 000955 1773235769.701181 trunc from 0x014591 to 0x01d64e (0x90be bytes) 000956 1773235769.705122 fallocate 0x026358 thru 0x02e7d5 (0x847e bytes) 000957 1773235769.709791 read 0x0145e0 thru 0x0204e1 (0xbf02 bytes) 000958 1773235769.717154 punch from 0x0330f2 to 0x034f73 (0x1e82 bytes) 000959 1773235769.729082 punch from 0x00111e to 0x010623 (0xf506 bytes) 000960 1773235769.733577 fallocate 0x037bcd thru 0x03ffff (0x8433 bytes) 000961 1773235769.737569 punch from 0x0110d7 to 0x0136d8 (0x2602 bytes) 000962 1773235769.742849 write 0x01633a thru 0x01ff23 (0x9bea bytes) 000964 1773235769.744262 trunc from 0x01d257 to 0x01edc7 (0x1b71 bytes) 000965 1773235769.760685 MIRROR_EXTEND to 11 mirrors 000968 1773235770.206044 write 0x03e565 thru 0x03ffff (0x1a9b bytes) 000970 1773235770.241142 mapread 0x02fb05 thru 0x033025 (0x3521 bytes) 000971 1773235770.255684 fallocate 0x03c6ce thru 0x03ffff (0x3932 bytes) 000972 1773235770.260367 punch from 0x00d3c9 to 0x018b3c (0xb774 bytes) 000973 1773235770.267107 trunc from 0x040000 to 0x021aa0 (0x1e55f bytes) 000974 1773235770.307007 trunc from 0x015c71 to 0x01656f (0x8ff bytes) 000975 1773235770.312329 fallocate 0x0023b6 thru 0x00c06e (0x9cb9 bytes) 000976 1773235770.318712 mapread 0x001add thru 0x006528 (0x4a4c bytes) 000978 1773235770.337145 MIRROR_RESYNC 11 mirrors 000979 1773235771.419095 mapwrite 0x01e050 thru 0x0274a2 (0x9453 bytes) 000981 1773235771.476169 punch from 0x012aa5 to 0x02100d (0xe569 bytes) 000982 1773235771.482618 punch from 0x00a0ca to 0x00d225 (0x315c bytes) 000983 1773235771.486229 read 0x02713e thru 0x0274a2 (0x365 bytes) 000984 1773235771.486929 mapread 0x00ae44 thru 0x00ca35 (0x1bf2 bytes) 000985 1773235771.505406 write 0x03ea3d thru 0x03ec13 (0x1d7 bytes) 000986 1773235771.506342 mapwrite 0x003485 thru 0x00763e (0x41ba bytes) 000987 1773235771.531981 mapread 0x003847 thru 0x00e6a2 (0xae5c bytes) 000988 1773235771.532990 punch from 0x03c04e to 0x03ec13 (0x2bc6 bytes) 000989 1773235771.539161 write 0x01fc40 thru 0x02ef04 (0xf2c5 bytes) 000990 1773235771.550720 write 0x0103d3 thru 0x0172c3 (0x6ef1 bytes) 000991 1773235771.556047 trunc from 0x03ec14 to 0x0218ba (0x1d359 bytes) 000992 1773235771.582357 mapread 0x004f1b thru 0x008ae9 (0x3bcf bytes) 000993 1773235771.585235 mapread 0x00df3f thru 0x01a384 (0xc446 bytes) 000994 1773235771.589851 MIRROR_SPLIT mirror 6 to 10 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x2ef:0x0]: Invalid argument (22) 000995 1773235771.763017 mapread 0x011a92 thru 0x02087b (0xedea bytes) 000996 1773235771.787774 punch from 0x01fb95 to 0x0218ba (0x1d26 bytes) 000997 1773235771.794633 trunc from 0x012621 to 0x01a016 (0x79f6 bytes) 000998 1773235771.801791 MIRROR_RESYNC 10 mirrors 000999 1773235773.551830 mapwrite 0x02015b thru 0x02ce27 (0xcccd bytes) 001000 1773235773.622053 read 0x008ecd thru 0x00ceca (0x3ffe bytes) All operations completed A-OK! PASS 70a (54s) == sanity-flr test 71: check mirror extend parent fid ==== 09:29:36 (1773235776) striped dir -i1 -c2 -H crush /mnt/lustre/d71.sanity-flr ** lfs fid2path /mnt/lustre [0x240000404:0x334:0x0] /mnt/lustre/d71.sanity-flr/f71.sanity-flr PASS 71 (6s) == sanity-flr test 200a: stress test ===================== 09:29:42 (1773235782) Starting client: oleg338-client.virtnet: -o user_xattr,flock 192.168.203.138@tcp:/lustre /mnt/lustre2 Starting client: oleg338-client.virtnet: -o user_xattr,flock 192.168.203.138@tcp:/lustre /mnt/lustre3 fail_loc=0x1A03 fail_loc=0x1A03 Extending file size to 1028544 .. lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8405408 .. Extending file size to 9000544 .. done Extending file size to 9168992 .. lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..Extending file size to 8680704 .. failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..Extending file size to 9264800 .. failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed Extending file size to 8918272 .. lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed Waiting 59830 59831 59833 59835 59837 192.168.203.138@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg338-client.virtnet /mnt/lustre2 (opts:) 192.168.203.138@tcp:/lustre /mnt/lustre3 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg338-client.virtnet /mnt/lustre3 (opts:) 3 PASS 200a (68s) == sanity-flr test 200b: racing IO, mirror extend and resync ========================================================== 09:30:50 (1773235850) Starting client: oleg338-client.virtnet: -o user_xattr,flock 192.168.203.138@tcp:/lustre /mnt/lustre2 Starting client: oleg338-client.virtnet: -o user_xattr,flock 192.168.203.138@tcp:/lustre /mnt/lustre3 fail_loc=0x1423 resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 1108768 .. Extending file size to 2472832 .. done Extending file size to 3225952 .. Extending file size to 3630304 .. resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed Extending file size to 9129440 .. Extending file size to 6928768 .. resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8404672 .. done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed Extending file size to 8582656 .. lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done Extending file size to 9343744 .. resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8652544 .. done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done Extending file size to 8945824 .. resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed Extending file size to 9042272 .. lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done lock to resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200b.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200b.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..Waiting 63027 63028 63029 63030 63034 done 192.168.203.138@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg338-client.virtnet /mnt/lustre2 (opts:) 192.168.203.138@tcp:/lustre /mnt/lustre3 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg338-client.virtnet /mnt/lustre3 (opts:) 3 PASS 200b (66s) == sanity-flr test 200c: layout change racing with open: LOVEA changes ========================================================== 09:31:56 (1773235916) Starting client: oleg338-client.virtnet: -o user_xattr,flock 192.168.203.138@tcp:/lustre /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2472 1268832 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 2428 1268876 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 22040 3584980 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 15940 3591080 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 37980 7176060 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 2472 1268832 1% /mnt/lustre2[MDT:0] lustre-MDT0001_UUID 1414116 2428 1268876 1% /mnt/lustre2[MDT:1] lustre-OST0000_UUID 3833116 22040 3584980 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3833116 15940 3591080 1% /mnt/lustre2[OST:1] filesystem_summary: 7666232 37980 7176060 1% /mnt/lustre2 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0466386 s, 45.0 MB/s fail_loc=0x80000175 fail_val=10 mirror extend mirror extend done fail_loc=0 fail_val=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0676033 s, 31.0 MB/s dd completed Chunks to be verified in /mnt/lustre/f200c.sanity-flr: [0, 0xffffffffffffffff) [1] 1 lfs verify: '/mnt/lustre/f200c.sanity-flr' chunk [0, 0xffffffffffffffff) is only valid in mirror 1: skipped 192.168.203.138@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg338-client.virtnet /mnt/lustre2 (opts:) PASS 200c (7s) SKIP: sanity-flr test_201 skipping ALWAYS excluded test 201 == sanity-flr test 202: lfs setstripe --add-component wide striping ========================================================== 09:32:04 (1773235924) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0527797 s, 39.7 MB/s PASS 202 (3s) == sanity-flr test 203: mirror file preserve mirror ID === 09:32:06 (1773235926) lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x60:0x0]: Invalid argument (22) /mnt/lustre/f203.sanity-flr lcm_layout_gen: 3 lcm_mirror_count: 1 lcm_entry_count: 1 lcme_id: 131074 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x8c:0x0] } /mnt/lustre/f203.sanity-flr lcm_layout_gen: 4 lcm_mirror_count: 2 lcm_entry_count: 2 lcme_id: 131074 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x8c:0x0] } lcme_id: 196609 lcme_mirror_id: 3 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x8d:0x0] } - 1: { l_ost_idx: 0, l_fid: [0x280000401:0xa9:0x0] } PASS 203 (3s) == sanity-flr test 204a: FLR write/stale/resync tests with self-extending mirror ========================================================== 09:32:09 (1773235929) striped dir -i0 -c2 -H crush2 /mnt/lustre/d204a.sanity-flr 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0382236 s, 54.9 MB/s /mnt/lustre/d204a.sanity-flr/f204a.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x409:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x40a:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 1048576 lcme_extent.e_end: 68157440 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x40b:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 68157440 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 PASS 204a (3s) == sanity-flr test 204b: FLR write/stale/resync tests with self-extending primary ========================================================== 09:32:12 (1773235932) striped dir -i0 -c2 -H all_char /mnt/lustre/d204b.sanity-flr 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0424068 s, 49.5 MB/s /mnt/lustre/d204b.sanity-flr/f204b.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x40a:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: 68157440 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x40b:0x0] } lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 68157440 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x40c:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 /mnt/lustre/d204b.sanity-flr/f204b.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x40a:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: 68157440 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000400:0x40b:0x0] } lcme_id: 65539 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 68157440 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x40c:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000400:0x40d:0x0] } PASS 204b (3s) == sanity-flr test 204c: FLR write/stale/resync test with component removal ========================================================== 09:32:15 (1773235935) striped dir -i0 -c2 -H all_char /mnt/lustre/d204c.sanity-flr 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0321049 s, 65.3 MB/s /mnt/lustre/d204c.sanity-flr/f204c.sanity-flr lcm_layout_gen: 7 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x8e:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xaa:0x0] } lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 /mnt/lustre/d204c.sanity-flr/f204c.sanity-flr lcm_layout_gen: 9 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x8e:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 1048576 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xaa:0x0] } lcme_id: 131078 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 1048576 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xab:0x0] } PASS 204c (25s) == sanity-flr test 204d: FLR write/stale/resync sel test with repeated comp ========================================================== 09:32:40 (1773235960) Waiting for MDT destroys to complete Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg338-server mds-ost sync done. striped dir -i0 -c2 -H fnv_1a_64 /mnt/lustre/d204d.sanity-flr 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257793 s, 40.7 MB/s /mnt/lustre/d204d.sanity-flr/f204d.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x8f:0x0] } lcme_id: 65541 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xad:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xac:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: stale lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 /mnt/lustre/d204d.sanity-flr/f204d.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x8f:0x0] } lcme_id: 65541 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xad:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 10485760 lmm_stripe_count: 1 lmm_stripe_size: 2097152 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xac:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 10485760 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x90:0x0] } PASS 204d (37s) == sanity-flr test 204e: FLR write/stale/resync sel test with repeated comp ========================================================== 09:33:17 (1773235997) Waiting for MDT destroys to complete Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg338-server mds-ost sync done. striped dir -i0 -c2 -H crush2 /mnt/lustre/d204e.sanity-flr /mnt/lustre/d204e.sanity-flr/f204e.sanity-flr lcm_layout_gen: 4 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xae:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xaf:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 67108864 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0361104 s, 58.1 MB/s /mnt/lustre/d204e.sanity-flr/f204e.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xae:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xaf:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x91:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 /mnt/lustre/d204e.sanity-flr/f204e.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 5 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xae:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 67108864 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xaf:0x0] } lcme_id: 131077 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 67108864 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x91:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 PASS 204e (35s) == sanity-flr test 204f: FLR write/stale/resync sel w/forced extension ========================================================== 09:33:52 (1773236032) Waiting for MDT destroys to complete Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg338-server mds-ost sync done. striped dir -i0 -c2 -H crush2 /mnt/lustre/d204f.sanity-flr Creating new pool test_204f oleg338-server: Pool lustre.test_204f created Adding targets to pool oleg338-server: OST lustre-OST0000_UUID added to pool lustre.test_204f oleg338-server: OST lustre-OST0001_UUID added to pool lustre.test_204f 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0387641 s, 54.1 MB/s /mnt/lustre/d204f.sanity-flr/f204f.sanity-flr lcm_layout_gen: 6 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xb0:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init,stale lcme_extent.e_start: 0 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_pool: test_204f lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x92:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lmm_pool: test_204f /mnt/lustre/d204f.sanity-flr/f204f.sanity-flr lcm_layout_gen: 8 lcm_mirror_count: 2 lcm_entry_count: 4 lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 104857600 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 lmm_objects: - 0: { l_ost_idx: 0, l_fid: [0x280000401:0xb0:0x0] } lcme_id: 65538 lcme_mirror_id: 1 lcme_flags: 0 lcme_extent.e_start: 104857600 lcme_extent.e_end: EOF lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lcme_id: 131075 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 134217728 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_pool: test_204f lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x92:0x0] } lcme_id: 131076 lcme_mirror_id: 2 lcme_flags: extension lcme_extent.e_start: 134217728 lcme_extent.e_end: EOF lmm_stripe_count: 0 lmm_extension_size: 67108864 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lmm_pool: test_204f Destroy the created pools: test_204f lustre.test_204f oleg338-server: OST lustre-OST0000_UUID removed from pool lustre.test_204f oleg338-server: OST lustre-OST0001_UUID removed from pool lustre.test_204f oleg338-server: Pool lustre.test_204f destroyed Waiting 90s for 'foo' PASS 204f (50s) == sanity-flr test 205a: lfs mirror extend to set prefer flag ========================================================== 09:34:42 (1773236082) PASS 205a (3s) == sanity-flr test 205b: lfs mirror extend to set nocompr flag ========================================================== 09:34:45 (1773236085) PASS 205b (3s) == sanity-flr test 206: lfs setstripe -pool .. --comp-flags=.. ========================================================== 09:34:48 (1773236088) oleg338-server: Pool lustre.test_206 created Adding targets to pool oleg338-server: OST lustre-OST0000_UUID added to pool lustre.test_206 oleg338-server: OST lustre-OST0001_UUID added to pool lustre.test_206 Destroy the created pools: test_206 lustre.test_206 oleg338-server: OST lustre-OST0000_UUID removed from pool lustre.test_206 oleg338-server: OST lustre-OST0001_UUID removed from pool lustre.test_206 oleg338-server: Pool lustre.test_206 destroyed Waiting 90s for 'foo' PASS 206 (17s) == sanity-flr test 207: create another replica with existing out-of-sync one ========================================================== 09:35:05 (1773236105) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332378 s, 31.5 MB/s 2 mirror IDs: 1 2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159879 s, 65.6 MB/s 2 mirror IDs: 1 2 2 mirror IDs: 1 2 3 mirror IDs: 1 2 3 CRC-32 checksum value for chunk [0, 0x100000): Mirror 1: 0x173b4697 Mirror 3: 0x173b4697 PASS 207 (4s) == sanity-flr test 208a: mirror selection to prefer non-rotational devices for reads ========================================================== 09:35:09 (1773236109) SKIP: sanity-flr test_208a needs >= 4 OSTs SKIP 208a (1s) == sanity-flr test 208b: mirror selection to prefer non-rotational devices for writes ========================================================== 09:35:10 (1773236110) SKIP: sanity-flr test_208b needs >= 4 OSTs SKIP 208b (2s) == sanity-flr test 209a: skip fast reads after layout invalidation ========================================================== 09:35:12 (1773236112) Starting client: oleg338-client.virtnet: -o user_xattr,flock 192.168.203.138@tcp:/lustre /mnt/lustre2 osd-ldiskfs.lustre-OST0000.nonrotational=0 osd-ldiskfs.lustre-OST0001.nonrotational=0 osd-ldiskfs.lustre-OST0001.nonrotational=1 replicated file created open(O_RDONLY) and first read from OST first read complete 4+0 records in 4+0 records out 4 bytes copied, 0.0156706 s, 0.3 kB/s BBBB written which made replica on ost1 stale fast read from pagecache in the original process read via /mnt/lustre2 new open(2) Buffer address : 0x1ab0000 Read this (4 bytes): BBBB fast read from pagecache after 5s in the original process Buffer address : 0x1000000 Read this (4 bytes): AAAA PAUSING Buffer address : 0x1000000 Read this (4 bytes): BBBB PAUSING Buffer address : 0x1000000 Read this (4 bytes): BBBB read via new open(2) Buffer address : 0x2000000 Read this (4 bytes): BBBB 192.168.203.138@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg338-client.virtnet /mnt/lustre2 (opts:) PASS 209a (17s) == sanity-flr test 209b: pagecache can be used after LL cancellation ========================================================== 09:35:29 (1773236129) 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.00525953 s, 1.6 MB/s the very first read read with warm cache PASS 209b (3s) == sanity-flr test 210a: handle broken mirrored lovea ==== 09:35:32 (1773236132) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197144 s, 53.2 MB/s fail_loc=0x1428 /mnt/lustre/f210a.sanity-flr composite_header: lcm_magic: 0x0BD60BD0 lcm_size: 240 lcm_flags: ro lcm_layout_gen: 1 lcm_mirror_count: 2 lcm_entry_count: 2 components: - lcme_id: 65537 lcme_mirror_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lcme_offset: 128 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x81 lmm_fid: [0x200000405:0x81:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: -1 lmm_objects: - 0: { l_ost_idx: -1, l_fid: [0:0x0:0x0] } - lcme_id: 131073 lcme_mirror_id: 2 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: EOF lcme_offset: 184 lcme_size: 56 sub_layout: lmm_magic: 0x0BD10BD0 lmm_seq: 0x200000405 lmm_object_id: 0x81 lmm_fid: [0x200000405:0x81:0x0] lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 lmm_objects: - 0: { l_ost_idx: 1, l_fid: [0x2c0000401:0x9c:0x0] } File: /mnt/lustre/f210a.sanity-flr Size: 1048576 Blocks: 4096 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205339611265 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-03-11 09:35:33.000000000 -0400 Modify: 2026-03-11 09:35:33.000000000 -0400 Change: 2026-03-11 09:35:33.000000000 -0400 Birth: 2026-03-11 09:35:32.000000000 -0400 PASS 210a (4s) == sanity-flr test 210b: handle broken mirrored lovea (unlink) ========================================================== 09:35:36 (1773236136) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152578 s, 68.7 MB/s oleg338-server: debugfs 1.47.3-wc2 (11-Nov-2025) fail_loc=0x1428 Waiting for MDT destroys to complete oleg338-server: debugfs 1.47.3-wc2 (11-Nov-2025) oleg338-server: O/280000401/d27/bb: File not found by ext2_lookup PASS 210b (14s) == sanity-flr test 211: mirror delete should not cause bad size ========================================================== 09:35:51 (1773236151) 10+0 records in 10+0 records out 40960 bytes (41 kB, 40 KiB) copied, 0.0525096 s, 780 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00787472 s, 520 kB/s size after second write -rw-r--r-- 1 root root 4096 Mar 11 09:35 /mnt/lustre/f211.sanity-flr lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000405:0x86:0x0]: Invalid argument (22) size after mirror delete -rw-r--r-- 1 root root 4096 Mar 11 09:35 /mnt/lustre/f211.sanity-flr PASS 211 (3s) == sanity-flr test complete, duration 1084 sec =========== 09:35:53 (1773236153) === sanity-flr: start cleanup 09:35:54 (1773236154) === === sanity-flr: finish cleanup 09:35:57 (1773236157) ===