== sanity-flr test 41: lfs mirror resync check =========== 10:56:20 (1754405780) **create two FLR files /mnt/lustre/f41.sanity-flr /mnt/lustre/f41.sanity-flr-1 DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[0, 2097152] DEBUG: prev component: NULL DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=2097152 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[2097152, 4194304] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[0, 2097152] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 2097152], curr=[2097152, 4194304] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=2097152, end=4194304 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[4194304, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[2097152, 4194304] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[2097152, 4194304], curr=[4194304, 18446744073709551615] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=4194304, end=18446744073709551615 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[0, 1048576] DEBUG: prev component: NULL DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=1048576 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[1048576, 2097152] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[0, 1048576] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 1048576], curr=[1048576, 2097152] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=1048576, end=2097152 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[2097152, 3145728] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[1048576, 2097152] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[1048576, 2097152], curr=[2097152, 3145728] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=2097152, end=3145728 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[3145728, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[2097152, 3145728] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[2097152, 3145728], curr=[3145728, 18446744073709551615] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=3145728, end=18446744073709551615 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[0, 2097152] DEBUG: prev component: NULL DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=2097152 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[2097152, 4194304] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[0, 2097152] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 2097152], curr=[2097152, 4194304] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=2097152, end=4194304 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[4194304, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[2097152, 4194304] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[2097152, 4194304], curr=[4194304, 18446744073709551615] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=4194304, end=18446744073709551615 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[0, 1048576] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[4194304, 18446744073709551615] DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=1048576 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[1048576, 2097152] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[0, 1048576] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 1048576], curr=[1048576, 2097152] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=1048576, end=2097152 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[2097152, 3145728] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[1048576, 2097152] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[1048576, 2097152], curr=[2097152, 3145728] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=2097152, end=3145728 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[3145728, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[2097152, 3145728] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[2097152, 3145728], curr=[3145728, 18446744073709551615] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=3145728, end=18446744073709551615 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[0, 2097152] DEBUG: prev component: NULL DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=2097152 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[2097152, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[0, 2097152] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 2097152], curr=[2097152, 18446744073709551615] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=2097152, end=18446744073709551615 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[0, 1048576] DEBUG: prev component: NULL DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=1048576 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[1048576, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[0, 1048576] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 1048576], curr=[1048576, 18446744073709551615] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=1048576, end=18446744073709551615 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[0, 4194304] DEBUG: prev component: NULL DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=4194304 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[4194304, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[0, 4194304] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 4194304], curr=[4194304, 18446744073709551615] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=4194304, end=18446744073709551615 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[0, 2097152] DEBUG: prev component: NULL DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=2097152 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[2097152, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[0, 2097152] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 2097152], curr=[2097152, 18446744073709551615] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=2097152, end=18446744073709551615 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x6, extent=[0, 1048576] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[2097152, 18446744073709551615] DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=1048576 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x6, extent=[1048576, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x6, extent=[0, 1048576] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 1048576], curr=[1048576, 18446744073709551615] DEBUG: prev flags=0x6, curr flags=0x6 DEBUG: Checking start <= end: start=1048576, end=18446744073709551615 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[0, 4194304] DEBUG: prev component: found DEBUG: prev component flags=0x6, extent=[1048576, 18446744073709551615] DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=4194304 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[4194304, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[0, 4194304] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 4194304], curr=[4194304, 18446744073709551615] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=4194304, end=18446744073709551615 DEBUG: Component validation passed, continuing **verify files be RDONLY 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.127948 s, 16.4 MB/s 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.257188 s, 16.3 MB/s **verify files be WRITE_PENDING **verify files have stale component **full resync **verify /mnt/lustre/f41.sanity-flr-1 data consistency in all mirrors **verify files be RDONLY **verify files do not contain stale component **write /mnt/lustre/f41.sanity-flr-1 for partial resync test 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.186935 s, 11.2 MB/s **only resync mirror 2 **resync mirror 2 again **verify /mnt/lustre/f41.sanity-flr-1 contains stale component **full resync /mnt/lustre/f41.sanity-flr-1 **full resync /mnt/lustre/f41.sanity-flr-1 again **verify /mnt/lustre/f41.sanity-flr-1 does not contain stale component