== sanity-flr test 200a: stress test ===================== 11:04:11 (1754406251) DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[0, 1048576] DEBUG: prev component: NULL DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=1048576 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[1048576, 2097152] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[0, 1048576] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 1048576], curr=[1048576, 2097152] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=1048576, end=2097152 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[2097152, 4194304] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[1048576, 2097152] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[1048576, 2097152], curr=[2097152, 4194304] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=2097152, end=4194304 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[4194304, 16777216] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[2097152, 4194304] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[2097152, 4194304], curr=[4194304, 16777216] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=4194304, end=16777216 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[16777216, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[4194304, 16777216] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[4194304, 16777216], curr=[16777216, 18446744073709551615] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=16777216, end=18446744073709551615 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[0, 2097152] DEBUG: prev component: NULL DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=2097152 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[2097152, 6291456] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[0, 2097152] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 2097152], curr=[2097152, 6291456] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=2097152, end=6291456 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[6291456, 8388608] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[2097152, 6291456] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[2097152, 6291456], curr=[6291456, 8388608] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=6291456, end=8388608 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[8388608, 33554432] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[6291456, 8388608] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[6291456, 8388608], curr=[8388608, 33554432] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=8388608, end=33554432 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[33554432, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[8388608, 33554432] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[8388608, 33554432], curr=[33554432, 18446744073709551615] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=33554432, end=18446744073709551615 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[0, 4194304] DEBUG: prev component: NULL DEBUG: Component starts at 0, checking if new mirror DEBUG: Setting first_comp = true (new mirror) DEBUG: Setting prev = NULL (new mirror) DEBUG: About to check adjacency, prev=NULL DEBUG: Checking start <= end: start=0, end=4194304 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[4194304, 8388608] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[0, 4194304] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[0, 4194304], curr=[4194304, 8388608] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=4194304, end=8388608 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[8388608, 67108864] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[4194304, 8388608] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[4194304, 8388608], curr=[8388608, 67108864] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=8388608, end=67108864 DEBUG: Component validation passed, continuing DEBUG: llapi_layout_sanity_cb: component flags=0x0, extent=[67108864, 18446744073709551615] DEBUG: prev component: found DEBUG: prev component flags=0x0, extent=[8388608, 67108864] DEBUG: About to check adjacency, prev=exists DEBUG: Checking adjacency: prev=[8388608, 67108864], curr=[67108864, 18446744073709551615] DEBUG: prev flags=0x0, curr flags=0x0 DEBUG: Checking start <= end: start=67108864, end=18446744073709551615 DEBUG: Component validation passed, continuing Starting client: oleg156-client.virtnet: -o user_xattr,flock 192.168.201.156@tcp:/lustre /mnt/lustre2 Starting client: oleg156-client.virtnet: -o user_xattr,flock 192.168.201.156@tcp:/lustre /mnt/lustre3 fail_loc=0x1A03 fail_loc=0x1A03 lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..Extending file size to 2918624 .. Extending file size to 1298624 .. Extending file size to 2656704 .. Extending file size to 4868768 .. Extending file size to 7397760 .. failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed Extending file size to 5285088 .. Extending file size to 8974048 .. Extending file size to 8065952 .. resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8737984 .. done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed Extending file size to 8831168 .. resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done Extending file size to 8912864 .. resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..Extending file size to 9097376 .. failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 9075296 .. done lock to resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done resync file /mnt/lustre3/f200a.sanity-flr with 'mirror_io resync -e resync_start' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed lock to resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed resync file /mnt/lustre3/f200a.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed Waiting 60053 60054 60055 60059 60060 192.168.201.156@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg156-client.virtnet /mnt/lustre2 (opts:) 192.168.201.156@tcp:/lustre /mnt/lustre3 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg156-client.virtnet /mnt/lustre3 (opts:) 3