== sanity-flr test 70a: flr mode fsx test ================ 10:41:46 (1787496106) Seed set to 6108 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x2f664 000004 1787496108.845456 trunc from 00000000 to 0x02f663 (0x2f664 bytes) 000005 1787496108.888989 mapwrite 0x01cdc1 thru 0x028a5e (0xbc9e bytes) 000007 1787496109.099192 mapread 0x025477 thru 0x02ab81 (0x570b bytes) 000008 1787496109.100098 mapread 0x006c76 thru 0x0097a7 (0x2b32 bytes) 000010 1787496109.132735 mapwrite 0x037ed2 thru 0x03b0c8 (0x31f7 bytes) 000011 1787496109.246223 mapread 0x00ec2c thru 0x01ade9 (0xc1be bytes) 000012 1787496109.256825 mapwrite 0x028767 thru 0x02aed3 (0x276d bytes) 000013 1787496109.314350 mapread 0x012e6f thru 0x02144a (0xe5dc bytes) 000016 1787496109.317922 read 0x016b1e thru 0x018877 (0x1d5a bytes) 000017 1787496109.318662 mapwrite 0x024a70 thru 0x02a49f (0x5a30 bytes) 000019 1787496109.359540 read 0x0194fe thru 0x0253e0 (0xbee3 bytes) 000020 1787496109.360164 read 0x028d46 thru 0x0316c3 (0x897e bytes) 000021 1787496109.360682 trunc from 0x03b0c9 to 0x027f15 (0x131b3 bytes) 000023 1787496109.379538 read 0x0139fd thru 0x020170 (0xc774 bytes) 000025 1787496109.381641 mapwrite 0x00f849 thru 0x01a664 (0xae1c bytes) 000027 1787496109.433810 read 0x00687c thru 0x00e470 (0x7bf5 bytes) 000028 1787496109.435504 mapread 0x005286 thru 0x01369c (0xe417 bytes) 000031 1787496109.438471 write 0x01583f thru 0x021f24 (0xc6e6 bytes) 000032 1787496109.440093 write 0x03a090 thru 0x03ae08 (0xd79 bytes) 000033 1787496109.441288 read 0x00bc48 thru 0x01a0b8 (0xe471 bytes) truncating to largest ever: 0x37e12 000034 1787496109.442738 trunc from 0x03ae09 to 0x037e11 (0x2ff7 bytes) 000037 1787496109.458728 trunc from 0x037e12 to 0x017d73 (0x2009e bytes) 000041 1787496109.489600 mapwrite 0x012b95 thru 0x015704 (0x2b70 bytes) 000044 1787496109.621274 trunc from 0x017d74 to 0x01f312 (0x759f bytes) 000047 1787496109.657262 mapread 0x009424 thru 0x016ede (0xdabb bytes) 000050 1787496109.663773 mapread 0x00eb4a thru 0x01413e (0x55f5 bytes) truncating to largest ever: 0x3b5b5 000051 1787496109.671158 trunc from 0x01f313 to 0x03b5b4 (0x1c2a2 bytes) 000053 1787496109.706844 write 0x0272ab thru 0x02d765 (0x64bb bytes) 000054 1787496109.723659 read 0x01cf6b thru 0x02be44 (0xeeda bytes) 000055 1787496109.726886 mapwrite 0x035f83 thru 0x03ffff (0xa07d bytes) 000058 1787496109.943985 write 0x01de0f thru 0x02be3f (0xe031 bytes) 000059 1787496109.945253 read 0x03153c thru 0x03f336 (0xddfb bytes) 000060 1787496109.956039 write 0x03969d thru 0x03ffff (0x6963 bytes) 000061 1787496109.957976 mapread 0x01e399 thru 0x01f8f8 (0x1560 bytes) 000063 1787496109.962285 read 0x02366b thru 0x026dba (0x3750 bytes) 000064 1787496109.972699 mapread 0x0354c6 thru 0x03a496 (0x4fd1 bytes) 000065 1787496109.973782 mapread 0x03a38e thru 0x03a8a0 (0x513 bytes) 000069 1787496109.991330 mapread 0x0072a1 thru 0x010441 (0x91a1 bytes) 000070 1787496109.996554 trunc from 0x040000 to 0x01b544 (0x24abb bytes) 000071 1787496110.032111 write 0x027693 thru 0x0339c0 (0xc32e bytes) 000073 1787496110.034835 mapwrite 0x02a0ce thru 0x039036 (0xef69 bytes) 000074 1787496110.199958 write 0x035dc3 thru 0x03fe34 (0xa072 bytes) 000076 1787496110.207662 trunc from 0x03fe35 to 0x0122f6 (0x2db3e bytes) 000077 1787496110.274875 mapread 0x005dfa thru 0x009220 (0x3427 bytes) 000079 1787496110.277426 read 0x003a55 thru 0x01165a (0xdc06 bytes) 000080 1787496110.278771 mapread 0x004d6f thru 0x007b2e (0x2dc0 bytes) 000081 1787496110.281604 trunc from 0x0122f7 to 0x02efd6 (0x1cce0 bytes) 000082 1787496110.305591 read 0x010391 thru 0x0201aa (0xfe1a bytes) 000086 1787496110.320797 mapwrite 0x005f17 thru 0x014f02 (0xefec bytes) 000088 1787496110.389046 read 0x005225 thru 0x0074c4 (0x22a0 bytes) 000090 1787496110.391919 mapread 0x009547 thru 0x01483b (0xb2f5 bytes) 000092 1787496110.397570 mapwrite 0x0098e5 thru 0x00d468 (0x3b84 bytes) 000093 1787496110.456131 mapwrite 0x02d84a thru 0x0307b9 (0x2f70 bytes) 000098 1787496110.531904 mapread 0x010edc thru 0x0134ba (0x25df bytes) 000100 1787496110.535542 mapread 0x01fc16 thru 0x025864 (0x5c4f bytes) 000101 1787496110.540684 MIRROR_EXTEND to 2 mirrors 000104 1787496111.749113 write 0x008a28 thru 0x00f63f (0x6c18 bytes) 000105 1787496111.843351 write 0x00864f thru 0x00f381 (0x6d33 bytes) 000106 1787496111.844449 mapread 0x01b399 thru 0x01cd9b (0x1a03 bytes) 000109 1787496111.855465 trunc from 0x0307ba to 0x01535d (0x1b45c bytes) 000110 1787496111.881072 trunc from 0x01535e to 0x038266 (0x22f09 bytes) 000112 1787496111.910411 mapwrite 0x03c530 thru 0x03ffff (0x3ad0 bytes) 000113 1787496112.036622 mapread 0x035fb1 thru 0x03d9df (0x7a2f bytes) 000117 1787496112.038968 mapread 0x00750e thru 0x00edbe (0x78b1 bytes) 000119 1787496112.047575 mapwrite 0x028d5b thru 0x02f17c (0x6422 bytes) 000121 1787496112.135688 trunc from 0x040000 to 0x02c2ea (0x13d15 bytes) 000122 1787496112.179055 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) 000123 1787496112.406576 trunc from 0x02c2eb to 0x01c2c6 (0x10024 bytes) 000124 1787496112.424954 mapwrite 0x00263e thru 0x0097ae (0x7171 bytes) 000125 1787496112.500043 MIRROR_EXTEND to 2 mirrors 000128 1787496113.543372 trunc from 0x01c2c7 to 0x00201b (0x1a2ab bytes) 000129 1787496113.616011 trunc from 0x00201c to 0x00b339 (0x931e bytes) 000130 1787496113.637659 MIRROR_RESYNC 2 mirrors 000131 1787496114.880964 read 0x000a02 thru 0x003d02 (0x3301 bytes) 000133 1787496114.894746 trunc from 0x00b33a to 0x0154a7 (0xa16e bytes) 000134 1787496115.059718 mapwrite 0x030000 thru 0x031ee5 (0x1ee6 bytes) 000136 1787496115.213126 MIRROR_RESYNC 2 mirrors 000138 1787496115.923079 mapwrite 0x0194ff thru 0x025c1f (0xc721 bytes) 000139 1787496116.169567 mapwrite 0x03203f thru 0x03f3b3 (0xd375 bytes) 000140 1787496116.368720 MIRROR_RESYNC 2 mirrors 000141 1787496117.865493 read 0x01744a thru 0x026975 (0xf52c bytes) 000142 1787496117.883108 read 0x034d88 thru 0x03c582 (0x77fb bytes) 000143 1787496117.903017 write 0x01667a thru 0x019c85 (0x360c bytes) 000146 1787496118.046602 write 0x006849 thru 0x015ff5 (0xf7ad bytes) 000147 1787496118.056771 mapread 0x01200c thru 0x0136fa (0x16ef bytes) 000150 1787496118.058657 read 0x00beb5 thru 0x015e81 (0x9fcd bytes) 000151 1787496118.060557 mapwrite 0x032b36 thru 0x038171 (0x563c bytes) 000152 1787496118.133268 read 0x00e8ae thru 0x013e1f (0x5572 bytes) 000155 1787496118.134729 mapwrite 0x0180bf thru 0x01fba7 (0x7ae9 bytes) 000156 1787496118.185846 trunc from 0x03f3b4 to 0x00575b (0x39c58 bytes) 000160 1787496118.212769 read 0x004635 thru 0x00575b (0x1127 bytes) 000161 1787496118.218670 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x575c): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x575c): Mirror 2: 0xd5f6a4ef (stale) CRC-32 checksum value for chunk [0, 0x575c): Mirror 1: 0xd5f6a4ef Mirror 3: 0xd5f6a4ef lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x575c: skipped 000162 1787496119.493657 mapread 0x0028ec thru 0x00575b (0x2e70 bytes) 000163 1787496119.515572 mapwrite 0x0272d0 thru 0x03237e (0xb0af bytes) 000165 1787496119.682654 mapwrite 0x022636 thru 0x02772d (0x50f8 bytes) 000167 1787496119.739630 trunc from 0x03237f to 0x01d9e0 (0x1499e bytes) 000168 1787496119.763747 trunc from 0x01d9e1 to 0x02e521 (0x10b41 bytes) 000170 1787496119.781608 write 0x0030bb thru 0x011c8a (0xebd0 bytes) 000172 1787496119.797126 read 0x016dde thru 0x01c4db (0x56fe bytes) 000173 1787496119.806004 read 0x020111 thru 0x023eb2 (0x3da2 bytes) 000175 1787496119.817086 mapwrite 0x0009f7 thru 0x002d85 (0x238f bytes) 000177 1787496119.873475 MIRROR_RESYNC 3 mirrors 000179 1787496120.670880 mapread 0x025804 thru 0x02c131 (0x692e bytes) 000181 1787496120.690048 write 0x000315 thru 0x00b322 (0xb00e bytes) 000182 1787496120.763293 read 0x0001c6 thru 0x00e366 (0xe1a1 bytes) 000184 1787496120.774513 MIRROR_RESYNC 3 mirrors 000185 1787496121.487545 mapwrite 0x0304a8 thru 0x036232 (0x5d8b bytes) 000188 1787496121.614338 read 0x026822 thru 0x035a8d (0xf26c bytes) 000191 1787496121.628046 trunc from 0x036233 to 0x0008b2 (0x35980 bytes) truncating to largest ever: 0x3ca00 000192 1787496121.670630 trunc from 0x0008b3 to 0x03c9ff (0x3c14d bytes) 000193 1787496121.704915 trunc from 0x03ca00 to 0x0275ea (0x15415 bytes) 000195 1787496121.724597 write 0x02cc24 thru 0x032853 (0x5c30 bytes) 000196 1787496121.727125 mapread 0x000fb5 thru 0x0075aa (0x65f6 bytes) truncating to largest ever: 0x3e03b 000200 1787496121.739763 trunc from 0x032854 to 0x03e03a (0xb7e7 bytes) 000201 1787496121.758559 write 0x0241e3 thru 0x032c59 (0xea77 bytes) 000202 1787496121.766614 write 0x03c6f3 thru 0x03ffff (0x390d bytes) 000204 1787496121.772628 mapwrite 0x02d212 thru 0x03b63f (0xe42e bytes) 000206 1787496121.853519 read 0x01cd4f thru 0x027da5 (0xb057 bytes) 000208 1787496121.873050 read 0x00e063 thru 0x01acbf (0xcc5d bytes) 000209 1787496121.874351 trunc from 0x040000 to 0x00363a (0x3c9c5 bytes) 000210 1787496121.897584 mapread 0x0000a1 thru 0x00363a (0x359a bytes) 000211 1787496121.899912 mapread 0x001027 thru 0x00363a (0x2614 bytes) 000214 1787496121.903612 mapread 0x00018b thru 0x00363a (0x34b0 bytes) 000217 1787496121.906064 trunc from 0x00363b to 0x034741 (0x31107 bytes) 000218 1787496121.928715 mapwrite 0x01244f thru 0x01834f (0x5f01 bytes) 000219 1787496122.008729 mapwrite 0x009285 thru 0x012472 (0x91ee bytes) 000221 1787496122.082796 write 0x024071 thru 0x02ac9c (0x6c2c bytes) 000222 1787496122.084692 trunc from 0x034742 to 0x0264f4 (0xe24d bytes) 000223 1787496122.115755 mapwrite 0x02850a thru 0x0329fa (0xa4f1 bytes) 000224 1787496122.290117 read 0x024a84 thru 0x02a881 (0x5dfe bytes) truncating to largest ever: 0x3e656 000226 1787496122.297693 trunc from 0x0329fb to 0x03e655 (0xbc5b bytes) 000227 1787496122.311682 mapread 0x003984 thru 0x005cc8 (0x2345 bytes) 000228 1787496122.316939 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3e656): Mirror 3: 0x66158980 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e656): Mirror 3: 0x66158980 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e656: skipped 000230 1787496122.581451 trunc from 0x03e656 to 0x02fa4c (0xec09 bytes) 000232 1787496122.599987 trunc from 0x02fa4d to 0x035a30 (0x5fe4 bytes) 000233 1787496122.616553 trunc from 0x035a31 to 0x009b39 (0x2bef7 bytes) 000235 1787496122.658314 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) 000237 1787496122.907236 write 0x00dd27 thru 0x013e96 (0x6170 bytes) 000239 1787496122.927190 MIRROR_EXTEND to 2 mirrors 000240 1787496123.969868 mapread 0x007ed0 thru 0x013e96 (0xbfc7 bytes) 000241 1787496124.012527 mapread 0x011809 thru 0x013e96 (0x268e bytes) 000244 1787496124.051521 read 0x000cf6 thru 0x00c22f (0xb53a bytes) 000247 1787496124.088478 read 0x0064eb thru 0x0087cb (0x22e1 bytes) 000249 1787496124.114911 trunc from 0x013e97 to 0x02af89 (0x170f3 bytes) 000251 1787496124.223413 mapread 0x012c1a thru 0x0214b3 (0xe89a bytes) 000253 1787496124.237740 mapread 0x005d5c thru 0x011384 (0xb629 bytes) 000254 1787496124.238895 mapread 0x008ec4 thru 0x0095a6 (0x6e3 bytes) 000255 1787496124.240501 trunc from 0x02af8a to 0x007a76 (0x23513 bytes) 000256 1787496124.263651 trunc from 0x007a77 to 0x0100bf (0x8649 bytes) 000257 1787496124.281011 mapwrite 0x025503 thru 0x02f964 (0xa462 bytes) 000258 1787496124.405902 mapread 0x01ad14 thru 0x01c87e (0x1b6b bytes) 000261 1787496124.422406 read 0x005b89 thru 0x00ba90 (0x5f08 bytes) 000262 1787496124.433574 write 0x03a100 thru 0x03ffff (0x5f00 bytes) 000264 1787496124.436652 mapwrite 0x02c579 thru 0x0350ec (0x8b74 bytes) 000266 1787496124.498277 read 0x0003c0 thru 0x005c10 (0x5851 bytes) 000267 1787496124.499032 trunc from 0x040000 to 0x037cea (0x8315 bytes) 000270 1787496124.517078 read 0x00b69d thru 0x014a4b (0x93af bytes) 000271 1787496124.517960 read 0x00ccdf thru 0x01c5b0 (0xf8d2 bytes) 000272 1787496124.519744 mapread 0x017f32 thru 0x0260fa (0xe1c9 bytes) 000276 1787496124.523708 write 0x03cbc5 thru 0x03ffff (0x343b bytes) 000277 1787496124.528392 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xdecfedfe (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xf2158da9 Mirror 3: 0xf2158da9 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000279 1787496125.508160 mapwrite 0x02c84a thru 0x032ae1 (0x6298 bytes) 000280 1787496125.663427 read 0x033523 thru 0x036843 (0x3321 bytes) 000283 1787496125.673027 read 0x02c59c thru 0x0322d9 (0x5d3e bytes) 000285 1787496125.674098 mapwrite 0x0032e2 thru 0x006e84 (0x3ba3 bytes) 000288 1787496125.752607 mapwrite 0x02488e thru 0x02fdde (0xb551 bytes) 000290 1787496125.824640 read 0x012dfa thru 0x01d626 (0xa82d bytes) 000294 1787496125.827609 mapwrite 0x0080d2 thru 0x00fd1c (0x7c4b bytes) 000295 1787496125.889414 read 0x03fafc thru 0x03ffff (0x504 bytes) 000296 1787496125.890360 read 0x037216 thru 0x03af69 (0x3d54 bytes) 000297 1787496125.894941 mapread 0x030c13 thru 0x033a97 (0x2e85 bytes) 000298 1787496125.896482 trunc from 0x040000 to 0x00c39c (0x33c63 bytes) 000299 1787496125.932118 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0xc39d): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xc39d): Mirror 2: 0x98b51ab5 (stale) Mirror 3: 0x830dd009 (stale) CRC-32 checksum value for chunk [0, 0xc39d): Mirror 1: 0x103aae6c Mirror 4: 0x103aae6c lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xc39d: skipped 000301 1787496127.022347 read 0x004035 thru 0x00c39c (0x8368 bytes) 000302 1787496127.032778 write 0x035d2f thru 0x03ac56 (0x4f28 bytes) 000304 1787496127.118510 mapread 0x028a82 thru 0x032950 (0x9ecf bytes) 000305 1787496127.145115 read 0x01fabb thru 0x024eb4 (0x53fa bytes) 000306 1787496127.162058 MIRROR_RESYNC 4 mirrors 000310 1787496128.739782 mapwrite 0x021227 thru 0x0309c8 (0xf7a2 bytes) 000312 1787496128.929925 write 0x004a9a thru 0x00a4a2 (0x5a09 bytes) 000313 1787496128.940242 mapwrite 0x00478a thru 0x0069f7 (0x226e bytes) 000316 1787496128.999872 mapread 0x014a00 thru 0x0155b3 (0xbb4 bytes) 000317 1787496129.010462 mapwrite 0x0002d8 thru 0x003e31 (0x3b5a bytes) 000320 1787496129.077898 read 0x009bf4 thru 0x00a86c (0xc79 bytes) 000321 1787496129.084346 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3ac57): Mirror 2: 0x77d7abbe (stale) Mirror 3: 0x77d7abbe (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3ac57): Mirror 2: 0x77d7abbe (stale) Mirror 3: 0x77d7abbe (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3ac57: skipped 000323 1787496129.445734 MIRROR_RESYNC 3 mirrors 000325 1787496130.657327 write 0x03267d thru 0x03f878 (0xd1fc bytes) 000328 1787496130.732300 trunc from 0x03f879 to 0x03360c (0xc26c bytes) 000331 1787496130.758060 write 0x00c25c thru 0x01b28b (0xf030 bytes) 000332 1787496130.773741 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3360d): Mirror 3: 0xa1489f04 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3360d): Mirror 3: 0xa1489f04 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3360d: skipped 000333 1787496131.255505 MIRROR_RESYNC 2 mirrors 000339 1787496132.566862 mapread 0x0028da thru 0x00c68f (0x9db6 bytes) 000341 1787496132.618860 read 0x005883 thru 0x014763 (0xeee1 bytes) 000342 1787496132.630301 MIRROR_EXTEND to 3 mirrors 000345 1787496133.623700 mapread 0x02007b thru 0x025997 (0x591d bytes) 000346 1787496133.647375 mapread 0x0040e0 thru 0x008429 (0x434a bytes) 000347 1787496133.658170 mapwrite 0x0336a6 thru 0x03c112 (0x8a6d bytes) 000348 1787496133.818495 write 0x01d828 thru 0x024242 (0x6a1b bytes) 000350 1787496133.834006 mapwrite 0x0271ab thru 0x02c0f4 (0x4f4a bytes) 000354 1787496133.943682 MIRROR_RESYNC 3 mirrors 000355 1787496135.470032 trunc from 0x03c113 to 0x010998 (0x2b77a bytes) 000356 1787496135.604159 mapread 0x00d805 thru 0x010998 (0x3194 bytes) 000357 1787496135.632897 trunc from 0x010999 to 0x00e155 (0x2843 bytes) 000361 1787496135.676601 read 0x00c82b thru 0x00db53 (0x1329 bytes) 000364 1787496135.684527 mapwrite 0x021085 thru 0x02ddbd (0xcd39 bytes) 000365 1787496135.889128 read 0x0249d2 thru 0x02b6bb (0x6cea bytes) 000369 1787496135.893618 trunc from 0x02ddbe to 0x0083d2 (0x259eb bytes) 000370 1787496135.908767 trunc from 0x0083d3 to 0x00b58f (0x31bd bytes) 000371 1787496135.924172 read 0x0029c3 thru 0x0039a5 (0xfe3 bytes) 000373 1787496135.933221 trunc from 0x00b590 to 0x02f2a2 (0x23d13 bytes) 000374 1787496135.947413 mapread 0x001bef thru 0x005ce8 (0x40fa bytes) 000378 1787496135.949540 read 0x02260c thru 0x02b7f2 (0x91e7 bytes) 000379 1787496135.959263 read 0x001d83 thru 0x006e1b (0x5099 bytes) 000380 1787496135.959863 trunc from 0x02f2a3 to 0x01a2ab (0x14ff7 bytes) 000381 1787496135.977517 mapwrite 0x009b4f thru 0x00e51e (0x49d0 bytes) 000382 1787496136.022082 read 0x00955f thru 0x00f03b (0x5add bytes) 000383 1787496136.028097 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x1a2ac): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1a2ac): Mirror 3: 0xe97270dc (stale) Mirror 4: 0xe97270dc (stale) CRC-32 checksum value for chunk [0, 0x1a2ac): Mirror 1: 0xcf6ab55a Mirror 5: 0xcf6ab55a lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1a2ac: skipped 000384 1787496137.359656 write 0x00a9f5 thru 0x013563 (0x8b6f bytes) 000385 1787496137.548060 MIRROR_RESYNC 4 mirrors 000386 1787496138.781191 read 0x0081b0 thru 0x011849 (0x969a bytes) 000387 1787496138.793546 mapwrite 0x00b07b thru 0x012b0d (0x7a93 bytes) 000389 1787496139.035287 mapread 0x0077de thru 0x016361 (0xeb84 bytes) 000392 1787496139.061114 mapread 0x0146ce thru 0x01a2ab (0x5bde bytes) 000393 1787496139.064057 trunc from 0x01a2ac to 0x00f919 (0xa992 bytes) 000396 1787496139.102397 write 0x01fa4e thru 0x025c85 (0x6238 bytes) 000397 1787496139.109414 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x25c86): Mirror 3: 0xf7a3637f (stale) Mirror 4: 0xf7a3637f (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x25c86): Mirror 3: 0xf7a3637f (stale) Mirror 4: 0xf7a3637f (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x25c86: skipped 000398 1787496139.412816 write 0x021cd3 thru 0x02b120 (0x944e bytes) 000400 1787496139.422963 read 0x014654 thru 0x01ab70 (0x651d bytes) 000401 1787496139.431986 mapwrite 0x03d403 thru 0x03efb1 (0x1baf bytes) 000402 1787496139.507836 read 0x006c70 thru 0x014226 (0xd5b7 bytes) 000405 1787496139.519262 read 0x02d819 thru 0x035d5c (0x8544 bytes) 000406 1787496139.520231 write 0x007309 thru 0x00ada2 (0x3a9a bytes) 000407 1787496139.522339 mapwrite 0x03a5a8 thru 0x03ffff (0x5a58 bytes) 000408 1787496139.583216 write 0x005e60 thru 0x00c3d3 (0x6574 bytes) 000409 1787496139.584593 write 0x00172b thru 0x008801 (0x70d7 bytes) 000411 1787496139.587406 trunc from 0x040000 to 0x026c99 (0x19366 bytes) 000412 1787496139.607174 read 0x014b01 thru 0x01cbd9 (0x80d9 bytes) 000413 1787496139.608332 read 0x007153 thru 0x010516 (0x93c4 bytes) 000414 1787496139.609739 mapread 0x013db7 thru 0x01c54f (0x8799 bytes) 000415 1787496139.611310 read 0x01a5cd thru 0x01ef36 (0x496a bytes) 000416 1787496139.615770 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x26c9a): Mirror 3: 0x15bf1be1 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x26c9a): Mirror 3: 0x15bf1be1 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x26c9a: skipped 000417 1787496139.892776 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x26c9a): Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x26c9a): Mirror 3: 0x15bf1be1 (stale) CRC-32 checksum value for chunk [0, 0x26c9a): Mirror 1: 0xf9e1063c Mirror 4: 0xf9e1063c lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x26c9a: skipped 000421 1787496140.903400 trunc from 0x026c9a to 0x005b62 (0x21137 bytes) 000422 1787496140.999353 read 0x000a1f thru 0x005b62 (0x5144 bytes) 000423 1787496141.010747 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x5b63): Mirror 3: 0x8ebd4308 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x5b63): Mirror 3: 0x8ebd4308 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x5b63: skipped 000424 1787496141.274520 write 0x01e39b thru 0x02d55f (0xf1c5 bytes) 000425 1787496141.281966 mapwrite 0x02ff71 thru 0x03b0fd (0xb18d bytes) 000426 1787496141.394966 mapread 0x034aae thru 0x03b0fd (0x6650 bytes) 000427 1787496141.400843 MIRROR_RESYNC 2 mirrors 000428 1787496142.118252 mapwrite 0x03e22e thru 0x03eb9d (0x970 bytes) 000429 1787496142.220715 write 0x012a4e thru 0x013c78 (0x122b bytes) 000431 1787496142.229927 mapwrite 0x008768 thru 0x00b7d3 (0x306c bytes) 000435 1787496142.284194 mapwrite 0x022adf thru 0x02d773 (0xac95 bytes) 000436 1787496142.342733 write 0x009561 thru 0x009b8c (0x62c bytes) 000438 1787496142.345397 mapwrite 0x00969e thru 0x00e2e0 (0x4c43 bytes) 000439 1787496142.394870 mapwrite 0x03016b thru 0x033780 (0x3616 bytes) 000441 1787496142.446435 trunc from 0x03eb9e to 0x021baf (0x1cfee bytes) 000444 1787496142.472963 trunc from 0x021bb0 to 0x00bf22 (0x15c8d bytes) 000446 1787496142.494658 write 0x03d757 thru 0x03ffff (0x28a9 bytes) 000449 1787496142.498398 read 0x00016c thru 0x00cf5d (0xcdf2 bytes) 000450 1787496142.507347 write 0x00e010 thru 0x017142 (0x9133 bytes) 000451 1787496142.518011 mapwrite 0x026c82 thru 0x035c56 (0xefd5 bytes) 000453 1787496142.594001 mapread 0x0192f3 thru 0x023e93 (0xaba1 bytes) 000454 1787496142.601023 mapread 0x024e89 thru 0x034906 (0xfa7e bytes) 000455 1787496142.602194 read 0x028deb thru 0x02d210 (0x4426 bytes) 000457 1787496142.610602 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) 000459 1787496142.775436 mapwrite 0x027767 thru 0x033a33 (0xc2cd bytes) 000460 1787496142.912967 trunc from 0x040000 to 0x037745 (0x88ba bytes) 000465 1787496142.960945 mapread 0x00ed78 thru 0x0180a1 (0x932a bytes) 000466 1787496142.994821 mapwrite 0x024236 thru 0x02f7e0 (0xb5ab bytes) 000469 1787496143.111521 write 0x0293ab thru 0x035217 (0xbe6d bytes) 000470 1787496143.115665 read 0x031aa2 thru 0x037745 (0x5ca4 bytes) 000472 1787496143.120426 mapread 0x0264d1 thru 0x02f19f (0x8ccf bytes) 000473 1787496143.121330 mapwrite 0x00010d thru 0x00dbc0 (0xdab4 bytes) 000476 1787496143.208163 trunc from 0x037746 to 0x010290 (0x274b5 bytes) 000477 1787496143.233319 read 0x002978 thru 0x00cb5e (0xa1e7 bytes) 000480 1787496143.235129 read 0x00ce1b thru 0x010290 (0x3476 bytes) 000482 1787496143.236260 mapwrite 0x026754 thru 0x02d9f9 (0x72a6 bytes) 000484 1787496143.335247 mapread 0x022ecc thru 0x02cdd4 (0x9f09 bytes) 000486 1787496143.337247 read 0x00cfa6 thru 0x016601 (0x965c bytes) 000487 1787496143.338336 trunc from 0x02d9fa to 0x01bf9f (0x11a5a bytes) 000488 1787496143.354021 write 0x002546 thru 0x006ae0 (0x459b bytes) 000490 1787496143.355497 trunc from 0x01bfa0 to 0x00ebb0 (0xd3ef bytes) 000491 1787496143.375015 trunc from 0x00ebb1 to 0x00b6f5 (0x34bb bytes) 000492 1787496143.393362 mapwrite 0x036624 thru 0x03ffff (0x99dc bytes) 000494 1787496143.473874 write 0x034658 thru 0x0394ab (0x4e54 bytes) 000495 1787496143.475695 read 0x01dd9e thru 0x02c072 (0xe2d5 bytes) 000499 1787496143.479613 mapread 0x02ea67 thru 0x02ee76 (0x410 bytes) 000500 1787496143.480787 read 0x0267e2 thru 0x02f54c (0x8d6b bytes) 000503 1787496143.482788 mapwrite 0x015c83 thru 0x024526 (0xe8a4 bytes) 000504 1787496143.558693 read 0x014ef6 thru 0x01ce7c (0x7f87 bytes) 000508 1787496143.563329 mapwrite 0x001e99 thru 0x0023cc (0x534 bytes) 000509 1787496143.603472 read 0x004cd7 thru 0x00fad5 (0xadff bytes) 000514 1787496143.608295 read 0x007f24 thru 0x00fe94 (0x7f71 bytes) 000515 1787496143.612495 MIRROR_EXTEND to 2 mirrors 000517 1787496144.315577 trunc from 0x040000 to 0x0279ef (0x18610 bytes) 000519 1787496144.414217 MIRROR_RESYNC 2 mirrors 000522 1787496145.184871 MIRROR_EXTEND to 3 mirrors 000524 1787496145.622347 read 0x013190 thru 0x014e55 (0x1cc6 bytes) 000525 1787496145.627283 mapread 0x02168a thru 0x0279ef (0x6366 bytes) 000526 1787496145.645243 read 0x001f15 thru 0x002529 (0x615 bytes) 000527 1787496145.659482 MIRROR_EXTEND to 4 mirrors 000530 1787496147.090456 trunc from 0x0279f0 to 0x0184a5 (0xf54a bytes) 000531 1787496147.166314 MIRROR_RESYNC 4 mirrors 000532 1787496148.350281 trunc from 0x0184a6 to 0x00523a (0x1326b bytes) 000534 1787496148.440560 write 0x02e633 thru 0x03d7de (0xf1ac bytes) 000536 1787496148.452031 read 0x0101e1 thru 0x019571 (0x9391 bytes) 000537 1787496148.468678 read 0x02c45c thru 0x034012 (0x7bb7 bytes) 000538 1787496148.482305 write 0x025c6e thru 0x0346ad (0xea40 bytes) 000539 1787496148.483923 read 0x03ad7e thru 0x03d7de (0x2a61 bytes) 000540 1787496148.484525 write 0x000797 thru 0x00cdd6 (0xc640 bytes) 000541 1787496148.490082 trunc from 0x03d7df to 0x0378e2 (0x5efc bytes) 000542 1787496148.509460 write 0x01c3ed thru 0x023275 (0x6e89 bytes) 000545 1787496148.510951 write 0x0104f5 thru 0x017201 (0x6d0d bytes) 000546 1787496148.512974 write 0x0312be thru 0x03a2f9 (0x903c bytes) truncating to largest ever: 0x3f841 000550 1787496148.515065 trunc from 0x03a2fa to 0x03f840 (0x5547 bytes) 000552 1787496148.576706 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3f841): Mirror 3: 0x5d25d07c (stale) Mirror 4: 0x5d25d07c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3f841): Mirror 3: 0x5d25d07c (stale) Mirror 4: 0x5d25d07c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3f841: skipped 000553 1787496148.981574 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x3f841): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3f841): Mirror 3: 0x5d25d07c (stale) Mirror 4: 0x5d25d07c (stale) CRC-32 checksum value for chunk [0, 0x3f841): Mirror 1: 0xa2f9c4c8 Mirror 5: 0xa2f9c4c8 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3f841: skipped 000554 1787496150.421643 mapwrite 0x01b1c6 thru 0x029725 (0xe560 bytes) 000555 1787496150.648062 trunc from 0x03f841 to 0x0160b3 (0x2978d bytes) 000556 1787496150.673830 read 0x00ca3f thru 0x0160b3 (0x9675 bytes) 000558 1787496150.687030 MIRROR_RESYNC 4 mirrors 000560 1787496151.883807 mapwrite 0x00a056 thru 0x00b19e (0x1149 bytes) 000561 1787496152.088725 write 0x03d75c thru 0x03ffff (0x28a4 bytes) 000562 1787496152.091241 write 0x027e53 thru 0x028f90 (0x113e bytes) 000563 1787496152.099420 mapread 0x03a619 thru 0x03ffff (0x59e7 bytes) 000565 1787496152.107855 trunc from 0x040000 to 0x0211dd (0x1ee22 bytes) 000566 1787496152.131427 read 0x01c60d thru 0x0211dd (0x4bd1 bytes) 000567 1787496152.142697 MIRROR_RESYNC 4 mirrors 000569 1787496153.314722 write 0x01e180 thru 0x02c25b (0xe0dc bytes) 000570 1787496153.407933 write 0x036ecf thru 0x03f983 (0x8ab5 bytes) 000571 1787496153.409498 mapwrite 0x030f4a thru 0x03d795 (0xc84c bytes) 000574 1787496153.525236 mapwrite 0x003a32 thru 0x011a01 (0xdfd0 bytes) 000577 1787496153.677847 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3f984): Mirror 3: 0x838b9b92 (stale) Mirror 4: 0x838b9b92 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3f984): Mirror 3: 0x838b9b92 (stale) Mirror 4: 0x838b9b92 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3f984: skipped 000578 1787496154.000521 read 0x02f57e thru 0x03ca0b (0xd48e bytes) 000580 1787496154.015775 write 0x0168a7 thru 0x01e171 (0x78cb bytes) 000581 1787496154.027323 trunc from 0x03f984 to 0x00b6de (0x342a5 bytes) 000582 1787496154.072219 read 0x00b670 thru 0x00b6de (0x6f bytes) 000585 1787496154.083115 mapwrite 0x03e743 thru 0x03ffff (0x18bd bytes) 000586 1787496154.214911 mapread 0x02cb0d thru 0x03623e (0x9732 bytes) 000587 1787496154.216307 write 0x03ad4e thru 0x03e122 (0x33d5 bytes) 000588 1787496154.241761 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x8a091e56 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x8a091e56 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000589 1787496154.673991 read 0x012081 thru 0x01a2f4 (0x8274 bytes) 000590 1787496154.683420 MIRROR_RESYNC 2 mirrors 000592 1787496155.655081 trunc from 0x040000 to 0x013235 (0x2cdca bytes) 000593 1787496155.775085 trunc from 0x013236 to 0x007879 (0xb9bc bytes) 000597 1787496155.835481 trunc from 0x00787a to 0x02cec8 (0x2564f bytes) 000598 1787496155.875628 mapwrite 0x00bb7f thru 0x019021 (0xd4a3 bytes) 000601 1787496155.996035 mapwrite 0x03e207 thru 0x03fd44 (0x1b3e bytes) 000602 1787496156.083994 mapwrite 0x00b54c thru 0x013f78 (0x8a2d bytes) 000604 1787496156.145626 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) 000605 1787496156.298865 MIRROR_EXTEND to 2 mirrors 000606 1787496157.315905 write 0x035d1e thru 0x03ffff (0xa2e2 bytes) 000607 1787496157.397239 mapread 0x021645 thru 0x02d510 (0xbecc bytes) 000609 1787496157.432746 read 0x01f518 thru 0x0279f4 (0x84dd bytes) 000610 1787496157.433672 write 0x01522e thru 0x0163ec (0x11bf bytes) 000612 1787496157.450368 mapread 0x033d1b thru 0x039465 (0x574b bytes) 000613 1787496157.451777 trunc from 0x040000 to 0x0144b1 (0x2bb4e bytes) 000615 1787496157.475797 read 0x00c676 thru 0x0144b1 (0x7e3c bytes) 000616 1787496157.484707 mapwrite 0x00fb62 thru 0x012cfa (0x3199 bytes) 000618 1787496157.541200 mapwrite 0x01eab6 thru 0x02e786 (0xfcd1 bytes) 000620 1787496157.747266 mapread 0x004ef1 thru 0x00a374 (0x5484 bytes) 000622 1787496157.751540 read 0x006ea9 thru 0x0130af (0xc207 bytes) 000623 1787496157.752348 trunc from 0x02e787 to 0x015ef2 (0x18894 bytes) 000624 1787496157.783869 trunc from 0x015ef3 to 0x030ab4 (0x1abc2 bytes) 000625 1787496157.849931 write 0x02b714 thru 0x035d53 (0xa640 bytes) 000626 1787496157.908065 MIRROR_RESYNC 2 mirrors 000629 1787496159.430432 mapread 0x0342eb thru 0x035d53 (0x1a69 bytes) 000630 1787496159.452728 MIRROR_RESYNC 2 mirrors 000631 1787496160.132766 MIRROR_RESYNC 2 mirrors 000632 1787496160.646255 mapread 0x00da50 thru 0x01d377 (0xf928 bytes) 000634 1787496160.688391 MIRROR_RESYNC 2 mirrors 000636 1787496160.989943 mapwrite 0x0149be thru 0x01af79 (0x65bc bytes) 000638 1787496161.149827 read 0x01e543 thru 0x02c11b (0xdbd9 bytes) 000640 1787496161.159907 mapread 0x025ff9 thru 0x02ee9b (0x8ea3 bytes) 000642 1787496161.169794 write 0x017ed2 thru 0x0261e0 (0xe30f bytes) 000644 1787496161.172597 read 0x0156bd thru 0x020726 (0xb06a bytes) 000645 1787496161.178394 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x35d54): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x35d54): Mirror 2: 0x43f65282 (stale) CRC-32 checksum value for chunk [0, 0x35d54): Mirror 1: 0x29ef8296 Mirror 3: 0x29ef8296 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x35d54: skipped 000648 1787496162.107854 mapread 0x004344 thru 0x00b1f9 (0x6eb6 bytes) 000649 1787496162.127999 trunc from 0x035d54 to 0x000d6d (0x34fe6 bytes) 000650 1787496162.257773 mapread 0x0003d5 thru 0x000d6d (0x999 bytes) 000652 1787496162.264656 write 0x00de4e thru 0x019e0a (0xbfbd bytes) 000653 1787496162.266936 write 0x02c3cd thru 0x02e2d4 (0x1f08 bytes) 000654 1787496162.273363 MIRROR_RESYNC 3 mirrors 000655 1787496163.876015 mapwrite 0x010b87 thru 0x01efe4 (0xe45e bytes) 000656 1787496164.068184 mapwrite 0x026fe1 thru 0x031f00 (0xaf20 bytes) 000657 1787496164.275767 trunc from 0x031f01 to 0x00dfd9 (0x23f27 bytes) 000658 1787496164.313950 write 0x00a716 thru 0x00f2c1 (0x4bac bytes) 000662 1787496164.330242 MIRROR_RESYNC 3 mirrors 000664 1787496165.594652 trunc from 0x00f2c2 to 0x02c444 (0x1d183 bytes) 000666 1787496165.685743 mapwrite 0x03bdeb thru 0x03fcd0 (0x3ee6 bytes) 000669 1787496165.936828 mapread 0x014fae thru 0x01507e (0xd1 bytes) 000670 1787496165.947786 mapread 0x03039d thru 0x0391a6 (0x8e0a bytes) 000672 1787496165.958380 write 0x020a0a thru 0x02255a (0x1b51 bytes) 000673 1787496165.968554 write 0x018619 thru 0x01beef (0x38d7 bytes) 000674 1787496165.969831 read 0x0134a1 thru 0x01f784 (0xc2e4 bytes) 000679 1787496165.981611 mapwrite 0x029dbb thru 0x02a208 (0x44e bytes) 000680 1787496166.109537 read 0x0203e5 thru 0x023e31 (0x3a4d bytes) 000681 1787496166.120726 read 0x00ec86 thru 0x019cc4 (0xb03f bytes) 000682 1787496166.121601 read 0x0021da thru 0x0116b3 (0xf4da bytes) 000687 1787496166.125624 mapread 0x0363a8 thru 0x039728 (0x3381 bytes) 000689 1787496166.127126 trunc from 0x03fcd1 to 0x0159c6 (0x2a30a bytes) 000690 1787496166.155804 MIRROR_RESYNC 3 mirrors 000692 1787496167.068784 write 0x001a8e thru 0x00388f (0x1e02 bytes) 000694 1787496167.157665 read 0x009120 thru 0x0159c6 (0xc8a7 bytes) 000697 1787496167.172792 write 0x01c1d9 thru 0x01eb3b (0x2963 bytes) 000699 1787496167.176835 mapread 0x0030d9 thru 0x007090 (0x3fb8 bytes) 000700 1787496167.200316 mapwrite 0x007121 thru 0x011385 (0xa265 bytes) 000702 1787496167.264213 mapread 0x0094f8 thru 0x019386 (0xfe8f bytes) 000706 1787496167.276532 mapread 0x01bb79 thru 0x01eb3b (0x2fc3 bytes) 000707 1787496167.282507 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1eb3c): Mirror 3: 0x21ef0693 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1eb3c): Mirror 3: 0x21ef0693 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1eb3c: skipped 000711 1787496167.598664 read 0x01a464 thru 0x01be38 (0x19d5 bytes) 000712 1787496167.603886 mapwrite 0x02491e thru 0x0343bd (0xfaa0 bytes) 000714 1787496167.739012 mapread 0x0306a1 thru 0x0343bd (0x3d1d bytes) 000715 1787496167.740880 write 0x018595 thru 0x021204 (0x8c70 bytes) 000717 1787496167.747575 read 0x02509f thru 0x028ebb (0x3e1d bytes) 000720 1787496167.751969 write 0x004bac thru 0x00c5c4 (0x7a19 bytes) 000721 1787496167.762825 mapwrite 0x017045 thru 0x01c8c9 (0x5885 bytes) 000722 1787496167.827923 read 0x00102e thru 0x007638 (0x660b bytes) 000723 1787496167.828916 mapwrite 0x005998 thru 0x0147a1 (0xee0a bytes) 000724 1787496167.916503 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) 000725 1787496168.064053 write 0x02be71 thru 0x02f1de (0x336e bytes) 000727 1787496168.079008 trunc from 0x0343be to 0x02e45f (0x5f5e bytes) 000729 1787496168.098742 mapwrite 0x02409d thru 0x027f3c (0x3ea0 bytes) 000730 1787496168.182241 mapwrite 0x019465 thru 0x01bab5 (0x2651 bytes) 000731 1787496168.243460 mapwrite 0x00e955 thru 0x01e6cf (0xfd7b bytes) 000733 1787496168.338800 mapwrite 0x02a8dc thru 0x038efb (0xe620 bytes) 000737 1787496168.530664 mapread 0x008aa3 thru 0x00b2df (0x283d bytes) 000740 1787496168.540132 mapwrite 0x030340 thru 0x03761c (0x72dd bytes) 000741 1787496168.627628 mapwrite 0x024f49 thru 0x0255c7 (0x67f bytes) 000744 1787496168.680636 write 0x0073a3 thru 0x0138f2 (0xc550 bytes) 000745 1787496168.686126 mapwrite 0x003b82 thru 0x00cc3e (0x90bd bytes) 000746 1787496168.811522 write 0x030fda thru 0x03a317 (0x933e bytes) 000749 1787496168.820078 read 0x027227 thru 0x035110 (0xdeea bytes) 000750 1787496168.827720 MIRROR_EXTEND to 2 mirrors 000751 1787496169.774562 mapread 0x0340c3 thru 0x03a317 (0x6255 bytes) truncating to largest ever: 0x3fe7c 000753 1787496169.818426 trunc from 0x03a318 to 0x03fe7b (0x5b64 bytes) 000754 1787496169.980538 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3fe7c): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3fe7c): Mirror 2: 0x9702d2a1 (stale) CRC-32 checksum value for chunk [0, 0x3fe7c): Mirror 1: 0x9702d2a1 Mirror 3: 0x9702d2a1 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3fe7c: skipped 000755 1787496171.882705 read 0x01cdbd thru 0x01fedd (0x3121 bytes) 000758 1787496171.908580 mapwrite 0x01a88c thru 0x01cf93 (0x2708 bytes) 000763 1787496172.140379 mapwrite 0x038feb thru 0x03ffff (0x7015 bytes) 000764 1787496172.265385 mapwrite 0x007d5d thru 0x0164b5 (0xe759 bytes) 000765 1787496172.328553 write 0x00094f thru 0x0067d2 (0x5e84 bytes) 000766 1787496172.334698 mapread 0x029df6 thru 0x02e336 (0x4541 bytes) 000767 1787496172.342885 mapwrite 0x00cf96 thru 0x012327 (0x5392 bytes) 000768 1787496172.391594 trunc from 0x040000 to 0x01842d (0x27bd2 bytes) 000770 1787496172.409468 mapwrite 0x008f00 thru 0x017458 (0xe559 bytes) 000773 1787496172.463210 mapread 0x01109c thru 0x01842d (0x7392 bytes) 000774 1787496172.465817 read 0x0083bf thru 0x00ed96 (0x69d8 bytes) 000775 1787496172.467605 read 0x014680 thru 0x01842d (0x3dae bytes) 000776 1787496172.468550 read 0x017a2a thru 0x01842d (0xa04 bytes) 000777 1787496172.469890 write 0x03f9e9 thru 0x03ffff (0x617 bytes) 000779 1787496172.473590 mapread 0x016818 thru 0x02657a (0xfd63 bytes) 000782 1787496172.486062 mapread 0x00aab1 thru 0x01229f (0x77ef bytes) 000783 1787496172.487536 trunc from 0x040000 to 0x0093e3 (0x36c1c bytes) 000784 1787496172.504757 mapread 0x000968 thru 0x0063ad (0x5a46 bytes) 000786 1787496172.511721 mapread 0x000556 thru 0x0093e3 (0x8e8e bytes) 000787 1787496172.512931 mapwrite 0x016359 thru 0x02442a (0xe0d2 bytes) 000788 1787496172.697003 MIRROR_RESYNC 3 mirrors 000789 1787496173.828978 mapwrite 0x00be97 thru 0x01b5d1 (0xf73b bytes) 000791 1787496174.072548 mapwrite 0x0043b2 thru 0x00bff7 (0x7c46 bytes) 000794 1787496174.132632 mapread 0x014d8d thru 0x01da7b (0x8cef bytes) 000795 1787496174.139758 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2442b): Mirror 2: 0xb901d54a (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2442b): Mirror 2: 0xb901d54a (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2442b: skipped 000796 1787496174.462679 read 0x016eb2 thru 0x0234af (0xc5fe bytes) 000799 1787496174.473988 write 0x012995 thru 0x016a6d (0x40d9 bytes) 000802 1787496174.483008 trunc from 0x02442b to 0x00e074 (0x163b6 bytes) 000803 1787496174.497953 mapwrite 0x027851 thru 0x02aa3a (0x31ea bytes) 000804 1787496174.572492 read 0x0118b6 thru 0x01250b (0xc56 bytes) 000805 1787496174.584331 read 0x010639 thru 0x01aea7 (0xa86f bytes) 000806 1787496174.585249 trunc from 0x02aa3b to 0x035cf2 (0xb2b8 bytes) 000807 1787496174.602824 trunc from 0x035cf3 to 0x031cdd (0x4015 bytes) 000809 1787496174.621771 read 0x012499 thru 0x017da1 (0x5909 bytes) 000811 1787496174.623424 mapread 0x02217d thru 0x02b3b5 (0x9239 bytes) 000812 1787496174.636657 read 0x015e14 thru 0x01ed20 (0x8f0d bytes) 000813 1787496174.637516 read 0x010d46 thru 0x01fa6d (0xed28 bytes) 000814 1787496174.638433 read 0x029c9d thru 0x030a01 (0x6d65 bytes) 000817 1787496174.646835 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) 000818 1787496174.847924 write 0x007efe thru 0x00c678 (0x477b bytes) 000819 1787496174.877848 read 0x020aa6 thru 0x02fa65 (0xefc0 bytes) 000820 1787496174.889794 mapwrite 0x02f7b1 thru 0x037f01 (0x8751 bytes) 000821 1787496175.057616 trunc from 0x037f02 to 0x00ffc6 (0x27f3b bytes) 000823 1787496175.095809 write 0x0200bf thru 0x021e13 (0x1d55 bytes) 000826 1787496175.100243 trunc from 0x021e14 to 0x015f71 (0xbea2 bytes) 000827 1787496175.133803 read 0x0152bc thru 0x015f71 (0xcb6 bytes) 000831 1787496175.149484 trunc from 0x015f72 to 0x03184f (0x1b8de bytes) 000835 1787496175.168354 mapread 0x0065a8 thru 0x00d882 (0x72db bytes) 000836 1787496175.170754 mapwrite 0x03e1a2 thru 0x03ffff (0x1e5e bytes) 000837 1787496175.278320 trunc from 0x040000 to 0x01acca (0x25335 bytes) 000838 1787496175.303323 read 0x01a08c thru 0x01acca (0xc3f bytes) 000839 1787496175.304202 mapread 0x016413 thru 0x01acca (0x48b8 bytes) 000840 1787496175.305857 trunc from 0x01accb to 0x00e65a (0xc670 bytes) 000842 1787496175.330707 mapwrite 0x00eb44 thru 0x00fe15 (0x12d2 bytes) 000843 1787496175.420944 trunc from 0x00fe16 to 0x013bb2 (0x3d9d bytes) 000844 1787496175.438991 read 0x013227 thru 0x013bb2 (0x98c bytes) 000845 1787496175.450701 read 0x00de2c thru 0x013bb2 (0x5d87 bytes) 000846 1787496175.451876 trunc from 0x013bb3 to 0x03b076 (0x274c4 bytes) 000847 1787496175.469255 read 0x004874 thru 0x00eabd (0xa24a bytes) 000848 1787496175.470773 write 0x03d6fe thru 0x03ffff (0x2902 bytes) 000850 1787496175.474374 mapread 0x022d7a thru 0x02f3b1 (0xc638 bytes) 000854 1787496175.488387 mapwrite 0x03be45 thru 0x03ffff (0x41bb bytes) 000855 1787496175.551200 write 0x020d7c thru 0x02808c (0x7311 bytes) 000857 1787496175.552959 mapwrite 0x03b2e2 thru 0x03ffff (0x4d1e bytes) 000858 1787496175.603202 mapread 0x028f3c thru 0x02b6bc (0x2781 bytes) 000863 1787496175.611621 mapread 0x03ca6b thru 0x03ffff (0x3595 bytes) 000864 1787496175.613476 mapwrite 0x02de91 thru 0x030658 (0x27c8 bytes) 000865 1787496175.665823 mapwrite 0x02bb68 thru 0x033ce6 (0x817f bytes) 000867 1787496175.727994 trunc from 0x040000 to 0x02e9d8 (0x11627 bytes) 000868 1787496175.751441 mapwrite 0x02224b thru 0x02d865 (0xb61b bytes) 000869 1787496175.809224 trunc from 0x02e9d9 to 0x02c429 (0x25af bytes) 000870 1787496175.830445 write 0x03ea77 thru 0x03ffff (0x1589 bytes) 000871 1787496175.832371 mapread 0x034221 thru 0x03770c (0x34ec bytes) 000873 1787496175.857425 write 0x01b534 thru 0x01e783 (0x3250 bytes) 000874 1787496175.859583 trunc from 0x040000 to 0x01d61e (0x229e1 bytes) 000876 1787496175.878373 mapwrite 0x02e7e8 thru 0x03b68a (0xcea3 bytes) 000879 1787496176.036665 mapread 0x01dfcd thru 0x02c090 (0xe0c4 bytes) 000880 1787496176.120273 read 0x019da5 thru 0x028d92 (0xefee bytes) 000881 1787496176.121616 trunc from 0x03b68b to 0x00a184 (0x31506 bytes) 000882 1787496176.149616 write 0x028d62 thru 0x0360a9 (0xd348 bytes) 000883 1787496176.156463 write 0x01c73e thru 0x02b0b4 (0xe977 bytes) 000884 1787496176.169360 read 0x031645 thru 0x0360a9 (0x4a65 bytes) 000886 1787496176.171738 write 0x015183 thru 0x0223ab (0xd229 bytes) 000887 1787496176.174758 mapwrite 0x000698 thru 0x000d22 (0x68b bytes) 000888 1787496176.232127 mapread 0x02d160 thru 0x034d7b (0x7c1c bytes) 000890 1787496176.233641 read 0x01b456 thru 0x01d7df (0x238a bytes) 000891 1787496176.234210 mapread 0x014097 thru 0x0207e8 (0xc752 bytes) 000893 1787496176.241575 mapread 0x011891 thru 0x01ca79 (0xb1e9 bytes) 000894 1787496176.243311 trunc from 0x0360aa to 0x032645 (0x3a64 bytes) 000895 1787496176.261533 write 0x00dc9a thru 0x011faf (0x4316 bytes) 000896 1787496176.262834 trunc from 0x032646 to 0x00ab2c (0x27b19 bytes) 000899 1787496176.286334 trunc from 0x00ab2d to 0x00eb40 (0x4014 bytes) 000900 1787496176.309711 write 0x015907 thru 0x022ec9 (0xd5c3 bytes) 000901 1787496176.314055 mapwrite 0x036c88 thru 0x038b40 (0x1eb9 bytes) 000904 1787496176.453957 read 0x00e6ca thru 0x0107a5 (0x20dc bytes) 000909 1787496176.457321 mapwrite 0x005779 thru 0x006aa5 (0x132d bytes) 000911 1787496176.511417 read 0x00635b thru 0x00cbe5 (0x688b bytes) 000912 1787496176.512799 mapread 0x0363ce thru 0x038b40 (0x2773 bytes) 000914 1787496176.515463 read 0x0314da thru 0x038b40 (0x7667 bytes) 000915 1787496176.519076 mapwrite 0x01261d thru 0x015fe4 (0x39c8 bytes) 000920 1787496176.595695 trunc from 0x038b41 to 0x02e84c (0xa2f4 bytes) 000921 1787496176.613297 mapwrite 0x00a9bd thru 0x01093e (0x5f82 bytes) 000922 1787496176.655318 write 0x01ff23 thru 0x025183 (0x5261 bytes) 000923 1787496176.656531 trunc from 0x02e84d to 0x038fcb (0xa77f bytes) 000925 1787496176.671523 trunc from 0x038fcc to 0x029f7d (0xf04e bytes) 000926 1787496176.688265 mapwrite 0x03efed thru 0x03ffff (0x1013 bytes) 000928 1787496176.765055 mapwrite 0x01fefe thru 0x02849a (0x859d bytes) 000931 1787496176.828848 mapread 0x030193 thru 0x0330f0 (0x2f5e bytes) 000933 1787496176.830624 trunc from 0x040000 to 0x00c7b1 (0x3384e bytes) 000935 1787496176.861363 read 0x00947b thru 0x00c7b1 (0x3337 bytes) 000938 1787496176.867748 mapread 0x008e91 thru 0x009505 (0x675 bytes) 000941 1787496176.888592 mapwrite 0x032fcb thru 0x035bcf (0x2c05 bytes) 000942 1787496177.023661 mapread 0x009aa0 thru 0x0166f5 (0xcc56 bytes) 000943 1787496177.031023 mapwrite 0x036e39 thru 0x03ffff (0x91c7 bytes) 000944 1787496177.207507 mapread 0x031fe9 thru 0x03d570 (0xb588 bytes) 000945 1787496177.210352 mapread 0x02cb60 thru 0x0319ac (0x4e4d bytes) 000947 1787496177.215675 mapread 0x022919 thru 0x026b93 (0x427b bytes) 000949 1787496177.218741 mapread 0x00265d thru 0x00744b (0x4def bytes) 000950 1787496177.224730 MIRROR_EXTEND to 2 mirrors 000955 1787496178.384634 mapwrite 0x03a0b9 thru 0x03ffff (0x5f47 bytes) 000956 1787496178.666951 write 0x00488e thru 0x008903 (0x4076 bytes) 000959 1787496178.696594 mapwrite 0x038245 thru 0x03ffff (0x7dbb bytes) 000960 1787496178.765788 trunc from 0x040000 to 0x0328a3 (0xd75c bytes) 000961 1787496178.827420 write 0x022334 thru 0x023adb (0x17a8 bytes) 000963 1787496178.848699 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) 000964 1787496179.012836 write 0x03d14f thru 0x03ffff (0x2eb1 bytes) 000965 1787496179.028286 MIRROR_EXTEND to 2 mirrors 000970 1787496180.321684 read 0x02d4e2 thru 0x03a5f5 (0xd114 bytes) 000972 1787496180.350217 trunc from 0x040000 to 0x016c47 (0x293b8 bytes) 000973 1787496180.415578 mapwrite 0x017b19 thru 0x02262d (0xab15 bytes) 000974 1787496180.528553 read 0x011361 thru 0x020bb3 (0xf853 bytes) 000975 1787496180.542318 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x182:0x0]: Invalid argument (22) 000976 1787496180.694020 trunc from 0x02262e to 0x0398ce (0x172a1 bytes) 000977 1787496180.710698 trunc from 0x0398cf to 0x03fd23 (0x6455 bytes) 000979 1787496180.729974 write 0x026cb3 thru 0x028765 (0x1ab3 bytes) 000981 1787496180.737462 trunc from 0x03fd24 to 0x03c55a (0x37c9 bytes) 000986 1787496180.759428 mapread 0x025fa9 thru 0x02b6b2 (0x570a bytes) 000987 1787496180.775794 mapwrite 0x0362fa thru 0x03c8a9 (0x65b0 bytes) 000991 1787496180.846946 MIRROR_EXTEND to 2 mirrors 000992 1787496181.804648 trunc from 0x03c8aa to 0x003aca (0x38ddf bytes) 000996 1787496181.923916 write 0x00e95f thru 0x012c69 (0x430b bytes) 000998 1787496181.926351 mapread 0x0023ae thru 0x01212e (0xfd81 bytes) 000999 1787496181.950682 trunc from 0x012c6a to 0x02c99d (0x19d34 bytes) All operations completed A-OK!