== sanity-flr test 70a: flr mode fsx test ================ 18:53:14 (1787698394) Seed set to 8395 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000002 1787698395.814224 MIRROR_EXTEND to 2 mirrors skipping zero size read truncating to largest ever: 0x11f3c 000006 1787698396.164545 trunc from 00000000 to 0x011f3b (0x11f3c bytes) 000009 1787698396.275397 MIRROR_RESYNC 2 mirrors 000010 1787698396.933585 write 0x000605 thru 0x000e9b (0x897 bytes) truncating to largest ever: 0x2a3e3 000011 1787698397.027887 trunc from 0x011f3c to 0x02a3e2 (0x184a7 bytes) 000012 1787698397.053656 read 0x005859 thru 0x007b0f (0x22b7 bytes) 000014 1787698397.059139 mapread 0x019f07 thru 0x027cd2 (0xddcc bytes) 000015 1787698397.086142 mapwrite 0x002af3 thru 0x008c87 (0x6195 bytes) 000017 1787698397.140848 mapwrite 0x01f91d thru 0x02def6 (0xe5da bytes) 000019 1787698397.221842 mapwrite 0x0084f7 thru 0x01119e (0x8ca8 bytes) 000023 1787698397.331526 mapread 0x024b4f thru 0x02bf59 (0x740b bytes) 000024 1787698397.341994 read 0x0044d7 thru 0x0093d4 (0x4efe bytes) 000029 1787698397.348204 mapwrite 0x0087fe thru 0x011c0d (0x9410 bytes) 000030 1787698397.417178 read 0x014561 thru 0x020717 (0xc1b7 bytes) 000034 1787698397.432439 MIRROR_RESYNC 2 mirrors truncating to largest ever: 0x32323 000035 1787698397.987631 trunc from 0x02def7 to 0x032322 (0x442c bytes) 000037 1787698398.079707 mapread 0x013631 thru 0x01a25f (0x6c2f bytes) 000038 1787698398.102066 write 0x034415 thru 0x035c7f (0x186b bytes) 000039 1787698398.103160 read 0x009bc7 thru 0x00bc73 (0x20ad bytes) 000041 1787698398.114335 mapwrite 0x019c7c thru 0x01d111 (0x3496 bytes) 000042 1787698398.187446 MIRROR_RESYNC 2 mirrors 000044 1787698398.825110 read 0x03196a thru 0x035c7f (0x4316 bytes) 000045 1787698398.841140 MIRROR_RESYNC 2 mirrors 000047 1787698399.135055 write 0x033873 thru 0x03fa97 (0xc225 bytes) 000048 1787698399.207412 MIRROR_RESYNC 2 mirrors 000049 1787698399.816934 trunc from 0x03fa98 to 0x02a66c (0x1542b bytes) 000050 1787698399.889543 MIRROR_RESYNC 2 mirrors 000052 1787698400.603653 mapwrite 0x007133 thru 0x014c01 (0xdacf bytes) 000056 1787698400.806296 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x2a66d): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2a66d): Mirror 2: 0xf35287a2 (stale) CRC-32 checksum value for chunk [0, 0x2a66d): Mirror 1: 0xac5bd9e Mirror 3: 0xac5bd9e lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2a66d: skipped 000057 1787698401.512903 write 0x02283a thru 0x028748 (0x5f0f bytes) 000058 1787698401.612367 write 0x0004e6 thru 0x004b09 (0x4624 bytes) 000060 1787698401.624791 read 0x0020b1 thru 0x00f776 (0xd6c6 bytes) 000061 1787698401.645174 mapread 0x004db3 thru 0x00eb05 (0x9d53 bytes) 000063 1787698401.649790 mapread 0x0115e2 thru 0x01b660 (0xa07f bytes) 000064 1787698401.699599 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2a66d): Mirror 3: 0xac5bd9e (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2a66d): Mirror 3: 0xac5bd9e (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2a66d: skipped 000068 1787698402.061193 read 0x007d3f thru 0x00fb34 (0x7df6 bytes) 000069 1787698402.076500 trunc from 0x02a66d to 0x01d719 (0xcf53 bytes) 000070 1787698402.106106 write 0x0199a2 thru 0x01f185 (0x57e4 bytes) 000071 1787698402.115546 mapwrite 0x03df10 thru 0x03ffff (0x20f0 bytes) 000072 1787698402.274994 read 0x02d46f thru 0x0361fe (0x8d90 bytes) 000074 1787698402.323093 mapread 0x038e96 thru 0x03ffff (0x716a bytes) 000076 1787698402.330176 write 0x020e4a thru 0x020eec (0xa3 bytes) 000079 1787698402.332987 read 0x00594b thru 0x010238 (0xa8ee bytes) 000080 1787698402.335476 trunc from 0x040000 to 0x025223 (0x1addc bytes) 000081 1787698402.367843 mapwrite 0x00c906 thru 0x011128 (0x4823 bytes) 000083 1787698402.424444 read 0x0168e4 thru 0x01b004 (0x4721 bytes) 000085 1787698402.425537 mapread 0x010624 thru 0x014041 (0x3a1e bytes) 000086 1787698402.426903 read 0x016b8b thru 0x017c5e (0x10d4 bytes) 000092 1787698402.430807 mapwrite 0x032030 thru 0x03f8d2 (0xd8a3 bytes) 000093 1787698402.541358 mapwrite 0x01a837 thru 0x01bc8c (0x1456 bytes) 000094 1787698402.610819 write 0x03216d thru 0x03f092 (0xcf26 bytes) 000095 1787698402.612049 mapwrite 0x00f052 thru 0x013898 (0x4847 bytes) 000096 1787698402.678500 write 0x00111d thru 0x00cb5a (0xba3e bytes) 000099 1787698402.680618 trunc from 0x03f8d3 to 0x009fa7 (0x3592b bytes) 000103 1787698402.730749 read 0x009953 thru 0x009fa7 (0x655 bytes) 000106 1787698402.740781 mapread 0x002256 thru 0x009fa7 (0x7d52 bytes) 000108 1787698402.762854 MIRROR_RESYNC 2 mirrors 000109 1787698403.609147 mapread 0x00901e thru 0x009fa7 (0xf8a bytes) 000111 1787698403.618065 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) 000112 1787698403.750348 mapread 0x00682d thru 0x009e7f (0x3653 bytes) 000113 1787698403.771764 trunc from 0x009fa8 to 0x0009e5 (0x95c2 bytes) 000114 1787698403.812628 mapwrite 0x00020e thru 0x009c65 (0x9a58 bytes) 000115 1787698403.936828 MIRROR_EXTEND to 2 mirrors 000116 1787698404.617532 mapwrite 0x03e1db thru 0x03ffff (0x1e25 bytes) 000118 1787698404.739172 mapread 0x03772e thru 0x03eb11 (0x73e4 bytes) 000120 1787698404.751677 trunc from 0x040000 to 0x02ba4a (0x145b5 bytes) 000123 1787698404.774464 MIRROR_RESYNC 2 mirrors 000124 1787698405.662207 mapwrite 0x00e590 thru 0x0165a2 (0x8013 bytes) 000127 1787698406.101689 mapread 0x00a154 thru 0x00b115 (0xfc2 bytes) 000128 1787698406.114120 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x2ba4b): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2ba4b): Mirror 2: 0x6d242d6d (stale) CRC-32 checksum value for chunk [0, 0x2ba4b): Mirror 1: 0xc3fa7053 Mirror 3: 0xc3fa7053 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2ba4b: skipped 000129 1787698407.441295 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2ba4b): Mirror 2: 0x6d242d6d (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2ba4b): Mirror 2: 0x6d242d6d (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2ba4b: skipped 000130 1787698407.806630 mapread 0x0047e3 thru 0x00e131 (0x994f bytes) 000131 1787698407.872224 trunc from 0x02ba4b to 0x02f260 (0x3816 bytes) 000132 1787698407.902251 mapread 0x008733 thru 0x015de7 (0xd6b5 bytes) 000134 1787698407.932801 trunc from 0x02f261 to 0x02d25e (0x2002 bytes) 000136 1787698407.979371 mapwrite 0x038750 thru 0x03d444 (0x4cf5 bytes) 000137 1787698408.076153 trunc from 0x03d445 to 0x02c6cd (0x10d77 bytes) 000141 1787698408.094273 read 0x000623 thru 0x001d14 (0x16f2 bytes) 000142 1787698408.099464 mapread 0x023b81 thru 0x028ca7 (0x5127 bytes) 000143 1787698408.108132 write 0x029a16 thru 0x039a10 (0xfffb bytes) 000144 1787698408.109830 mapwrite 0x02bec3 thru 0x02e296 (0x23d4 bytes) 000146 1787698408.146711 mapread 0x000cb8 thru 0x005eca (0x5213 bytes) 000148 1787698408.148907 trunc from 0x039a11 to 0x0210da (0x18936 bytes) 000149 1787698408.163059 mapwrite 0x006d8d thru 0x015fa5 (0xf219 bytes) 000151 1787698408.209582 mapwrite 0x003ec2 thru 0x004741 (0x880 bytes) 000152 1787698408.248116 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x210db): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x210db): Mirror 2: 0x526d6814 (stale) CRC-32 checksum value for chunk [0, 0x210db): Mirror 1: 0xca1dad48 Mirror 3: 0xca1dad48 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x210db: skipped 000155 1787698409.228281 mapread 0x01f8d2 thru 0x0210da (0x1809 bytes) 000157 1787698409.253662 trunc from 0x0210db to 0x01007f (0x1105b bytes) 000158 1787698409.428971 mapread 0x000394 thru 0x00c87b (0xc4e8 bytes) 000159 1787698409.437681 write 0x021ad4 thru 0x02cfba (0xb4e7 bytes) 000162 1787698409.442873 write 0x02b31e thru 0x039a1d (0xe700 bytes) 000163 1787698409.444552 read 0x005b10 thru 0x00beb4 (0x63a5 bytes) 000164 1787698409.446327 read 0x0237d3 thru 0x02a119 (0x6947 bytes) 000165 1787698409.455657 MIRROR_RESYNC 3 mirrors truncating to largest ever: 0x3f98a 000166 1787698410.536007 trunc from 0x039a1e to 0x03f989 (0x5f6c bytes) 000167 1787698410.605771 write 0x02a37f thru 0x036736 (0xc3b8 bytes) 000170 1787698410.621255 mapread 0x02ff99 thru 0x03f989 (0xf9f1 bytes) 000171 1787698410.651302 mapwrite 0x029750 thru 0x02c403 (0x2cb4 bytes) 000173 1787698410.719191 trunc from 0x03f98a to 0x03583d (0xa14c bytes) 000174 1787698410.746666 mapwrite 0x00455f thru 0x00dffd (0x9a9f bytes) 000176 1787698410.804306 mapread 0x029d95 thru 0x03068b (0x68f7 bytes) 000177 1787698410.805576 write 0x03cbb7 thru 0x03ffff (0x3449 bytes) 000178 1787698410.807099 trunc from 0x040000 to 0x0193ba (0x26c45 bytes) 000184 1787698410.846831 mapread 0x011d7f thru 0x0193ba (0x763c bytes) 000185 1787698410.886686 write 0x026036 thru 0x035b44 (0xfb0f bytes) 000186 1787698410.890827 mapread 0x013939 thru 0x017af3 (0x41bb bytes) 000188 1787698410.893288 read 0x03371f thru 0x035b44 (0x2426 bytes) 000190 1787698410.900094 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x35b45): Mirror 2: 0xb079c2c5 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x35b45): Mirror 2: 0xb079c2c5 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x35b45: skipped 000195 1787698411.231705 mapwrite 0x030163 thru 0x038651 (0x84ef bytes) 000196 1787698411.446643 read 0x029d56 thru 0x038651 (0xe8fc bytes) 000197 1787698411.455289 mapwrite 0x00ae46 thru 0x00f30e (0x44c9 bytes) 000198 1787698411.559637 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x38652): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x38652): Mirror 2: 0xaff88503 (stale) CRC-32 checksum value for chunk [0, 0x38652): Mirror 1: 0xc59ef1a8 Mirror 3: 0xc59ef1a8 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x38652: skipped 000199 1787698412.530101 mapwrite 0x0316c2 thru 0x03c62a (0xaf69 bytes) 000200 1787698412.867568 mapwrite 0x033cbf thru 0x03d18e (0x94d0 bytes) 000201 1787698412.961244 write 0x00fc80 thru 0x01dce5 (0xe066 bytes) 000203 1787698412.970951 mapread 0x00a9a5 thru 0x014f27 (0xa583 bytes) 000209 1787698412.981120 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3d18f): Mirror 3: 0x1172fc8a (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3d18f): Mirror 3: 0x1172fc8a (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3d18f: skipped 000210 1787698413.254049 mapwrite 0x014e03 thru 0x02398d (0xeb8b bytes) 000211 1787698413.330693 read 0x0066ea thru 0x00757d (0xe94 bytes) 000212 1787698413.344246 read 0x024aa4 thru 0x02b087 (0x65e4 bytes) 000214 1787698413.365569 read 0x02603c thru 0x02906d (0x3032 bytes) 000216 1787698413.366768 mapread 0x001a4b thru 0x001c0a (0x1c0 bytes) 000217 1787698413.374105 write 0x03fb3f thru 0x03ffff (0x4c1 bytes) 000219 1787698413.376367 mapwrite 0x019e8f thru 0x0216db (0x784d bytes) 000220 1787698413.421200 mapwrite 0x02aaea thru 0x032f19 (0x8430 bytes) 000221 1787698413.481380 read 0x00c72d thru 0x013041 (0x6915 bytes) 000222 1787698413.486831 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xe094241e (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xaf74b087 Mirror 4: 0xaf74b087 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000225 1787698414.554885 write 0x018b88 thru 0x018dfb (0x274 bytes) 000226 1787698414.806296 write 0x025c35 thru 0x02a324 (0x46f0 bytes) 000228 1787698414.821794 mapread 0x027e21 thru 0x0282ec (0x4cc bytes) 000229 1787698414.824114 mapwrite 0x026c65 thru 0x03495f (0xdcfb bytes) 000230 1787698415.037884 MIRROR_RESYNC 3 mirrors 000231 1787698416.134573 trunc from 0x040000 to 0x02402d (0x1bfd2 bytes) 000232 1787698416.200913 write 0x000a4e thru 0x00ac9b (0xa24e bytes) 000238 1787698416.225198 read 0x012353 thru 0x01cba8 (0xa856 bytes) 000239 1787698416.233966 mapread 0x016631 thru 0x016fd4 (0x9a4 bytes) 000240 1787698416.235208 mapread 0x00de92 thru 0x0176d1 (0x9840 bytes) 000243 1787698416.255465 trunc from 0x02402e to 0x0226a0 (0x198d bytes) 000244 1787698416.276863 read 0x015361 thru 0x01a2de (0x4f7e bytes) 000248 1787698416.280488 trunc from 0x0226a1 to 0x00341c (0x1f284 bytes) 000250 1787698416.317546 write 0x02d761 thru 0x035d8a (0x862a bytes) 000252 1787698416.320482 write 0x03e27e thru 0x03ffff (0x1d82 bytes) 000253 1787698416.329667 mapread 0x03a2a1 thru 0x03ffff (0x5d5f bytes) 000254 1787698416.341493 read 0x01b670 thru 0x0266c2 (0xb053 bytes) 000258 1787698416.343550 write 0x02f707 thru 0x03b2e2 (0xbbdc bytes) 000259 1787698416.350529 trunc from 0x040000 to 0x0293fe (0x16c01 bytes) 000260 1787698416.367921 mapwrite 0x0372e9 thru 0x038b19 (0x1831 bytes) 000261 1787698416.450025 read 0x019cda thru 0x01cee3 (0x320a bytes) 000262 1787698416.450582 read 0x0200dc thru 0x025941 (0x5866 bytes) 000263 1787698416.451102 mapwrite 0x00d212 thru 0x0102c8 (0x30b7 bytes) 000265 1787698416.486101 mapwrite 0x017d9b thru 0x01c81f (0x4a85 bytes) 000266 1787698416.524045 MIRROR_RESYNC 3 mirrors 000267 1787698417.207501 trunc from 0x038b1a to 0x03ef51 (0x6438 bytes) 000268 1787698417.302587 trunc from 0x03ef52 to 0x03ee40 (0x111 bytes) 000269 1787698417.324382 mapwrite 0x01ec92 thru 0x02b5cb (0xc93a bytes) 000271 1787698417.389840 mapwrite 0x01e1d3 thru 0x02a6fe (0xc52c bytes) 000272 1787698417.430737 mapread 0x03b885 thru 0x03ee40 (0x35bc bytes) 000273 1787698417.443553 read 0x011564 thru 0x01c670 (0xb10d bytes) 000274 1787698417.452897 mapread 0x033053 thru 0x03ee40 (0xbdee bytes) 000275 1787698417.465364 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3ee41): Mirror 4: 0x6a43b953 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3ee41): Mirror 4: 0x6a43b953 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3ee41: skipped 000276 1787698417.813630 write 0x0353af thru 0x03bf64 (0x6bb6 bytes) 000277 1787698417.823432 mapwrite 0x02e233 thru 0x038392 (0xa160 bytes) 000281 1787698417.886214 read 0x008a89 thru 0x012459 (0x99d1 bytes) 000283 1787698417.896792 write 0x03b1c4 thru 0x03ffff (0x4e3c bytes) 000284 1787698417.898959 trunc from 0x040000 to 0x00c44a (0x33bb5 bytes) 000290 1787698417.917344 trunc from 0x00c44b to 0x007dcf (0x467b bytes) 000291 1787698417.935457 mapread 0x0072a4 thru 0x007dcf (0xb2c bytes) 000292 1787698417.945651 mapread 0x006a43 thru 0x007dcf (0x138d bytes) 000294 1787698417.948295 trunc from 0x007dd0 to 0x00e28c (0x64bd bytes) 000296 1787698417.985637 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0xe28d): Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xe28d): Mirror 4: 0xb7285a98 (stale) CRC-32 checksum value for chunk [0, 0xe28d): Mirror 1: 0x9feb3ead Mirror 5: 0x9feb3ead lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xe28d: skipped 000297 1787698419.097984 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0xe28d): Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xe28d): Mirror 4: 0xb7285a98 (stale) CRC-32 checksum value for chunk [0, 0xe28d): Mirror 1: 0x9feb3ead Mirror 5: 0x9feb3ead Mirror 6: 0x9feb3ead lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xe28d: skipped 000299 1787698420.326631 trunc from 0x00e28d to 0x01341c (0x5190 bytes) 000302 1787698420.454443 trunc from 0x01341d to 0x03372e (0x20312 bytes) 000303 1787698420.476932 trunc from 0x03372f to 0x03c065 (0x8937 bytes) 000304 1787698420.510162 mapwrite 0x00af69 thru 0x00b3f8 (0x490 bytes) 000305 1787698420.609045 mapwrite 0x0154f9 thru 0x021f60 (0xca68 bytes) 000306 1787698420.696454 read 0x020dca thru 0x030ac2 (0xfcf9 bytes) 000307 1787698420.706280 mapwrite 0x014e8c thru 0x018a0e (0x3b83 bytes) 000310 1787698420.824907 write 0x02d595 thru 0x02e3c5 (0xe31 bytes) 000311 1787698420.827081 read 0x029b19 thru 0x037949 (0xde31 bytes) 000312 1787698420.838435 write 0x01a0e9 thru 0x01fb75 (0x5a8d bytes) 000314 1787698420.839857 mapread 0x01adc3 thru 0x0295b4 (0xe7f2 bytes) 000316 1787698420.846704 trunc from 0x03c066 to 0x015a5b (0x2660a bytes) 000318 1787698420.875622 read 0x00100c thru 0x001dd7 (0xdcc bytes) 000320 1787698420.886299 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x15a5c): Mirror 4: 0x685668f0 (stale) Mirror 6: 0x57828d57 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x15a5c): Mirror 4: 0x685668f0 (stale) Mirror 6: 0x57828d57 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x15a5c: skipped 000321 1787698421.353560 write 0x0372b7 thru 0x03ffff (0x8d49 bytes) 000322 1787698421.358026 mapread 0x00801e thru 0x012627 (0xa60a bytes) 000323 1787698421.393397 write 0x02ad06 thru 0x02e92f (0x3c2a bytes) 000327 1787698421.408998 mapread 0x02c4dd thru 0x03b12e (0xec52 bytes) 000329 1787698421.425927 read 0x008034 thru 0x00abc3 (0x2b90 bytes) 000330 1787698421.426531 trunc from 0x040000 to 0x001386 (0x3ec79 bytes) 000331 1787698421.452583 mapread 0x000f2f thru 0x001386 (0x458 bytes) 000332 1787698421.465370 mapwrite 0x0173ac thru 0x01bee1 (0x4b36 bytes) 000333 1787698421.577165 write 0x035ff7 thru 0x03dc2b (0x7c35 bytes) 000334 1787698421.579013 write 0x00105e thru 0x00f824 (0xe7c7 bytes) truncating to largest ever: 0x3fe57 000335 1787698421.589088 trunc from 0x03dc2c to 0x03fe56 (0x222b bytes) 000338 1787698421.610413 write 0x03e44c thru 0x03ffff (0x1bb4 bytes) 000339 1787698421.622545 mapwrite 0x00b303 thru 0x00f0b3 (0x3db1 bytes) 000340 1787698421.670947 write 0x00b131 thru 0x0188ab (0xd77b bytes) 000341 1787698421.675241 read 0x00e0bf thru 0x010f78 (0x2eba bytes) 000342 1787698421.678266 write 0x026e15 thru 0x0328e5 (0xbad1 bytes) 000345 1787698421.695669 MIRROR_RESYNC 3 mirrors 000346 1787698422.898011 mapread 0x02c305 thru 0x02f763 (0x345f bytes) 000348 1787698422.960458 mapread 0x03a50f thru 0x03ffff (0x5af1 bytes) 000349 1787698422.972496 read 0x0396b2 thru 0x03ffff (0x694e bytes) 000350 1787698422.973010 read 0x012b0e thru 0x01e11d (0xb610 bytes) 000351 1787698423.009666 trunc from 0x040000 to 0x00ec0a (0x313f5 bytes) 000353 1787698423.205757 write 0x00f06a thru 0x017d7c (0x8d13 bytes) 000354 1787698423.208649 write 0x036a45 thru 0x03bd78 (0x5334 bytes) 000356 1787698423.214566 mapwrite 0x0007ab thru 0x00efb3 (0xe809 bytes) 000357 1787698423.523471 mapread 0x02714c thru 0x027baa (0xa5f bytes) 000359 1787698423.534884 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3bd79): Mirror 6: 0xab34fe61 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3bd79): Mirror 6: 0xab34fe61 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3bd79: skipped 000360 1787698423.896943 MIRROR_RESYNC 2 mirrors 000361 1787698424.604542 trunc from 0x03bd79 to 0x03beb3 (0x13b bytes) 000362 1787698424.682715 trunc from 0x03beb4 to 0x03ce7a (0xfc7 bytes) 000363 1787698424.725361 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3ce7b): Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3ce7b): Mirror 6: 0xabdcc720 (stale) CRC-32 checksum value for chunk [0, 0x3ce7b): Mirror 1: 0xabdcc720 Mirror 7: 0xabdcc720 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3ce7b: skipped 000367 1787698425.655997 write 0x03243f thru 0x0334ed (0x10af bytes) 000371 1787698425.745245 mapread 0x010182 thru 0x01e840 (0xe6bf bytes) 000374 1787698425.772473 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3ce7b): Mirror 7: 0xabdcc720 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3ce7b): Mirror 7: 0xabdcc720 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3ce7b: skipped 000375 1787698425.987410 write 0x03d121 thru 0x03d746 (0x626 bytes) 000376 1787698426.002049 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) 000377 1787698426.209878 write 0x031fd8 thru 0x038435 (0x645e bytes) 000378 1787698426.224387 mapwrite 0x032868 thru 0x03f742 (0xcedb bytes) 000379 1787698426.370549 mapwrite 0x010918 thru 0x013939 (0x3022 bytes) 000381 1787698426.477702 mapread 0x016c26 thru 0x01cde9 (0x61c4 bytes) 000382 1787698426.491788 mapwrite 0x0321d8 thru 0x0341dd (0x2006 bytes) 000386 1787698426.540991 trunc from 0x03f743 to 0x0136b5 (0x2c08d bytes) 000387 1787698426.560532 read 0x00f7c6 thru 0x0136b5 (0x3ef0 bytes) 000388 1787698426.561148 trunc from 0x0136b6 to 0x018297 (0x4be2 bytes) 000389 1787698426.579455 mapread 0x00a0b0 thru 0x00f048 (0x4f99 bytes) 000390 1787698426.580294 read 0x005845 thru 0x008d80 (0x353c bytes) 000391 1787698426.593506 trunc from 0x018298 to 0x001c68 (0x1662f bytes) 000393 1787698426.624933 mapread 0x00094d thru 0x001c68 (0x131c bytes) 000394 1787698426.626109 mapread 0x0009a2 thru 0x001c68 (0x12c7 bytes) 000395 1787698426.634937 mapread 0x0009fa thru 0x001c68 (0x126f bytes) 000396 1787698426.635910 read 0x0003ea thru 0x001c68 (0x187f bytes) 000397 1787698426.638380 read 0x000a18 thru 0x001c68 (0x1251 bytes) 000398 1787698426.641619 read 0x001b7d thru 0x001c68 (0xec bytes) 000400 1787698426.648465 write 0x021d9b thru 0x024efb (0x3161 bytes) 000401 1787698426.652652 read 0x020bed thru 0x021578 (0x98c bytes) 000404 1787698426.667135 write 0x008067 thru 0x011722 (0x96bc bytes) 000406 1787698426.673820 mapwrite 0x03803c thru 0x03bc2e (0x3bf3 bytes) 000414 1787698426.832204 MIRROR_EXTEND to 2 mirrors 000415 1787698428.165831 mapwrite 0x000566 thru 0x00d6d0 (0xd16b bytes) 000416 1787698428.491855 write 0x000b20 thru 0x00e6df (0xdbc0 bytes) 000421 1787698428.496418 mapwrite 0x0121ac thru 0x01a252 (0x80a7 bytes) 000422 1787698428.548676 write 0x015199 thru 0x024a95 (0xf8fd bytes) 000425 1787698428.568892 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3bc2f): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3bc2f): Mirror 2: 0x9f12ef35 (stale) CRC-32 checksum value for chunk [0, 0x3bc2f): Mirror 1: 0xdeb810cc Mirror 3: 0xdeb810cc lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3bc2f: skipped 000426 1787698429.517757 MIRROR_RESYNC 3 mirrors 000428 1787698430.711457 mapwrite 0x00c080 thru 0x013b06 (0x7a87 bytes) 000429 1787698431.010801 read 0x011c28 thru 0x015b15 (0x3eee bytes) 000431 1787698431.017490 trunc from 0x03bc2f to 0x01f1cf (0x1ca5f bytes) 000433 1787698431.051011 read 0x00f515 thru 0x01932a (0x9e16 bytes) 000434 1787698431.095411 mapread 0x01a4dd thru 0x01deb9 (0x39dd bytes) 000436 1787698431.099936 write 0x022dd9 thru 0x0258e2 (0x2b0a bytes) 000437 1787698431.108474 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x258e3): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x258e3): Mirror 2: 0xbd516e02 (stale) Mirror 3: 0xbd516e02 (stale) CRC-32 checksum value for chunk [0, 0x258e3): Mirror 1: 0xd2a9f708 Mirror 4: 0xd2a9f708 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x258e3: skipped 000438 1787698431.926215 mapwrite 0x03c1a3 thru 0x03f2e2 (0x3140 bytes) 000439 1787698432.088556 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3f2e3): Mirror 3: 0xe6f88eb (stale) Mirror 4: 0xf18cc363 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3f2e3): Mirror 3: 0xe6f88eb (stale) Mirror 4: 0xf18cc363 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3f2e3: skipped 000440 1787698432.366474 trunc from 0x03f2e3 to 0x01f157 (0x2018b bytes) 000441 1787698432.387491 mapwrite 0x0368e9 thru 0x03ffff (0x9717 bytes) 000442 1787698432.469457 mapwrite 0x00d7fe thru 0x0112c5 (0x3ac8 bytes) 000445 1787698432.523780 mapwrite 0x00d401 thru 0x017ce9 (0xa8e9 bytes) 000446 1787698432.572790 read 0x007bd7 thru 0x00ba1a (0x3e44 bytes) 000447 1787698432.577499 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x918fb34e (stale) Mirror 4: 0x3a64d052 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x11eb86c6 Mirror 5: 0x11eb86c6 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000449 1787698433.281518 MIRROR_RESYNC 4 mirrors 000450 1787698434.297174 mapwrite 0x010935 thru 0x01be5f (0xb52b bytes) 000451 1787698434.560540 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x11eb86c6 (stale) Mirror 4: 0x11eb86c6 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x11eb86c6 (stale) Mirror 4: 0x11eb86c6 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000452 1787698434.976548 trunc from 0x040000 to 0x0330a2 (0xcf5d bytes) 000453 1787698435.015468 mapwrite 0x0302be thru 0x03faea (0xf82d bytes) 000454 1787698435.132994 write 0x007f17 thru 0x010dd3 (0x8ebd bytes) 000456 1787698435.169043 mapwrite 0x015496 thru 0x0159dd (0x548 bytes) 000457 1787698435.252170 mapread 0x02cc41 thru 0x038ccf (0xc08f bytes) 000459 1787698435.255759 read 0x022e23 thru 0x0302d6 (0xd4b4 bytes) 000460 1787698435.270905 mapwrite 0x011574 thru 0x01622c (0x4cb9 bytes) 000461 1787698435.371626 trunc from 0x03faeb to 0x0003c4 (0x3f726 bytes) 000462 1787698435.403982 MIRROR_RESYNC 3 mirrors 000464 1787698436.167390 mapwrite 0x03490b thru 0x0392f5 (0x49eb bytes) 000465 1787698436.401960 write 0x00a0c9 thru 0x019bfe (0xfb36 bytes) 000468 1787698436.414962 MIRROR_RESYNC 3 mirrors 000470 1787698437.411443 trunc from 0x0392f6 to 0x02b76f (0xdb86 bytes) 000472 1787698437.484993 mapread 0x004076 thru 0x00b1ba (0x7145 bytes) 000477 1787698437.510244 trunc from 0x02b770 to 0x01733c (0x14433 bytes) 000479 1787698437.531141 mapwrite 0x0330ac thru 0x0370e3 (0x4038 bytes) 000480 1787698437.626975 write 0x018cb5 thru 0x01cc2c (0x3f78 bytes) 000481 1787698437.640156 write 0x00077b thru 0x0013c8 (0xc4e bytes) 000482 1787698437.646966 trunc from 0x0370e4 to 0x01f6ce (0x17a15 bytes) 000484 1787698437.675547 trunc from 0x01f6cf to 0x03f8da (0x2020c bytes) 000487 1787698437.715226 trunc from 0x03f8db to 0x019038 (0x268a2 bytes) 000489 1787698437.738618 read 0x0041c5 thru 0x00ea99 (0xa8d5 bytes) 000490 1787698437.741204 read 0x00bc9d thru 0x00fd45 (0x40a9 bytes) 000492 1787698437.747919 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x19039): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x19039): Mirror 3: 0x22503fae (stale) Mirror 4: 0x22503fae (stale) CRC-32 checksum value for chunk [0, 0x19039): Mirror 1: 0x1db5ca33 Mirror 5: 0x1db5ca33 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x19039: skipped 000493 1787698438.582874 trunc from 0x019039 to 0x031d0e (0x18cd6 bytes) 000495 1787698438.673494 write 0x0022e2 thru 0x004d9d (0x2abc bytes) 000497 1787698438.687996 MIRROR_RESYNC 4 mirrors 000498 1787698439.601156 mapread 0x026e72 thru 0x02c37d (0x550c bytes) 000500 1787698439.622084 mapread 0x001833 thru 0x00bff6 (0xa7c4 bytes) 000503 1787698439.631121 write 0x038512 thru 0x03ffff (0x7aee bytes) 000504 1787698439.709554 trunc from 0x040000 to 0x03591e (0xa6e1 bytes) 000505 1787698439.745347 mapwrite 0x0135bf thru 0x019314 (0x5d56 bytes) 000506 1787698439.882241 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3591f): Mirror 4: 0x8fdc2db6 (stale) Mirror 5: 0x8fdc2db6 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3591f): Mirror 4: 0x8fdc2db6 (stale) Mirror 5: 0x8fdc2db6 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3591f: skipped 000507 1787698440.223423 mapread 0x0334e4 thru 0x03591e (0x243b bytes) 000510 1787698440.241128 mapread 0x01f419 thru 0x026d92 (0x797a bytes) 000511 1787698440.250519 read 0x0135f8 thru 0x0183dc (0x4de5 bytes) 000513 1787698440.258611 mapread 0x0309a0 thru 0x03591e (0x4f7f bytes) 000515 1787698440.263808 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x3591f): Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3591f): Mirror 4: 0x8fdc2db6 (stale) Mirror 5: 0x8fdc2db6 (stale) CRC-32 checksum value for chunk [0, 0x3591f): Mirror 1: 0x4a41b9bd Mirror 6: 0x4a41b9bd lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3591f: skipped 000516 1787698441.395548 mapwrite 0x00db48 thru 0x01775d (0x9c16 bytes) 000517 1787698441.560632 mapwrite 0x01f9be thru 0x01fecc (0x50f bytes) 000518 1787698441.595028 read 0x01f9f9 thru 0x024ae3 (0x50eb bytes) 000519 1787698441.602879 mapwrite 0x03a35c thru 0x03ffff (0x5ca4 bytes) 000520 1787698441.718229 trunc from 0x040000 to 0x02855c (0x17aa3 bytes) 000521 1787698441.740766 mapwrite 0x019ca1 thru 0x01b542 (0x18a2 bytes) 000524 1787698441.797649 trunc from 0x02855d to 0x030419 (0x7ebd bytes) 000525 1787698441.815945 mapread 0x018bd7 thru 0x0220bf (0x94e9 bytes) 000526 1787698441.817258 trunc from 0x03041a to 0x01fc9c (0x1077d bytes) 000527 1787698441.835119 write 0x0081e4 thru 0x015071 (0xce8e bytes) 000529 1787698441.837052 read 0x009f4e thru 0x0122ce (0x8381 bytes) 000530 1787698441.842782 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1fc9d): Mirror 4: 0x3bc03900 (stale) Mirror 5: 0x3bc03900 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1fc9d): Mirror 4: 0x3bc03900 (stale) Mirror 5: 0x3bc03900 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1fc9d: skipped 000532 1787698442.196632 MIRROR_RESYNC 3 mirrors 000533 1787698442.962430 write 0x02b582 thru 0x036e80 (0xb8ff bytes) 000534 1787698443.017306 mapread 0x009beb thru 0x01415f (0xa575 bytes) 000535 1787698443.038714 mapread 0x02c983 thru 0x036e80 (0xa4fe bytes) 000536 1787698443.040445 write 0x0092a4 thru 0x013dd6 (0xab33 bytes) 000539 1787698443.043109 mapread 0x00daa0 thru 0x018cfd (0xb25e bytes) 000542 1787698443.050005 MIRROR_RESYNC 3 mirrors 000543 1787698443.937416 write 0x01589f thru 0x020262 (0xa9c4 bytes) 000545 1787698444.015565 mapwrite 0x03b59f thru 0x03ffff (0x4a61 bytes) 000549 1787698444.156224 mapread 0x031133 thru 0x0393da (0x82a8 bytes) 000554 1787698444.170926 mapwrite 0x033d7a thru 0x03fce7 (0xbf6e bytes) 000556 1787698444.248430 write 0x01399b thru 0x020383 (0xc9e9 bytes) 000558 1787698444.254050 write 0x030dac thru 0x03ffff (0xf254 bytes) 000559 1787698444.265642 mapread 0x003702 thru 0x00c51d (0x8e1c bytes) 000560 1787698444.299599 mapwrite 0x039528 thru 0x03ffff (0x6ad8 bytes) 000562 1787698444.465498 mapread 0x03fdc4 thru 0x03ffff (0x23c bytes) 000563 1787698444.466353 write 0x0369f6 thru 0x03ffff (0x960a bytes) 000566 1787698444.472166 mapwrite 0x011d63 thru 0x01b9ab (0x9c49 bytes) 000568 1787698444.572090 mapwrite 0x009fcf thru 0x0171e3 (0xd215 bytes) 000572 1787698444.632664 write 0x025683 thru 0x02dd7b (0x86f9 bytes) 000573 1787698444.638072 read 0x02eeba thru 0x03a0e0 (0xb227 bytes) 000574 1787698444.638860 trunc from 0x040000 to 0x00b565 (0x34a9a bytes) 000575 1787698444.658932 read 0x002aa1 thru 0x00b565 (0x8ac5 bytes) 000577 1787698444.660152 write 0x01ae14 thru 0x025d7a (0xaf67 bytes) 000578 1787698444.662587 mapread 0x011d4c thru 0x01dc9a (0xbf4f bytes) 000580 1787698444.672545 mapread 0x016c18 thru 0x01f2db (0x86c4 bytes) 000584 1787698444.675934 write 0x0192e3 thru 0x01d6df (0x43fd bytes) 000586 1787698444.679235 trunc from 0x025d7b to 0x03b66c (0x158f2 bytes) 000587 1787698444.699675 write 0x03e7e6 thru 0x03f7cd (0xfe8 bytes) 000589 1787698444.702208 mapread 0x0203c7 thru 0x026ccd (0x6907 bytes) 000593 1787698444.718009 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3f7ce): Mirror 5: 0x503f4d9f (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3f7ce): Mirror 5: 0x503f4d9f (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3f7ce: skipped 000597 1787698445.129475 mapwrite 0x004fd9 thru 0x00a06c (0x5094 bytes) 000600 1787698445.236815 mapread 0x00c4c1 thru 0x00d4f1 (0x1031 bytes) 000602 1787698445.239452 trunc from 0x03f7ce to 0x034264 (0xb569 bytes) 000603 1787698445.268526 write 0x03ba96 thru 0x03bf9a (0x505 bytes) 000605 1787698445.273481 read 0x00ea59 thru 0x018240 (0x97e8 bytes) 000606 1787698445.290516 read 0x0078dd thru 0x00de17 (0x653b bytes) 000607 1787698445.292795 mapwrite 0x003867 thru 0x0040fd (0x897 bytes) 000608 1787698445.360335 write 0x03930c thru 0x03ffff (0x6cf4 bytes) 000609 1787698445.370532 read 0x02bd77 thru 0x032589 (0x6813 bytes) 000611 1787698445.371691 mapwrite 0x02d430 thru 0x02de7f (0xa50 bytes) 000613 1787698445.420589 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0xeb5fa8b8 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x162b884a Mirror 6: 0x162b884a lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000615 1787698446.371392 read 0x02cad9 thru 0x03637a (0x98a2 bytes) 000616 1787698446.390296 read 0x012882 thru 0x015164 (0x28e3 bytes) 000621 1787698446.416043 read 0x014f9d thru 0x02366e (0xe6d2 bytes) 000623 1787698446.416942 mapread 0x02e25e thru 0x030837 (0x25da bytes) 000624 1787698446.418059 write 0x017c16 thru 0x019211 (0x15fc bytes) 000625 1787698446.496126 write 0x027316 thru 0x0363d3 (0xf0be bytes) 000627 1787698446.499512 read 0x0258ee thru 0x02cb96 (0x72a9 bytes) 000629 1787698446.501105 read 0x01b0d6 thru 0x024295 (0x91c0 bytes) 000630 1787698446.502178 write 0x012773 thru 0x012f30 (0x7be bytes) 000632 1787698446.504492 write 0x03898b thru 0x039c78 (0x12ee bytes) 000633 1787698446.505605 read 0x026e3b thru 0x02f4a8 (0x866e bytes) 000634 1787698446.506630 write 0x00ac9a thru 0x00e3b4 (0x371b bytes) 000635 1787698446.508378 write 0x0301d4 thru 0x0323ee (0x221b bytes) 000636 1787698446.509531 write 0x02a0ba thru 0x03049a (0x63e1 bytes) 000637 1787698446.510779 write 0x008bd5 thru 0x015c94 (0xd0c0 bytes) 000639 1787698446.516699 MIRROR_RESYNC 3 mirrors 000640 1787698447.270446 mapread 0x008b4e thru 0x00d88c (0x4d3f bytes) 000643 1787698447.310875 trunc from 0x040000 to 0x009d75 (0x3628a bytes) 000644 1787698447.431106 MIRROR_RESYNC 3 mirrors 000645 1787698448.298630 write 0x012a73 thru 0x01837d (0x590b bytes) 000647 1787698448.370885 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x1837e): Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1837e): Mirror 5: 0x5d9a0a88 (stale) Mirror 6: 0x5d9a0a88 (stale) CRC-32 checksum value for chunk [0, 0x1837e): Mirror 1: 0x4751d719 Mirror 7: 0x4751d719 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1837e: skipped 000648 1787698449.345655 trunc from 0x01837e to 0x0300b2 (0x17d35 bytes) 000649 1787698449.428564 mapwrite 0x0396d2 thru 0x03ffff (0x692e bytes) 000650 1787698449.555186 read 0x001fdb thru 0x00fbb6 (0xdbdc bytes) 000651 1787698449.565574 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: 0x8d9c54a8 (stale) Mirror 7: 0x421c644b (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: 0x8d9c54a8 (stale) Mirror 7: 0x421c644b (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000652 1787698449.871059 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: (stale - read error) Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: 0x8d9c54a8 (stale) Mirror 7: 0x421c644b (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x7add9af8 Mirror 8: 0x7add9af8 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000655 1787698451.059407 read 0x0179d5 thru 0x02660e (0xec3a bytes) 000656 1787698451.077060 mapread 0x03c9bc thru 0x03c9d2 (0x17 bytes) 000658 1787698451.097559 write 0x028598 thru 0x028e6e (0x8d7 bytes) 000660 1787698451.163005 trunc from 0x040000 to 0x01cbcc (0x23433 bytes) 000663 1787698451.181787 read 0x008016 thru 0x00ab39 (0x2b24 bytes) 000664 1787698451.189599 mapwrite 0x024132 thru 0x02a4dc (0x63ab bytes) 000666 1787698451.260978 trunc from 0x02a4dd to 0x029059 (0x1483 bytes) 000668 1787698451.280344 write 0x01ccb6 thru 0x022654 (0x599f bytes) 000670 1787698451.282878 write 0x008774 thru 0x00ec97 (0x6524 bytes) 000673 1787698451.285647 mapread 0x023aaa thru 0x029059 (0x55b0 bytes) 000676 1787698451.288544 write 0x025315 thru 0x032460 (0xd14c bytes) 000681 1787698451.292976 trunc from 0x032461 to 0x02811f (0xa341 bytes) 000683 1787698451.327171 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x28120): Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x28120): Mirror 6: 0x798b7996 (stale) Mirror 7: 0xc0e12b31 (stale) Mirror 8: 0xc0e12b31 (stale) CRC-32 checksum value for chunk [0, 0x28120): Mirror 1: 0xaf4dfb8a Mirror 9: 0xaf4dfb8a lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x28120: skipped 000684 1787698451.972881 read 0x021132 thru 0x02811f (0x6fee bytes) 000685 1787698451.985980 trunc from 0x028120 to 0x00f100 (0x1901f bytes) 000686 1787698452.073525 write 0x03ba75 thru 0x03ffff (0x458b bytes) 000687 1787698452.074813 read 0x03d531 thru 0x03ffff (0x2acf bytes) 000688 1787698452.080285 read 0x026092 thru 0x02c848 (0x67b7 bytes) 000691 1787698452.100297 MIRROR_RESYNC 5 mirrors 000698 1787698453.367527 write 0x02c3be thru 0x032358 (0x5f9b bytes) 000699 1787698453.507377 mapwrite 0x0184ac thru 0x01f7b6 (0x730b bytes) 000700 1787698453.654386 MIRROR_RESYNC 5 mirrors 000702 1787698455.791133 write 0x0081c6 thru 0x00a81e (0x2659 bytes) 000704 1787698455.927412 read 0x00a6ec thru 0x00b5d3 (0xee8 bytes) 000705 1787698455.938587 mapread 0x017c8d thru 0x01d0ea (0x545e bytes) 000707 1787698455.993565 mapread 0x0222aa thru 0x030f77 (0xecce bytes) 000708 1787698456.026343 trunc from 0x040000 to 0x03648f (0x9b70 bytes) 000713 1787698456.053284 trunc from 0x036490 to 0x03f965 (0x94d6 bytes) 000717 1787698456.071024 trunc from 0x03f966 to 0x027e3b (0x17b2a bytes) 000719 1787698456.093952 mapread 0x0170f7 thru 0x01f8b2 (0x87bc bytes) 000721 1787698456.101222 MIRROR_RESYNC 5 mirrors 000723 1787698458.198788 write 0x01de21 thru 0x02c923 (0xeb03 bytes) 000724 1787698458.281192 mapwrite 0x008046 thru 0x00c1b6 (0x4171 bytes) 000725 1787698458.370816 MIRROR_RESYNC 5 mirrors 000726 1787698459.356907 MIRROR_EXTEND to 6 mirrors 000727 1787698459.986667 trunc from 0x02c924 to 0x0176d3 (0x15250 bytes) 000728 1787698460.074847 write 0x000d09 thru 0x002958 (0x1c50 bytes) 000730 1787698460.082958 read 0x00c69d thru 0x0127ba (0x611e bytes) 000733 1787698460.094517 write 0x02e8e0 thru 0x03396e (0x508f bytes) 000734 1787698460.096442 trunc from 0x03396f to 0x015fa7 (0x1d9c7 bytes) 000735 1787698460.121283 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x15fa8): Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x15fa8): Mirror 6: 0x117dbbbd (stale) Mirror 7: 0x117dbbbd (stale) Mirror 8: 0x117dbbbd (stale) Mirror 9: 0x117dbbbd (stale) Mirror 10: 0x117dbbbd (stale) CRC-32 checksum value for chunk [0, 0x15fa8): Mirror 1: 0xb01d6157 Mirror 11: 0xb01d6157 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x15fa8: skipped 000737 1787698460.998257 MIRROR_RESYNC 7 mirrors 000739 1787698462.159063 mapwrite 0x02c908 thru 0x02f9fe (0x30f7 bytes) 000740 1787698462.332514 trunc from 0x02f9ff to 0x03fbeb (0x101ed bytes) 000742 1787698462.355140 write 0x01d5d6 thru 0x026a73 (0x949e bytes) 000745 1787698462.369973 read 0x017771 thru 0x022fda (0xb86a bytes) 000746 1787698462.389021 MIRROR_RESYNC 7 mirrors 000747 1787698464.437691 write 0x025179 thru 0x027418 (0x22a0 bytes) 000748 1787698464.517365 mapwrite 0x02e36b thru 0x034267 (0x5efd bytes) 000749 1787698464.579701 trunc from 0x03fbec to 0x013343 (0x2c8a8 bytes) 000751 1787698464.606476 trunc from 0x013344 to 0x03e14e (0x2ae0b bytes) 000752 1787698464.631659 write 0x0122f8 thru 0x01962e (0x7337 bytes) 000754 1787698464.645312 write 0x031b3a thru 0x03379a (0x1c61 bytes) 000755 1787698464.657442 mapwrite 0x010ecb thru 0x012e27 (0x1f5d bytes) 000758 1787698464.705112 write 0x01bbb7 thru 0x01caf1 (0xf3b bytes) 000759 1787698464.727443 write 0x00b5bb thru 0x00fd8d (0x47d3 bytes) 000760 1787698464.742157 MIRROR_SPLIT mirror 1 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3e14f): Mirror 7: 0x4cf71480 (stale) Mirror 8: 0x4cf71480 (stale) Mirror 9: 0x4cf71480 (stale) Mirror 10: 0x4cf71480 (stale) Mirror 11: 0x4cf71480 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e14f): Mirror 7: 0x4cf71480 (stale) Mirror 8: 0x4cf71480 (stale) Mirror 9: 0x4cf71480 (stale) Mirror 10: 0x4cf71480 (stale) Mirror 11: 0x4cf71480 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e14f: skipped 000761 1787698465.241383 write 0x02069a thru 0x02bcd6 (0xb63d bytes) 000762 1787698465.254187 write 0x001a71 thru 0x00e682 (0xcc12 bytes) 000765 1787698465.270424 mapread 0x016674 thru 0x01b7e7 (0x5174 bytes) 000767 1787698465.303073 trunc from 0x03e14f to 0x03a832 (0x391c bytes) 000769 1787698465.339625 mapwrite 0x02bea4 thru 0x02e40b (0x2568 bytes) 000770 1787698465.435875 read 0x01197d thru 0x01f522 (0xdba6 bytes) 000772 1787698465.437369 write 0x0171ff thru 0x026abb (0xf8bd bytes) 000773 1787698465.441415 write 0x002fc7 thru 0x0071c5 (0x41ff bytes) 000774 1787698465.452397 read 0x016b13 thru 0x0255ab (0xea99 bytes) 000775 1787698465.455389 mapread 0x00319c thru 0x0045ad (0x1412 bytes) 000776 1787698465.459327 mapwrite 0x03be5e thru 0x03ffff (0x41a2 bytes) 000778 1787698465.602564 mapwrite 0x014e56 thru 0x01bedd (0x7088 bytes) 000779 1787698465.670802 write 0x014988 thru 0x01b5e2 (0x6c5b bytes) 000780 1787698465.687557 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 7: 0x3af064d5 (stale) Mirror 8: 0x3af064d5 (stale) Mirror 9: 0x3af064d5 (stale) Mirror 10: 0x3af064d5 (stale) Mirror 11: 0x3af064d5 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x47689a94 Mirror 12: 0x47689a94 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000781 1787698467.065939 MIRROR_SPLIT mirror 6 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 7: 0x3af064d5 (stale) Mirror 8: 0x3af064d5 (stale) Mirror 9: 0x3af064d5 (stale) Mirror 10: 0x3af064d5 (stale) Mirror 11: 0x3af064d5 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 7: 0x3af064d5 (stale) Mirror 8: 0x3af064d5 (stale) Mirror 9: 0x3af064d5 (stale) Mirror 10: 0x3af064d5 (stale) Mirror 11: 0x3af064d5 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000783 1787698467.607261 read 0x01a302 thru 0x023a1a (0x9719 bytes) 000784 1787698467.617139 read 0x01eef3 thru 0x02348c (0x459a bytes) 000785 1787698467.618346 mapread 0x03b244 thru 0x03ffff (0x4dbc bytes) 000786 1787698467.640677 trunc from 0x040000 to 0x001f12 (0x3e0ed bytes) 000787 1787698467.682763 mapread 0x001132 thru 0x001f12 (0xde1 bytes) 000788 1787698467.695839 MIRROR_RESYNC 6 mirrors 000789 1787698468.776988 trunc from 0x001f13 to 0x036393 (0x34481 bytes) 000790 1787698468.889361 mapread 0x02ae99 thru 0x03018a (0x52f2 bytes) 000791 1787698468.922780 read 0x01aa7b thru 0x0227c8 (0x7d4e bytes) 000792 1787698468.935112 mapwrite 0x021e30 thru 0x022ee8 (0x10b9 bytes) 000793 1787698468.985946 write 0x02f8eb thru 0x03f418 (0xfb2e bytes) 000794 1787698468.989080 trunc from 0x03f419 to 0x0087e8 (0x36c30 bytes) 000796 1787698469.022171 mapread 0x0053d1 thru 0x0087e8 (0x3418 bytes) 000797 1787698469.041701 read 0x0028d5 thru 0x0087e8 (0x5f14 bytes) 000799 1787698469.045112 mapread 0x001163 thru 0x0087e8 (0x7686 bytes) 000800 1787698469.048895 trunc from 0x0087e9 to 0x011e77 (0x968f bytes) 000801 1787698469.073566 mapread 0x005251 thru 0x005563 (0x313 bytes) 000804 1787698469.089837 MIRROR_RESYNC 6 mirrors 000805 1787698470.306694 mapread 0x00d5e1 thru 0x011e77 (0x4897 bytes) 000807 1787698470.336015 mapread 0x00b6c7 thru 0x00ddc3 (0x26fd bytes) 000808 1787698470.336684 read 0x00bd68 thru 0x011e77 (0x6110 bytes) 000812 1787698470.338622 trunc from 0x011e78 to 0x008cb3 (0x91c4 bytes) 000814 1787698470.497746 mapwrite 0x024496 thru 0x02fc36 (0xb7a1 bytes) 000815 1787698470.652462 MIRROR_SPLIT mirror 5 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2fc37): Mirror 7: 0x9866b7f0 (stale) Mirror 8: 0x9866b7f0 (stale) Mirror 9: 0x9866b7f0 (stale) Mirror 10: 0x9866b7f0 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2fc37): Mirror 7: 0x9866b7f0 (stale) Mirror 8: 0x9866b7f0 (stale) Mirror 9: 0x9866b7f0 (stale) Mirror 10: 0x9866b7f0 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2fc37: skipped 000816 1787698471.046552 read 0x00fcb4 thru 0x019103 (0x9450 bytes) 000820 1787698471.058906 mapwrite 0x0130a5 thru 0x01ad95 (0x7cf1 bytes) 000821 1787698471.119166 trunc from 0x02fc37 to 0x0083a6 (0x27890 bytes) 000822 1787698471.151432 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x83a7): Mirror 8: 0xf4de0f5a (stale) Mirror 9: 0xf4de0f5a (stale) Mirror 10: 0xf4de0f5a (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x83a7): Mirror 8: 0xf4de0f5a (stale) Mirror 9: 0xf4de0f5a (stale) Mirror 10: 0xf4de0f5a (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x83a7: skipped 000823 1787698471.521551 mapwrite 0x02830d thru 0x02bd06 (0x39fa bytes) 000824 1787698471.622315 mapwrite 0x022190 thru 0x02ef94 (0xce05 bytes) 000826 1787698471.746232 mapwrite 0x006364 thru 0x010c6c (0xa909 bytes) 000828 1787698471.821924 mapread 0x02d1fa thru 0x02ef94 (0x1d9b bytes) 000829 1787698471.823012 read 0x028588 thru 0x0293ba (0xe33 bytes) 000830 1787698471.823826 write 0x007a84 thru 0x014f8b (0xd508 bytes) 000831 1787698471.832375 MIRROR_RESYNC 4 mirrors 000834 1787698472.792395 write 0x03b0a0 thru 0x03ffff (0x4f60 bytes) 000837 1787698472.849603 trunc from 0x040000 to 0x00677f (0x39880 bytes) 000843 1787698472.878880 mapwrite 0x020b7f thru 0x023e22 (0x32a4 bytes) 000844 1787698472.996411 write 0x019cfa thru 0x023bf8 (0x9eff bytes) 000847 1787698473.008453 mapread 0x01d8d9 thru 0x023e22 (0x654a bytes) 000848 1787698473.011636 mapread 0x01c7e9 thru 0x02197f (0x5197 bytes) 000849 1787698473.013543 trunc from 0x023e23 to 0x034437 (0x10615 bytes) 000851 1787698473.030884 trunc from 0x034438 to 0x0216cc (0x12d6b bytes) 000852 1787698473.055518 mapwrite 0x01332d thru 0x01b7a6 (0x847a bytes) 000853 1787698473.098904 mapwrite 0x008032 thru 0x0153f1 (0xd3c0 bytes) 000856 1787698473.149545 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x216cd): Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x216cd): Mirror 8: 0x716c675b (stale) Mirror 9: 0x716c675b (stale) Mirror 10: 0x716c675b (stale) CRC-32 checksum value for chunk [0, 0x216cd): Mirror 1: 0x39c9bb11 Mirror 11: 0x39c9bb11 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x216cd: skipped 000857 1787698473.787653 mapread 0x0040f6 thru 0x00886f (0x477a bytes) 000859 1787698473.818209 mapwrite 0x02b5c0 thru 0x02e667 (0x30a8 bytes) 000860 1787698474.065355 mapwrite 0x02bf0b thru 0x030008 (0x40fe bytes) 000861 1787698474.192138 read 0x00f12a thru 0x01cde0 (0xdcb7 bytes) 000863 1787698474.219699 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x30009): Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x30009): Mirror 8: 0x1ba5d4a0 (stale) Mirror 9: 0x1ba5d4a0 (stale) Mirror 10: 0x1ba5d4a0 (stale) Mirror 11: 0x9283763 (stale) CRC-32 checksum value for chunk [0, 0x30009): Mirror 1: 0x658c6b0c Mirror 12: 0x658c6b0c lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x30009: skipped 000864 1787698474.833280 MIRROR_RESYNC 6 mirrors 000866 1787698476.168983 mapread 0x022b29 thru 0x030008 (0xd4e0 bytes) 000867 1787698476.212883 trunc from 0x030009 to 0x028203 (0x7e05 bytes) 000869 1787698476.366453 MIRROR_SPLIT mirror 5 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x28204): Mirror 8: 0xa344bede (stale) Mirror 9: 0xa344bede (stale) Mirror 10: 0xa344bede (stale) Mirror 11: 0xa344bede (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x28204): Mirror 8: 0xa344bede (stale) Mirror 9: 0xa344bede (stale) Mirror 10: 0xa344bede (stale) Mirror 11: 0xa344bede (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x28204: skipped 000870 1787698476.907857 mapread 0x026f82 thru 0x028203 (0x1282 bytes) 000872 1787698476.944319 mapread 0x00ea9d thru 0x00ef1c (0x480 bytes) 000875 1787698476.964688 mapread 0x01d26a thru 0x026eb2 (0x9c49 bytes) 000876 1787698476.995269 mapwrite 0x001a31 thru 0x011604 (0xfbd4 bytes) 000878 1787698477.076880 write 0x03002b thru 0x03afd0 (0xafa6 bytes) 000881 1787698477.086111 MIRROR_RESYNC 5 mirrors 000882 1787698478.258668 read 0x038d54 thru 0x03afd0 (0x227d bytes) 000884 1787698478.273155 MIRROR_SPLIT mirror 4 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) 000885 1787698478.888289 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) 000888 1787698479.282572 read 0x0356f7 thru 0x03ad33 (0x563d bytes) 000890 1787698479.294111 read 0x00a7c8 thru 0x011a6d (0x72a6 bytes) 000894 1787698479.322836 MIRROR_EXTEND to 4 mirrors 000895 1787698480.569533 read 0x017934 thru 0x025843 (0xdf10 bytes) 000896 1787698480.605709 MIRROR_RESYNC 4 mirrors 000897 1787698481.204593 trunc from 0x03afd1 to 0x01ccf3 (0x1e2dd bytes) 000898 1787698481.285500 write 0x00ac21 thru 0x011d32 (0x7112 bytes) 000902 1787698481.300587 mapread 0x003686 thru 0x00efb2 (0xb92d bytes) 000904 1787698481.337112 write 0x00b000 thru 0x012792 (0x7793 bytes) 000906 1787698481.353959 trunc from 0x01ccf4 to 0x03b923 (0x1ec30 bytes) 000907 1787698481.376030 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3b924): Mirror 10: 0x5b7f1b9c (stale) Mirror 11: 0x5b7f1b9c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3b924): Mirror 10: 0x5b7f1b9c (stale) Mirror 11: 0x5b7f1b9c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3b924: skipped 000908 1787698481.678672 mapwrite 0x020058 thru 0x029ea0 (0x9e49 bytes) 000910 1787698481.751185 read 0x01c793 thru 0x02244f (0x5cbd bytes) 000911 1787698481.751759 mapread 0x02e8b8 thru 0x037240 (0x8989 bytes) 000914 1787698481.773834 trunc from 0x03b924 to 0x03bc81 (0x35e bytes) 000920 1787698481.794620 mapread 0x0022ec thru 0x00cc19 (0xa92e bytes) 000922 1787698481.803039 mapwrite 0x0333eb thru 0x037933 (0x4549 bytes) 000924 1787698481.873787 MIRROR_RESYNC 3 mirrors 000925 1787698482.781062 mapwrite 0x010d59 thru 0x020438 (0xf6e0 bytes) 000927 1787698482.923451 read 0x01b557 thru 0x02a1df (0xec89 bytes) 000929 1787698482.934698 mapwrite 0x026101 thru 0x0304f0 (0xa3f0 bytes) 000930 1787698482.986416 mapwrite 0x031862 thru 0x03ffff (0xe79e bytes) 000931 1787698483.058075 mapwrite 0x03e81d thru 0x03ffff (0x17e3 bytes) 000932 1787698483.089340 read 0x00ef86 thru 0x010e4f (0x1eca bytes) 000934 1787698483.090418 mapwrite 0x0384f5 thru 0x03ffff (0x7b0b bytes) 000936 1787698483.128232 mapwrite 0x020739 thru 0x02ce6a (0xc732 bytes) 000937 1787698483.166223 mapread 0x020308 thru 0x020533 (0x22c bytes) 000940 1787698483.167972 mapwrite 0x009724 thru 0x0187f5 (0xf0d2 bytes) 000941 1787698483.221993 mapwrite 0x024cbb thru 0x026f02 (0x2248 bytes) 000942 1787698483.254520 mapread 0x01d508 thru 0x028c99 (0xb792 bytes) 000944 1787698483.255920 mapwrite 0x03c00f thru 0x03ffff (0x3ff1 bytes) 000946 1787698483.290574 mapread 0x01b75f thru 0x023f1f (0x87c1 bytes) 000948 1787698483.291970 write 0x03f8e0 thru 0x03ffff (0x720 bytes) 000950 1787698483.294596 read 0x019463 thru 0x022107 (0x8ca5 bytes) 000952 1787698483.296071 read 0x0083a8 thru 0x00ddc3 (0x5a1c bytes) 000954 1787698483.304479 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 10: (stale - read error) Mirror 11: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 10: 0x97b7cd48 (stale) Mirror 11: 0x97b7cd48 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x18cba4ee Mirror 12: 0x18cba4ee lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000955 1787698483.920580 trunc from 0x040000 to 0x018a3b (0x275c4 bytes) 000956 1787698484.025229 mapwrite 0x007b46 thru 0x0097ee (0x1ca9 bytes) 000962 1787698484.106480 MIRROR_RESYNC 4 mirrors 000963 1787698485.016437 trunc from 0x018a3c to 0x03e060 (0x25625 bytes) 000964 1787698485.121611 trunc from 0x03e061 to 0x022fa9 (0x1b0b7 bytes) 000965 1787698485.148610 MIRROR_RESYNC 4 mirrors 000967 1787698486.107678 mapread 0x003e5e thru 0x009f87 (0x612a bytes) 000970 1787698486.126672 MIRROR_RESYNC 4 mirrors 000971 1787698486.470553 read 0x021828 thru 0x022fa9 (0x1782 bytes) 000972 1787698486.477311 trunc from 0x022faa to 0x021ee1 (0x10c8 bytes) 000976 1787698486.555335 trunc from 0x021ee2 to 0x005fcb (0x1bf16 bytes) 000977 1787698486.583313 write 0x03b1e3 thru 0x03febf (0x4cdd bytes) 000979 1787698486.585415 write 0x0245a8 thru 0x026a15 (0x246e bytes) 000980 1787698486.624582 MIRROR_RESYNC 4 mirrors 000984 1787698488.192697 trunc from 0x03fec0 to 0x03a57b (0x5944 bytes) 000985 1787698488.286163 mapwrite 0x02c336 thru 0x0347d5 (0x84a0 bytes) 000986 1787698488.353009 trunc from 0x03a57c to 0x01e4bd (0x1c0be bytes) 000987 1787698488.376129 read 0x019cea thru 0x01e4bd (0x47d4 bytes) 000990 1787698488.384808 mapread 0x00426a thru 0x00c99c (0x8733 bytes) 000991 1787698488.396802 read 0x010b7c thru 0x01850f (0x7994 bytes) 000995 1787698488.407804 read 0x015f52 thru 0x01c320 (0x63cf bytes) 000997 1787698488.409137 read 0x00c98c thru 0x00e973 (0x1fe8 bytes) 000998 1787698488.413435 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1de:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1e4be): Mirror 11: 0x877ca0b4 (stale) Mirror 12: 0x877ca0b4 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1e4be): Mirror 11: 0x877ca0b4 (stale) Mirror 12: 0x877ca0b4 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1e4be: skipped 001000 1787698488.712399 write 0x01ee80 thru 0x028b5e (0x9cdf bytes) All operations completed A-OK!