== sanity-flr test 70a: flr mode fsx test ================ 12:28:50 (1790785730) Seed set to 5732 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x36743 000001 1790785733.220666 trunc from 00000000 to 0x036742 (0x36743 bytes) 000002 1790785733.238267 write 0x03cc5c thru 0x03ffff (0x33a4 bytes) 000008 1790785733.244568 mapread 0x027d5a thru 0x032dab (0xb052 bytes) 000009 1790785733.268835 read 0x036df5 thru 0x03ffff (0x920b bytes) 000012 1790785733.275875 write 0x031e9b thru 0x034ed3 (0x3039 bytes) 000013 1790785733.277286 read 0x00c323 thru 0x01947a (0xd158 bytes) 000014 1790785733.288334 trunc from 0x040000 to 0x0177ee (0x28811 bytes) 000016 1790785733.309691 mapwrite 0x00d4f8 thru 0x019988 (0xc491 bytes) 000019 1790785733.384211 trunc from 0x019989 to 0x005200 (0x14788 bytes) 000021 1790785733.405759 trunc from 0x005201 to 0x02acbc (0x25abc bytes) 000023 1790785733.423167 trunc from 0x02acbd to 0x020103 (0xabb9 bytes) 000024 1790785733.446168 mapread 0x00d7c3 thru 0x01a08b (0xc8c9 bytes) 000026 1790785733.524075 mapwrite 0x0328f3 thru 0x03ffff (0xd70d bytes) 000028 1790785733.724241 punch from 0x00b7f8 to 0x01a33a (0xeb43 bytes) 000029 1790785733.741377 trunc from 0x040000 to 0x011858 (0x2e7a7 bytes) 000031 1790785733.764162 read 0x00d240 thru 0x011858 (0x4619 bytes) 000034 1790785733.784085 trunc from 0x011859 to 0x00132b (0x1052d bytes) 000035 1790785733.804486 punch from 0x001157 to 0x00132b (0x1d5 bytes) 000037 1790785733.814222 mapread 0x00041d thru 0x00132b (0xf0f bytes) 000038 1790785733.829360 mapread 0x00017b thru 0x00132b (0x11b1 bytes) 000040 1790785733.831953 mapread 0x000388 thru 0x00132b (0xfa4 bytes) 000042 1790785733.835429 mapwrite 0x00ce29 thru 0x016c53 (0x9e2b bytes) 000043 1790785733.941657 mapwrite 0x030888 thru 0x03ffff (0xf778 bytes) 000044 1790785734.070147 mapwrite 0x014a90 thru 0x0232e2 (0xe853 bytes) 000045 1790785734.140786 trunc from 0x040000 to 0x02eb54 (0x114ab bytes) 000047 1790785734.182868 trunc from 0x02eb55 to 0x00441e (0x2a736 bytes) 000050 1790785734.213262 write 0x00e409 thru 0x00ed38 (0x930 bytes) 000051 1790785734.214306 mapwrite 0x02ba17 thru 0x0335b6 (0x7ba0 bytes) 000053 1790785734.511891 read 0x00bd92 thru 0x00e463 (0x26d2 bytes) 000055 1790785734.531469 trunc from 0x0335b7 to 0x02e28f (0x5327 bytes) 000056 1790785734.559133 mapwrite 0x03b37d thru 0x03ffff (0x4c83 bytes) 000057 1790785734.726124 mapwrite 0x0049d9 thru 0x00a14f (0x5777 bytes) 000058 1790785734.833399 read 0x03b876 thru 0x03ffff (0x478a bytes) 000060 1790785734.834842 punch from 0x008fb3 to 0x013e7f (0xaecd bytes) 000061 1790785734.850930 MIRROR_EXTEND to 2 mirrors 000063 1790785736.166991 mapwrite 0x02caa4 thru 0x03c809 (0xfd66 bytes) 000064 1790785736.472540 punch from 0x024ef6 to 0x025bb6 (0xcc1 bytes) 000066 1790785736.484039 read 0x0208ba thru 0x02b848 (0xaf8f bytes) truncating to largest ever: 0x3e219 000068 1790785736.508199 trunc from 0x040000 to 0x03e218 (0x1de7 bytes) 000071 1790785736.566228 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) 000073 1790785736.701115 mapread 0x00faba thru 0x01aec4 (0xb40b bytes) 000075 1790785736.738071 read 0x006514 thru 0x00b389 (0x4e76 bytes) 000076 1790785736.747353 mapwrite 0x009f90 thru 0x00ba21 (0x1a92 bytes) 000077 1790785736.782021 mapread 0x01c092 thru 0x022bb6 (0x6b25 bytes) 000078 1790785736.783148 trunc from 0x03e219 to 0x00ae01 (0x33417 bytes) 000079 1790785736.805370 write 0x01e8ab thru 0x02159a (0x2cf0 bytes) 000082 1790785736.811061 write 0x009aaa thru 0x00f860 (0x5db7 bytes) 000084 1790785736.817160 punch from 0x0050c5 to 0x0148db (0xf817 bytes) 000086 1790785736.834164 mapwrite 0x03477e thru 0x03b2e9 (0x6b6c bytes) 000088 1790785736.980451 mapwrite 0x02e4b4 thru 0x03d3fa (0xef47 bytes) 000089 1790785737.193607 punch from 0x0001c3 to 0x00d2ac (0xd0ea bytes) 000091 1790785737.206598 mapwrite 0x03b70e thru 0x03ffff (0x48f2 bytes) 000092 1790785737.340208 trunc from 0x040000 to 0x013937 (0x2c6c8 bytes) 000093 1790785737.392696 mapread 0x00ce04 thru 0x013937 (0x6b34 bytes) 000094 1790785737.404781 read 0x008e90 thru 0x013937 (0xaaa8 bytes) 000095 1790785737.405409 write 0x03ac20 thru 0x03ffff (0x53e0 bytes) 000096 1790785737.420573 MIRROR_EXTEND to 2 mirrors 000097 1790785738.419163 write 0x01f587 thru 0x02db82 (0xe5fc bytes) 000098 1790785738.500202 punch from 0x0159db to 0x023742 (0xdd68 bytes) 000099 1790785738.513186 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xeeb6101 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x94caf2c6 Mirror 3: 0x94caf2c6 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000101 1790785740.331835 MIRROR_RESYNC 3 mirrors 000102 1790785741.668637 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) 000103 1790785742.229392 trunc from 0x040000 to 0x028476 (0x17b89 bytes) 000104 1790785742.402959 write 0x03411a thru 0x0343ea (0x2d1 bytes) 000105 1790785742.407847 punch from 0x017362 to 0x01dadb (0x677a bytes) 000106 1790785742.421782 mapread 0x0308d2 thru 0x03192b (0x105a bytes) 000107 1790785742.435342 write 0x025903 thru 0x030092 (0xa790 bytes) 000108 1790785742.481103 mapread 0x02d344 thru 0x02f6bd (0x237a bytes) 000109 1790785742.495278 trunc from 0x0343eb to 0x01c194 (0x18256 bytes) 000110 1790785742.549484 trunc from 0x01c195 to 0x039d4b (0x1dbb7 bytes) 000112 1790785742.610501 read 0x026911 thru 0x0365c0 (0xfcb0 bytes) 000113 1790785742.625983 read 0x028425 thru 0x02de00 (0x59dc bytes) 000114 1790785742.632622 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x39d4c): Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x39d4c): Mirror 3: 0x13410b53 (stale) CRC-32 checksum value for chunk [0, 0x39d4c): Mirror 1: 0xd624d110 Mirror 4: 0xd624d110 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x39d4c: skipped 000115 1790785744.838223 read 0x03564d thru 0x036f30 (0x18e4 bytes) 000118 1790785744.890025 write 0x036b5d thru 0x03bbe4 (0x5088 bytes) 000122 1790785745.007425 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x3bbe5): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3bbe5): Mirror 3: 0xd3cf8fa6 (stale) Mirror 4: 0x743bbe1 (stale) CRC-32 checksum value for chunk [0, 0x3bbe5): Mirror 1: 0x32eb7e07 Mirror 5: 0x32eb7e07 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3bbe5: skipped 000123 1790785746.815385 mapwrite 0x02e257 thru 0x03d2dc (0xf086 bytes) 000125 1790785747.112972 read 0x00f606 thru 0x0111cb (0x1bc6 bytes) 000126 1790785747.119735 read 0x005c77 thru 0x0114be (0xb848 bytes) 000129 1790785747.139426 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x3d2dd): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3d2dd): Mirror 3: 0x861d0a31 (stale) Mirror 4: 0xee26bf39 (stale) Mirror 5: 0xb3b7f946 (stale) CRC-32 checksum value for chunk [0, 0x3d2dd): Mirror 1: 0xda7a3ee3 Mirror 6: 0xda7a3ee3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3d2dd: skipped 000131 1790785748.357946 mapwrite 0x0328f8 thru 0x03e3ce (0xbad7 bytes) 000132 1790785748.747347 punch from 0x012f4c to 0x01dc81 (0xad36 bytes) 000133 1790785748.766027 write 0x03c554 thru 0x03ff7b (0x3a28 bytes) 000134 1790785748.771736 trunc from 0x03ff7c to 0x00c286 (0x33cf5 bytes) 000135 1790785748.815254 write 0x007eda thru 0x00e1c8 (0x62ef bytes) 000137 1790785748.827869 read 0x003254 thru 0x00e1c8 (0xaf75 bytes) 000138 1790785748.852898 MIRROR_SPLIT mirror 4 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0xe1c9): Mirror 3: 0x527e8b (stale) Mirror 4: 0x527e8b (stale) Mirror 5: 0x527e8b (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xe1c9): Mirror 3: 0x527e8b (stale) Mirror 4: 0x527e8b (stale) Mirror 5: 0x527e8b (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xe1c9: skipped 000139 1790785749.390348 read 0x0032b5 thru 0x0052a4 (0x1ff0 bytes) 000141 1790785749.397686 punch from 0x00cdaa to 0x00e1c8 (0x141f bytes) 000142 1790785749.407445 MIRROR_RESYNC 4 mirrors 000143 1790785751.277662 write 0x034c06 thru 0x03ffff (0xb3fa bytes) 000144 1790785751.441021 write 0x022eeb thru 0x0294b4 (0x65ca bytes) 000146 1790785751.472715 mapread 0x01e5e3 thru 0x01f697 (0x10b5 bytes) 000147 1790785751.509831 read 0x00b987 thru 0x0163ad (0xaa27 bytes) 000148 1790785751.540441 mapread 0x00b4b3 thru 0x016507 (0xb055 bytes) 000150 1790785751.551234 trunc from 0x040000 to 0x0254d9 (0x1ab26 bytes) 000151 1790785751.603343 trunc from 0x0254da to 0x01d3ae (0x812b bytes) 000152 1790785751.640953 punch from 0x00a5d1 to 0x00f9b3 (0x53e3 bytes) 000153 1790785751.651413 mapread 0x0159f2 thru 0x0179a9 (0x1fb8 bytes) 000154 1790785751.672319 read 0x003662 thru 0x00cefd (0x989c bytes) 000155 1790785751.703648 mapwrite 0x01611d thru 0x025075 (0xef59 bytes) 000156 1790785751.964997 punch from 0x014614 to 0x020cda (0xc6c7 bytes) 000157 1790785751.972894 trunc from 0x025076 to 0x018c78 (0xc3fd bytes) 000159 1790785752.034989 punch from 0x0172b0 to 0x018c78 (0x19c9 bytes) 000160 1790785752.067206 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x18c79): Mirror 3: 0xf37a3a4c (stale) Mirror 5: 0xf37a3a4c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x18c79): Mirror 3: 0xf37a3a4c (stale) Mirror 5: 0xf37a3a4c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x18c79: skipped 000161 1790785752.879025 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x18c79): Mirror 5: 0xf37a3a4c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x18c79): Mirror 5: 0xf37a3a4c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x18c79: skipped 000162 1790785753.607495 punch from 0x013e64 to 0x013fb9 (0x156 bytes) 000163 1790785753.616063 mapread 0x015e4e thru 0x018c78 (0x2e2b bytes) 000164 1790785753.672955 write 0x00e719 thru 0x00e8f2 (0x1da bytes) 000166 1790785753.679490 mapread 0x00ac7f thru 0x01375a (0x8adc bytes) 000167 1790785753.696004 mapwrite 0x036ffa thru 0x03ffff (0x9006 bytes) 000168 1790785753.912372 mapwrite 0x007262 thru 0x013d26 (0xcac5 bytes) 000169 1790785754.059417 MIRROR_RESYNC 2 mirrors 000170 1790785755.726407 write 0x020ca7 thru 0x026f15 (0x626f bytes) 000171 1790785755.960210 trunc from 0x040000 to 0x0047c5 (0x3b83a bytes) 000173 1790785756.024496 mapread 0x000b21 thru 0x0047c5 (0x3ca5 bytes) 000176 1790785756.110597 trunc from 0x0047c6 to 0x0289d8 (0x24213 bytes) 000178 1790785756.158467 punch from 0x017959 to 0x023923 (0xbfcb bytes) 000179 1790785756.180005 write 0x0265ab thru 0x033f13 (0xd969 bytes) 000181 1790785756.210054 write 0x03428e thru 0x0362c7 (0x203a bytes) 000182 1790785756.211394 trunc from 0x0362c8 to 0x007ba6 (0x2e721 bytes) 000183 1790785756.289711 MIRROR_RESYNC 2 mirrors 000184 1790785757.764049 trunc from 0x007ba7 to 0x02483f (0x1cc99 bytes) 000185 1790785757.885499 punch from 0x0233b1 to 0x02483f (0x148f bytes) 000187 1790785757.903199 MIRROR_RESYNC 2 mirrors 000189 1790785758.967597 mapread 0x0207ca thru 0x02483f (0x4076 bytes) 000190 1790785758.996831 write 0x02ff0f thru 0x032f11 (0x3003 bytes) 000191 1790785759.086153 write 0x03ccbe thru 0x03ffff (0x3342 bytes) 000192 1790785759.088253 write 0x034a21 thru 0x03f2f0 (0xa8d0 bytes) 000193 1790785759.096190 mapwrite 0x03f01e thru 0x03f94b (0x92e bytes) 000194 1790785759.168035 trunc from 0x040000 to 0x011d6e (0x2e291 bytes) 000195 1790785759.199330 write 0x007bdd thru 0x00946a (0x188e bytes) 000196 1790785759.214576 mapread 0x004b7f thru 0x00bb81 (0x7003 bytes) 000198 1790785759.228365 write 0x012158 thru 0x01a753 (0x85fc bytes) 000199 1790785759.231285 write 0x020e10 thru 0x027c6e (0x6e5f bytes) 000203 1790785759.240142 mapread 0x000276 thru 0x00ab90 (0xa91b bytes) 000204 1790785759.247596 mapwrite 0x02f4de thru 0x0317ee (0x2311 bytes) 000205 1790785759.335979 mapwrite 0x028725 thru 0x030b9a (0x8476 bytes) 000206 1790785759.390939 read 0x007d78 thru 0x00d10a (0x5393 bytes) 000207 1790785759.392168 write 0x0069b8 thru 0x013b82 (0xd1cb bytes) 000209 1790785759.398598 mapwrite 0x03377c thru 0x0367dc (0x3061 bytes) 000211 1790785759.520291 read 0x00234a thru 0x00ff34 (0xdbeb bytes) 000212 1790785759.521269 write 0x01406e thru 0x01f0e7 (0xb07a bytes) 000213 1790785759.531019 trunc from 0x0367dd to 0x037e53 (0x1677 bytes) 000214 1790785759.553000 mapread 0x023260 thru 0x032d00 (0xfaa1 bytes) 000215 1790785759.555792 punch from 0x0085d1 to 0x008705 (0x135 bytes) 000216 1790785759.572195 read 0x001762 thru 0x007466 (0x5d05 bytes) 000217 1790785759.573444 mapread 0x028a7d thru 0x02dfea (0x556e bytes) 000218 1790785759.576432 write 0x0382f3 thru 0x03cca3 (0x49b1 bytes) 000219 1790785759.579409 mapread 0x00e33a thru 0x0153b2 (0x7079 bytes) 000221 1790785759.582109 mapread 0x02f976 thru 0x0344af (0x4b3a bytes) 000223 1790785759.585073 punch from 0x03b3bb to 0x03cca3 (0x18e9 bytes) 000224 1790785759.601438 read 0x0369a9 thru 0x03cca3 (0x62fb bytes) 000225 1790785759.614138 write 0x027152 thru 0x034db8 (0xdc67 bytes) 000226 1790785759.624470 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) 000229 1790785759.803079 punch from 0x0124b0 to 0x015b24 (0x3675 bytes) 000230 1790785759.810060 trunc from 0x03cca4 to 0x009b71 (0x33132 bytes) 000231 1790785759.833620 trunc from 0x009b72 to 0x02c4e7 (0x22976 bytes) 000232 1790785759.860015 write 0x01c608 thru 0x023345 (0x6d3e bytes) 000233 1790785759.873477 mapwrite 0x02b8eb thru 0x03221a (0x6930 bytes) 000234 1790785760.102977 mapread 0x00e0c1 thru 0x016bfa (0x8b3a bytes) 000235 1790785760.126309 write 0x034054 thru 0x03ffff (0xbfac bytes) 000236 1790785760.129193 mapread 0x00c0dd thru 0x01780c (0xb730 bytes) 000239 1790785760.156000 trunc from 0x040000 to 0x039c00 (0x63ff bytes) 000240 1790785760.204966 mapread 0x01e1a3 thru 0x01ec8a (0xae8 bytes) 000241 1790785760.206196 trunc from 0x039c01 to 0x0273df (0x12821 bytes) 000242 1790785760.244058 punch from 0x020293 to 0x0273df (0x714d bytes) 000243 1790785760.251936 trunc from 0x0273e0 to 0x0098d1 (0x1db0e bytes) 000245 1790785760.299462 MIRROR_EXTEND to 2 mirrors 000246 1790785761.243592 mapwrite 0x034e0f thru 0x0375a8 (0x279a bytes) 000247 1790785761.441848 write 0x032bf4 thru 0x03ffff (0xd40c bytes) 000248 1790785761.457101 MIRROR_RESYNC 2 mirrors 000250 1790785762.829865 read 0x039c3f thru 0x03ffff (0x63c1 bytes) 000256 1790785762.842552 mapread 0x037b03 thru 0x03ffff (0x84fd bytes) 000259 1790785762.858870 write 0x0208e4 thru 0x027bbb (0x72d8 bytes) 000260 1790785762.994463 write 0x009ccc thru 0x00ced4 (0x3209 bytes) 000261 1790785763.010728 read 0x01842d thru 0x0203da (0x7fae bytes) 000262 1790785763.040092 MIRROR_RESYNC 2 mirrors 000265 1790785764.282276 read 0x01a1b3 thru 0x01a290 (0xde bytes) 000266 1790785764.300106 read 0x011cb1 thru 0x017924 (0x5c74 bytes) 000267 1790785764.332826 MIRROR_RESYNC 2 mirrors 000272 1790785764.872902 punch from 0x03f49b to 0x03ffff (0xb65 bytes) 000273 1790785764.945146 read 0x02b43d thru 0x0336fe (0x82c2 bytes) 000274 1790785764.955071 trunc from 0x040000 to 0x02dd56 (0x122a9 bytes) 000275 1790785765.015033 trunc from 0x02dd57 to 0x00cccb (0x2108b bytes) 000278 1790785765.040291 punch from 0x0001f2 to 0x000259 (0x68 bytes) 000284 1790785765.052394 punch from 0x002491 to 0x00cccb (0xa83b bytes) 000285 1790785765.063193 MIRROR_RESYNC 2 mirrors 000287 1790785766.143769 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) 000289 1790785766.409443 punch from 0x00684c to 0x006b9b (0x350 bytes) 000293 1790785766.481634 MIRROR_EXTEND to 2 mirrors 000294 1790785767.826354 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) 000295 1790785768.052561 mapwrite 0x032bce thru 0x03318d (0x5c0 bytes) 000298 1790785768.237861 mapread 0x00dc27 thru 0x01cc9f (0xf079 bytes) 000299 1790785768.273037 mapread 0x02155f thru 0x0282b5 (0x6d57 bytes) 000300 1790785768.297064 mapwrite 0x0161f2 thru 0x018f71 (0x2d80 bytes) 000301 1790785768.367248 read 0x013734 thru 0x0149c2 (0x128f bytes) 000302 1790785768.367889 read 0x02d285 thru 0x03318d (0x5f09 bytes) 000303 1790785768.376475 mapread 0x01c223 thru 0x01f03f (0x2e1d bytes) 000305 1790785768.377880 mapread 0x000eeb thru 0x0090db (0x81f1 bytes) 000306 1790785768.379308 mapread 0x0160df thru 0x01874a (0x266c bytes) 000307 1790785768.380559 trunc from 0x03318e to 0x00270d (0x30a80 bytes) 000308 1790785768.403080 mapread 0x000395 thru 0x00270d (0x2379 bytes) 000309 1790785768.405075 mapread 0x000f50 thru 0x00270d (0x17be bytes) 000310 1790785768.407077 write 0x01890b thru 0x024807 (0xbefd bytes) 000312 1790785768.410542 mapread 0x013ffa thru 0x01c499 (0x84a0 bytes) 000313 1790785768.433650 punch from 0x00d52d to 0x014219 (0x6ced bytes) 000314 1790785768.460535 mapread 0x01502b thru 0x01c31d (0x72f3 bytes) 000316 1790785768.462254 trunc from 0x024808 to 0x01633d (0xe4ca bytes) 000317 1790785768.503015 trunc from 0x01633e to 0x035bea (0x1f8ad bytes) 000318 1790785768.556028 trunc from 0x035beb to 0x03a26e (0x4684 bytes) 000320 1790785768.599158 mapread 0x031e1f thru 0x037d91 (0x5f73 bytes) 000321 1790785768.637813 write 0x005eb7 thru 0x014818 (0xe962 bytes) 000324 1790785768.668265 read 0x00e94b thru 0x015afd (0x71b3 bytes) 000325 1790785768.669158 read 0x0346f0 thru 0x03a26e (0x5b7f bytes) 000326 1790785768.670420 read 0x003ac5 thru 0x00429e (0x7da bytes) 000327 1790785768.691328 read 0x006d8a thru 0x012778 (0xb9ef bytes) 000330 1790785768.700156 mapwrite 0x02ef70 thru 0x038109 (0x919a bytes) 000332 1790785768.900180 mapwrite 0x02d187 thru 0x02e04d (0xec7 bytes) 000333 1790785769.038817 write 0x0011c6 thru 0x00479b (0x35d6 bytes) 000334 1790785769.044959 mapwrite 0x00853e thru 0x016329 (0xddec bytes) 000336 1790785769.210741 trunc from 0x03a26f to 0x025f06 (0x14368 bytes) 000337 1790785769.235565 punch from 0x021ebd to 0x025f06 (0x404a bytes) 000338 1790785769.242119 mapread 0x007473 thru 0x0169d5 (0xf563 bytes) 000339 1790785769.244074 trunc from 0x025f07 to 0x00a0b3 (0x1be53 bytes) 000340 1790785769.287033 write 0x026a8e thru 0x02b7d8 (0x4d4b bytes) 000342 1790785769.291229 punch from 0x01301a to 0x01743b (0x4422 bytes) 000343 1790785769.307461 trunc from 0x02b7d9 to 0x022d4f (0x8a89 bytes) 000345 1790785769.336851 mapwrite 0x01f349 thru 0x0217bd (0x2475 bytes) 000346 1790785769.509590 trunc from 0x022d50 to 0x02326d (0x51e bytes) 000348 1790785769.577639 mapwrite 0x038fde thru 0x03aa13 (0x1a36 bytes) 000349 1790785769.772239 trunc from 0x03aa14 to 0x0373d5 (0x363e bytes) 000350 1790785769.823965 trunc from 0x0373d6 to 0x01d0bd (0x1a318 bytes) 000351 1790785769.891025 punch from 0x00a750 to 0x01434b (0x9bfc bytes) 000352 1790785769.907082 mapwrite 0x016e56 thru 0x020db9 (0x9f64 bytes) 000357 1790785770.113389 trunc from 0x020dba to 0x01b72d (0x568c bytes) 000358 1790785770.150314 trunc from 0x01b72e to 0x038b22 (0x1d3f5 bytes) 000364 1790785770.217974 read 0x0207d4 thru 0x028198 (0x79c5 bytes) 000365 1790785770.235894 punch from 0x021631 to 0x029119 (0x7ae9 bytes) 000367 1790785770.246948 mapwrite 0x01fc60 thru 0x02da90 (0xde31 bytes) 000368 1790785770.462072 mapread 0x00dc16 thru 0x0193d6 (0xb7c1 bytes) 000370 1790785770.473882 read 0x02d028 thru 0x038b22 (0xbafb bytes) 000372 1790785770.479062 mapwrite 0x01498e thru 0x022609 (0xdc7c bytes) 000373 1790785770.617720 write 0x02b336 thru 0x02b77f (0x44a bytes) 000375 1790785770.619169 mapread 0x0186c5 thru 0x02428c (0xbbc8 bytes) 000376 1790785770.633800 MIRROR_EXTEND to 2 mirrors 000378 1790785772.411680 write 0x0112f5 thru 0x01ceca (0xbbd6 bytes) 000379 1790785772.556046 trunc from 0x038b23 to 0x03499f (0x4183 bytes) 000380 1790785772.594001 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x349a0): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x349a0): Mirror 2: 0xa577eebc (stale) CRC-32 checksum value for chunk [0, 0x349a0): Mirror 1: 0x30d79806 Mirror 3: 0x30d79806 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x349a0: skipped 000381 1790785774.568913 trunc from 0x0349a0 to 0x0258b3 (0xf0ec bytes) 000382 1790785774.666735 mapwrite 0x01b84e thru 0x01e0a6 (0x2859 bytes) 000384 1790785774.800758 mapwrite 0x00caf5 thru 0x01ca2b (0xff37 bytes) 000385 1790785774.926915 read 0x017316 thru 0x025050 (0xdd3b bytes) 000386 1790785774.933455 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x258b4): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x258b4): Mirror 2: 0xf147fe40 (stale) Mirror 3: 0x36678e75 (stale) CRC-32 checksum value for chunk [0, 0x258b4): Mirror 1: 0xffdc0d68 Mirror 4: 0xffdc0d68 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x258b4: skipped 000388 1790785776.884989 read 0x023efb thru 0x0258b3 (0x19b9 bytes) 000389 1790785776.901925 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x258b4): Mirror 2: 0xf147fe40 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x258b4): Mirror 2: 0xf147fe40 (stale) CRC-32 checksum value for chunk [0, 0x258b4): Mirror 1: 0xffdc0d68 Mirror 4: 0xffdc0d68 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x258b4: skipped 000391 1790785777.599372 mapwrite 0x00e8be thru 0x0167b0 (0x7ef3 bytes) 000392 1790785777.743258 punch from 0x00589d to 0x005bd6 (0x33a bytes) 000393 1790785777.747206 mapwrite 0x028b36 thru 0x0325c8 (0x9a93 bytes) 000394 1790785777.827795 trunc from 0x0325c9 to 0x0110e5 (0x214e3 bytes) 000396 1790785777.843011 mapread 0x003455 thru 0x010a89 (0xd635 bytes) 000397 1790785777.853875 punch from 0x0021ef to 0x0110e5 (0xeef7 bytes) 000398 1790785777.862408 MIRROR_RESYNC 3 mirrors 000399 1790785779.150190 write 0x03201a thru 0x03f269 (0xd250 bytes) 000400 1790785779.228669 trunc from 0x03f26a to 0x01a23e (0x2502b bytes) 000401 1790785779.261925 write 0x00d0c1 thru 0x00f5ba (0x24fa bytes) 000404 1790785779.278107 trunc from 0x01a23f to 0x037208 (0x1cfca bytes) 000405 1790785779.338579 mapwrite 0x02b02e thru 0x02da0d (0x29e0 bytes) 000409 1790785779.507688 mapwrite 0x01daff thru 0x02ad9b (0xd29d bytes) 000410 1790785779.609310 trunc from 0x037209 to 0x015c39 (0x215cf bytes) 000411 1790785779.659122 read 0x005b12 thru 0x00e411 (0x8900 bytes) 000412 1790785779.685943 punch from 0x004c21 to 0x00e997 (0x9d77 bytes) 000414 1790785779.703376 write 0x00f35f thru 0x0161b6 (0x6e58 bytes) 000415 1790785779.711890 read 0x00feea thru 0x014337 (0x444e bytes) 000416 1790785779.713804 punch from 0x00cac5 to 0x0115d3 (0x4b0f bytes) 000417 1790785779.753841 mapwrite 0x01a06d thru 0x01cff6 (0x2f8a bytes) 000418 1790785779.873466 read 0x0113d0 thru 0x01589f (0x44d0 bytes) 000419 1790785779.874256 read 0x009ea1 thru 0x01032a (0x648a bytes) 000420 1790785779.876078 read 0x000a01 thru 0x002174 (0x1774 bytes) 000421 1790785779.877298 mapwrite 0x030080 thru 0x0352ce (0x524f bytes) 000422 1790785780.000373 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x352cf): Mirror 2: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x352cf): Mirror 2: 0xaf27d6e0 (stale) Mirror 4: 0xaf27d6e0 (stale) CRC-32 checksum value for chunk [0, 0x352cf): Mirror 1: 0x8553d499 Mirror 5: 0x8553d499 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x352cf: skipped 000424 1790785781.807059 mapread 0x0219f0 thru 0x0228a7 (0xeb8 bytes) 000425 1790785781.820882 mapread 0x00c50a thru 0x019459 (0xcf50 bytes) 000426 1790785781.848872 read 0x00a4a6 thru 0x018f15 (0xea70 bytes) 000427 1790785781.850292 punch from 0x02e1aa to 0x0352ce (0x7125 bytes) 000429 1790785781.974121 punch from 0x01a9cb to 0x029024 (0xe65a bytes) 000430 1790785781.980262 read 0x0331b8 thru 0x033d46 (0xb8f bytes) 000431 1790785782.021669 mapwrite 0x00e665 thru 0x018db5 (0xa751 bytes) 000433 1790785782.099475 punch from 0x02d272 to 0x0352ce (0x805d bytes) 000434 1790785782.109891 mapread 0x0054dd thru 0x0074d7 (0x1ffb bytes) 000435 1790785782.111128 write 0x02dd4c thru 0x0340bf (0x6374 bytes) 000436 1790785782.137840 mapread 0x005b6a thru 0x00f160 (0x95f7 bytes) 000438 1790785782.142782 trunc from 0x0352cf to 0x01d54e (0x17d80 bytes) 000440 1790785782.182781 write 0x0028e3 thru 0x00ff61 (0xd67f bytes) 000442 1790785782.192836 mapwrite 0x03323e thru 0x0366b1 (0x3474 bytes) 000444 1790785782.346382 mapread 0x02ae9d thru 0x0343e9 (0x954d bytes) 000445 1790785782.421834 mapread 0x01039a thru 0x01462e (0x4295 bytes) 000446 1790785782.423893 mapwrite 0x02feda thru 0x034cc7 (0x4dee bytes) 000447 1790785782.508980 write 0x00ca58 thru 0x00e643 (0x1bec bytes) 000448 1790785782.512259 mapread 0x00c601 thru 0x011c8a (0x568a bytes) 000450 1790785782.513880 mapwrite 0x039531 thru 0x03ffff (0x6acf bytes) 000451 1790785782.598113 mapwrite 0x013df5 thru 0x01cee8 (0x90f4 bytes) 000453 1790785782.651652 mapread 0x002395 thru 0x007149 (0x4db5 bytes) 000454 1790785782.652687 punch from 0x037578 to 0x03b4af (0x3f38 bytes) 000455 1790785782.661367 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x31f39437 (stale) Mirror 4: 0x31f39437 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x31f39437 (stale) Mirror 4: 0x31f39437 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000458 1790785783.044168 read 0x0247e4 thru 0x028e64 (0x4681 bytes) 000459 1790785783.057662 mapread 0x03126e thru 0x034bd4 (0x3967 bytes) 000461 1790785783.080553 mapwrite 0x01b98f thru 0x027178 (0xb7ea bytes) 000462 1790785783.152202 write 0x0384d5 thru 0x03ffff (0x7b2b bytes) 000464 1790785783.157112 trunc from 0x040000 to 0x004570 (0x3ba8f bytes) 000466 1790785783.187292 write 0x037cba thru 0x038fe1 (0x1328 bytes) 000468 1790785783.189048 read 0x0018f9 thru 0x00b5c2 (0x9cca bytes) 000469 1790785783.210643 mapread 0x03459b thru 0x038fe1 (0x4a47 bytes) 000470 1790785783.214174 write 0x0328c2 thru 0x03c708 (0x9e47 bytes) 000472 1790785783.217181 mapwrite 0x0241fb thru 0x0278f0 (0x36f6 bytes) 000473 1790785783.336613 punch from 0x02e16a to 0x02fbc0 (0x1a57 bytes) 000474 1790785783.350024 mapread 0x01df78 thru 0x0240b6 (0x613f bytes) 000475 1790785783.353358 punch from 0x025126 to 0x02c7d3 (0x76ae bytes) 000476 1790785783.372505 write 0x03dde1 thru 0x03ffff (0x221f bytes) 000477 1790785783.378561 punch from 0x03f6e3 to 0x03ffff (0x91d bytes) 000480 1790785783.417950 trunc from 0x040000 to 0x02e4b5 (0x11b4a bytes) 000481 1790785783.456982 read 0x008752 thru 0x0142c7 (0xbb76 bytes) 000482 1790785783.474235 MIRROR_RESYNC 3 mirrors 000483 1790785784.512134 punch from 0x02dbec to 0x02e4b5 (0x8ca bytes) 000484 1790785784.579551 write 0x0186b5 thru 0x02370d (0xb059 bytes) 000485 1790785784.592193 read 0x002183 thru 0x0041c3 (0x2041 bytes) 000486 1790785784.598387 read 0x009076 thru 0x015177 (0xc102 bytes) 000487 1790785784.612741 read 0x0064a2 thru 0x008f53 (0x2ab2 bytes) 000488 1790785784.615928 mapwrite 0x024a6e thru 0x02a280 (0x5813 bytes) 000489 1790785784.669391 write 0x0304d0 thru 0x036864 (0x6395 bytes) 000490 1790785784.675665 trunc from 0x036865 to 0x029319 (0xd54b bytes) 000494 1790785784.710371 trunc from 0x02931a to 0x00474a (0x24bcf bytes) 000495 1790785784.735684 punch from 0x002231 to 0x00474a (0x251a bytes) 000496 1790785784.741959 mapread 0x0034e1 thru 0x00474a (0x126a bytes) 000497 1790785784.753418 read 0x001ba4 thru 0x00474a (0x2ba7 bytes) 000498 1790785784.754447 punch from 0x00253a to 0x00474a (0x2211 bytes) 000500 1790785784.761394 mapwrite 0x02d908 thru 0x035e19 (0x8512 bytes) 000501 1790785784.847388 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x35e1a): Mirror 2: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x35e1a): Mirror 2: 0xfe5a98cb (stale) Mirror 4: 0xfe5a98cb (stale) CRC-32 checksum value for chunk [0, 0x35e1a): Mirror 1: 0xb85443c6 Mirror 5: 0xb85443c6 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x35e1a: skipped 000502 1790785785.935191 trunc from 0x035e1a to 0x02b6ec (0xa72d bytes) 000503 1790785786.007596 mapwrite 0x02e383 thru 0x03b716 (0xd394 bytes) 000504 1790785786.107936 trunc from 0x03b717 to 0x00144f (0x3a2c7 bytes) 000505 1790785786.126943 trunc from 0x001450 to 0x011dd4 (0x10985 bytes) 000506 1790785786.148714 write 0x001477 thru 0x004144 (0x2cce bytes) 000511 1790785786.162865 punch from 0x007aab to 0x00ec58 (0x71ae bytes) 000512 1790785786.174400 read 0x006c02 thru 0x011dd4 (0xb1d3 bytes) 000514 1790785786.183794 mapwrite 0x01741b thru 0x025fc2 (0xeba8 bytes) 000515 1790785786.270893 mapwrite 0x023bbe thru 0x0315ff (0xda42 bytes) 000516 1790785786.377317 read 0x008baf thru 0x00ea75 (0x5ec7 bytes) 000517 1790785786.378039 write 0x0177f7 thru 0x01aaa3 (0x32ad bytes) 000520 1790785786.386792 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x31600): Mirror 2: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x31600): Mirror 2: 0xc6f4c491 (stale) Mirror 4: 0xc6f4c491 (stale) Mirror 5: 0x8afaa0da (stale) CRC-32 checksum value for chunk [0, 0x31600): Mirror 1: 0x8873bd76 Mirror 6: 0x8873bd76 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x31600: skipped 000521 1790785787.544725 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x31600): Mirror 4: 0xc6f4c491 (stale) Mirror 5: 0x8afaa0da (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x31600): Mirror 4: 0xc6f4c491 (stale) Mirror 5: 0x8afaa0da (stale) CRC-32 checksum value for chunk [0, 0x31600): Mirror 1: 0x8873bd76 Mirror 6: 0x8873bd76 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x31600: skipped 000522 1790785788.046105 read 0x00b16f thru 0x016a79 (0xb90b bytes) 000523 1790785788.057263 mapwrite 0x0393f1 thru 0x03ffff (0x6c0f bytes) 000524 1790785788.224169 mapwrite 0x03d5a9 thru 0x03ffff (0x2a57 bytes) 000525 1790785788.280946 read 0x0008e9 thru 0x00e2be (0xd9d6 bytes) 000526 1790785788.297213 trunc from 0x040000 to 0x01a4d0 (0x25b2f bytes) 000529 1790785788.326261 read 0x005f02 thru 0x008d03 (0x2e02 bytes) 000530 1790785788.333112 MIRROR_RESYNC 4 mirrors 000531 1790785790.084867 MIRROR_EXTEND to 5 mirrors 000533 1790785790.959718 write 0x03e91c thru 0x03ffff (0x16e4 bytes) 000534 1790785791.046560 punch from 0x02c767 to 0x0365ec (0x9e86 bytes) 000535 1790785791.062339 trunc from 0x040000 to 0x015c55 (0x2a3aa bytes) 000537 1790785791.097394 read 0x006541 thru 0x015c55 (0xf715 bytes) 000538 1790785791.120001 mapwrite 0x0301fb thru 0x0348d3 (0x46d9 bytes) 000539 1790785791.255042 trunc from 0x0348d4 to 0x02f123 (0x57b0 bytes) 000541 1790785791.284036 mapread 0x02c444 thru 0x02f123 (0x2ce0 bytes) 000542 1790785791.286204 trunc from 0x02f124 to 0x0108c1 (0x1e862 bytes) 000543 1790785791.308119 write 0x009fc4 thru 0x017b09 (0xdb46 bytes) 000544 1790785791.310128 read 0x000edf thru 0x002a8d (0x1baf bytes) 000545 1790785791.320363 mapread 0x00a543 thru 0x010a0a (0x64c8 bytes) 000546 1790785791.322221 write 0x0124de thru 0x01f939 (0xd45c bytes) 000547 1790785791.324740 mapwrite 0x0149f7 thru 0x01ef34 (0xa53e bytes) 000548 1790785791.388848 mapread 0x01441d thru 0x01f939 (0xb51d bytes) 000549 1790785791.390460 punch from 0x01dc45 to 0x01f939 (0x1cf5 bytes) 000550 1790785791.397621 mapwrite 0x00080d thru 0x003039 (0x282d bytes) 000553 1790785791.455629 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1f93a): Mirror 5: 0x5e6ab4aa (stale) Mirror 6: 0x5e6ab4aa (stale) Mirror 7: 0x5e6ab4aa (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1f93a): Mirror 5: 0x5e6ab4aa (stale) Mirror 6: 0x5e6ab4aa (stale) Mirror 7: 0x5e6ab4aa (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1f93a: skipped 000555 1790785791.944491 punch from 0x00f888 to 0x01a384 (0xaafd bytes) 000558 1790785791.958684 trunc from 0x01f93a to 0x0259d0 (0x6097 bytes) 000559 1790785791.981984 trunc from 0x0259d1 to 0x0140bc (0x11914 bytes) 000560 1790785792.027154 punch from 0x00e1bd to 0x00f38e (0x11d2 bytes) 000561 1790785792.062593 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x140bd): Mirror 5: 0x92422e71 (stale) Mirror 7: 0x92422e71 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x140bd): Mirror 5: 0x92422e71 (stale) Mirror 7: 0x92422e71 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x140bd: skipped 000562 1790785792.513806 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x140bd): Mirror 7: 0x92422e71 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x140bd): Mirror 7: 0x92422e71 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x140bd: skipped 000563 1790785792.856823 mapread 0x00f616 thru 0x01275e (0x3149 bytes) 000564 1790785792.880278 trunc from 0x0140bd to 0x012cc6 (0x13f6 bytes) 000565 1790785792.900201 trunc from 0x012cc7 to 0x0101d6 (0x2af0 bytes) 000566 1790785792.920147 punch from 0x005ea6 to 0x0101d6 (0xa331 bytes) 000567 1790785792.927244 write 0x004205 thru 0x00dec1 (0x9cbd bytes) 000568 1790785792.940172 read 0x007066 thru 0x0101d6 (0x9171 bytes) 000569 1790785792.947565 mapread 0x00b8cb thru 0x0101d6 (0x490c bytes) 000570 1790785792.949082 trunc from 0x0101d7 to 0x032576 (0x223a0 bytes) 000571 1790785792.970835 mapwrite 0x013b07 thru 0x014c7f (0x1179 bytes) 000573 1790785793.055128 mapread 0x028826 thru 0x02d257 (0x4a32 bytes) 000575 1790785793.069178 mapwrite 0x0235e3 thru 0x02ccfd (0x971b bytes) 000576 1790785793.128492 mapwrite 0x02e152 thru 0x03c052 (0xdf01 bytes) 000577 1790785793.219207 MIRROR_RESYNC 2 mirrors 000578 1790785794.616182 trunc from 0x03c053 to 0x028659 (0x139f9 bytes) 000579 1790785794.741736 write 0x035fb7 thru 0x03ffff (0xa049 bytes) 000580 1790785794.745225 mapread 0x02e304 thru 0x03bcb0 (0xd9ad bytes) 000581 1790785794.787383 write 0x001054 thru 0x00913b (0x80e8 bytes) 000582 1790785794.813352 punch from 0x02a726 to 0x034f47 (0xa822 bytes) 000583 1790785794.850462 write 0x039249 thru 0x03ffff (0x6db7 bytes) 000584 1790785794.859185 mapwrite 0x034ff9 thru 0x03afe7 (0x5fef bytes) 000585 1790785795.018068 write 0x017118 thru 0x01d99c (0x6885 bytes) 000587 1790785795.033414 mapread 0x03a1b9 thru 0x03c8be (0x2706 bytes) 000588 1790785795.035440 write 0x017eb8 thru 0x01969e (0x17e7 bytes) 000589 1790785795.036555 write 0x03bdad thru 0x03ffff (0x4253 bytes) 000590 1790785795.037539 mapwrite 0x02ff37 thru 0x032c6f (0x2d39 bytes) 000591 1790785795.095060 read 0x02ba43 thru 0x034bd9 (0x9197 bytes) 000592 1790785795.109127 mapread 0x038f23 thru 0x03ffff (0x70dd bytes) 000593 1790785795.113410 write 0x013ebc thru 0x01d5a0 (0x96e5 bytes) 000594 1790785795.122307 mapread 0x0199ce thru 0x01db95 (0x41c8 bytes) 000595 1790785795.124967 write 0x01ef40 thru 0x02d754 (0xe815 bytes) 000597 1790785795.129205 mapwrite 0x021e40 thru 0x025de2 (0x3fa3 bytes) truncating to largest ever: 0x3e9d9 000598 1790785795.205230 trunc from 0x040000 to 0x03e9d8 (0x1627 bytes) 000600 1790785795.252752 write 0x02f39b thru 0x0328d0 (0x3536 bytes) 000603 1790785795.263195 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) 000604 1790785795.467005 read 0x038681 thru 0x03e890 (0x6210 bytes) 000605 1790785795.479229 trunc from 0x03e9d9 to 0x0130e7 (0x2b8f1 bytes) 000606 1790785795.509124 write 0x01ed32 thru 0x0206f2 (0x19c1 bytes) 000607 1790785795.528311 MIRROR_EXTEND to 2 mirrors 000608 1790785796.693816 mapwrite 0x0319b2 thru 0x03a6d2 (0x8d21 bytes) 000609 1790785796.876178 read 0x00b4d7 thru 0x010761 (0x528b bytes) 000610 1790785796.885423 trunc from 0x03a6d3 to 0x02cd10 (0xd9c2 bytes) 000612 1790785796.909291 mapwrite 0x0194d7 thru 0x0213e0 (0x7f0a bytes) 000615 1790785796.963185 trunc from 0x02cd11 to 0x024845 (0x84cb bytes) 000616 1790785796.979850 mapread 0x00b47a thru 0x01760b (0xc192 bytes) 000618 1790785796.989565 mapread 0x00c511 thru 0x019bf7 (0xd6e7 bytes) 000621 1790785796.992468 punch from 0x010527 to 0x019b7d (0x9657 bytes) 000623 1790785796.998270 trunc from 0x024846 to 0x0244c7 (0x37e bytes) 000625 1790785797.018861 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x244c8): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x244c8): Mirror 2: 0xa18017ba (stale) CRC-32 checksum value for chunk [0, 0x244c8): Mirror 1: 0x6e52ec0c Mirror 3: 0x6e52ec0c lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x244c8: skipped 000626 1790785798.476857 punch from 0x00e139 to 0x01d3bf (0xf287 bytes) 000627 1790785798.564195 punch from 0x0063e0 to 0x01276c (0xc38d bytes) 000630 1790785798.576142 trunc from 0x0244c8 to 0x025bd2 (0x170b bytes) 000631 1790785798.609828 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x25bd3): Mirror 3: 0xb2ce3c04 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x25bd3): Mirror 3: 0xb2ce3c04 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x25bd3: skipped 000632 1790785798.901669 mapwrite 0x005cb6 thru 0x00adbe (0x5109 bytes) 000633 1790785798.966944 mapread 0x01650e thru 0x0219ac (0xb49f bytes) 000635 1790785798.985292 read 0x01066f thru 0x015ebf (0x5851 bytes) 000637 1790785798.988223 write 0x01fa53 thru 0x020074 (0x622 bytes) 000638 1790785798.989802 mapwrite 0x02ed1a thru 0x039ce9 (0xafd0 bytes) 000641 1790785799.083064 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) 000646 1790785799.246915 punch from 0x02ab8e to 0x036d12 (0xc185 bytes) 000647 1790785799.252582 punch from 0x021b21 to 0x02e7a0 (0xcc80 bytes) 000648 1790785799.258685 read 0x01564a thru 0x01dd12 (0x86c9 bytes) 000651 1790785799.272632 mapwrite 0x017db5 thru 0x0211c2 (0x940e bytes) 000653 1790785799.340711 write 0x03a5c9 thru 0x03ffff (0x5a37 bytes) 000655 1790785799.344667 punch from 0x03bbfe to 0x03ffff (0x4402 bytes) 000658 1790785799.361239 write 0x013d45 thru 0x015c57 (0x1f13 bytes) 000659 1790785799.368255 punch from 0x03fe9f to 0x03ffff (0x161 bytes) 000661 1790785799.377934 write 0x0162d4 thru 0x01b459 (0x5186 bytes) 000663 1790785799.381424 punch from 0x033940 to 0x03ffff (0xc6c0 bytes) 000664 1790785799.397383 trunc from 0x040000 to 0x02630a (0x19cf5 bytes) 000665 1790785799.419593 punch from 0x011fcd to 0x021a35 (0xfa69 bytes) 000666 1790785799.429140 read 0x02231a thru 0x02630a (0x3ff1 bytes) 000667 1790785799.430266 punch from 0x00bff0 to 0x014463 (0x8474 bytes) 000668 1790785799.436416 trunc from 0x02630b to 0x0079f1 (0x1e919 bytes) 000669 1790785799.455498 punch from 0x0072f1 to 0x0079f1 (0x701 bytes) 000670 1790785799.464315 write 0x02c7c3 thru 0x033bf3 (0x7431 bytes) 000673 1790785799.466906 mapwrite 0x03f164 thru 0x03ffff (0xe9c bytes) 000674 1790785799.576129 mapread 0x0396e5 thru 0x03ffff (0x691b bytes) 000675 1790785799.577112 write 0x0229b5 thru 0x030182 (0xd7ce bytes) 000677 1790785799.586695 punch from 0x00a216 to 0x013554 (0x933f bytes) 000679 1790785799.603108 punch from 0x01300c to 0x01e792 (0xb787 bytes) 000680 1790785799.609150 mapwrite 0x009192 thru 0x0120fc (0x8f6b bytes) 000682 1790785799.709296 punch from 0x007a9a to 0x00ee42 (0x73a9 bytes) 000683 1790785799.717351 write 0x035cbf thru 0x03ac1b (0x4f5d bytes) 000684 1790785799.718606 read 0x0217f8 thru 0x02ab9c (0x93a5 bytes) 000686 1790785799.719695 punch from 0x00bf15 to 0x010cfa (0x4de6 bytes) 000689 1790785799.736485 punch from 0x00f6bd to 0x0103f2 (0xd36 bytes) 000691 1790785799.744376 punch from 0x00d611 to 0x00e961 (0x1351 bytes) 000692 1790785799.749949 mapread 0x001896 thru 0x00a1e2 (0x894d bytes) 000693 1790785799.760768 mapwrite 0x000126 thru 0x00f94d (0xf828 bytes) 000694 1790785799.829825 read 0x03a99f thru 0x03ffff (0x5661 bytes) 000698 1790785799.834008 mapwrite 0x018173 thru 0x02392a (0xb7b8 bytes) 000700 1790785799.882937 read 0x0218b0 thru 0x02b538 (0x9c89 bytes) 000701 1790785799.883758 punch from 0x02e59a to 0x039cf6 (0xb75d bytes) 000704 1790785799.892742 read 0x00ee8b thru 0x01a201 (0xb377 bytes) 000705 1790785799.894214 read 0x033607 thru 0x03ffff (0xc9f9 bytes) 000706 1790785799.934044 write 0x027499 thru 0x02e69d (0x7205 bytes) 000709 1790785799.946568 trunc from 0x040000 to 0x0115af (0x2ea50 bytes) 000712 1790785799.977374 read 0x002a68 thru 0x00f5be (0xcb57 bytes) 000714 1790785799.979967 mapread 0x00a028 thru 0x0115af (0x7588 bytes) 000715 1790785799.982289 mapread 0x006b01 thru 0x00bc52 (0x5152 bytes) 000717 1790785799.985161 read 0x0004ad thru 0x00ee0c (0xe960 bytes) 000718 1790785799.986268 punch from 0x00182d to 0x003429 (0x1bfd bytes) 000719 1790785799.992476 punch from 0x00c2fb to 0x00c4f6 (0x1fc bytes) 000721 1790785799.999433 mapread 0x006570 thru 0x00eaf6 (0x8587 bytes) 000722 1790785800.011231 read 0x001da5 thru 0x00f2e5 (0xd541 bytes) 000724 1790785800.013471 mapread 0x008eb2 thru 0x0115af (0x86fe bytes) 000727 1790785800.020939 mapread 0x00c86a thru 0x01101c (0x47b3 bytes) 000731 1790785800.026413 trunc from 0x0115b0 to 0x02d730 (0x1c181 bytes) 000732 1790785800.054051 trunc from 0x02d731 to 0x030551 (0x2e21 bytes) 000734 1790785800.078727 write 0x03a6f7 thru 0x03ffff (0x5909 bytes) 000735 1790785800.082893 write 0x030578 thru 0x031982 (0x140b bytes) 000736 1790785800.094863 punch from 0x009dde to 0x0183ba (0xe5dd bytes) 000737 1790785800.117262 mapwrite 0x026479 thru 0x02ed39 (0x88c1 bytes) 000738 1790785800.164437 write 0x01a271 thru 0x025163 (0xaef3 bytes) 000740 1790785800.166487 read 0x020d40 thru 0x02c81b (0xbadc bytes) 000741 1790785800.169040 mapread 0x03406e thru 0x03ffff (0xbf92 bytes) 000742 1790785800.172661 trunc from 0x040000 to 0x0239b4 (0x1c64b bytes) 000743 1790785800.193875 mapwrite 0x03fd38 thru 0x03ffff (0x2c8 bytes) 000744 1790785800.288730 punch from 0x037a09 to 0x03ffff (0x85f7 bytes) 000745 1790785800.298784 MIRROR_EXTEND to 2 mirrors 000746 1790785801.563940 punch from 0x012b6e to 0x01f7db (0xcc6e bytes) 000747 1790785801.748542 write 0x02868f thru 0x031de7 (0x9759 bytes) 000748 1790785801.781217 punch from 0x01c90c to 0x025733 (0x8e28 bytes) 000749 1790785801.815368 mapwrite 0x037e73 thru 0x03ffff (0x818d bytes) 000750 1790785801.959490 read 0x01374d thru 0x01d393 (0x9c47 bytes) 000752 1790785801.987123 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) 000756 1790785802.151602 read 0x032be5 thru 0x0393ad (0x67c9 bytes) 000757 1790785802.161612 punch from 0x028faf to 0x02eda6 (0x5df8 bytes) 000758 1790785802.167073 read 0x00f877 thru 0x011afb (0x2285 bytes) 000759 1790785802.179491 mapread 0x01de55 thru 0x028fd9 (0xb185 bytes) 000760 1790785802.180890 write 0x018d33 thru 0x027c67 (0xef35 bytes) 000763 1790785802.185699 read 0x02433b thru 0x033613 (0xf2d9 bytes) 000764 1790785802.186979 mapread 0x001ba1 thru 0x00eee0 (0xd340 bytes) 000765 1790785802.188976 punch from 0x01c94c to 0x01d731 (0xde6 bytes) 000766 1790785802.203226 mapread 0x03804a thru 0x03ffff (0x7fb6 bytes) 000769 1790785802.206740 mapwrite 0x03b5bf thru 0x03c1a1 (0xbe3 bytes) 000770 1790785802.257690 mapwrite 0x02f61e thru 0x03e799 (0xf17c bytes) 000771 1790785802.398572 mapread 0x01dd54 thru 0x022631 (0x48de bytes) 000772 1790785802.428686 punch from 0x01b7ff to 0x0205b9 (0x4dbb bytes) 000773 1790785802.447626 write 0x0289af thru 0x0363a7 (0xd9f9 bytes) 000774 1790785802.449175 trunc from 0x040000 to 0x0035fc (0x3ca03 bytes) 000776 1790785802.483283 mapwrite 0x02c57f thru 0x03726a (0xacec bytes) 000778 1790785802.650368 mapwrite 0x006a87 thru 0x009df1 (0x336b bytes) 000780 1790785802.734986 trunc from 0x03726b to 0x01c808 (0x1aa62 bytes) 000781 1790785802.765737 mapread 0x009ca1 thru 0x011bad (0x7f0d bytes) 000782 1790785802.804744 read 0x00a424 thru 0x012452 (0x802f bytes) 000783 1790785802.815220 mapwrite 0x037e6a thru 0x03ffff (0x8196 bytes) 000784 1790785802.883493 read 0x03598e thru 0x0393cd (0x3a40 bytes) 000786 1790785802.885101 read 0x02f017 thru 0x034efb (0x5ee5 bytes) 000787 1790785802.885753 punch from 0x025db7 to 0x02e37c (0x85c6 bytes) 000788 1790785802.891763 read 0x00959c thru 0x0190da (0xfb3f bytes) 000790 1790785802.893870 punch from 0x007883 to 0x01737f (0xfafd bytes) 000791 1790785802.898547 trunc from 0x040000 to 0x01c589 (0x23a76 bytes) 000793 1790785802.919725 write 0x015457 thru 0x01a732 (0x52dc bytes) 000794 1790785802.931038 punch from 0x00921d to 0x009770 (0x554 bytes) 000795 1790785802.944639 trunc from 0x01c58a to 0x02f365 (0x12ddc bytes) 000796 1790785802.963968 read 0x00816a thru 0x00c274 (0x410b bytes) 000797 1790785802.974881 trunc from 0x02f366 to 0x01e45b (0x10f0a bytes) 000800 1790785802.993740 read 0x00133d thru 0x0104c9 (0xf18d bytes) 000802 1790785802.995666 mapwrite 0x00d97c thru 0x00e6bd (0xd42 bytes) 000804 1790785803.039355 punch from 0x00de7f to 0x00f615 (0x1797 bytes) 000805 1790785803.045663 read 0x009539 thru 0x00be63 (0x292b bytes) 000806 1790785803.047225 trunc from 0x01e45c to 0x001a45 (0x1ca16 bytes) 000807 1790785803.070182 mapread 0x0009ce thru 0x001a45 (0x1078 bytes) 000808 1790785803.073122 punch from 0x00086a to 0x001a45 (0x11dc bytes) 000810 1790785803.079151 mapread 0x000b0d thru 0x001a45 (0xf39 bytes) 000814 1790785803.091784 trunc from 0x001a46 to 0x03a680 (0x38c3b bytes) 000818 1790785803.113731 mapwrite 0x03e924 thru 0x03ffff (0x16dc bytes) 000819 1790785803.183997 write 0x00dbd1 thru 0x01281a (0x4c4a bytes) 000820 1790785803.185346 punch from 0x01d329 to 0x0245b6 (0x728e bytes) 000825 1790785803.202908 read 0x039906 thru 0x03f2dc (0x59d7 bytes) 000826 1790785803.203725 write 0x02a3e7 thru 0x02a8ab (0x4c5 bytes) 000827 1790785803.205387 punch from 0x011957 to 0x016a74 (0x511e bytes) 000828 1790785803.214084 write 0x0361d4 thru 0x03ffff (0x9e2c bytes) 000829 1790785803.215385 read 0x03bc5b thru 0x03ffff (0x43a5 bytes) 000830 1790785803.216712 mapread 0x036990 thru 0x03ffff (0x9670 bytes) 000832 1790785803.222191 MIRROR_EXTEND to 2 mirrors 000833 1790785804.344228 mapwrite 0x0057d4 thru 0x011a1c (0xc249 bytes) 000834 1790785804.572493 trunc from 0x040000 to 0x001999 (0x3e666 bytes) 000835 1790785804.596768 mapwrite 0x014a68 thru 0x0213c3 (0xc95c bytes) 000836 1790785804.724721 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x213c4): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x213c4): Mirror 2: 0x1486048c (stale) CRC-32 checksum value for chunk [0, 0x213c4): Mirror 1: 0xa7d45100 Mirror 3: 0xa7d45100 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x213c4: skipped 000837 1790785806.476615 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x213c4): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x213c4): Mirror 2: 0x1486048c (stale) CRC-32 checksum value for chunk [0, 0x213c4): Mirror 1: 0xa7d45100 Mirror 3: 0xa7d45100 Mirror 4: 0xa7d45100 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x213c4: skipped 000844 1790785808.031692 mapread 0x00cfbe thru 0x01aaa9 (0xdaec bytes) 000845 1790785808.071465 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x213c4): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x213c4): Mirror 2: 0x1486048c (stale) CRC-32 checksum value for chunk [0, 0x213c4): Mirror 1: 0xa7d45100 Mirror 3: 0xa7d45100 Mirror 4: 0xa7d45100 Mirror 5: 0xa7d45100 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x213c4: skipped 000847 1790785809.547654 trunc from 0x0213c4 to 0x00d27b (0x14148 bytes) 000848 1790785809.616252 mapread 0x003857 thru 0x00552e (0x1cd8 bytes) 000850 1790785809.641341 trunc from 0x00d27c to 0x0141b4 (0x6f39 bytes) 000851 1790785809.679052 write 0x02cdd6 thru 0x0387c6 (0xb9f1 bytes) 000853 1790785809.692872 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x387c7): Mirror 3: 0x4ef13a7f (stale) Mirror 4: 0x4ef13a7f (stale) Mirror 5: 0x4ef13a7f (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x387c7): Mirror 3: 0x4ef13a7f (stale) Mirror 4: 0x4ef13a7f (stale) Mirror 5: 0x4ef13a7f (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x387c7: skipped truncating to largest ever: 0x3ea42 000855 1790785810.209384 trunc from 0x0387c7 to 0x03ea41 (0x627b bytes) 000856 1790785810.242954 mapwrite 0x018bb7 thru 0x022c43 (0xa08d bytes) 000857 1790785810.373276 read 0x006dbb thru 0x009e26 (0x306c bytes) 000861 1790785810.406360 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3ea42): Mirror 3: 0xf0bc2ffc (stale) Mirror 4: 0xf0bc2ffc (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3ea42): Mirror 3: 0xf0bc2ffc (stale) Mirror 4: 0xf0bc2ffc (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3ea42: skipped 000862 1790785810.767138 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x3ea42): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3ea42): Mirror 3: 0xf0bc2ffc (stale) Mirror 4: 0xf0bc2ffc (stale) CRC-32 checksum value for chunk [0, 0x3ea42): Mirror 1: 0xcb9320c9 Mirror 5: 0xcb9320c9 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3ea42: skipped 000864 1790785812.398137 read 0x00f4eb thru 0x014ede (0x59f4 bytes) 000865 1790785812.414418 mapwrite 0x02dd95 thru 0x02f3c0 (0x162c bytes) 000866 1790785812.661346 trunc from 0x03ea42 to 0x02eccf (0xfd72 bytes) 000868 1790785812.709372 mapwrite 0x030355 thru 0x035e16 (0x5ac2 bytes) 000869 1790785812.824233 mapwrite 0x009f86 thru 0x015f1e (0xbf99 bytes) 000872 1790785812.934385 mapread 0x035d16 thru 0x035e16 (0x101 bytes) 000873 1790785812.943006 MIRROR_RESYNC 4 mirrors 000874 1790785816.352434 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) 000875 1790785817.025722 MIRROR_EXTEND to 4 mirrors 000876 1790785818.326039 mapread 0x0110ff thru 0x01c3c0 (0xb2c2 bytes) 000877 1790785818.384647 MIRROR_RESYNC 4 mirrors 000878 1790785819.236354 mapwrite 0x00a1cf thru 0x012e01 (0x8c33 bytes) 000879 1790785819.476236 punch from 0x010a8a to 0x01bc74 (0xb1eb bytes) 000880 1790785819.484060 mapwrite 0x035d9a thru 0x03c01e (0x6285 bytes) 000881 1790785819.749629 mapwrite 0x0337f1 thru 0x03ffff (0xc80f bytes) 000882 1790785819.900836 write 0x0159a9 thru 0x016a2b (0x1083 bytes) 000884 1790785819.916094 trunc from 0x040000 to 0x0110f9 (0x2ef06 bytes) 000886 1790785819.946815 write 0x028dc1 thru 0x0369da (0xdc1a bytes) 000887 1790785819.948593 write 0x02c577 thru 0x02e0d4 (0x1b5e bytes) 000890 1790785819.958352 trunc from 0x0369db to 0x028961 (0xe079 bytes) 000892 1790785819.981222 write 0x00df78 thru 0x01616e (0x81f7 bytes) 000894 1790785819.989787 mapwrite 0x031854 thru 0x03e1ab (0xc958 bytes) 000896 1790785820.127355 punch from 0x0367e1 to 0x03b095 (0x48b5 bytes) 000897 1790785820.135348 write 0x03e804 thru 0x03ffff (0x17fc bytes) 000898 1790785820.136484 mapread 0x0251aa thru 0x030bac (0xba03 bytes) 000900 1790785820.150999 write 0x003484 thru 0x00aa93 (0x7610 bytes) 000902 1790785820.160418 punch from 0x03f26d to 0x03ffff (0xd93 bytes) 000905 1790785820.182839 mapwrite 0x02c43b thru 0x036e4a (0xaa10 bytes) 000906 1790785820.302435 mapwrite 0x011961 thru 0x016807 (0x4ea7 bytes) 000907 1790785820.400577 MIRROR_RESYNC 4 mirrors 000910 1790785824.226239 punch from 0x03fa10 to 0x03ffff (0x5f0 bytes) 000912 1790785824.382100 MIRROR_RESYNC 4 mirrors 000913 1790785828.177032 write 0x013384 thru 0x014180 (0xdfd bytes) 000917 1790785828.388117 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0x6b3a6a5 (stale) Mirror 6: 0x6b3a6a5 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0x6b3a6a5 (stale) Mirror 6: 0x6b3a6a5 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000918 1790785828.873434 read 0x02b2bf thru 0x03b039 (0xfd7b bytes) 000919 1790785828.883648 read 0x01467d thru 0x018748 (0x40cc bytes) 000920 1790785828.895107 trunc from 0x040000 to 0x00285a (0x3d7a5 bytes) 000923 1790785828.912179 write 0x020d94 thru 0x02d919 (0xcb86 bytes) 000925 1790785828.916228 punch from 0x005d03 to 0x00fd9e (0xa09c bytes) 000926 1790785828.932309 read 0x0295c1 thru 0x02b98d (0x23cd bytes) 000927 1790785828.936727 MIRROR_RESYNC 3 mirrors 000928 1790785831.908259 read 0x01c75f thru 0x024b4f (0x83f1 bytes) 000929 1790785831.936219 read 0x01bb80 thru 0x029eb7 (0xe338 bytes) 000930 1790785832.008917 MIRROR_RESYNC 3 mirrors 000931 1790785832.951729 mapread 0x02b88c thru 0x02d919 (0x208e bytes) 000932 1790785832.952680 mapwrite 0x03e5ca thru 0x03ffff (0x1a36 bytes) 000933 1790785833.302307 mapwrite 0x02762c thru 0x02b2b5 (0x3c8a bytes) 000934 1790785833.396010 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0x87f34191 (stale) Mirror 6: 0x87f34191 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xaa54382 Mirror 7: 0xaa54382 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000935 1790785835.417278 MIRROR_RESYNC 4 mirrors 000937 1790785838.301777 read 0x03e1c0 thru 0x03ffff (0x1e40 bytes) 000938 1790785838.313373 read 0x02ce88 thru 0x038ddd (0xbf56 bytes) 000941 1790785838.365059 MIRROR_RESYNC 4 mirrors 000942 1790785839.309571 mapread 0x02f13b thru 0x03ce7a (0xdd40 bytes) 000946 1790785839.311468 write 0x00f67e thru 0x01e820 (0xf1a3 bytes) 000947 1790785839.492490 trunc from 0x040000 to 0x02bfd8 (0x14027 bytes) 000948 1790785839.547761 trunc from 0x02bfd9 to 0x02d496 (0x14be bytes) 000949 1790785839.603539 mapread 0x01e237 thru 0x0200b8 (0x1e82 bytes) 000951 1790785839.656027 read 0x0264cb thru 0x02d496 (0x6fcc bytes) 000953 1790785839.682671 write 0x0065b4 thru 0x007b46 (0x1593 bytes) 000954 1790785839.734574 mapwrite 0x002ac5 thru 0x011374 (0xe8b0 bytes) 000955 1790785839.961226 trunc from 0x02d497 to 0x019e01 (0x13695 bytes) 000956 1790785840.080241 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x19e02): Mirror 5: 0x961c0e4a (stale) Mirror 7: 0x961c0e4a (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x19e02): Mirror 5: 0x961c0e4a (stale) Mirror 7: 0x961c0e4a (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x19e02: skipped 000957 1790785840.800120 write 0x003a25 thru 0x012b30 (0xf10c bytes) 000958 1790785840.813437 read 0x007f40 thru 0x017a55 (0xfb16 bytes) 000959 1790785840.825743 punch from 0x009e83 to 0x01545c (0xb5da bytes) 000960 1790785840.841554 punch from 0x018c89 to 0x019e01 (0x1179 bytes) 000961 1790785840.847173 mapread 0x00535a thru 0x0108fb (0xb5a2 bytes) 000962 1790785840.867768 mapwrite 0x02ae0b thru 0x0310e5 (0x62db bytes) 000963 1790785840.960924 MIRROR_RESYNC 3 mirrors 000964 1790785843.759206 read 0x00e08b thru 0x01c109 (0xe07f bytes) 000966 1790785843.797640 trunc from 0x0310e6 to 0x021e82 (0xf263 bytes) 000967 1790785843.960768 write 0x01f354 thru 0x0237a3 (0x4450 bytes) 000968 1790785843.986555 trunc from 0x0237a4 to 0x004d64 (0x1ea3f bytes) 000969 1790785844.028474 mapwrite 0x034711 thru 0x03a830 (0x6120 bytes) 000970 1790785844.220174 mapwrite 0x02fa63 thru 0x038bb5 (0x9153 bytes) 000971 1790785844.287197 read 0x00c020 thru 0x013819 (0x77fa bytes) 000972 1790785844.310706 write 0x000dbb thru 0x008400 (0x7646 bytes) 000973 1790785844.311843 read 0x027434 thru 0x036dbf (0xf98c bytes) 000975 1790785844.314342 read 0x0119a3 thru 0x0128e0 (0xf3e bytes) 000976 1790785844.315078 mapwrite 0x00f0bf thru 0x015599 (0x64db bytes) 000979 1790785844.420225 punch from 0x00be59 to 0x00e916 (0x2abe bytes) 000980 1790785844.445424 punch from 0x00bb10 to 0x015015 (0x9506 bytes) 000981 1790785844.452575 punch from 0x00a76e to 0x01000e (0x58a1 bytes) 000984 1790785844.475383 trunc from 0x03a831 to 0x00ef0b (0x2b925 bytes) 000985 1790785844.531689 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0xef0c): Mirror 7: 0xedaa6f0c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xef0c): Mirror 7: 0xedaa6f0c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xef0c: skipped 000987 1790785844.826065 punch from 0x00a017 to 0x00ef0b (0x4ef5 bytes) 000988 1790785844.832105 mapread 0x007be9 thru 0x00ef0b (0x7323 bytes) 000989 1790785844.863140 mapread 0x005141 thru 0x009edd (0x4d9d bytes) 000991 1790785844.864896 write 0x02e185 thru 0x02e524 (0x3a0 bytes) 000993 1790785844.869816 read 0x01e270 thru 0x025a18 (0x77a9 bytes) 000994 1790785844.882870 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x186:0x0]: Invalid argument (22) 000995 1790785845.041961 write 0x016aa2 thru 0x018469 (0x19c8 bytes) 000996 1790785845.057156 mapread 0x009ab1 thru 0x00fcec (0x623c bytes) 000997 1790785845.081331 MIRROR_EXTEND to 2 mirrors 000998 1790785846.326061 read 0x00f541 thru 0x01d770 (0xe230 bytes) All operations completed A-OK!