== sanity-flr test 70a: flr mode fsx test ================ 20:00:25 (1777593625) Seed set to 3626 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002 1777593626.429245 write 0x000ce0 thru 0x009865 (0x8b86 bytes) 000005 1777593626.432891 mapwrite 0x019f3a thru 0x01e127 (0x41ee bytes) 000007 1777593626.471291 mapread 0x00332f thru 0x009fc9 (0x6c9b bytes) 000008 1777593626.472383 mapwrite 0x021fbc thru 0x025d49 (0x3d8e bytes) 000009 1777593626.500533 mapwrite 0x008759 thru 0x00ecfa (0x65a2 bytes) 000011 1777593626.519864 mapread 0x00a3f4 thru 0x0146aa (0xa2b7 bytes) 000013 1777593626.523728 mapwrite 0x0357d1 thru 0x035fd4 (0x804 bytes) 000015 1777593626.549321 mapwrite 0x01e474 thru 0x01e8d0 (0x45d bytes) 000017 1777593626.566097 mapread 0x00a3d1 thru 0x01811a (0xdd4a bytes) 000018 1777593626.567087 mapwrite 0x0153f8 thru 0x024e1f (0xfa28 bytes) 000020 1777593626.584168 mapwrite 0x00d5a0 thru 0x013132 (0x5b93 bytes) 000022 1777593626.600264 mapread 0x0057b7 thru 0x0106de (0xaf28 bytes) 000023 1777593626.601182 write 0x0080ad thru 0x015f68 (0xdebc bytes) 000025 1777593626.602969 write 0x01a0c8 thru 0x01b53f (0x1478 bytes) 000027 1777593626.604077 read 0x02d9b2 thru 0x035fd4 (0x8623 bytes) truncating to largest ever: 0x1558a 000028 1777593626.607173 trunc from 0x035fd5 to 0x015589 (0x20a4b bytes) truncating to largest ever: 0x18d78 000029 1777593626.613463 trunc from 0x01558a to 0x018d77 (0x37ee bytes) 000030 1777593626.618143 read 0x003cdf thru 0x01380c (0xfb2e bytes) 000032 1777593626.619354 write 0x01ab66 thru 0x025292 (0xa72d bytes) 000034 1777593626.621227 mapread 0x00a7ca thru 0x0134a1 (0x8cd8 bytes) 000035 1777593626.622082 trunc from 0x025293 to 0x003ea1 (0x213f1 bytes) 000039 1777593626.629854 mapread 0x003717 thru 0x003ea1 (0x78b bytes) 000040 1777593626.630548 write 0x001821 thru 0x00e99d (0xd17d bytes) 000041 1777593626.631959 mapwrite 0x029b95 thru 0x02b638 (0x1aa4 bytes) 000043 1777593626.659483 mapwrite 0x00ac1c thru 0x010880 (0x5c65 bytes) 000046 1777593626.677989 write 0x00b250 thru 0x019ca1 (0xea52 bytes) truncating to largest ever: 0x2b2dc 000047 1777593626.679465 trunc from 0x02b639 to 0x02b2db (0x35d bytes) 000048 1777593626.685406 mapwrite 0x01b910 thru 0x01d095 (0x1786 bytes) 000050 1777593626.703062 mapwrite 0x01bca4 thru 0x01d33e (0x169b bytes) 000051 1777593626.717880 mapread 0x00c01a thru 0x015d9f (0x9d86 bytes) 000052 1777593626.719017 write 0x03f671 thru 0x03ffff (0x98f bytes) 000054 1777593626.720208 mapread 0x0168ac thru 0x0220e6 (0xb83b bytes) 000056 1777593626.721526 read 0x02e426 thru 0x0308e4 (0x24bf bytes) 000057 1777593626.725707 write 0x032c05 thru 0x03454b (0x1947 bytes) truncating to largest ever: 0x2f964 000059 1777593626.727679 trunc from 0x040000 to 0x02f963 (0x1069c bytes) 000062 1777593626.735951 write 0x017e66 thru 0x0216c9 (0x9864 bytes) 000063 1777593626.737142 mapwrite 0x037f3c thru 0x038544 (0x609 bytes) 000064 1777593626.764686 mapread 0x02380e thru 0x02d597 (0x9d8a bytes) 000065 1777593626.765859 mapread 0x01fb7c thru 0x02dacb (0xdf50 bytes) 000067 1777593626.767383 mapwrite 0x01a219 thru 0x02262f (0x8417 bytes) 000068 1777593626.786511 mapwrite 0x03da69 thru 0x03ffff (0x2597 bytes) 000069 1777593626.816138 mapwrite 0x0226ec thru 0x030448 (0xdd5d bytes) 000070 1777593626.837318 mapwrite 0x0311ba thru 0x0360a7 (0x4eee bytes) 000071 1777593626.870428 mapwrite 0x010beb thru 0x01b378 (0xa78e bytes) 000074 1777593626.891935 mapwrite 0x02eeef thru 0x0304ea (0x15fc bytes) 000075 1777593626.909939 write 0x03eda3 thru 0x03fe6c (0x10ca bytes) 000077 1777593626.911604 mapread 0x02e62b thru 0x034f8a (0x6960 bytes) 000078 1777593626.912606 write 0x0246de thru 0x02f2d6 (0xabf9 bytes) 000079 1777593626.913610 mapwrite 0x0356e3 thru 0x03a9b3 (0x52d1 bytes) 000083 1777593626.944100 mapread 0x027514 thru 0x03409b (0xcb88 bytes) 000087 1777593626.946828 read 0x01ed02 thru 0x024720 (0x5a1f bytes) 000088 1777593626.947476 trunc from 0x040000 to 0x0274ab (0x18b54 bytes) 000089 1777593626.958949 mapwrite 0x0232ac thru 0x028b8f (0x58e4 bytes) 000090 1777593626.988252 MIRROR_EXTEND to 2 mirrors 000091 1777593627.235839 MIRROR_RESYNC 2 mirrors 000095 1777593627.374550 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) 000096 1777593627.426688 write 0x023918 thru 0x032aba (0xf1a3 bytes) 000097 1777593627.433737 write 0x0161ff thru 0x01d638 (0x743a bytes) 000098 1777593627.439985 trunc from 0x032abb to 0x002a84 (0x30036 bytes) 000099 1777593627.447530 trunc from 0x002a85 to 0x00b8c9 (0x8e45 bytes) 000100 1777593627.453921 mapwrite 0x00d7c3 thru 0x00f287 (0x1ac5 bytes) 000101 1777593627.485374 read 0x003f11 thru 0x00e908 (0xa9f8 bytes) 000102 1777593627.488879 mapread 0x004870 thru 0x007d2b (0x34bc bytes) 000107 1777593627.492445 mapread 0x0086f9 thru 0x00dcaa (0x55b2 bytes) 000108 1777593627.493427 mapwrite 0x02e6d3 thru 0x03e610 (0xff3e bytes) 000109 1777593627.529518 mapread 0x00c890 thru 0x01a172 (0xd8e3 bytes) 000110 1777593627.536168 write 0x005ead thru 0x015460 (0xf5b4 bytes) 000112 1777593627.537849 read 0x018dd5 thru 0x01dadc (0x4d08 bytes) 000115 1777593627.539484 mapread 0x030d7a thru 0x031603 (0x88a bytes) 000116 1777593627.540243 mapread 0x003810 thru 0x012ec8 (0xf6b9 bytes) 000117 1777593627.541108 read 0x006e64 thru 0x00bfbe (0x515b bytes) 000119 1777593627.542133 mapread 0x0192c3 thru 0x01fb1e (0x685c bytes) 000123 1777593627.544096 write 0x00a56c thru 0x014a44 (0xa4d9 bytes) 000124 1777593627.545110 mapwrite 0x037374 thru 0x03ffff (0x8c8c bytes) 000125 1777593627.573321 mapread 0x0261d0 thru 0x0331a2 (0xcfd3 bytes) 000126 1777593627.574555 write 0x0161ee thru 0x023736 (0xd549 bytes) 000127 1777593627.575815 mapread 0x028641 thru 0x02ba29 (0x33e9 bytes) 000131 1777593627.577967 mapread 0x0212eb thru 0x029d17 (0x8a2d bytes) 000133 1777593627.579627 write 0x000813 thru 0x003c62 (0x3450 bytes) 000139 1777593627.583288 mapwrite 0x012291 thru 0x0219d8 (0xf748 bytes) 000142 1777593627.608170 MIRROR_EXTEND to 2 mirrors 000143 1777593627.814239 mapread 0x010f87 thru 0x01b484 (0xa4fe bytes) 000145 1777593627.829196 read 0x002be9 thru 0x0075eb (0x4a03 bytes) 000147 1777593627.833333 mapwrite 0x033bfb thru 0x03ffff (0xc405 bytes) 000148 1777593627.905571 trunc from 0x040000 to 0x00f669 (0x30996 bytes) truncating to largest ever: 0x37ec6 000149 1777593627.921620 trunc from 0x00f66a to 0x037ec5 (0x2885c bytes) 000150 1777593627.927247 read 0x00a196 thru 0x019acd (0xf938 bytes) 000151 1777593627.933474 read 0x01374e thru 0x0176a9 (0x3f5c bytes) 000153 1777593627.934596 write 0x033563 thru 0x03884f (0x52ed bytes) 000155 1777593627.936172 read 0x020825 thru 0x0299cb (0x91a7 bytes) 000156 1777593627.936771 trunc from 0x038850 to 0x00a793 (0x2e0bc bytes) 000158 1777593627.943545 write 0x03c122 thru 0x03ffff (0x3ede bytes) 000159 1777593627.944581 write 0x01a77d thru 0x01fffe (0x5882 bytes) 000160 1777593627.950805 mapwrite 0x03b258 thru 0x03ffff (0x4da8 bytes) 000164 1777593627.971153 write 0x002707 thru 0x008ba3 (0x649d bytes) 000165 1777593627.972240 read 0x03b915 thru 0x03c9e4 (0x10d0 bytes) 000166 1777593627.972772 trunc from 0x040000 to 0x02685f (0x197a0 bytes) 000170 1777593627.981486 mapwrite 0x012a7b thru 0x01b787 (0x8d0d bytes) 000171 1777593628.005872 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x26860): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x26860): Mirror 2: 0x5131bf15 (stale) CRC-32 checksum value for chunk [0, 0x26860): Mirror 1: 0x58bd3fc1 Mirror 3: 0x58bd3fc1 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x26860: skipped 000172 1777593628.308300 read 0x025853 thru 0x02685f (0x100d bytes) 000177 1777593628.324929 mapread 0x018af0 thru 0x01bca2 (0x31b3 bytes) 000178 1777593628.337564 MIRROR_RESYNC 3 mirrors truncating to largest ever: 0x3abc1 000179 1777593628.560183 trunc from 0x026860 to 0x03abc0 (0x14361 bytes) 000182 1777593628.602716 MIRROR_RESYNC 3 mirrors 000184 1777593628.884059 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) 000187 1777593629.017421 mapwrite 0x03c607 thru 0x03ffff (0x39f9 bytes) 000189 1777593629.070077 read 0x03eda3 thru 0x03ffff (0x125d bytes) 000191 1777593629.071312 trunc from 0x040000 to 0x02db1b (0x124e4 bytes) 000194 1777593629.080805 mapwrite 0x02d212 thru 0x038e62 (0xbc51 bytes) 000195 1777593629.126453 mapwrite 0x02a26e thru 0x034695 (0xa428 bytes) 000196 1777593629.150264 mapwrite 0x01bfdf thru 0x02ab74 (0xeb96 bytes) 000197 1777593629.186147 MIRROR_RESYNC 2 mirrors 000199 1777593629.428269 trunc from 0x038e63 to 0x0385f3 (0x86f bytes) 000202 1777593629.474106 mapread 0x025c0f thru 0x02baa9 (0x5e9b bytes) 000204 1777593629.482384 mapread 0x01eb39 thru 0x021d6e (0x3236 bytes) 000205 1777593629.489946 MIRROR_RESYNC 2 mirrors 000210 1777593629.717458 mapwrite 0x0289a7 thru 0x02cf60 (0x45ba bytes) 000212 1777593629.817094 read 0x014e95 thru 0x021f95 (0xd101 bytes) 000213 1777593629.823777 mapread 0x02764f thru 0x027e6e (0x820 bytes) 000214 1777593629.827254 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x385f4): Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x385f4): Mirror 3: 0x23db9d29 (stale) CRC-32 checksum value for chunk [0, 0x385f4): Mirror 1: 0x2d7ee949 Mirror 4: 0x2d7ee949 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x385f4: skipped 000215 1777593630.192637 mapread 0x017e9e thru 0x0185ae (0x711 bytes) 000216 1777593630.203576 write 0x0354af thru 0x03ffff (0xab51 bytes) 000221 1777593630.263727 MIRROR_RESYNC 3 mirrors 000223 1777593630.622181 mapwrite 0x01c11a thru 0x020e52 (0x4d39 bytes) 000224 1777593630.700388 write 0x03b833 thru 0x03ffff (0x47cd bytes) 000225 1777593630.709077 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xb93ec8d9 (stale) Mirror 4: 0xb93ec8d9 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xa408872f Mirror 5: 0xa408872f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000227 1777593631.048484 read 0x003066 thru 0x0032ce (0x269 bytes) 000228 1777593631.055095 mapwrite 0x014ac1 thru 0x022c85 (0xe1c5 bytes) 000230 1777593631.132452 write 0x0211aa thru 0x027ee7 (0x6d3e bytes) 000231 1777593631.136103 trunc from 0x040000 to 0x01a5ab (0x25a54 bytes) 000234 1777593631.145058 write 0x000ad4 thru 0x00518a (0x46b7 bytes) 000237 1777593631.150082 write 0x00785a thru 0x011638 (0x9ddf bytes) 000242 1777593631.152930 mapwrite 0x023566 thru 0x029b7a (0x6615 bytes) 000243 1777593631.186839 mapread 0x0024c9 thru 0x0025ac (0xe4 bytes) 000246 1777593631.188624 trunc from 0x029b7b to 0x035777 (0xbbfd bytes) 000248 1777593631.199942 write 0x03568a thru 0x03cde4 (0x775b bytes) 000249 1777593631.206675 read 0x0223a5 thru 0x0226ce (0x32a bytes) 000250 1777593631.207319 trunc from 0x03cde5 to 0x02e47f (0xe965 bytes) 000252 1777593631.216468 read 0x016584 thru 0x022789 (0xc206 bytes) 000255 1777593631.221330 mapread 0x016376 thru 0x01df5c (0x7be7 bytes) 000257 1777593631.222514 mapread 0x02600f thru 0x02e47f (0x8471 bytes) 000258 1777593631.223532 write 0x032162 thru 0x035113 (0x2fb2 bytes) 000261 1777593631.225309 trunc from 0x035114 to 0x016ec2 (0x1e251 bytes) 000264 1777593631.233310 mapwrite 0x00ee26 thru 0x014211 (0x53ec bytes) 000265 1777593631.253469 read 0x00e71f thru 0x016ec2 (0x87a4 bytes) 000266 1777593631.254122 read 0x0109e5 thru 0x016ec2 (0x64de bytes) 000269 1777593631.255486 mapwrite 0x002ac3 thru 0x007ae1 (0x501f bytes) 000270 1777593631.277979 mapwrite 0x01fd34 thru 0x02cc4a (0xcf17 bytes) 000271 1777593631.319795 write 0x01220d thru 0x018aee (0x68e2 bytes) 000274 1777593631.322238 read 0x02c827 thru 0x02cc4a (0x424 bytes) 000276 1777593631.323466 write 0x005905 thru 0x00ea9c (0x9198 bytes) 000278 1777593631.325548 read 0x00d4dd thru 0x011e87 (0x49ab bytes) 000280 1777593631.331318 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x2cc4b): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2cc4b): Mirror 3: 0x2325d5a7 (stale) Mirror 4: 0x2325d5a7 (stale) Mirror 5: 0x90d5a5cd (stale) CRC-32 checksum value for chunk [0, 0x2cc4b): Mirror 1: 0x5952235b Mirror 6: 0x5952235b lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2cc4b: skipped 000281 1777593631.623190 read 0x000cc1 thru 0x008ed6 (0x8216 bytes) 000282 1777593631.629570 read 0x02bfb7 thru 0x02cc4a (0xc94 bytes) 000284 1777593631.637492 read 0x0080cb thru 0x0160c5 (0xdffb bytes) 000286 1777593631.638799 write 0x037de6 thru 0x03ffff (0x821a bytes) 000288 1777593631.682225 trunc from 0x040000 to 0x015946 (0x2a6b9 bytes) 000289 1777593631.700073 trunc from 0x015947 to 0x016198 (0x852 bytes) 000290 1777593631.707463 mapread 0x00caeb thru 0x01584c (0x8d62 bytes) 000291 1777593631.719240 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x16199): Mirror 4: 0x27d6d6ea (stale) Mirror 5: 0x27d6d6ea (stale) Mirror 6: 0x33600f80 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x16199): Mirror 4: 0x27d6d6ea (stale) Mirror 5: 0x27d6d6ea (stale) Mirror 6: 0x33600f80 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x16199: skipped 000293 1777593631.866050 read 0x00df17 thru 0x012247 (0x4331 bytes) 000294 1777593631.872611 read 0x0055a4 thru 0x00c435 (0x6e92 bytes) 000297 1777593631.879095 mapwrite 0x000894 thru 0x009b51 (0x92be bytes) 000300 1777593631.909786 MIRROR_RESYNC 4 mirrors 000301 1777593632.257133 read 0x0127b5 thru 0x016198 (0x39e4 bytes) 000302 1777593632.262421 write 0x017c89 thru 0x01eb84 (0x6efc bytes) 000303 1777593632.294087 mapread 0x0181c2 thru 0x01a105 (0x1f44 bytes) 000305 1777593632.295519 trunc from 0x01eb85 to 0x00bcbb (0x12ec9 bytes) 000306 1777593632.306540 read 0x009cbe thru 0x00bcbb (0x1ffe bytes) 000307 1777593632.311478 trunc from 0x00bcbc to 0x00f076 (0x33bb bytes) 000310 1777593632.319432 mapwrite 0x01ff58 thru 0x02d893 (0xd93c bytes) 000312 1777593632.364081 read 0x003a39 thru 0x009577 (0x5b3f bytes) 000313 1777593632.364640 mapread 0x004141 thru 0x00637b (0x223b bytes) 000314 1777593632.365459 write 0x028c37 thru 0x02f294 (0x665e bytes) 000315 1777593632.366436 mapwrite 0x028013 thru 0x03173f (0x972d bytes) 000316 1777593632.397537 mapwrite 0x017305 thru 0x019bde (0x28da bytes) 000317 1777593632.426174 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x31740): Mirror 4: 0x16c56d8c (stale) Mirror 5: 0x16c56d8c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x31740): Mirror 4: 0x16c56d8c (stale) Mirror 5: 0x16c56d8c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x31740: skipped 000319 1777593632.586458 mapread 0x00ae93 thru 0x018dc3 (0xdf31 bytes) 000320 1777593632.600054 write 0x01dd4b thru 0x01e046 (0x2fc bytes) 000322 1777593632.604366 mapwrite 0x0284dc thru 0x02ac50 (0x2775 bytes) 000324 1777593632.625349 mapread 0x02b75d thru 0x02f5a8 (0x3e4c bytes) 000325 1777593632.626402 trunc from 0x031740 to 0x03010a (0x1635 bytes) 000326 1777593632.635782 read 0x029ceb thru 0x03010a (0x6420 bytes) 000328 1777593632.637144 trunc from 0x03010b to 0x003712 (0x2c9f8 bytes) 000329 1777593632.645444 trunc from 0x003713 to 0x014909 (0x111f7 bytes) 000330 1777593632.655664 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x1490a): Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1490a): Mirror 4: 0x2285eb3f (stale) Mirror 5: 0x2285eb3f (stale) CRC-32 checksum value for chunk [0, 0x1490a): Mirror 1: 0xc55b33bb Mirror 6: 0xc55b33bb lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1490a: skipped 000331 1777593632.962179 trunc from 0x01490a to 0x027685 (0x12d7c bytes) 000333 1777593633.004611 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x27686): Mirror 4: 0xdbf9af4f (stale) Mirror 5: 0xdbf9af4f (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x27686): Mirror 4: 0xdbf9af4f (stale) Mirror 5: 0xdbf9af4f (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x27686: skipped 000335 1777593633.139933 read 0x0114a3 thru 0x016587 (0x50e5 bytes) 000338 1777593633.147349 write 0x025cdc thru 0x02be26 (0x614b bytes) 000341 1777593633.152312 mapwrite 0x024074 thru 0x02c74b (0x86d8 bytes) 000343 1777593633.192132 MIRROR_RESYNC 3 mirrors 000345 1777593633.547587 mapwrite 0x02f473 thru 0x03c0c5 (0xcc53 bytes) 000346 1777593633.649942 write 0x014296 thru 0x021ebf (0xdc2a bytes) 000347 1777593633.659678 read 0x03aabf thru 0x03c0c5 (0x1607 bytes) 000348 1777593633.665234 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x3c0c6): Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3c0c6): Mirror 4: 0x8657fb71 (stale) Mirror 5: 0x8657fb71 (stale) CRC-32 checksum value for chunk [0, 0x3c0c6): Mirror 1: 0x2a18690b Mirror 6: 0x2a18690b lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3c0c6: skipped 000351 1777593633.994344 mapwrite 0x00887f thru 0x00af1b (0x269d bytes) 000355 1777593634.079608 mapwrite 0x0096a3 thru 0x00a9d8 (0x1336 bytes) 000356 1777593634.101926 mapread 0x0198c4 thru 0x01a886 (0xfc3 bytes) 000357 1777593634.109239 mapwrite 0x007228 thru 0x00fe78 (0x8c51 bytes) 000358 1777593634.138808 read 0x02a53e thru 0x038892 (0xe355 bytes) 000360 1777593634.146372 mapread 0x035e9a thru 0x03c0c5 (0x622c bytes) 000363 1777593634.152054 write 0x00e9ab thru 0x00f0f6 (0x74c bytes) 000364 1777593634.153051 mapread 0x02363e thru 0x02b089 (0x7a4c bytes) 000366 1777593634.160114 mapwrite 0x03a60e thru 0x03b064 (0xa57 bytes) 000367 1777593634.189360 write 0x02a2c0 thru 0x031eb9 (0x7bfa bytes) 000369 1777593634.190820 write 0x0342b4 thru 0x035d61 (0x1aae bytes) 000371 1777593634.195010 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x3c0c6): Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3c0c6): Mirror 4: 0x8657fb71 (stale) Mirror 5: 0x8657fb71 (stale) Mirror 6: 0x2a18690b (stale) CRC-32 checksum value for chunk [0, 0x3c0c6): Mirror 1: 0xc82a7e82 Mirror 7: 0xc82a7e82 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3c0c6: skipped 000373 1777593634.527746 read 0x029d5f thru 0x034f70 (0xb212 bytes) 000374 1777593634.531714 trunc from 0x03c0c6 to 0x0102d0 (0x2bdf5 bytes) 000375 1777593634.586007 write 0x034056 thru 0x03e903 (0xa8ae bytes) 000376 1777593634.587259 mapwrite 0x03a542 thru 0x03a831 (0x2f0 bytes) 000379 1777593634.613927 MIRROR_RESYNC 5 mirrors 000381 1777593634.989476 read 0x006fd9 thru 0x012aa9 (0xbad1 bytes) 000382 1777593634.994854 read 0x037aa1 thru 0x03e903 (0x6e63 bytes) 000383 1777593634.999791 read 0x029197 thru 0x02baa6 (0x2910 bytes) 000385 1777593635.000584 read 0x0000d6 thru 0x0049d2 (0x48fd bytes) 000386 1777593635.001006 trunc from 0x03e904 to 0x01be34 (0x22acf bytes) 000387 1777593635.045312 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1be35): Mirror 5: 0x341519c5 (stale) Mirror 6: 0x341519c5 (stale) Mirror 7: 0x341519c5 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1be35): Mirror 5: 0x341519c5 (stale) Mirror 6: 0x341519c5 (stale) Mirror 7: 0x341519c5 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1be35: skipped 000388 1777593635.209603 read 0x01198f thru 0x0183e4 (0x6a56 bytes) 000391 1777593635.215907 mapwrite 0x032aa1 thru 0x037ef3 (0x5453 bytes) 000393 1777593635.257546 mapwrite 0x01eb65 thru 0x02d9ee (0xee8a bytes) 000394 1777593635.306227 write 0x02e99b thru 0x0310ae (0x2714 bytes) 000396 1777593635.307873 read 0x00b354 thru 0x01096e (0x561b bytes) truncating to largest ever: 0x3e354 000397 1777593635.315884 trunc from 0x037ef4 to 0x03e353 (0x6460 bytes) 000398 1777593635.330134 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3e354): Mirror 5: 0x4a4a50d9 (stale) Mirror 7: 0x4a4a50d9 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e354): Mirror 5: 0x4a4a50d9 (stale) Mirror 7: 0x4a4a50d9 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e354: skipped 000399 1777593635.478284 trunc from 0x03e354 to 0x01aa69 (0x238ea bytes) 000400 1777593635.486745 trunc from 0x01aa6a to 0x01baff (0x1096 bytes) 000401 1777593635.495817 mapwrite 0x03e0ff thru 0x03e3a1 (0x2a3 bytes) 000402 1777593635.520251 read 0x0032fb thru 0x004c75 (0x197b bytes) 000403 1777593635.523329 mapwrite 0x0355e2 thru 0x03ffff (0xaa1e bytes) 000404 1777593635.568008 read 0x02fe28 thru 0x03a86e (0xaa47 bytes) 000405 1777593635.568640 mapwrite 0x02bff0 thru 0x03860b (0xc61c bytes) 000407 1777593635.596504 read 0x031cb0 thru 0x0368f4 (0x4c45 bytes) 000410 1777593635.597871 mapread 0x03d8f0 thru 0x03ffff (0x2710 bytes) 000411 1777593635.598520 mapwrite 0x03bf38 thru 0x03ffff (0x40c8 bytes) 000413 1777593635.618642 write 0x001b1c thru 0x00dab8 (0xbf9d bytes) 000416 1777593635.623553 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: (stale - read error) Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0xdca3a33f (stale) Mirror 7: 0xdca3a33f (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xe1262801 Mirror 8: 0xe1262801 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000417 1777593635.947777 mapwrite 0x01f616 thru 0x02efff (0xf9ea bytes) 000418 1777593636.041536 read 0x02e823 thru 0x034e44 (0x6622 bytes) 000419 1777593636.044536 trunc from 0x040000 to 0x0142e4 (0x2bd1b bytes) 000421 1777593636.062752 mapwrite 0x008432 thru 0x00b0ea (0x2cb9 bytes) 000422 1777593636.090835 MIRROR_RESYNC 4 mirrors 000423 1777593636.467944 trunc from 0x0142e5 to 0x00c8b5 (0x7a2f bytes) 000425 1777593636.526065 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0xc8b6): Mirror 7: 0x3f5b2dca (stale) Mirror 8: 0x3f5b2dca (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xc8b6): Mirror 7: 0x3f5b2dca (stale) Mirror 8: 0x3f5b2dca (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xc8b6: skipped 000428 1777593636.680404 mapwrite 0x03ecfc thru 0x03ffff (0x1304 bytes) 000429 1777593636.727013 MIRROR_RESYNC 3 mirrors 000430 1777593637.032759 mapread 0x033d40 thru 0x034c24 (0xee5 bytes) 000432 1777593637.045587 write 0x00f216 thru 0x01c6ac (0xd497 bytes) 000434 1777593637.091678 mapread 0x0328d4 thru 0x03e418 (0xbb45 bytes) 000436 1777593637.106760 read 0x00be32 thru 0x018885 (0xca54 bytes) 000437 1777593637.107321 write 0x01ad08 thru 0x01ef88 (0x4281 bytes) 000438 1777593637.113649 trunc from 0x040000 to 0x020eea (0x1f115 bytes) 000439 1777593637.144900 write 0x02ae4d thru 0x031a4d (0x6c01 bytes) 000440 1777593637.146125 write 0x033e3e thru 0x037251 (0x3414 bytes) 000441 1777593637.147179 mapwrite 0x01d35d thru 0x0237f0 (0x6494 bytes) 000442 1777593637.187754 trunc from 0x037252 to 0x03bfef (0x4d9e bytes) 000443 1777593637.199753 MIRROR_RESYNC 3 mirrors 000444 1777593637.490482 read 0x02b32e thru 0x034dbf (0x9a92 bytes) 000445 1777593637.495992 trunc from 0x03bff0 to 0x0337f1 (0x87fe bytes) 000446 1777593637.536371 write 0x00b1e4 thru 0x015ede (0xacfb bytes) 000448 1777593637.542997 mapwrite 0x008f27 thru 0x014827 (0xb901 bytes) 000449 1777593637.568554 mapwrite 0x00162c thru 0x001c54 (0x629 bytes) 000450 1777593637.586473 write 0x03c237 thru 0x03dce2 (0x1aac bytes) 000451 1777593637.587370 mapwrite 0x03b6d4 thru 0x03ffcd (0x48fa bytes) 000452 1777593637.626340 MIRROR_RESYNC 3 mirrors 000453 1777593637.915735 mapwrite 0x023a11 thru 0x0328d9 (0xeec9 bytes) 000454 1777593637.990267 read 0x00cc90 thru 0x01218f (0x5500 bytes) 000457 1777593637.995442 mapread 0x02a5d6 thru 0x0325ec (0x8017 bytes) 000459 1777593637.997053 write 0x020f95 thru 0x02d2b0 (0xc31c bytes) 000460 1777593637.998314 trunc from 0x03ffce to 0x009221 (0x36dac bytes) 000461 1777593638.015269 mapread 0x003ff0 thru 0x009221 (0x5232 bytes) 000462 1777593638.023498 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x9222): Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x9222): Mirror 7: 0x6b856065 (stale) Mirror 8: 0x6b856065 (stale) CRC-32 checksum value for chunk [0, 0x9222): Mirror 1: 0x6b856065 Mirror 9: 0x6b856065 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x9222: skipped 000464 1777593638.365477 read 0x006ce7 thru 0x007130 (0x44a bytes) 000465 1777593638.373791 read 0x00292f thru 0x002e08 (0x4da bytes) 000466 1777593638.381923 read 0x00491f thru 0x009221 (0x4903 bytes) 000467 1777593638.386274 write 0x007596 thru 0x0095b7 (0x2022 bytes) 000468 1777593638.421842 trunc from 0x0095b8 to 0x028773 (0x1f1bc bytes) 000470 1777593638.434624 MIRROR_RESYNC 4 mirrors 000473 1777593638.749977 write 0x0276d2 thru 0x02c889 (0x51b8 bytes) 000475 1777593638.774187 mapwrite 0x0239f5 thru 0x026564 (0x2b70 bytes) 000476 1777593638.797662 mapwrite 0x027172 thru 0x02870c (0x159b bytes) 000477 1777593638.815951 write 0x0343d6 thru 0x03c0b1 (0x7cdc bytes) 000479 1777593638.822925 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3c0b2): Mirror 8: 0x211b7a3e (stale) Mirror 9: 0x211b7a3e (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3c0b2): Mirror 8: 0x211b7a3e (stale) Mirror 9: 0x211b7a3e (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3c0b2: skipped 000480 1777593638.967620 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x3c0b2): Mirror 8: (stale - read error) Mirror 9: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3c0b2): Mirror 8: 0x211b7a3e (stale) Mirror 9: 0x211b7a3e (stale) CRC-32 checksum value for chunk [0, 0x3c0b2): Mirror 1: 0x9a5e38eb Mirror 10: 0x9a5e38eb lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3c0b2: skipped 000481 1777593639.293916 read 0x0084e2 thru 0x00912f (0xc4e bytes) 000483 1777593639.305085 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3c0b2): Mirror 9: 0x211b7a3e (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3c0b2): Mirror 9: 0x211b7a3e (stale) CRC-32 checksum value for chunk [0, 0x3c0b2): Mirror 1: 0x9a5e38eb Mirror 10: 0x9a5e38eb lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3c0b2: skipped 000492 1777593639.656596 write 0x0130dc thru 0x01cea3 (0x9dc8 bytes) 000494 1777593639.698575 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x3c0b2): Mirror 9: (stale - read error) Mirror 10: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3c0b2): Mirror 9: 0x211b7a3e (stale) Mirror 10: 0x9a5e38eb (stale) CRC-32 checksum value for chunk [0, 0x3c0b2): Mirror 1: 0x5f98dabc Mirror 11: 0x5f98dabc lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3c0b2: skipped 000495 1777593640.122819 mapread 0x02ea24 thru 0x03835d (0x993a bytes) 000496 1777593640.149970 mapread 0x03ad46 thru 0x03c0b1 (0x136c bytes) 000497 1777593640.164755 mapwrite 0x01ea34 thru 0x0232d6 (0x48a3 bytes) 000498 1777593640.304635 write 0x0056f1 thru 0x0111f3 (0xbb03 bytes) 000500 1777593640.315975 read 0x014b49 thru 0x01cec7 (0x837f bytes) 000501 1777593640.321338 mapread 0x023dde thru 0x02c437 (0x865a bytes) 000503 1777593640.330510 trunc from 0x03c0b2 to 0x02aefc (0x111b5 bytes) 000504 1777593640.343766 read 0x013dfd thru 0x01fc12 (0xbe16 bytes) 000507 1777593640.351841 read 0x00461f thru 0x0133be (0xeda0 bytes) 000512 1777593640.354745 mapread 0x0144c9 thru 0x01bc92 (0x77ca bytes) 000514 1777593640.356435 mapwrite 0x03f076 thru 0x03ffff (0xf8a bytes) 000517 1777593640.416796 mapread 0x03de22 thru 0x03ffff (0x21de bytes) 000518 1777593640.417887 read 0x03d5a9 thru 0x03ec3b (0x1693 bytes) 000519 1777593640.418425 mapwrite 0x00e25a thru 0x01287c (0x4623 bytes) 000522 1777593640.455957 trunc from 0x040000 to 0x01306d (0x2cf92 bytes) 000523 1777593640.473206 mapwrite 0x00853b thru 0x01180f (0x92d5 bytes) 000525 1777593640.510962 trunc from 0x01306e to 0x0162b9 (0x324c bytes) 000526 1777593640.526698 MIRROR_RESYNC 4 mirrors 000527 1777593640.932197 mapwrite 0x01f78c thru 0x02cfe2 (0xd857 bytes) 000529 1777593641.051878 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x2cfe3): Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2cfe3): Mirror 9: 0x380374fa (stale) Mirror 10: 0x380374fa (stale) Mirror 11: 0x380374fa (stale) CRC-32 checksum value for chunk [0, 0x2cfe3): Mirror 1: 0xb3e20f00 Mirror 12: 0xb3e20f00 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2cfe3: skipped 000530 1777593641.604187 mapwrite 0x01c649 thru 0x0260cc (0x9a84 bytes) 000531 1777593641.717218 mapwrite 0x02c1eb thru 0x02da32 (0x1848 bytes) 000532 1777593641.766455 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2da33): Mirror 10: 0x56b0f27 (stale) Mirror 11: 0x56b0f27 (stale) Mirror 12: 0x2e970986 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2da33): Mirror 10: 0x56b0f27 (stale) Mirror 11: 0x56b0f27 (stale) Mirror 12: 0x2e970986 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2da33: skipped 000533 1777593641.977567 mapwrite 0x009432 thru 0x0121f5 (0x8dc4 bytes) 000534 1777593642.009659 mapwrite 0x00e0d9 thru 0x011135 (0x305d bytes) 000536 1777593642.036046 read 0x028240 thru 0x028af8 (0x8b9 bytes) 000537 1777593642.039971 write 0x002c00 thru 0x008813 (0x5c14 bytes) 000538 1777593642.044770 trunc from 0x02da33 to 0x019a63 (0x13fcf bytes) 000539 1777593642.054715 mapread 0x003c47 thru 0x00d7a2 (0x9b5c bytes) 000540 1777593642.059135 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x19a64): Mirror 11: 0x5dedde10 (stale) Mirror 12: 0x5dedde10 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x19a64): Mirror 11: 0x5dedde10 (stale) Mirror 12: 0x5dedde10 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x19a64: skipped 000544 1777593642.253380 trunc from 0x019a64 to 0x01506b (0x49f8 bytes) 000545 1777593642.266115 mapread 0x004f7b thru 0x007251 (0x22d7 bytes) 000546 1777593642.284617 mapwrite 0x0264f9 thru 0x0352fe (0xee06 bytes) 000548 1777593642.357642 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x352ff): Mirror 11: (stale - read error) Mirror 12: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x352ff): Mirror 11: 0x3d8e9dfb (stale) Mirror 12: 0x553ca1c9 (stale) CRC-32 checksum value for chunk [0, 0x352ff): Mirror 1: 0x96390142 Mirror 13: 0x96390142 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x352ff: skipped 000551 1777593642.756476 write 0x039991 thru 0x03ffff (0x666f bytes) 000553 1777593642.800113 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: 0xface5ccc (stale) Mirror 13: 0xb98d78fb (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: 0xface5ccc (stale) Mirror 13: 0xb98d78fb (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000555 1777593643.005387 MIRROR_RESYNC 3 mirrors 000556 1777593643.475581 mapwrite 0x01883b thru 0x021f8d (0x9753 bytes) 000559 1777593643.595641 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: (stale - read error) Mirror 13: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: 0xf5f9c71b (stale) Mirror 13: 0xf5f9c71b (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xe04765f3 Mirror 14: 0xe04765f3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000560 1777593644.099380 trunc from 0x040000 to 0x02c343 (0x13cbc bytes) 000562 1777593644.152183 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x2c344): Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2c344): Mirror 12: 0x7d408f25 (stale) Mirror 13: 0x7d408f25 (stale) Mirror 14: 0x3b1e1f46 (stale) CRC-32 checksum value for chunk [0, 0x2c344): Mirror 1: 0x3b1e1f46 Mirror 15: 0x3b1e1f46 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2c344: skipped 000567 1777593644.686973 mapwrite 0x031acd thru 0x03445d (0x2991 bytes) 000569 1777593644.795586 read 0x02e9d2 thru 0x03245c (0x3a8b bytes) 000570 1777593644.796371 mapread 0x005dcc thru 0x00d650 (0x7885 bytes) 000573 1777593644.810951 trunc from 0x03445e to 0x002c08 (0x31855 bytes) 000574 1777593644.834643 mapwrite 0x015374 thru 0x01573a (0x3c7 bytes) 000575 1777593644.888383 write 0x03ce45 thru 0x03ffff (0x31bb bytes) 000576 1777593644.889577 trunc from 0x040000 to 0x00176b (0x3e894 bytes) 000578 1777593644.903888 trunc from 0x00176c to 0x019a2c (0x182c1 bytes) 000579 1777593644.910875 mapwrite 0x022c3c thru 0x02ef60 (0xc325 bytes) 000582 1777593644.969241 read 0x01afb8 thru 0x02af7f (0xffc8 bytes) 000584 1777593644.986171 mapwrite 0x03dcef thru 0x03ffff (0x2311 bytes) 000585 1777593645.056815 write 0x005c60 thru 0x007871 (0x1c12 bytes) 000587 1777593645.060102 mapwrite 0x007fd2 thru 0x013bac (0xbbdb bytes) 000588 1777593645.105998 mapread 0x024a28 thru 0x02be43 (0x741c bytes) 000590 1777593645.107793 trunc from 0x040000 to 0x033136 (0xcec9 bytes) 000591 1777593645.123003 mapread 0x00da69 thru 0x017802 (0x9d9a bytes) 000594 1777593645.127153 mapread 0x002e29 thru 0x0069cb (0x3ba3 bytes) 000596 1777593645.129006 read 0x02560d thru 0x02f6a8 (0xa09c bytes) 000597 1777593645.130535 trunc from 0x033137 to 0x00565a (0x2dadc bytes) 000598 1777593645.147949 mapwrite 0x018d7b thru 0x01e847 (0x5acd bytes) 000599 1777593645.224883 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x1e848): Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1e848): Mirror 12: 0xb8c5a3d1 (stale) Mirror 13: 0xb8c5a3d1 (stale) Mirror 14: 0x7948ca01 (stale) Mirror 15: 0x7948ca01 (stale) CRC-32 checksum value for chunk [0, 0x1e848): Mirror 1: 0xb7c9ccd9 Mirror 16: 0xb7c9ccd9 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1e848: skipped 000601 1777593645.724101 trunc from 0x01e848 to 0x01f4be (0xc77 bytes) 000603 1777593645.777770 mapread 0x0085ab thru 0x014953 (0xc3a9 bytes) 000605 1777593645.797717 MIRROR_RESYNC 6 mirrors 000606 1777593646.440946 write 0x03afe6 thru 0x03ffff (0x501a bytes) 000607 1777593646.482064 trunc from 0x040000 to 0x00cf28 (0x330d7 bytes) 000609 1777593646.493009 mapread 0x00c70d thru 0x00cf28 (0x81c bytes) 000610 1777593646.496919 read 0x009261 thru 0x00cf28 (0x3cc8 bytes) 000612 1777593646.501048 mapread 0x003940 thru 0x00cf28 (0x95e9 bytes) 000613 1777593646.508641 write 0x03afb8 thru 0x03ffff (0x5048 bytes) 000619 1777593646.512218 read 0x03a487 thru 0x03ffff (0x5b79 bytes) 000621 1777593646.513419 mapread 0x004a37 thru 0x006c54 (0x221e bytes) 000623 1777593646.514920 write 0x017d66 thru 0x01de74 (0x610f bytes) 000628 1777593646.524964 mapread 0x03a527 thru 0x03ffff (0x5ad9 bytes) 000629 1777593646.526353 write 0x03e6a2 thru 0x03ffff (0x195e bytes) 000630 1777593646.527576 write 0x029f4d thru 0x02bb6c (0x1c20 bytes) 000631 1777593646.531812 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: 0x9846dfee (stale) Mirror 13: 0x9846dfee (stale) Mirror 14: 0x9846dfee (stale) Mirror 15: 0x9846dfee (stale) Mirror 16: 0x9846dfee (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x2afaa6d8 Mirror 17: 0x2afaa6d8 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000632 1777593646.999368 trunc from 0x040000 to 0x00e82f (0x317d0 bytes) 000635 1777593647.049739 write 0x00c725 thru 0x01a338 (0xdc14 bytes) 000636 1777593647.054473 mapwrite 0x02b5b0 thru 0x033f1d (0x896e bytes) 000639 1777593647.106413 trunc from 0x033f1e to 0x034dc7 (0xeaa bytes) 000640 1777593647.117199 MIRROR_RESYNC 7 mirrors 000641 1777593647.644388 MIRROR_EXTEND to 8 mirrors 000643 1777593648.007462 mapwrite 0x00e86b thru 0x01964d (0xade3 bytes) 000645 1777593648.112837 MIRROR_EXTEND to 9 mirrors CRC-32 checksum value for chunk [0, 0x34dc8): Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) Mirror 17: (stale - read error) Mirror 18: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x34dc8): Mirror 12: 0x9f9682b7 (stale) Mirror 13: 0x9f9682b7 (stale) Mirror 14: 0x9f9682b7 (stale) Mirror 15: 0x9f9682b7 (stale) Mirror 16: 0x9f9682b7 (stale) Mirror 17: 0x9f9682b7 (stale) Mirror 18: 0x9f9682b7 (stale) CRC-32 checksum value for chunk [0, 0x34dc8): Mirror 1: 0xd806e4a6 Mirror 19: 0xd806e4a6 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x34dc8: skipped 000646 1777593648.536873 write 0x031dc7 thru 0x03ffff (0xe239 bytes) 000650 1777593648.586185 MIRROR_EXTEND to 10 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) Mirror 17: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: 0xf03fd560 (stale) Mirror 13: 0xf03fd560 (stale) Mirror 14: 0xf03fd560 (stale) Mirror 15: 0xf03fd560 (stale) Mirror 16: 0xf03fd560 (stale) Mirror 17: 0xf03fd560 (stale) Mirror 18: 0xf03fd560 (stale) Mirror 19: 0x234b715e (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x90b0a201 Mirror 20: 0x90b0a201 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000651 1777593649.039521 read 0x02664f thru 0x034a6d (0xe41f bytes) 000652 1777593649.045992 trunc from 0x040000 to 0x0129b6 (0x2d649 bytes) 000653 1777593649.085481 write 0x02938b thru 0x02dfcb (0x4c41 bytes) 000657 1777593649.087818 mapwrite 0x017882 thru 0x01b426 (0x3ba5 bytes) 000658 1777593649.127324 read 0x0136ce thru 0x01a54e (0x6e81 bytes) 000660 1777593649.128337 trunc from 0x02dfcc to 0x02c8cc (0x16ff bytes) 000662 1777593649.144695 MIRROR_EXTEND to 11 mirrors CRC-32 checksum value for chunk [0, 0x2c8cd): Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) Mirror 17: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) Mirror 20: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2c8cd): Mirror 12: 0x11647dec (stale) Mirror 13: 0x11647dec (stale) Mirror 14: 0x11647dec (stale) Mirror 15: 0x11647dec (stale) Mirror 16: 0x11647dec (stale) Mirror 17: 0x11647dec (stale) Mirror 18: 0x11647dec (stale) Mirror 19: 0xab0f5d1b (stale) Mirror 20: 0xab0f5d1b (stale) CRC-32 checksum value for chunk [0, 0x2c8cd): Mirror 1: 0xb0fec27c Mirror 21: 0xb0fec27c lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2c8cd: skipped 000664 1777593649.705736 write 0x01d59e thru 0x01fbad (0x2610 bytes) 000665 1777593649.749077 MIRROR_SPLIT mirror 10 to 10 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2c8cd): Mirror 12: 0x11647dec (stale) Mirror 13: 0x11647dec (stale) Mirror 14: 0x11647dec (stale) Mirror 15: 0x11647dec (stale) Mirror 16: 0x11647dec (stale) Mirror 17: 0x11647dec (stale) Mirror 18: 0x11647dec (stale) Mirror 19: 0xab0f5d1b (stale) Mirror 20: 0xab0f5d1b (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2c8cd): Mirror 12: 0x11647dec (stale) Mirror 13: 0x11647dec (stale) Mirror 14: 0x11647dec (stale) Mirror 15: 0x11647dec (stale) Mirror 16: 0x11647dec (stale) Mirror 17: 0x11647dec (stale) Mirror 18: 0x11647dec (stale) Mirror 19: 0xab0f5d1b (stale) Mirror 20: 0xab0f5d1b (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2c8cd: skipped 000668 1777593650.020660 mapread 0x019cef thru 0x01d7d1 (0x3ae3 bytes) 000669 1777593650.038127 write 0x030c6e thru 0x03f2db (0xe66e bytes) 000673 1777593650.041330 mapwrite 0x01da6e thru 0x02093c (0x2ecf bytes) 000675 1777593650.068243 mapwrite 0x013d99 thru 0x016ec7 (0x312f bytes) 000678 1777593650.097799 trunc from 0x03f2dc to 0x01e609 (0x20cd2 bytes) 000680 1777593650.105589 read 0x003356 thru 0x010c6f (0xd91a bytes) 000681 1777593650.111537 read 0x0042cc thru 0x006be2 (0x2917 bytes) 000684 1777593650.113258 trunc from 0x01e60a to 0x005cd9 (0x18930 bytes) 000685 1777593650.120623 read 0x004ac9 thru 0x005cd9 (0x1211 bytes) 000686 1777593650.121154 write 0x029968 thru 0x0340e5 (0xa77e bytes) 000687 1777593650.122342 mapread 0x00f027 thru 0x016d05 (0x7cdf bytes) 000688 1777593650.128901 mapwrite 0x01d333 thru 0x026747 (0x9415 bytes) 000689 1777593650.153246 write 0x00beb0 thru 0x012d02 (0x6e53 bytes) 000690 1777593650.154246 read 0x01f08b thru 0x025322 (0x6298 bytes) 000691 1777593650.154822 read 0x00b27b thru 0x00e1e2 (0x2f68 bytes) 000692 1777593650.155408 write 0x006651 thru 0x00c5c3 (0x5f73 bytes) 000694 1777593650.156604 read 0x00f451 thru 0x01a2fd (0xaead bytes) 000695 1777593650.157200 trunc from 0x0340e6 to 0x02d479 (0x6c6c bytes) 000699 1777593650.166418 write 0x0194da thru 0x024b9e (0xb6c5 bytes) 000700 1777593650.167651 mapread 0x028cc7 thru 0x02d479 (0x47b3 bytes) 000701 1777593650.168620 trunc from 0x02d47a to 0x036cd3 (0x985a bytes) 000702 1777593650.182970 MIRROR_EXTEND to 11 mirrors CRC-32 checksum value for chunk [0, 0x36cd4): Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) Mirror 17: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) Mirror 20: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x36cd4): Mirror 12: 0x921e6eb (stale) Mirror 13: 0x921e6eb (stale) Mirror 14: 0x921e6eb (stale) Mirror 15: 0x921e6eb (stale) Mirror 16: 0x921e6eb (stale) Mirror 17: 0x921e6eb (stale) Mirror 18: 0x921e6eb (stale) Mirror 19: 0xc801bbdc (stale) Mirror 20: 0xc46145ea (stale) CRC-32 checksum value for chunk [0, 0x36cd4): Mirror 1: 0x2594eb33 Mirror 21: 0x2594eb33 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x36cd4: skipped 000704 1777593650.597662 mapread 0x026f44 thru 0x02c530 (0x55ed bytes) 000705 1777593650.608779 write 0x0139e2 thru 0x01bed5 (0x84f4 bytes) 000706 1777593650.640073 write 0x012746 thru 0x014a3b (0x22f6 bytes) 000708 1777593650.641451 read 0x03411f thru 0x036402 (0x22e4 bytes) 000709 1777593650.645905 read 0x011415 thru 0x01c582 (0xb16e bytes) 000711 1777593650.652804 write 0x02319a thru 0x02bce5 (0x8b4c bytes) 000713 1777593650.657342 MIRROR_RESYNC 11 mirrors 000714 1777593651.294741 mapwrite 0x0207e1 thru 0x0280c4 (0x78e4 bytes) 000717 1777593651.367995 mapread 0x00d767 thru 0x014321 (0x6bbb bytes) 000719 1777593651.373072 read 0x013050 thru 0x01f456 (0xc407 bytes) 000721 1777593651.376527 trunc from 0x036cd4 to 0x012ca0 (0x24033 bytes) 000723 1777593651.384998 trunc from 0x012ca1 to 0x0290bc (0x1641c bytes) 000724 1777593651.391406 mapwrite 0x03abab thru 0x03e68b (0x3ae1 bytes) 000725 1777593651.427101 MIRROR_RESYNC 11 mirrors 000726 1777593651.994735 mapwrite 0x00ad81 thru 0x01a7a1 (0xfa21 bytes) 000728 1777593652.083668 write 0x016d34 thru 0x0259aa (0xec77 bytes) 000730 1777593652.088932 read 0x01b1c4 thru 0x0260cd (0xaf0a bytes) 000731 1777593652.092326 mapread 0x015f99 thru 0x021f6a (0xbfd2 bytes) 000732 1777593652.094203 trunc from 0x03e68c to 0x01207d (0x2c60e bytes) 000733 1777593652.117646 mapwrite 0x031243 thru 0x03a4c7 (0x9285 bytes) 000734 1777593652.173076 trunc from 0x03a4c8 to 0x01aa86 (0x1fa41 bytes) 000735 1777593652.186743 trunc from 0x01aa87 to 0x0143df (0x66a7 bytes) 000740 1777593652.195903 trunc from 0x0143e0 to 0x01a6e6 (0x6307 bytes) 000742 1777593652.203713 mapread 0x008a97 thru 0x009d63 (0x12cd bytes) 000743 1777593652.204555 mapwrite 0x002499 thru 0x007a29 (0x5591 bytes) 000744 1777593652.227902 read 0x0176a0 thru 0x01a6e6 (0x3047 bytes) 000745 1777593652.232544 write 0x03ef24 thru 0x03ffff (0x10dc bytes) 000747 1777593652.234080 mapread 0x029762 thru 0x02b3f9 (0x1c98 bytes) 000750 1777593652.240616 trunc from 0x040000 to 0x00c214 (0x33deb bytes) 000753 1777593652.251816 read 0x00aabc thru 0x00bd17 (0x125c bytes) 000755 1777593652.252754 read 0x005ced thru 0x00c214 (0x6528 bytes) 000756 1777593652.253235 mapread 0x00b01f thru 0x00c214 (0x11f6 bytes) 000759 1777593652.254803 read 0x00134a thru 0x004df7 (0x3aae bytes) 000762 1777593652.256299 read 0x002137 thru 0x009dae (0x7c78 bytes) 000763 1777593652.256881 read 0x003bbc thru 0x005319 (0x175e bytes) 000764 1777593652.257368 mapwrite 0x0035eb thru 0x0123e8 (0xedfe bytes) 000765 1777593652.293509 mapwrite 0x01c772 thru 0x01e4e1 (0x1d70 bytes) 000766 1777593652.327150 trunc from 0x01e4e2 to 0x012ead (0xb634 bytes) 000769 1777593652.337638 mapwrite 0x0303da thru 0x03bc44 (0xb86b bytes) 000770 1777593652.367177 trunc from 0x03bc45 to 0x0077d6 (0x3446e bytes) 000771 1777593652.373930 trunc from 0x0077d7 to 0x0075d6 (0x200 bytes) 000772 1777593652.383873 MIRROR_EXTEND to 12 mirrors CRC-32 checksum value for chunk [0, 0x75d7): Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) Mirror 17: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) Mirror 20: (stale - read error) Mirror 21: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x75d7): Mirror 12: 0x7e3c7de (stale) Mirror 13: 0x7e3c7de (stale) Mirror 14: 0x7e3c7de (stale) Mirror 15: 0x7e3c7de (stale) Mirror 16: 0x7e3c7de (stale) Mirror 17: 0x7e3c7de (stale) Mirror 18: 0x7e3c7de (stale) Mirror 19: 0x7e3c7de (stale) Mirror 20: 0x7e3c7de (stale) Mirror 21: 0x7e3c7de (stale) CRC-32 checksum value for chunk [0, 0x75d7): Mirror 1: 0x4f0d6ac5 Mirror 22: 0x4f0d6ac5 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x75d7: skipped 000774 1777593652.862164 trunc from 0x0075d7 to 0x0233e8 (0x1be12 bytes) 000775 1777593652.919730 trunc from 0x0233e9 to 0x019fd6 (0x9412 bytes) 000776 1777593652.931636 read 0x00316d thru 0x009149 (0x5fdd bytes) 000779 1777593652.938462 mapread 0x00129b thru 0x003b58 (0x28be bytes) 000781 1777593652.947765 read 0x009885 thru 0x00e01c (0x4798 bytes) 000782 1777593652.954911 mapread 0x00f0bf thru 0x019602 (0xa544 bytes) 000784 1777593652.956755 mapread 0x012588 thru 0x019fd6 (0x7a4f bytes) 000785 1777593652.957762 trunc from 0x019fd7 to 0x01f1cf (0x51f9 bytes) 000790 1777593652.969959 mapwrite 0x013b97 thru 0x021aef (0xdf59 bytes) 000791 1777593653.013616 write 0x0021f0 thru 0x011797 (0xf5a8 bytes) 000792 1777593653.015263 trunc from 0x021af0 to 0x0264e2 (0x49f3 bytes) 000793 1777593653.025634 MIRROR_EXTEND to 13 mirrors CRC-32 checksum value for chunk [0, 0x264e3): Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) Mirror 17: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) Mirror 20: (stale - read error) Mirror 21: (stale - read error) Mirror 22: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x264e3): Mirror 12: 0x932c39a4 (stale) Mirror 13: 0x932c39a4 (stale) Mirror 14: 0x932c39a4 (stale) Mirror 15: 0x932c39a4 (stale) Mirror 16: 0x932c39a4 (stale) Mirror 17: 0x932c39a4 (stale) Mirror 18: 0x932c39a4 (stale) Mirror 19: 0x932c39a4 (stale) Mirror 20: 0x932c39a4 (stale) Mirror 21: 0x932c39a4 (stale) Mirror 22: 0xbc7269e7 (stale) CRC-32 checksum value for chunk [0, 0x264e3): Mirror 1: 0xcf5784f5 Mirror 23: 0xcf5784f5 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x264e3: skipped 000796 1777593653.526500 mapwrite 0x00cf22 thru 0x01376b (0x684a bytes) 000797 1777593653.602276 read 0x019118 thru 0x01c01f (0x2f08 bytes) 000798 1777593653.605430 mapread 0x012534 thru 0x01b889 (0x9356 bytes) 000800 1777593653.610361 read 0x023350 thru 0x0264e2 (0x3193 bytes) 000801 1777593653.614602 trunc from 0x0264e3 to 0x028ad3 (0x25f1 bytes) 000805 1777593653.624919 read 0x017f17 thru 0x01acd7 (0x2dc1 bytes) 000807 1777593653.626107 trunc from 0x028ad4 to 0x0186a9 (0x1042a bytes) 000808 1777593653.633321 mapwrite 0x03c571 thru 0x03fcc1 (0x3751 bytes) 000809 1777593653.662936 mapread 0x004a9a thru 0x00cde3 (0x834a bytes) 000815 1777593653.669808 MIRROR_RESYNC 13 mirrors 000818 1777593654.475462 mapwrite 0x002949 thru 0x01099f (0xe057 bytes) 000821 1777593654.566079 write 0x0302f2 thru 0x036d26 (0x6a35 bytes) 000824 1777593654.574646 mapwrite 0x0194d3 thru 0x025219 (0xbd47 bytes) 000825 1777593654.626374 trunc from 0x03fcc2 to 0x027082 (0x18c3f bytes) 000826 1777593654.646706 MIRROR_RESYNC 13 mirrors 000829 1777593655.482509 MIRROR_EXTEND to 14 mirrors 000830 1777593655.854458 write 0x035cb6 thru 0x037789 (0x1ad4 bytes) 000832 1777593655.891811 mapread 0x00b87e thru 0x015da6 (0xa529 bytes) 000833 1777593655.905067 read 0x01891c thru 0x028595 (0xfc7a bytes) 000834 1777593655.915496 MIRROR_RESYNC 14 mirrors 000835 1777593656.632404 MIRROR_RESYNC 14 mirrors 000836 1777593656.903117 write 0x03836d thru 0x03ffff (0x7c93 bytes) 000837 1777593656.930768 read 0x03afc4 thru 0x03ffff (0x503c bytes) 000838 1777593656.931448 read 0x005f2c thru 0x00e76e (0x8843 bytes) 000839 1777593656.939154 MIRROR_SPLIT mirror 6 to 13 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: 0x47385854 (stale) Mirror 13: 0x47385854 (stale) Mirror 14: 0x47385854 (stale) Mirror 15: 0x47385854 (stale) Mirror 16: 0x47385854 (stale) Mirror 18: 0x47385854 (stale) Mirror 19: 0x47385854 (stale) Mirror 20: 0x47385854 (stale) Mirror 21: 0x47385854 (stale) Mirror 22: 0x47385854 (stale) Mirror 23: 0x47385854 (stale) Mirror 24: 0x47385854 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: 0x47385854 (stale) Mirror 13: 0x47385854 (stale) Mirror 14: 0x47385854 (stale) Mirror 15: 0x47385854 (stale) Mirror 16: 0x47385854 (stale) Mirror 18: 0x47385854 (stale) Mirror 19: 0x47385854 (stale) Mirror 20: 0x47385854 (stale) Mirror 21: 0x47385854 (stale) Mirror 22: 0x47385854 (stale) Mirror 23: 0x47385854 (stale) Mirror 24: 0x47385854 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000840 1777593657.205368 mapread 0x035ce0 thru 0x036d17 (0x1038 bytes) 000841 1777593657.210751 MIRROR_EXTEND to 14 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) Mirror 20: (stale - read error) Mirror 21: (stale - read error) Mirror 22: (stale - read error) Mirror 23: (stale - read error) Mirror 24: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: 0x47385854 (stale) Mirror 13: 0x47385854 (stale) Mirror 14: 0x47385854 (stale) Mirror 15: 0x47385854 (stale) Mirror 16: 0x47385854 (stale) Mirror 18: 0x47385854 (stale) Mirror 19: 0x47385854 (stale) Mirror 20: 0x47385854 (stale) Mirror 21: 0x47385854 (stale) Mirror 22: 0x47385854 (stale) Mirror 23: 0x47385854 (stale) Mirror 24: 0x47385854 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xf0971c41 Mirror 25: 0xf0971c41 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000842 1777593657.604492 mapwrite 0x03054c thru 0x037613 (0x70c8 bytes) 000843 1777593657.671016 read 0x00c938 thru 0x01c82c (0xfef5 bytes) 000844 1777593657.675581 mapread 0x02f023 thru 0x038c0b (0x9be9 bytes) 000845 1777593657.676675 mapwrite 0x018051 thru 0x01841c (0x3cc bytes) 000846 1777593657.690741 trunc from 0x040000 to 0x021ccc (0x1e333 bytes) 000847 1777593657.700252 MIRROR_SPLIT mirror 4 to 13 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x21ccd): Mirror 12: 0x174f9dc4 (stale) Mirror 13: 0x174f9dc4 (stale) Mirror 14: 0x174f9dc4 (stale) Mirror 16: 0x174f9dc4 (stale) Mirror 18: 0x174f9dc4 (stale) Mirror 19: 0x174f9dc4 (stale) Mirror 20: 0x174f9dc4 (stale) Mirror 21: 0x174f9dc4 (stale) Mirror 22: 0x174f9dc4 (stale) Mirror 23: 0x174f9dc4 (stale) Mirror 24: 0x174f9dc4 (stale) Mirror 25: 0x174f9dc4 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x21ccd): Mirror 12: 0x174f9dc4 (stale) Mirror 13: 0x174f9dc4 (stale) Mirror 14: 0x174f9dc4 (stale) Mirror 16: 0x174f9dc4 (stale) Mirror 18: 0x174f9dc4 (stale) Mirror 19: 0x174f9dc4 (stale) Mirror 20: 0x174f9dc4 (stale) Mirror 21: 0x174f9dc4 (stale) Mirror 22: 0x174f9dc4 (stale) Mirror 23: 0x174f9dc4 (stale) Mirror 24: 0x174f9dc4 (stale) Mirror 25: 0x174f9dc4 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x21ccd: skipped 000848 1777593657.953095 mapwrite 0x038340 thru 0x03ffff (0x7cc0 bytes) 000849 1777593657.993928 mapwrite 0x02fd0b thru 0x03680b (0x6b01 bytes) 000851 1777593658.021349 read 0x00b430 thru 0x011771 (0x6342 bytes) 000852 1777593658.026305 mapwrite 0x01aa2b thru 0x02276c (0x7d42 bytes) 000854 1777593658.048430 write 0x0393ef thru 0x03ffff (0x6c11 bytes) 000858 1777593658.051333 mapwrite 0x022900 thru 0x02f9d9 (0xd0da bytes) 000859 1777593658.074544 mapread 0x01a9f0 thru 0x01dafa (0x310b bytes) 000860 1777593658.075605 write 0x024b66 thru 0x02f35d (0xa7f8 bytes) 000861 1777593658.076549 mapwrite 0x032e90 thru 0x03a9c9 (0x7b3a bytes) 000863 1777593658.096476 MIRROR_SPLIT mirror 8 to 12 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: 0x47385854 (stale) Mirror 13: 0x47385854 (stale) Mirror 14: 0x47385854 (stale) Mirror 16: 0x47385854 (stale) Mirror 18: 0x47385854 (stale) Mirror 19: 0x47385854 (stale) Mirror 20: 0x47385854 (stale) Mirror 22: 0x47385854 (stale) Mirror 23: 0x47385854 (stale) Mirror 24: 0x47385854 (stale) Mirror 25: 0xf0971c41 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 12: 0x47385854 (stale) Mirror 13: 0x47385854 (stale) Mirror 14: 0x47385854 (stale) Mirror 16: 0x47385854 (stale) Mirror 18: 0x47385854 (stale) Mirror 19: 0x47385854 (stale) Mirror 20: 0x47385854 (stale) Mirror 22: 0x47385854 (stale) Mirror 23: 0x47385854 (stale) Mirror 24: 0x47385854 (stale) Mirror 25: 0xf0971c41 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000865 1777593658.301781 read 0x00f4d8 thru 0x01cdd9 (0xd902 bytes) 000867 1777593658.305874 mapwrite 0x0246bf thru 0x03443a (0xfd7c bytes) 000871 1777593658.331162 write 0x037d27 thru 0x03ffff (0x82d9 bytes) 000873 1777593658.332653 mapwrite 0x02d8f0 thru 0x02fa9c (0x21ad bytes) 000874 1777593658.350259 MIRROR_RESYNC 12 mirrors 000875 1777593658.848448 MIRROR_SPLIT mirror 8 to 11 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) 000876 1777593659.029015 mapread 0x031478 thru 0x0375bc (0x6145 bytes) 000877 1777593659.036333 trunc from 0x040000 to 0x0308ff (0xf700 bytes) 000878 1777593659.076879 mapwrite 0x038b41 thru 0x03ffff (0x74bf bytes) 000879 1777593659.108943 MIRROR_RESYNC 11 mirrors 000881 1777593659.506675 read 0x014240 thru 0x01d7b8 (0x9579 bytes) 000882 1777593659.509520 mapwrite 0x034807 thru 0x03731b (0x2b15 bytes) 000884 1777593659.561189 mapwrite 0x00252a thru 0x0115a7 (0xf07e bytes) 000885 1777593659.584087 mapread 0x033fee thru 0x034507 (0x51a bytes) 000886 1777593659.584872 read 0x008e38 thru 0x00fd10 (0x6ed9 bytes) 000888 1777593659.585866 mapread 0x01f170 thru 0x023341 (0x41d2 bytes) 000890 1777593659.590915 read 0x0233c3 thru 0x028674 (0x52b2 bytes) 000892 1777593659.591810 mapread 0x006763 thru 0x0132e3 (0xcb81 bytes) 000893 1777593659.592752 mapread 0x020601 thru 0x02fe66 (0xf866 bytes) 000895 1777593659.595911 MIRROR_RESYNC 11 mirrors 000896 1777593659.994277 write 0x03a0c0 thru 0x03fbc0 (0x5b01 bytes) 000897 1777593660.016099 write 0x0148af thru 0x01788c (0x2fde bytes) 000898 1777593660.019539 read 0x002b6b thru 0x00a7c7 (0x7c5d bytes) truncating to largest ever: 0x3fa48 000899 1777593660.022918 trunc from 0x040000 to 0x03fa47 (0x5b8 bytes) 000900 1777593660.035468 MIRROR_SPLIT mirror 10 to 10 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3fa48): Mirror 12: 0xe5eaa788 (stale) Mirror 13: 0xe5eaa788 (stale) Mirror 14: 0xe5eaa788 (stale) Mirror 16: 0xe5eaa788 (stale) Mirror 18: 0xe5eaa788 (stale) Mirror 19: 0xe5eaa788 (stale) Mirror 20: 0xe5eaa788 (stale) Mirror 23: 0xe5eaa788 (stale) Mirror 24: 0xe5eaa788 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3fa48): Mirror 12: 0xe5eaa788 (stale) Mirror 13: 0xe5eaa788 (stale) Mirror 14: 0xe5eaa788 (stale) Mirror 16: 0xe5eaa788 (stale) Mirror 18: 0xe5eaa788 (stale) Mirror 19: 0xe5eaa788 (stale) Mirror 20: 0xe5eaa788 (stale) Mirror 23: 0xe5eaa788 (stale) Mirror 24: 0xe5eaa788 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3fa48: skipped 000901 1777593660.205321 MIRROR_EXTEND to 11 mirrors CRC-32 checksum value for chunk [0, 0x3fa48): Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 16: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) Mirror 20: (stale - read error) Mirror 23: (stale - read error) Mirror 24: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3fa48): Mirror 12: 0xe5eaa788 (stale) Mirror 13: 0xe5eaa788 (stale) Mirror 14: 0xe5eaa788 (stale) Mirror 16: 0xe5eaa788 (stale) Mirror 18: 0xe5eaa788 (stale) Mirror 19: 0xe5eaa788 (stale) Mirror 20: 0xe5eaa788 (stale) Mirror 23: 0xe5eaa788 (stale) Mirror 24: 0xe5eaa788 (stale) CRC-32 checksum value for chunk [0, 0x3fa48): Mirror 1: 0xd1512f4f Mirror 25: 0xd1512f4f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3fa48: skipped 000902 1777593660.499846 trunc from 0x03fa48 to 0x030224 (0xf823 bytes) 000903 1777593660.530512 write 0x007a9d thru 0x00f542 (0x7aa6 bytes) 000907 1777593660.537608 MIRROR_EXTEND to 12 mirrors CRC-32 checksum value for chunk [0, 0x30225): Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 16: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) Mirror 20: (stale - read error) Mirror 23: (stale - read error) Mirror 24: (stale - read error) Mirror 25: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x30225): Mirror 12: 0x5d9eb08e (stale) Mirror 13: 0x5d9eb08e (stale) Mirror 14: 0x5d9eb08e (stale) Mirror 16: 0x5d9eb08e (stale) Mirror 18: 0x5d9eb08e (stale) Mirror 19: 0x5d9eb08e (stale) Mirror 20: 0x5d9eb08e (stale) Mirror 23: 0x5d9eb08e (stale) Mirror 24: 0x5d9eb08e (stale) Mirror 25: 0x40d68577 (stale) CRC-32 checksum value for chunk [0, 0x30225): Mirror 1: 0x8259f59d Mirror 26: 0x8259f59d lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x30225: skipped 000908 1777593660.839252 trunc from 0x030225 to 0x002405 (0x2de1f bytes) 000909 1777593660.876145 write 0x03b734 thru 0x03e8ca (0x3197 bytes) 000911 1777593660.879577 MIRROR_SPLIT mirror 1 to 11 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3e8cb): Mirror 13: 0x89846052 (stale) Mirror 14: 0x89846052 (stale) Mirror 16: 0x89846052 (stale) Mirror 18: 0x89846052 (stale) Mirror 19: 0x89846052 (stale) Mirror 20: 0x89846052 (stale) Mirror 23: 0x89846052 (stale) Mirror 24: 0x89846052 (stale) Mirror 25: 0x911c18fc (stale) Mirror 26: 0x3bf6cf90 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e8cb): Mirror 13: 0x89846052 (stale) Mirror 14: 0x89846052 (stale) Mirror 16: 0x89846052 (stale) Mirror 18: 0x89846052 (stale) Mirror 19: 0x89846052 (stale) Mirror 20: 0x89846052 (stale) Mirror 23: 0x89846052 (stale) Mirror 24: 0x89846052 (stale) Mirror 25: 0x911c18fc (stale) Mirror 26: 0x3bf6cf90 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e8cb: skipped 000913 1777593661.132234 mapwrite 0x01da56 thru 0x023c0b (0x61b6 bytes) 000915 1777593661.154623 mapread 0x030a3b thru 0x03c8ec (0xbeb2 bytes) 000917 1777593661.163094 mapread 0x03d71e thru 0x03e8ca (0x11ad bytes) 000918 1777593661.166446 MIRROR_RESYNC 11 mirrors 000922 1777593661.695540 read 0x0281ca thru 0x0359ec (0xd823 bytes) 000923 1777593661.699254 mapread 0x02ca21 thru 0x034c29 (0x8209 bytes) 000924 1777593661.700061 mapread 0x005e6e thru 0x00eb78 (0x8d0b bytes) 000925 1777593661.710110 MIRROR_SPLIT mirror 1 to 10 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) 000926 1777593661.863476 trunc from 0x03e8cb to 0x03665f (0x826b bytes) 000927 1777593661.895924 mapwrite 0x0352ea thru 0x03b717 (0x642e bytes) 000929 1777593661.931925 trunc from 0x03b718 to 0x020884 (0x1ae93 bytes) 000931 1777593661.937744 write 0x00e169 thru 0x012e6a (0x4d02 bytes) 000933 1777593661.944381 MIRROR_EXTEND to 11 mirrors CRC-32 checksum value for chunk [0, 0x20885): Mirror 14: (stale - read error) Mirror 16: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) Mirror 20: (stale - read error) Mirror 23: (stale - read error) Mirror 24: (stale - read error) Mirror 25: (stale - read error) Mirror 26: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x20885): Mirror 14: 0xa7f14da9 (stale) Mirror 16: 0xa7f14da9 (stale) Mirror 18: 0xa7f14da9 (stale) Mirror 19: 0xa7f14da9 (stale) Mirror 20: 0xa7f14da9 (stale) Mirror 23: 0xa7f14da9 (stale) Mirror 24: 0xa7f14da9 (stale) Mirror 25: 0xa7f14da9 (stale) Mirror 26: 0xa7f14da9 (stale) CRC-32 checksum value for chunk [0, 0x20885): Mirror 1: 0x3166984f Mirror 27: 0x3166984f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x20885: skipped 000937 1777593662.326417 write 0x03d9de thru 0x03ffff (0x2622 bytes) 000940 1777593662.350985 MIRROR_SPLIT mirror 8 to 10 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 14: 0x9cb3a20d (stale) Mirror 16: 0x9cb3a20d (stale) Mirror 18: 0x9cb3a20d (stale) Mirror 19: 0x9cb3a20d (stale) Mirror 20: 0x9cb3a20d (stale) Mirror 23: 0x9cb3a20d (stale) Mirror 24: 0x9cb3a20d (stale) Mirror 26: 0x9cb3a20d (stale) Mirror 27: 0x46f339de (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 14: 0x9cb3a20d (stale) Mirror 16: 0x9cb3a20d (stale) Mirror 18: 0x9cb3a20d (stale) Mirror 19: 0x9cb3a20d (stale) Mirror 20: 0x9cb3a20d (stale) Mirror 23: 0x9cb3a20d (stale) Mirror 24: 0x9cb3a20d (stale) Mirror 26: 0x9cb3a20d (stale) Mirror 27: 0x46f339de (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000942 1777593662.538001 write 0x008a80 thru 0x00a90d (0x1e8e bytes) 000943 1777593662.541789 read 0x005b53 thru 0x008fd7 (0x3485 bytes) 000944 1777593662.544416 write 0x029f05 thru 0x02bc25 (0x1d21 bytes) 000945 1777593662.549352 trunc from 0x040000 to 0x0220df (0x1df20 bytes) 000947 1777593662.556774 read 0x01f7a6 thru 0x0220df (0x293a bytes) 000949 1777593662.561597 mapread 0x00e1a1 thru 0x0165d8 (0x8438 bytes) 000950 1777593662.562532 mapwrite 0x0148c4 thru 0x021d26 (0xd463 bytes) 000951 1777593662.584906 write 0x0195c2 thru 0x01ebb2 (0x55f1 bytes) 000952 1777593662.585998 write 0x011d67 thru 0x01ed27 (0xcfc1 bytes) 000954 1777593662.587629 mapwrite 0x012937 thru 0x01e204 (0xb8ce bytes) 000955 1777593662.605309 mapwrite 0x012579 thru 0x018e7d (0x6905 bytes) 000956 1777593662.619849 mapwrite 0x00fb40 thru 0x0173fc (0x78bd bytes) 000957 1777593662.635221 mapwrite 0x01ea23 thru 0x022414 (0x39f2 bytes) 000958 1777593662.655322 trunc from 0x022415 to 0x003645 (0x1edcf bytes) 000959 1777593662.661438 trunc from 0x003646 to 0x02389d (0x20258 bytes) 000960 1777593662.667193 mapwrite 0x01f092 thru 0x02c708 (0xd677 bytes) 000961 1777593662.697126 trunc from 0x02c709 to 0x0337a8 (0x70a0 bytes) 000962 1777593662.703502 read 0x02ebc1 thru 0x0337a8 (0x4be8 bytes) 000963 1777593662.711416 MIRROR_RESYNC 10 mirrors 000964 1777593663.181741 trunc from 0x0337a9 to 0x004628 (0x2f180 bytes) 000965 1777593663.205644 read 0x0020ad thru 0x004628 (0x257c bytes) 000966 1777593663.208159 read 0x0036db thru 0x004628 (0xf4e bytes) 000967 1777593663.208530 read 0x001738 thru 0x004628 (0x2ef1 bytes) 000968 1777593663.210382 trunc from 0x004629 to 0x033171 (0x2eb49 bytes) 000969 1777593663.218546 write 0x01d28c thru 0x0258c0 (0x8635 bytes) 000971 1777593663.224744 MIRROR_EXTEND to 11 mirrors CRC-32 checksum value for chunk [0, 0x33172): Mirror 14: (stale - read error) Mirror 16: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) Mirror 20: (stale - read error) Mirror 23: (stale - read error) Mirror 24: (stale - read error) Mirror 26: (stale - read error) Mirror 27: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x33172): Mirror 14: 0xb6ac0f0d (stale) Mirror 16: 0xb6ac0f0d (stale) Mirror 18: 0xb6ac0f0d (stale) Mirror 19: 0xb6ac0f0d (stale) Mirror 20: 0xb6ac0f0d (stale) Mirror 23: 0xb6ac0f0d (stale) Mirror 24: 0xb6ac0f0d (stale) Mirror 26: 0xb6ac0f0d (stale) Mirror 27: 0xb6ac0f0d (stale) CRC-32 checksum value for chunk [0, 0x33172): Mirror 1: 0xd64238fd Mirror 28: 0xd64238fd lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x33172: skipped 000972 1777593663.517678 write 0x026af9 thru 0x030eb9 (0xa3c1 bytes) 000973 1777593663.544888 read 0x003ece thru 0x005ce9 (0x1e1c bytes) 000974 1777593663.547214 write 0x01e433 thru 0x023373 (0x4f41 bytes) 000975 1777593663.551400 trunc from 0x033172 to 0x006934 (0x2c83d bytes) 000976 1777593663.565375 MIRROR_SPLIT mirror 6 to 10 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x6935): Mirror 14: 0xfd15b348 (stale) Mirror 16: 0xfd15b348 (stale) Mirror 18: 0xfd15b348 (stale) Mirror 19: 0xfd15b348 (stale) Mirror 20: 0xfd15b348 (stale) Mirror 24: 0xfd15b348 (stale) Mirror 26: 0xfd15b348 (stale) Mirror 27: 0xfd15b348 (stale) Mirror 28: 0xfd15b348 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x6935): Mirror 14: 0xfd15b348 (stale) Mirror 16: 0xfd15b348 (stale) Mirror 18: 0xfd15b348 (stale) Mirror 19: 0xfd15b348 (stale) Mirror 20: 0xfd15b348 (stale) Mirror 24: 0xfd15b348 (stale) Mirror 26: 0xfd15b348 (stale) Mirror 27: 0xfd15b348 (stale) Mirror 28: 0xfd15b348 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x6935: skipped 000977 1777593663.703115 MIRROR_SPLIT mirror 8 to 9 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x453:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x6935): Mirror 14: 0xfd15b348 (stale) Mirror 16: 0xfd15b348 (stale) Mirror 18: 0xfd15b348 (stale) Mirror 19: 0xfd15b348 (stale) Mirror 20: 0xfd15b348 (stale) Mirror 24: 0xfd15b348 (stale) Mirror 26: 0xfd15b348 (stale) Mirror 28: 0xfd15b348 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x6935): Mirror 14: 0xfd15b348 (stale) Mirror 16: 0xfd15b348 (stale) Mirror 18: 0xfd15b348 (stale) Mirror 19: 0xfd15b348 (stale) Mirror 20: 0xfd15b348 (stale) Mirror 24: 0xfd15b348 (stale) Mirror 26: 0xfd15b348 (stale) Mirror 28: 0xfd15b348 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x6935: skipped 000979 1777593663.843852 mapwrite 0x03e573 thru 0x03ffff (0x1a8d bytes) 000981 1777593663.867774 mapwrite 0x035a73 thru 0x036049 (0x5d7 bytes) 000982 1777593663.886001 mapread 0x03696e thru 0x03c20f (0x58a2 bytes) 000983 1777593663.886849 mapread 0x00eea3 thru 0x010d40 (0x1e9e bytes) 000984 1777593663.891672 read 0x021c2c thru 0x023256 (0x162b bytes) 000985 1777593663.894602 write 0x03c26e thru 0x03ec34 (0x29c7 bytes) 000986 1777593663.895469 mapread 0x024689 thru 0x02fdd5 (0xb74d bytes) 000987 1777593663.901893 mapwrite 0x02ed63 thru 0x03afda (0xc278 bytes) 000988 1777593663.918358 mapwrite 0x007b29 thru 0x00ac11 (0x30e9 bytes) 000990 1777593663.931947 mapwrite 0x02e465 thru 0x032b4f (0x46eb bytes) 000991 1777593663.944922 mapread 0x03b4e5 thru 0x03b4f1 (0xd bytes) 000993 1777593663.946507 mapread 0x001adf thru 0x00fd57 (0xe279 bytes) 000995 1777593663.950832 mapread 0x01c019 thru 0x021d86 (0x5d6e bytes) 000997 1777593663.954407 mapwrite 0x03e407 thru 0x03ffff (0x1bf9 bytes) 001000 1777593663.967816 mapwrite 0x0238c2 thru 0x033658 (0xfd97 bytes) All operations completed A-OK!