== sanity-flr test 70a: flr mode fsx test ================ 16:55:48 (1787604948) Seed set to 4949 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x25e01 000005 1787604950.145042 trunc from 00000000 to 0x025e00 (0x25e01 bytes) 000006 1787604950.161147 read 0x00c14a thru 0x00ea44 (0x28fb bytes) 000007 1787604950.166018 mapwrite 0x03936f thru 0x03ffff (0x6c91 bytes) 000016 1787604950.265880 write 0x033044 thru 0x033ba1 (0xb5e bytes) 000017 1787604950.269939 read 0x02b80d thru 0x03517d (0x9971 bytes) 000020 1787604950.281448 read 0x00fe5f thru 0x01859d (0x873f bytes) 000021 1787604950.282227 mapwrite 0x01274a thru 0x01d648 (0xaeff bytes) 000023 1787604950.339457 mapread 0x01333e thru 0x021b95 (0xe858 bytes) 000024 1787604950.345502 MIRROR_EXTEND to 2 mirrors 000025 1787604951.286589 MIRROR_EXTEND to 3 mirrors truncating to largest ever: 0x2e89c 000028 1787604952.108547 trunc from 0x040000 to 0x02e89b (0x11764 bytes) 000029 1787604952.193260 MIRROR_RESYNC 3 mirrors 000031 1787604953.069236 MIRROR_RESYNC 3 mirrors 000034 1787604953.412340 write 0x01e950 thru 0x021672 (0x2d23 bytes) 000036 1787604953.532300 trunc from 0x02e89c to 0x01a86c (0x1402f bytes) 000037 1787604953.552209 read 0x018990 thru 0x01a86c (0x1edd bytes) 000038 1787604953.558215 read 0x0160d1 thru 0x01a86c (0x479c bytes) 000040 1787604953.568887 read 0x00fa03 thru 0x013276 (0x3874 bytes) 000041 1787604953.575705 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1a86d): Mirror 3: 0xef8b8f6b (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1a86d): Mirror 3: 0xef8b8f6b (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1a86d: skipped 000042 1787604953.869789 mapwrite 0x00e1ea thru 0x012bed (0x4a04 bytes) 000045 1787604953.946492 mapwrite 0x004826 thru 0x00d136 (0x8911 bytes) 000046 1787604954.001971 mapwrite 0x03400d thru 0x038122 (0x4116 bytes) 000047 1787604954.116284 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000049 1787604954.306137 write 0x013e6b thru 0x01a844 (0x69da bytes) 000051 1787604954.318229 write 0x02e6cd thru 0x0324ab (0x3ddf bytes) 000054 1787604954.330673 read 0x008123 thru 0x00eee7 (0x6dc5 bytes) 000056 1787604954.342531 mapwrite 0x0221c6 thru 0x030fbb (0xedf6 bytes) 000057 1787604954.462087 write 0x00cce9 thru 0x0178fe (0xac16 bytes) 000058 1787604954.463679 read 0x02dc90 thru 0x038122 (0xa493 bytes) 000061 1787604954.477570 trunc from 0x038123 to 0x003cff (0x34423 bytes) 000064 1787604954.496725 mapread 0x002706 thru 0x003cff (0x15fa bytes) 000065 1787604954.498716 mapwrite 0x00c2c5 thru 0x010b50 (0x488c bytes) 000066 1787604954.625358 write 0x034e97 thru 0x03f907 (0xaa71 bytes) 000067 1787604954.627025 mapread 0x02bf9f thru 0x039e11 (0xde73 bytes) 000071 1787604954.709419 mapread 0x00185b thru 0x007b1b (0x62c1 bytes) 000073 1787604954.753270 trunc from 0x03f908 to 0x018f6c (0x2699b bytes) 000074 1787604954.775018 mapread 0x017f75 thru 0x018f6c (0xff8 bytes) 000075 1787604954.787415 read 0x015078 thru 0x018f6c (0x3ef5 bytes) 000076 1787604954.798099 MIRROR_EXTEND to 2 mirrors 000078 1787604955.548397 read 0x0046de thru 0x00ba8b (0x73ae bytes) 000079 1787604955.561562 mapread 0x006735 thru 0x010e0d (0xa6d9 bytes) 000080 1787604955.594778 mapwrite 0x0210ef thru 0x024602 (0x3514 bytes) 000081 1787604955.739973 write 0x01e531 thru 0x027949 (0x9419 bytes) 000083 1787604955.749637 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000086 1787604955.904604 trunc from 0x02794a to 0x015c57 (0x11cf2 bytes) 000089 1787604955.921586 trunc from 0x015c58 to 0x021b79 (0xbf22 bytes) 000092 1787604955.934658 trunc from 0x021b7a to 0x00530e (0x1c86b bytes) truncating to largest ever: 0x31076 000094 1787604955.950595 trunc from 0x00530f to 0x031075 (0x2bd67 bytes) 000096 1787604955.961565 trunc from 0x031076 to 0x00849b (0x28bda bytes) 000097 1787604955.973408 mapwrite 0x03b7ee thru 0x03ffff (0x4812 bytes) 000098 1787604956.029334 mapread 0x02f473 thru 0x03e504 (0xf092 bytes) 000099 1787604956.035716 mapwrite 0x01316e thru 0x02056f (0xd402 bytes) 000103 1787604956.089933 mapwrite 0x033245 thru 0x03d38c (0xa148 bytes) 000106 1787604956.145444 mapread 0x01c369 thru 0x022138 (0x5dd0 bytes) 000110 1787604956.149586 mapwrite 0x00f433 thru 0x01a94e (0xb51c bytes) 000114 1787604956.198621 mapwrite 0x0292f3 thru 0x035930 (0xc63e bytes) 000115 1787604956.249472 MIRROR_EXTEND to 2 mirrors 000116 1787604956.814041 mapread 0x012c87 thru 0x01fef9 (0xd273 bytes) 000117 1787604956.842095 read 0x0275be thru 0x033cac (0xc6ef bytes) 000118 1787604956.849353 mapread 0x03e8db thru 0x03ffff (0x1725 bytes) 000119 1787604956.859467 mapwrite 0x036bee thru 0x03d299 (0x66ac bytes) truncating to largest ever: 0x3a4b2 000122 1787604956.992544 trunc from 0x040000 to 0x03a4b1 (0x5b4e bytes) 000123 1787604957.044163 mapwrite 0x00c4ef thru 0x0122b0 (0x5dc2 bytes) 000126 1787604957.120380 write 0x019cd2 thru 0x029403 (0xf732 bytes) 000128 1787604957.135025 MIRROR_RESYNC 2 mirrors 000133 1787604957.871841 mapread 0x005db1 thru 0x00a49e (0x46ee bytes) 000135 1787604957.897341 mapwrite 0x01050f thru 0x012ada (0x25cc bytes) 000136 1787604958.047424 trunc from 0x03a4b2 to 0x02d7a0 (0xcd11 bytes) 000138 1787604958.099659 mapread 0x02d100 thru 0x02d7a0 (0x6a1 bytes) 000141 1787604958.111872 trunc from 0x02d7a1 to 0x01e35f (0xf441 bytes) 000142 1787604958.151621 write 0x0087f2 thru 0x017856 (0xf065 bytes) 000149 1787604958.162978 mapwrite 0x01a409 thru 0x029f1a (0xfb12 bytes) 000150 1787604958.299929 mapwrite 0x03f36f thru 0x03ffff (0xc91 bytes) 000151 1787604958.366985 mapread 0x02dceb thru 0x0357e7 (0x7afd bytes) 000154 1787604958.379270 write 0x0299c0 thru 0x032ce7 (0x9328 bytes) 000155 1787604958.381025 write 0x01572e thru 0x018a89 (0x335c bytes) 000157 1787604958.383069 mapwrite 0x0039d8 thru 0x008718 (0x4d41 bytes) 000158 1787604958.440566 mapread 0x0337d2 thru 0x036eab (0x36da bytes) 000162 1787604958.447055 write 0x00cafd thru 0x0120d3 (0x55d7 bytes) 000163 1787604958.448356 read 0x0185fe thru 0x02011a (0x7b1d bytes) 000164 1787604958.449339 mapwrite 0x01f62c thru 0x02c275 (0xcc4a bytes) 000165 1787604958.499079 trunc from 0x040000 to 0x0398bc (0x6743 bytes) 000170 1787604958.522909 trunc from 0x0398bd to 0x0370d6 (0x27e6 bytes) 000173 1787604958.562877 MIRROR_RESYNC 2 mirrors 000174 1787604959.288240 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000177 1787604959.411296 mapwrite 0x0178fe thru 0x02055d (0x8c60 bytes) 000178 1787604959.496791 write 0x0112c0 thru 0x016bd7 (0x5918 bytes) 000179 1787604959.509142 mapwrite 0x00cfb3 thru 0x0142a3 (0x72f1 bytes) 000180 1787604959.582181 write 0x0184ac thru 0x01899d (0x4f2 bytes) 000187 1787604959.588080 trunc from 0x0370d7 to 0x01db21 (0x195b5 bytes) 000188 1787604959.622910 MIRROR_EXTEND to 2 mirrors 000191 1787604960.503508 MIRROR_EXTEND to 3 mirrors 000192 1787604961.329558 read 0x019bad thru 0x01db21 (0x3f75 bytes) 000193 1787604961.339925 read 0x0162e4 thru 0x01db21 (0x783e bytes) 000194 1787604961.351581 trunc from 0x01db22 to 0x02b8c9 (0xdda8 bytes) 000196 1787604961.431997 write 0x03ff72 thru 0x03ffff (0x8e bytes) 000197 1787604961.433207 mapwrite 0x02949a thru 0x02d550 (0x40b7 bytes) 000198 1787604961.484522 MIRROR_RESYNC 3 mirrors 000199 1787604962.490962 mapread 0x00495e thru 0x012772 (0xde15 bytes) 000200 1787604962.511332 write 0x02ca5d thru 0x03bbdb (0xf17f bytes) 000201 1787604962.585977 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x302c208 (stale) Mirror 3: 0x302c208 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xf0a6b5a6 Mirror 4: 0xf0a6b5a6 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000204 1787604963.740903 read 0x0305b3 thru 0x03a041 (0x9a8f bytes) 000205 1787604963.758243 mapread 0x0230ed thru 0x03004a (0xcf5e bytes) 000206 1787604963.793799 write 0x031294 thru 0x0316ca (0x437 bytes) 000208 1787604963.877406 trunc from 0x040000 to 0x02c303 (0x13cfc bytes) 000210 1787604963.912629 trunc from 0x02c304 to 0x002e92 (0x29471 bytes) 000212 1787604963.939526 mapread 0x00004f thru 0x000ef2 (0xea4 bytes) 000213 1787604963.945831 mapread 0x001797 thru 0x002e92 (0x16fc bytes) 000214 1787604963.952053 write 0x006e26 thru 0x00725f (0x43a bytes) 000216 1787604963.981927 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x7260): Mirror 2: (stale - read error) Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x7260): Mirror 2: 0xe074d361 (stale) Mirror 3: 0xe074d361 (stale) Mirror 4: 0xe074d361 (stale) CRC-32 checksum value for chunk [0, 0x7260): Mirror 1: 0xdd51de2f Mirror 5: 0xdd51de2f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x7260: skipped 000219 1787604964.811745 mapread 0x005d4c thru 0x00661f (0x8d4 bytes) 000220 1787604964.848481 MIRROR_SPLIT mirror 2 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x7260): Mirror 2: 0xe074d361 (stale) Mirror 4: 0xe074d361 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x7260): Mirror 2: 0xe074d361 (stale) Mirror 4: 0xe074d361 (stale) CRC-32 checksum value for chunk [0, 0x7260): Mirror 1: 0xdd51de2f Mirror 5: 0xdd51de2f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x7260: skipped 000225 1787604965.592042 read 0x006d7f thru 0x00725f (0x4e1 bytes) 000229 1787604965.660269 read 0x006d63 thru 0x00725f (0x4fd bytes) 000230 1787604965.676440 read 0x0001dc thru 0x00725f (0x7084 bytes) 000232 1787604965.719828 mapread 0x000c3f thru 0x00725f (0x6621 bytes) 000237 1787604965.808288 trunc from 0x007260 to 0x01ad8f (0x13b30 bytes) 000238 1787604965.886107 write 0x028320 thru 0x02d085 (0x4d66 bytes) 000240 1787604965.889001 mapread 0x012bd3 thru 0x01cf5d (0xa38b bytes) 000241 1787604965.902063 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2d086): Mirror 4: 0x465ccfc (stale) Mirror 5: 0x36ebe89f (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2d086): Mirror 4: 0x465ccfc (stale) Mirror 5: 0x36ebe89f (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2d086: skipped 000242 1787604966.316389 trunc from 0x02d086 to 0x017088 (0x15ffd bytes) 000243 1787604966.341290 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x17089): Mirror 5: 0xfbf6b499 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x17089): Mirror 5: 0xfbf6b499 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x17089: skipped 000249 1787604966.567488 write 0x008383 thru 0x011d63 (0x99e1 bytes) 000250 1787604966.581148 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x17089): Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x17089): Mirror 5: 0xfbf6b499 (stale) CRC-32 checksum value for chunk [0, 0x17089): Mirror 1: 0xc6d57017 Mirror 6: 0xc6d57017 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x17089: skipped 000252 1787604967.564162 mapwrite 0x01ce71 thru 0x023973 (0x6b03 bytes) 000253 1787604967.860368 mapwrite 0x02d07d thru 0x03193a (0x48be bytes) 000254 1787604967.953669 trunc from 0x03193b to 0x006b52 (0x2ade8 bytes) 000259 1787604967.974199 write 0x005045 thru 0x00f901 (0xa8bd bytes) 000260 1787604967.983021 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0xf902): Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xf902): Mirror 5: 0x2659b5ad (stale) Mirror 6: 0x38a78bdf (stale) CRC-32 checksum value for chunk [0, 0xf902): Mirror 1: 0x553a1eb0 Mirror 7: 0x553a1eb0 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xf902: skipped 000263 1787604968.893218 trunc from 0x00f902 to 0x017542 (0x7c41 bytes) 000265 1787604968.966845 mapread 0x01361a thru 0x017542 (0x3f29 bytes) 000267 1787604968.999033 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x17543): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x17543): Mirror 5: 0x6399ef8e (stale) Mirror 6: 0xa51e3142 (stale) Mirror 7: 0x6309304f (stale) CRC-32 checksum value for chunk [0, 0x17543): Mirror 1: 0x6309304f Mirror 8: 0x6309304f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x17543: skipped 000268 1787604970.018963 read 0x0018b8 thru 0x005116 (0x385f bytes) 000269 1787604970.030947 mapwrite 0x03c99c thru 0x03f0e2 (0x2747 bytes) 000272 1787604970.164918 mapwrite 0x038d0f thru 0x03ffff (0x72f1 bytes) 000276 1787604970.240223 MIRROR_RESYNC 5 mirrors 000277 1787604971.767642 MIRROR_RESYNC 5 mirrors 000278 1787604972.231120 mapread 0x03e237 thru 0x03ffff (0x1dc9 bytes) 000279 1787604972.263314 mapwrite 0x00ca46 thru 0x0171f0 (0xa7ab bytes) 000283 1787604972.570307 read 0x02f6d1 thru 0x0362dc (0x6c0c bytes) 000284 1787604972.580661 mapwrite 0x005dbf thru 0x00a674 (0x48b6 bytes) 000285 1787604972.664918 trunc from 0x040000 to 0x003801 (0x3c7fe bytes) 000286 1787604972.706173 write 0x024227 thru 0x03357b (0xf355 bytes) 000289 1787604972.713395 read 0x026555 thru 0x03357b (0xd027 bytes) 000290 1787604972.714797 mapread 0x00c8e8 thru 0x014bbc (0x82d5 bytes) 000293 1787604972.733927 write 0x01a014 thru 0x020cae (0x6c9b bytes) 000294 1787604972.739884 write 0x02f69e thru 0x030226 (0xb89 bytes) 000295 1787604972.742153 trunc from 0x03357c to 0x02ae74 (0x8707 bytes) 000296 1787604972.766019 mapread 0x00afcd thru 0x0115a5 (0x65d9 bytes) 000297 1787604972.767089 read 0x002ddc thru 0x00493c (0x1b61 bytes) 000300 1787604972.770115 trunc from 0x02ae75 to 0x01d5e3 (0xd891 bytes) 000303 1787604972.816841 MIRROR_RESYNC 5 mirrors 000304 1787604974.103947 mapwrite 0x035071 thru 0x03ffff (0xaf8f bytes) 000305 1787604974.270949 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0x21bfc574 (stale) Mirror 6: 0x21bfc574 (stale) Mirror 7: 0x21bfc574 (stale) Mirror 8: 0x21bfc574 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xa1159c25 Mirror 9: 0xa1159c25 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000306 1787604975.419458 mapread 0x02208b thru 0x02d7cb (0xb741 bytes) 000308 1787604975.455199 MIRROR_SPLIT mirror 1 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: 0x21bfc574 (stale) Mirror 7: 0x21bfc574 (stale) Mirror 8: 0x21bfc574 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: 0x21bfc574 (stale) Mirror 7: 0x21bfc574 (stale) Mirror 8: 0x21bfc574 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xa1159c25 Mirror 9: 0xa1159c25 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000309 1787604976.027851 mapwrite 0x0040ab thru 0x00eeb7 (0xae0d bytes) 000310 1787604976.259552 mapread 0x018621 thru 0x027099 (0xea79 bytes) 000311 1787604976.281212 read 0x031930 thru 0x03c424 (0xaaf5 bytes) 000313 1787604976.293655 write 0x03df0b thru 0x03ffff (0x20f5 bytes) 000314 1787604976.307353 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 7: 0x21bfc574 (stale) Mirror 8: 0x21bfc574 (stale) Mirror 9: 0xa1159c25 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 7: 0x21bfc574 (stale) Mirror 8: 0x21bfc574 (stale) Mirror 9: 0xa1159c25 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000315 1787604976.780084 MIRROR_RESYNC 4 mirrors 000317 1787604978.200888 read 0x026d6c thru 0x029d88 (0x301d bytes) 000318 1787604978.207111 read 0x034016 thru 0x03ffff (0xbfea bytes) 000321 1787604978.224207 write 0x0353aa thru 0x03757e (0x21d5 bytes) 000323 1787604978.347178 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 7: 0xa7eb93ef (stale) Mirror 8: 0xa7eb93ef (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 7: 0xa7eb93ef (stale) Mirror 8: 0xa7eb93ef (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000328 1787604978.734263 trunc from 0x040000 to 0x0302d2 (0xfd2d bytes) 000330 1787604978.750904 write 0x01944f thru 0x01f028 (0x5bda bytes) 000332 1787604978.762222 mapread 0x0054e8 thru 0x014761 (0xf27a bytes) 000333 1787604978.787003 read 0x00de01 thru 0x01b3fd (0xd5fd bytes) 000335 1787604978.795736 MIRROR_RESYNC 3 mirrors 000338 1787604980.087725 write 0x02ce27 thru 0x02f6f1 (0x28cb bytes) 000339 1787604980.151509 mapwrite 0x03a661 thru 0x03e73b (0x40db bytes) 000342 1787604980.234380 write 0x026e71 thru 0x034461 (0xd5f1 bytes) 000343 1787604980.244116 write 0x02c39b thru 0x03407e (0x7ce4 bytes) 000347 1787604980.246841 read 0x0157ec thru 0x01e666 (0x8e7b bytes) 000348 1787604980.254021 mapwrite 0x03f398 thru 0x03ffff (0xc68 bytes) 000350 1787604980.307299 write 0x01c251 thru 0x028c65 (0xca15 bytes) 000351 1787604980.308803 mapwrite 0x016ba0 thru 0x01db94 (0x6ff5 bytes) 000352 1787604980.407140 write 0x031a8f thru 0x03ffff (0xe571 bytes) 000354 1787604980.410098 trunc from 0x040000 to 0x027dd9 (0x18226 bytes) 000355 1787604980.424328 mapwrite 0x03f1dc thru 0x03ffff (0xe24 bytes) 000357 1787604980.587337 mapwrite 0x0335f3 thru 0x03ffff (0xca0d bytes) 000359 1787604980.686724 mapread 0x02c30d thru 0x02fe5b (0x3b4f bytes) 000360 1787604980.687994 trunc from 0x040000 to 0x02892f (0x176d0 bytes) 000361 1787604980.706747 mapwrite 0x007f08 thru 0x014fa2 (0xd09b bytes) 000362 1787604980.791477 read 0x01518b thru 0x020f4c (0xbdc2 bytes) 000363 1787604980.796968 MIRROR_RESYNC 3 mirrors 000364 1787604981.781290 read 0x002bd8 thru 0x00fdbe (0xd1e7 bytes) 000365 1787604981.787509 write 0x030c54 thru 0x032532 (0x18df bytes) 000367 1787604981.849436 mapwrite 0x0228b9 thru 0x02eb42 (0xc28a bytes) 000369 1787604981.916231 read 0x002a80 thru 0x00f0f6 (0xc677 bytes) 000371 1787604981.925343 trunc from 0x032533 to 0x000b82 (0x319b0 bytes) 000373 1787604981.958271 MIRROR_RESYNC 3 mirrors 000374 1787604982.575286 read 0x000234 thru 0x000b82 (0x94f bytes) 000375 1787604982.580717 mapwrite 0x005a5c thru 0x00df86 (0x852b bytes) 000376 1787604982.740513 trunc from 0x00df87 to 0x000afd (0xd489 bytes) 000377 1787604982.761164 read 0x0006a5 thru 0x000afd (0x459 bytes) 000378 1787604982.769397 mapwrite 0x03f407 thru 0x03ffff (0xbf9 bytes) 000379 1787604982.835512 trunc from 0x040000 to 0x021206 (0x1edf9 bytes) 000380 1787604982.858148 mapwrite 0x00757e thru 0x00ee33 (0x78b6 bytes) 000381 1787604982.923645 read 0x0203b4 thru 0x021206 (0xe53 bytes) 000382 1787604982.924873 read 0x018557 thru 0x01978a (0x1234 bytes) 000386 1787604982.945611 MIRROR_RESYNC 3 mirrors 000387 1787604984.132409 read 0x015260 thru 0x021206 (0xbfa7 bytes) 000388 1787604984.149885 MIRROR_RESYNC 3 mirrors 000389 1787604984.656909 mapread 0x0053e6 thru 0x00d445 (0x8060 bytes) 000390 1787604984.694220 mapread 0x018175 thru 0x021206 (0x9092 bytes) 000392 1787604984.700135 write 0x004a16 thru 0x0099f3 (0x4fde bytes) 000395 1787604984.818646 read 0x00fc5f thru 0x01b4fc (0xb89e bytes) 000397 1787604984.827315 read 0x016f59 thru 0x021206 (0xa2ae bytes) 000398 1787604984.834859 read 0x0034ea thru 0x010d35 (0xd84c bytes) 000399 1787604984.835480 read 0x000868 thru 0x00ee62 (0xe5fb bytes) 000400 1787604984.836081 read 0x017ad5 thru 0x017e3b (0x367 bytes) 000402 1787604984.838292 read 0x01efcc thru 0x021206 (0x223b bytes) 000406 1787604984.841682 mapwrite 0x01a69a thru 0x027a0a (0xd371 bytes) 000407 1787604984.918741 write 0x037972 thru 0x03ad84 (0x3413 bytes) 000408 1787604984.920369 write 0x02e5db thru 0x03ab82 (0xc5a8 bytes) 000409 1787604984.928917 mapwrite 0x020450 thru 0x02bcf1 (0xb8a2 bytes) 000410 1787604984.990024 write 0x0022f0 thru 0x011d5b (0xfa6c bytes) 000412 1787604984.991880 write 0x00fc12 thru 0x016641 (0x6a30 bytes) 000413 1787604984.994179 read 0x025994 thru 0x028c4c (0x32b9 bytes) 000414 1787604984.998076 mapread 0x00b444 thru 0x00b7b2 (0x36f bytes) 000415 1787604985.000352 write 0x020ac7 thru 0x02e96a (0xdea4 bytes) 000416 1787604985.002363 mapread 0x013015 thru 0x01fdf4 (0xcde0 bytes) 000417 1787604985.003928 mapwrite 0x026e8f thru 0x02a694 (0x3806 bytes) 000418 1787604985.058155 mapread 0x00557a thru 0x012989 (0xd410 bytes) 000419 1787604985.059407 write 0x00fd52 thru 0x011bc6 (0x1e75 bytes) 000420 1787604985.064902 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x3ad85): Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3ad85): Mirror 7: 0x718f893b (stale) Mirror 8: 0x718f893b (stale) CRC-32 checksum value for chunk [0, 0x3ad85): Mirror 1: 0xb3af1b9f Mirror 9: 0xb3af1b9f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3ad85: skipped 000421 1787604985.983930 mapread 0x008ed3 thru 0x00d86f (0x499d bytes) 000422 1787604986.009638 read 0x00161c thru 0x0065c0 (0x4fa5 bytes) 000423 1787604986.018596 read 0x007ebf thru 0x01000f (0x8151 bytes) 000426 1787604986.037921 read 0x0040e5 thru 0x00505c (0xf78 bytes) 000427 1787604986.045072 read 0x00b4ed thru 0x018d3e (0xd852 bytes) 000428 1787604986.066222 read 0x01abd9 thru 0x023fb2 (0x93da bytes) 000429 1787604986.072673 read 0x0164e5 thru 0x02629a (0xfdb6 bytes) 000431 1787604986.086140 mapwrite 0x03ede2 thru 0x03ffff (0x121e bytes) 000433 1787604986.266223 mapwrite 0x0262a2 thru 0x032a87 (0xc7e6 bytes) 000435 1787604986.314399 write 0x032007 thru 0x03ffff (0xdff9 bytes) 000437 1787604986.325382 trunc from 0x040000 to 0x013f6a (0x2c095 bytes) 000438 1787604986.371099 mapwrite 0x01bf24 thru 0x02744f (0xb52c bytes) 000439 1787604986.562634 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x27450): Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x27450): Mirror 7: 0xfa1e83fe (stale) Mirror 8: 0xfa1e83fe (stale) Mirror 9: 0xa7eb20b5 (stale) CRC-32 checksum value for chunk [0, 0x27450): Mirror 1: 0x5ece859 Mirror 10: 0x5ece859 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x27450: skipped 000440 1787604987.424731 mapwrite 0x02ecd7 thru 0x038e58 (0xa182 bytes) 000441 1787604987.663088 mapread 0x02936e thru 0x02d8b8 (0x454b bytes) 000442 1787604987.664047 write 0x03c15d thru 0x03f362 (0x3206 bytes) 000445 1787604987.671444 trunc from 0x03f363 to 0x000586 (0x3eddc bytes) 000446 1787604987.704354 trunc from 0x000587 to 0x027454 (0x26ece bytes) 000447 1787604987.733676 trunc from 0x027455 to 0x02259f (0x4eb5 bytes) 000448 1787604987.757324 write 0x00d2f1 thru 0x013611 (0x6321 bytes) 000453 1787604987.770398 read 0x01331a thru 0x01fe64 (0xcb4b bytes) 000455 1787604987.777782 mapwrite 0x02ef41 thru 0x03a219 (0xb2d9 bytes) 000456 1787604987.887444 write 0x029b53 thru 0x035843 (0xbcf1 bytes) 000457 1787604987.889107 mapwrite 0x011224 thru 0x01aa6b (0x9848 bytes) 000459 1787604987.952348 mapwrite 0x0264a7 thru 0x030761 (0xa2bb bytes) truncating to largest ever: 0x3d0ef 000460 1787604988.016455 trunc from 0x03a21a to 0x03d0ee (0x2ed5 bytes) 000461 1787604988.028430 write 0x035d4c thru 0x03ffff (0xa2b4 bytes) 000462 1787604988.029690 trunc from 0x040000 to 0x004260 (0x3bd9f bytes) 000464 1787604988.046420 mapwrite 0x00a51f thru 0x011b3b (0x761d bytes) 000466 1787604988.101129 mapwrite 0x012ab2 thru 0x016256 (0x37a5 bytes) 000467 1787604988.150143 read 0x00591c thru 0x00ed95 (0x947a bytes) 000469 1787604988.161418 MIRROR_RESYNC 5 mirrors 000470 1787604989.137150 MIRROR_EXTEND to 6 mirrors 000471 1787604989.845568 mapwrite 0x027b8e thru 0x02982c (0x1c9f bytes) 000475 1787604990.046227 read 0x00609e thru 0x00d52f (0x7492 bytes) 000479 1787604990.061384 MIRROR_SPLIT mirror 4 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2982d): Mirror 7: 0x864df895 (stale) Mirror 8: 0x864df895 (stale) Mirror 9: 0x864df895 (stale) Mirror 11: 0x864df895 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2982d): Mirror 7: 0x864df895 (stale) Mirror 8: 0x864df895 (stale) Mirror 9: 0x864df895 (stale) Mirror 11: 0x864df895 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2982d: skipped 000481 1787604990.481015 read 0x01bf05 thru 0x01f038 (0x3134 bytes) 000483 1787604990.493303 mapwrite 0x001890 thru 0x0023ab (0xb1c bytes) truncating to largest ever: 0x3e880 000484 1787604990.575107 trunc from 0x02982d to 0x03e87f (0x15053 bytes) 000487 1787604990.608800 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x3e880): Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 11: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e880): Mirror 7: 0x9d5df278 (stale) Mirror 8: 0x9d5df278 (stale) Mirror 9: 0x9d5df278 (stale) Mirror 11: 0x9d5df278 (stale) CRC-32 checksum value for chunk [0, 0x3e880): Mirror 1: 0xee7f67c4 Mirror 12: 0xee7f67c4 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e880: skipped 000491 1787604991.650409 mapwrite 0x01473f thru 0x023a50 (0xf312 bytes) 000492 1787604991.834181 read 0x03b777 thru 0x03e87f (0x3109 bytes) 000493 1787604991.841985 trunc from 0x03e880 to 0x026c80 (0x17bff bytes) 000495 1787604991.889480 MIRROR_RESYNC 6 mirrors 000497 1787604994.039952 read 0x008588 thru 0x00b160 (0x2bd9 bytes) 000498 1787604994.047629 mapwrite 0x01e707 thru 0x02632f (0x7c29 bytes) 000499 1787604994.352385 trunc from 0x026c81 to 0x01ae0a (0xbe76 bytes) 000502 1787604994.401439 mapwrite 0x02699c thru 0x026ddd (0x442 bytes) 000505 1787604994.501236 trunc from 0x026dde to 0x01070e (0x166cf bytes) 000509 1787604994.538367 trunc from 0x01070f to 0x0323e3 (0x21cd5 bytes) 000510 1787604994.569807 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x323e4): Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x323e4): Mirror 7: 0x7fba2104 (stale) Mirror 8: 0x7fba2104 (stale) Mirror 9: 0x7fba2104 (stale) Mirror 11: 0x7fba2104 (stale) Mirror 12: 0x7fba2104 (stale) CRC-32 checksum value for chunk [0, 0x323e4): Mirror 1: 0x89f74f70 Mirror 13: 0x89f74f70 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x323e4: skipped 000511 1787604995.936347 read 0x01d277 thru 0x01dc71 (0x9fb bytes) 000512 1787604995.952198 trunc from 0x0323e4 to 0x001ce2 (0x30701 bytes) 000515 1787604996.068582 MIRROR_EXTEND to 8 mirrors CRC-32 checksum value for chunk [0, 0x1ce3): Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) Mirror 13: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1ce3): Mirror 7: 0xb8c95edf (stale) Mirror 8: 0xb8c95edf (stale) Mirror 9: 0xb8c95edf (stale) Mirror 11: 0xb8c95edf (stale) Mirror 12: 0xb8c95edf (stale) Mirror 13: 0xb8c95edf (stale) CRC-32 checksum value for chunk [0, 0x1ce3): Mirror 1: 0xb8c95edf Mirror 14: 0xb8c95edf lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1ce3: skipped 000517 1787604997.171861 mapwrite 0x0020b8 thru 0x007084 (0x4fcd bytes) 000518 1787604997.372254 trunc from 0x007085 to 0x02023f (0x191bb bytes) 000521 1787604997.397254 MIRROR_SPLIT mirror 7 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x20240): Mirror 7: 0xe8335def (stale) Mirror 8: 0xe8335def (stale) Mirror 9: 0xe8335def (stale) Mirror 11: 0xe8335def (stale) Mirror 12: 0xe8335def (stale) Mirror 13: 0x46c9c550 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x20240): Mirror 7: 0xe8335def (stale) Mirror 8: 0xe8335def (stale) Mirror 9: 0xe8335def (stale) Mirror 11: 0xe8335def (stale) Mirror 12: 0xe8335def (stale) Mirror 13: 0x46c9c550 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x20240: skipped 000522 1787604997.768546 mapwrite 0x02186f thru 0x02e6cd (0xce5f bytes) 000523 1787604997.873702 mapread 0x000392 thru 0x00b87a (0xb4e9 bytes) 000525 1787604997.893105 MIRROR_EXTEND to 8 mirrors CRC-32 checksum value for chunk [0, 0x2e6ce): Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) Mirror 13: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2e6ce): Mirror 7: 0x5a69d226 (stale) Mirror 8: 0x5a69d226 (stale) Mirror 9: 0x5a69d226 (stale) Mirror 11: 0x5a69d226 (stale) Mirror 12: 0x5a69d226 (stale) Mirror 13: 0x699bbdee (stale) CRC-32 checksum value for chunk [0, 0x2e6ce): Mirror 1: 0x6dfaa1db Mirror 14: 0x6dfaa1db lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2e6ce: skipped 000526 1787604998.943354 read 0x00fbc4 thru 0x01bed7 (0xc314 bytes) 000530 1787604998.982296 trunc from 0x02e6ce to 0x0106a4 (0x1e029 bytes) 000533 1787604999.105083 mapwrite 0x0082d7 thru 0x00b38b (0x30b5 bytes) 000534 1787604999.169534 mapwrite 0x034c43 thru 0x039196 (0x4554 bytes) 000536 1787604999.278318 mapread 0x0380ad thru 0x039196 (0x10ea bytes) 000538 1787604999.279727 mapread 0x00d41e thru 0x011438 (0x401b bytes) 000540 1787604999.301559 trunc from 0x039197 to 0x036c41 (0x2555 bytes) 000546 1787604999.328516 write 0x027dda thru 0x027fa3 (0x1ca bytes) 000547 1787604999.335859 mapread 0x02c33e thru 0x036c41 (0xa904 bytes) 000548 1787604999.337113 trunc from 0x036c42 to 0x016106 (0x20b3b bytes) 000549 1787604999.358643 MIRROR_SPLIT mirror 1 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x16107): Mirror 8: 0xe891ba62 (stale) Mirror 9: 0xe891ba62 (stale) Mirror 11: 0xe891ba62 (stale) Mirror 12: 0xe891ba62 (stale) Mirror 13: 0xffd99802 (stale) Mirror 14: 0x1bf4380f (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x16107): Mirror 8: 0xe891ba62 (stale) Mirror 9: 0xe891ba62 (stale) Mirror 11: 0xe891ba62 (stale) Mirror 12: 0xe891ba62 (stale) Mirror 13: 0xffd99802 (stale) Mirror 14: 0x1bf4380f (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x16107: skipped 000553 1787604999.869891 write 0x0308b1 thru 0x033ec8 (0x3618 bytes) 000557 1787604999.878941 mapwrite 0x032b73 thru 0x03ea7b (0xbf09 bytes) 000558 1787605000.001370 mapread 0x037cdb thru 0x039f70 (0x2296 bytes) 000561 1787605000.007851 write 0x0249a9 thru 0x02872b (0x3d83 bytes) 000563 1787605000.034919 mapwrite 0x03ba7b thru 0x03ffff (0x4585 bytes) 000565 1787605000.166824 trunc from 0x040000 to 0x00cbd9 (0x33426 bytes) 000567 1787605000.191215 trunc from 0x00cbda to 0x00a292 (0x2947 bytes) 000569 1787605000.227949 write 0x02a358 thru 0x02f94b (0x55f4 bytes) 000571 1787605000.230906 write 0x028380 thru 0x02fc60 (0x78e1 bytes) 000572 1787605000.248275 mapread 0x013936 thru 0x01d7e3 (0x9eae bytes) 000573 1787605000.286338 MIRROR_EXTEND to 8 mirrors CRC-32 checksum value for chunk [0, 0x2fc61): Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2fc61): Mirror 8: 0x7218fed1 (stale) Mirror 9: 0x7218fed1 (stale) Mirror 11: 0x7218fed1 (stale) Mirror 12: 0x7218fed1 (stale) Mirror 13: 0x1d1cd68c (stale) Mirror 14: 0x4262f797 (stale) CRC-32 checksum value for chunk [0, 0x2fc61): Mirror 1: 0xd1826701 Mirror 15: 0xd1826701 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2fc61: skipped 000574 1787605001.605619 mapwrite 0x037088 thru 0x03ffff (0x8f78 bytes) 000577 1787605001.810419 mapread 0x018c2a thru 0x01f0da (0x64b1 bytes) 000578 1787605001.829413 MIRROR_RESYNC 8 mirrors 000582 1787605004.570130 write 0x02feb7 thru 0x035f73 (0x60bd bytes) 000583 1787605004.643083 trunc from 0x040000 to 0x02cad3 (0x1352c bytes) 000584 1787605004.663186 write 0x027305 thru 0x032526 (0xb222 bytes) 000588 1787605004.675026 mapwrite 0x03bf50 thru 0x03e16f (0x2220 bytes) 000589 1787605004.801184 mapread 0x0062fd thru 0x00c9e4 (0x66e8 bytes) 000591 1787605004.809997 mapwrite 0x012d83 thru 0x01f7c5 (0xca43 bytes) 000594 1787605004.875301 mapwrite 0x034254 thru 0x038705 (0x44b2 bytes) 000595 1787605004.929621 read 0x013e15 thru 0x013e9a (0x86 bytes) 000596 1787605004.934334 MIRROR_SPLIT mirror 3 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3e170): Mirror 8: 0x5bc8fa9c (stale) Mirror 9: 0x5bc8fa9c (stale) Mirror 12: 0x5bc8fa9c (stale) Mirror 13: 0x5bc8fa9c (stale) Mirror 14: 0x5bc8fa9c (stale) Mirror 15: 0x5bc8fa9c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e170): Mirror 8: 0x5bc8fa9c (stale) Mirror 9: 0x5bc8fa9c (stale) Mirror 12: 0x5bc8fa9c (stale) Mirror 13: 0x5bc8fa9c (stale) Mirror 14: 0x5bc8fa9c (stale) Mirror 15: 0x5bc8fa9c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e170: skipped 000597 1787605005.506352 MIRROR_SPLIT mirror 4 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3e170): Mirror 8: 0x5bc8fa9c (stale) Mirror 9: 0x5bc8fa9c (stale) Mirror 12: 0x5bc8fa9c (stale) Mirror 14: 0x5bc8fa9c (stale) Mirror 15: 0x5bc8fa9c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e170): Mirror 8: 0x5bc8fa9c (stale) Mirror 9: 0x5bc8fa9c (stale) Mirror 12: 0x5bc8fa9c (stale) Mirror 14: 0x5bc8fa9c (stale) Mirror 15: 0x5bc8fa9c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e170: skipped 000598 1787605005.991999 mapwrite 0x01a079 thru 0x022d45 (0x8ccd bytes) 000600 1787605006.065802 read 0x02a589 thru 0x0349d6 (0xa44e bytes) 000605 1787605006.081574 read 0x0317ca thru 0x039a13 (0x824a bytes) 000606 1787605006.098776 read 0x02796e thru 0x036189 (0xe81c bytes) 000607 1787605006.099709 trunc from 0x03e170 to 0x00682b (0x37944 bytes) 000608 1787605006.126191 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x682c): Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 12: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x682c): Mirror 8: 0x899d8daf (stale) Mirror 9: 0x899d8daf (stale) Mirror 12: 0x899d8daf (stale) Mirror 14: 0x899d8daf (stale) Mirror 15: 0x899d8daf (stale) CRC-32 checksum value for chunk [0, 0x682c): Mirror 1: 0x899d8daf Mirror 16: 0x899d8daf lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x682c: skipped 000610 1787605006.985345 mapread 0x002a67 thru 0x003c7d (0x1217 bytes) 000611 1787605007.009376 mapwrite 0x010ccd thru 0x015828 (0x4b5c bytes) 000613 1787605007.141318 mapread 0x000f37 thru 0x0027b2 (0x187c bytes) 000614 1787605007.156885 mapwrite 0x028225 thru 0x03323d (0xb019 bytes) 000618 1787605007.236417 mapread 0x006f59 thru 0x00caf6 (0x5b9e bytes) 000620 1787605007.243443 MIRROR_RESYNC 7 mirrors 000622 1787605010.020797 mapwrite 0x030023 thru 0x03fa45 (0xfa23 bytes) 000624 1787605010.209274 trunc from 0x03fa46 to 0x00f33d (0x30708 bytes) 000625 1787605010.232703 read 0x009d92 thru 0x00f33d (0x55ac bytes) 000626 1787605010.239906 read 0x0075cf thru 0x00ebd1 (0x7603 bytes) 000628 1787605010.246962 mapread 0x00613f thru 0x007b1c (0x19de bytes) 000629 1787605010.251400 MIRROR_RESYNC 7 mirrors 000630 1787605011.740041 MIRROR_SPLIT mirror 1 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000631 1787605012.069998 read 0x007918 thru 0x00a8a7 (0x2f90 bytes) 000632 1787605012.084728 MIRROR_SPLIT mirror 1 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000637 1787605012.369112 mapwrite 0x008956 thru 0x015b9d (0xd248 bytes) 000639 1787605012.614008 read 0x0137e0 thru 0x015b9d (0x23be bytes) 000644 1787605012.621259 read 0x002da0 thru 0x009ca1 (0x6f02 bytes) 000645 1787605012.627280 mapread 0x0026cc thru 0x0066cd (0x4002 bytes) 000647 1787605012.629594 read 0x00e620 thru 0x014524 (0x5f05 bytes) 000648 1787605012.630203 write 0x004919 thru 0x00a526 (0x5c0e bytes) 000650 1787605012.632776 mapwrite 0x038daa thru 0x039aec (0xd43 bytes) 000651 1787605012.711519 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x39aed): Mirror 14: 0xb639699d (stale) Mirror 15: 0xb639699d (stale) Mirror 16: 0xb639699d (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x39aed): Mirror 14: 0xb639699d (stale) Mirror 15: 0xb639699d (stale) Mirror 16: 0xb639699d (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x39aed: skipped 000652 1787605013.120476 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x39aed): Mirror 14: 0xb639699d (stale) Mirror 16: 0xb639699d (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x39aed): Mirror 14: 0xb639699d (stale) Mirror 16: 0xb639699d (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x39aed: skipped 000653 1787605013.476516 write 0x02e769 thru 0x034198 (0x5a30 bytes) 000655 1787605013.494238 read 0x00f350 thru 0x0101fd (0xeae bytes) 000656 1787605013.505026 trunc from 0x039aed to 0x0230d7 (0x16a15 bytes) 000657 1787605013.532509 write 0x03b082 thru 0x03ffff (0x4f7e bytes) 000658 1787605013.534893 read 0x0218fc thru 0x025095 (0x379a bytes) 000659 1787605013.547595 mapwrite 0x01068a thru 0x01dc81 (0xd5f8 bytes) 000663 1787605013.617039 trunc from 0x040000 to 0x003945 (0x3c6ba bytes) 000664 1787605013.633553 mapread 0x003563 thru 0x003945 (0x3e3 bytes) 000665 1787605013.634383 write 0x029fc7 thru 0x039f67 (0xffa1 bytes) 000669 1787605013.640356 mapwrite 0x037fd1 thru 0x03ffff (0x802f bytes) 000670 1787605013.725333 trunc from 0x040000 to 0x010103 (0x2fefc bytes) 000671 1787605013.745220 trunc from 0x010104 to 0x03d973 (0x2d870 bytes) 000673 1787605013.803247 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x3d974): Mirror 14: (stale - read error) Mirror 16: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3d974): Mirror 14: 0xd1917592 (stale) Mirror 16: 0xd1917592 (stale) CRC-32 checksum value for chunk [0, 0x3d974): Mirror 1: 0xbec78e44 Mirror 17: 0xbec78e44 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3d974: skipped 000674 1787605015.000867 read 0x023350 thru 0x027764 (0x4415 bytes) 000676 1787605015.024681 read 0x000a36 thru 0x008cff (0x82ca bytes) 000677 1787605015.048037 trunc from 0x03d974 to 0x010334 (0x2d63f bytes) 000678 1787605015.156148 trunc from 0x010335 to 0x016e70 (0x6b3c bytes) 000679 1787605015.197575 write 0x01423f thru 0x0216a1 (0xd463 bytes) 000680 1787605015.211406 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x216a2): Mirror 16: 0xea3d44 (stale) Mirror 17: 0x7b89f31 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x216a2): Mirror 16: 0xea3d44 (stale) Mirror 17: 0x7b89f31 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x216a2: skipped 000681 1787605015.578060 mapwrite 0x001b1e thru 0x003a1d (0x1f00 bytes) 000684 1787605015.696729 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x216a2): Mirror 17: 0x7b89f31 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x216a2): Mirror 17: 0x7b89f31 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x216a2: skipped 000685 1787605015.970801 mapread 0x01b876 thru 0x0216a1 (0x5e2c bytes) 000686 1787605015.990528 read 0x00823a thru 0x010ead (0x8c74 bytes) 000687 1787605016.001914 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000689 1787605016.210172 trunc from 0x0216a2 to 0x015af0 (0xbbb1 bytes) 000690 1787605016.241570 mapwrite 0x02465d thru 0x028bcd (0x4571 bytes) 000691 1787605016.319715 write 0x03eede thru 0x03ffff (0x1122 bytes) 000693 1787605016.323456 mapwrite 0x023e0a thru 0x02f5d7 (0xb7ce bytes) 000695 1787605016.436773 mapwrite 0x00c264 thru 0x01b7ad (0xf54a bytes) 000697 1787605016.543428 trunc from 0x040000 to 0x00a670 (0x3598f bytes) 000699 1787605016.572941 read 0x0017c5 thru 0x00a670 (0x8eac bytes) 000700 1787605016.579390 read 0x005ea3 thru 0x00a2a1 (0x43ff bytes) 000701 1787605016.582673 read 0x005775 thru 0x006d35 (0x15c1 bytes) 000703 1787605016.584115 mapread 0x00812a thru 0x00a670 (0x2547 bytes) 000704 1787605016.585592 trunc from 0x00a671 to 0x017b3c (0xd4cc bytes) 000707 1787605016.616936 trunc from 0x017b3d to 0x039eac (0x22370 bytes) 000709 1787605016.632972 mapwrite 0x03eea1 thru 0x03ffff (0x115f bytes) 000712 1787605016.760277 mapwrite 0x018a7b thru 0x01b47a (0x2a00 bytes) 000713 1787605016.894716 mapwrite 0x0037ab thru 0x009729 (0x5f7f bytes) 000714 1787605017.007089 read 0x025345 thru 0x0337cb (0xe487 bytes) 000715 1787605017.013959 MIRROR_EXTEND to 2 mirrors 000716 1787605018.325288 write 0x036439 thru 0x03ffff (0x9bc7 bytes) 000720 1787605018.491684 trunc from 0x040000 to 0x03e07a (0x1f85 bytes) 000724 1787605018.529196 write 0x0358fc thru 0x03abb4 (0x52b9 bytes) 000728 1787605018.551038 MIRROR_RESYNC 2 mirrors 000730 1787605019.460834 read 0x03103a thru 0x03d313 (0xc2da bytes) 000731 1787605019.471847 trunc from 0x03e07b to 0x03c01d (0x205d bytes) 000732 1787605019.573945 trunc from 0x03c01e to 0x005f8b (0x36092 bytes) 000733 1787605019.590488 mapread 0x002b6b thru 0x005f8b (0x3421 bytes) 000734 1787605019.626437 read 0x00128d thru 0x005f8b (0x4cff bytes) 000736 1787605019.627494 write 0x02c1d7 thru 0x030ce1 (0x4b0b bytes) 000737 1787605019.638690 MIRROR_RESYNC 2 mirrors 000739 1787605020.514963 trunc from 0x030ce2 to 0x022c43 (0xe09e bytes) 000740 1787605020.621701 mapwrite 0x0349a2 thru 0x03ffff (0xb65e bytes) 000742 1787605020.722966 write 0x02756f thru 0x030e02 (0x9894 bytes) 000744 1787605020.736689 mapread 0x00a333 thru 0x00b368 (0x1036 bytes) 000745 1787605020.752224 read 0x00841b thru 0x00b6af (0x3295 bytes) 000746 1787605020.753236 mapread 0x003631 thru 0x006aed (0x34bd bytes) 000748 1787605020.758669 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000749 1787605020.978233 trunc from 0x040000 to 0x020706 (0x1f8f9 bytes) 000750 1787605021.013392 mapwrite 0x02d028 thru 0x033c64 (0x6c3d bytes) 000752 1787605021.125203 trunc from 0x033c65 to 0x00043b (0x33829 bytes) 000754 1787605021.147082 write 0x0145d2 thru 0x01b1c8 (0x6bf7 bytes) 000757 1787605021.150687 mapwrite 0x0213a1 thru 0x031316 (0xff76 bytes) 000760 1787605021.301058 mapread 0x028541 thru 0x02e8d2 (0x6392 bytes) 000761 1787605021.304258 trunc from 0x031317 to 0x0052fb (0x2c01b bytes) 000766 1787605021.344029 trunc from 0x0052fc to 0x026507 (0x2120c bytes) 000767 1787605021.387480 MIRROR_EXTEND to 2 mirrors 000768 1787605022.046216 write 0x009a7d thru 0x019a34 (0xffb8 bytes) 000769 1787605022.179370 read 0x00ded2 thru 0x01d8b4 (0xf9e3 bytes) 000770 1787605022.186427 mapwrite 0x00bc14 thru 0x00ecdc (0x30c9 bytes) 000774 1787605022.248522 mapwrite 0x034c97 thru 0x03ffff (0xb369 bytes) 000775 1787605022.377477 read 0x008806 thru 0x013bf5 (0xb3f0 bytes) 000776 1787605022.385270 mapread 0x01373e thru 0x01a35a (0x6c1d bytes) 000781 1787605022.397645 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000782 1787605022.526205 trunc from 0x040000 to 0x026b0e (0x194f1 bytes) 000784 1787605022.540531 mapwrite 0x03d4b3 thru 0x03ffff (0x2b4d bytes) 000786 1787605022.652119 trunc from 0x040000 to 0x01a89f (0x25760 bytes) 000788 1787605022.694050 mapwrite 0x007665 thru 0x00d10e (0x5aaa bytes) 000790 1787605022.750465 trunc from 0x01a8a0 to 0x00cc25 (0xdc7a bytes) 000792 1787605022.779092 mapwrite 0x023d86 thru 0x0278e2 (0x3b5d bytes) 000793 1787605022.858516 write 0x0300ae thru 0x03b253 (0xb1a6 bytes) 000794 1787605022.868228 MIRROR_EXTEND to 2 mirrors 000795 1787605023.665665 read 0x01a9c8 thru 0x01b4a2 (0xadb bytes) 000797 1787605023.679096 mapread 0x016594 thru 0x01b904 (0x5371 bytes) 000799 1787605023.708600 write 0x021b27 thru 0x029226 (0x7700 bytes) 000800 1787605023.817640 write 0x010142 thru 0x011292 (0x1151 bytes) 000801 1787605023.832476 read 0x03689b thru 0x03a124 (0x388a bytes) 000806 1787605023.851738 read 0x01dea6 thru 0x02989f (0xb9fa bytes) 000807 1787605023.856726 write 0x0326a9 thru 0x03eb28 (0xc480 bytes) 000808 1787605023.871840 MIRROR_RESYNC 2 mirrors 000812 1787605025.031208 mapread 0x009c7d thru 0x01384e (0x9bd2 bytes) 000814 1787605025.050400 read 0x0372c7 thru 0x03e84e (0x7588 bytes) 000815 1787605025.056738 mapread 0x00627e thru 0x00d239 (0x6fbc bytes) 000816 1787605025.057953 trunc from 0x03eb29 to 0x03bddb (0x2d4d bytes) 000817 1787605025.121898 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000818 1787605025.277658 read 0x014449 thru 0x0213d3 (0xcf8b bytes) 000819 1787605025.288292 read 0x02dc52 thru 0x032e02 (0x51b1 bytes) 000820 1787605025.324492 MIRROR_EXTEND to 2 mirrors 000823 1787605026.910320 trunc from 0x03bddc to 0x0107d5 (0x2b606 bytes) 000826 1787605027.066157 read 0x00e7a1 thru 0x0107d5 (0x2035 bytes) 000827 1787605027.072797 trunc from 0x0107d6 to 0x01f46d (0xec98 bytes) 000828 1787605027.097467 trunc from 0x01f46e to 0x0065ba (0x18eb3 bytes) 000831 1787605027.118813 mapwrite 0x0027d9 thru 0x00d973 (0xb19b bytes) 000833 1787605027.242558 trunc from 0x00d974 to 0x009c42 (0x3d31 bytes) 000834 1787605027.274579 MIRROR_RESYNC 2 mirrors 000835 1787605028.046568 trunc from 0x009c43 to 0x038b75 (0x2ef33 bytes) 000836 1787605028.109525 mapwrite 0x004d2e thru 0x00c0aa (0x737d bytes) 000837 1787605028.192564 mapwrite 0x00812a thru 0x00cae3 (0x49ba bytes) 000842 1787605028.245897 mapwrite 0x0154f7 thru 0x01ae1e (0x5928 bytes) 000843 1787605028.307210 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000844 1787605028.461017 write 0x0152bc thru 0x01cffd (0x7d42 bytes) 000845 1787605028.480248 mapread 0x01a80b thru 0x022851 (0x8047 bytes) 000847 1787605028.509032 read 0x008998 thru 0x00ea23 (0x608c bytes) 000848 1787605028.519504 write 0x023a1f thru 0x02a721 (0x6d03 bytes) 000849 1787605028.526668 mapread 0x02c7dc thru 0x032a75 (0x629a bytes) 000851 1787605028.543846 write 0x029b33 thru 0x02b666 (0x1b34 bytes) 000853 1787605028.545834 trunc from 0x038b76 to 0x0185e0 (0x20595 bytes) 000854 1787605028.580542 MIRROR_EXTEND to 2 mirrors 000855 1787605029.619126 read 0x007c10 thru 0x0163cd (0xe7be bytes) 000858 1787605029.638633 MIRROR_RESYNC 2 mirrors 000860 1787605030.018653 write 0x022641 thru 0x0305b3 (0xdf73 bytes) 000862 1787605030.143448 mapwrite 0x013a51 thru 0x02194c (0xdefc bytes) 000863 1787605030.267716 write 0x02c196 thru 0x02e014 (0x1e7f bytes) 000864 1787605030.269136 mapwrite 0x022ea2 thru 0x026384 (0x34e3 bytes) 000865 1787605030.428265 mapwrite 0x035d90 thru 0x036b40 (0xdb1 bytes) 000866 1787605030.620276 mapwrite 0x010d46 thru 0x019e44 (0x90ff bytes) 000868 1787605030.768327 write 0x030040 thru 0x033c1f (0x3be0 bytes) 000869 1787605030.769310 read 0x004d41 thru 0x00f226 (0xa4e6 bytes) 000871 1787605030.810143 write 0x00f12c thru 0x016fc3 (0x7e98 bytes) 000872 1787605030.811207 trunc from 0x036b41 to 0x00b0f0 (0x2ba50 bytes) 000873 1787605030.846529 read 0x009d20 thru 0x00a812 (0xaf3 bytes) 000874 1787605030.847516 mapread 0x003943 thru 0x008609 (0x4cc7 bytes) 000875 1787605030.877253 trunc from 0x00b0f1 to 0x01ba5a (0x1096a bytes) 000876 1787605030.907198 write 0x000ee6 thru 0x00e432 (0xd54d bytes) 000877 1787605030.943716 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x1ba5b): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1ba5b): Mirror 2: 0x74ebe320 (stale) CRC-32 checksum value for chunk [0, 0x1ba5b): Mirror 1: 0xe3f77f6d Mirror 3: 0xe3f77f6d lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1ba5b: skipped 000878 1787605032.169546 trunc from 0x01ba5b to 0x02c206 (0x107ac bytes) 000879 1787605032.256652 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2c207): Mirror 3: 0xe441c7da (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2c207): Mirror 3: 0xe441c7da (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2c207: skipped 000880 1787605032.482976 read 0x005942 thru 0x013578 (0xdc37 bytes) 000881 1787605032.490296 mapread 0x003903 thru 0x00a499 (0x6b97 bytes) 000885 1787605032.505026 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x2c207): Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2c207): Mirror 3: 0xe441c7da (stale) CRC-32 checksum value for chunk [0, 0x2c207): Mirror 1: 0xe441c7da Mirror 4: 0xe441c7da lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2c207: skipped 000886 1787605033.257708 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000887 1787605033.611224 mapread 0x00cf4c thru 0x01cbda (0xfc8f bytes) 000888 1787605033.646224 read 0x01ef77 thru 0x02c206 (0xd290 bytes) 000890 1787605033.663602 read 0x019f38 thru 0x0286cd (0xe796 bytes) 000893 1787605033.666907 mapwrite 0x022078 thru 0x02cd8f (0xad18 bytes) 000894 1787605033.932238 trunc from 0x02cd90 to 0x009c19 (0x23176 bytes) 000896 1787605033.958328 mapwrite 0x00a5bc thru 0x013d8a (0x97cf bytes) 000897 1787605034.126666 write 0x0057b4 thru 0x011f6c (0xc7b9 bytes) 000900 1787605034.135450 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000901 1787605034.297961 write 0x037a3a thru 0x03a590 (0x2b57 bytes) 000903 1787605034.299509 write 0x016113 thru 0x0196ed (0x35db bytes) 000904 1787605034.310966 read 0x037d3e thru 0x03a590 (0x2853 bytes) 000906 1787605034.312317 trunc from 0x03a591 to 0x02f873 (0xad1d bytes) 000907 1787605034.326648 write 0x0354e9 thru 0x03ffff (0xab17 bytes) 000909 1787605034.329981 trunc from 0x040000 to 0x033e1e (0xc1e1 bytes) 000912 1787605034.353094 mapwrite 0x0280bb thru 0x02d9f0 (0x5936 bytes) 000913 1787605034.414081 read 0x0228bc thru 0x022bb4 (0x2f9 bytes) 000915 1787605034.432686 write 0x021e38 thru 0x0239d5 (0x1b9e bytes) 000918 1787605034.439666 read 0x0089b8 thru 0x00ea8c (0x60d5 bytes) 000919 1787605034.450916 MIRROR_EXTEND to 2 mirrors 000923 1787605035.908170 mapwrite 0x00fc17 thru 0x01c206 (0xc5f0 bytes) 000924 1787605036.060914 MIRROR_RESYNC 2 mirrors 000925 1787605036.928644 mapread 0x000e5e thru 0x009334 (0x84d7 bytes) 000926 1787605036.957589 mapwrite 0x02219e thru 0x0241c4 (0x2027 bytes) 000927 1787605037.166313 MIRROR_RESYNC 2 mirrors 000930 1787605038.092025 trunc from 0x033e1f to 0x03105b (0x2dc3 bytes) 000931 1787605038.181738 read 0x019d0d thru 0x01d3a8 (0x369c bytes) 000932 1787605038.190711 read 0x02656c thru 0x02b41d (0x4eb2 bytes) 000937 1787605038.204836 mapwrite 0x000dd0 thru 0x001dd4 (0x1005 bytes) 000938 1787605038.242609 read 0x02004e thru 0x02f4d7 (0xf48a bytes) 000940 1787605038.244485 write 0x026934 thru 0x02c1c1 (0x588e bytes) 000941 1787605038.246018 trunc from 0x03105c to 0x017652 (0x19a09 bytes) 000946 1787605038.270584 trunc from 0x017653 to 0x01627e (0x13d4 bytes) 000947 1787605038.287944 trunc from 0x01627f to 0x039357 (0x230d9 bytes) 000948 1787605038.302454 write 0x037a51 thru 0x03ffff (0x85af bytes) 000950 1787605038.312909 trunc from 0x040000 to 0x026d31 (0x192ce bytes) 000951 1787605038.331537 mapwrite 0x00c5a0 thru 0x00f904 (0x3365 bytes) 000952 1787605038.369055 mapwrite 0x030090 thru 0x037ed3 (0x7e44 bytes) 000953 1787605038.463411 write 0x039608 thru 0x03ffff (0x69f8 bytes) 000954 1787605038.466174 read 0x018ee4 thru 0x01cea7 (0x3fc4 bytes) 000955 1787605038.467550 mapread 0x026972 thru 0x03299a (0xc029 bytes) 000956 1787605038.477934 mapwrite 0x019828 thru 0x025d8e (0xc567 bytes) 000962 1787605038.539958 mapwrite 0x01009c thru 0x017e01 (0x7d66 bytes) 000963 1787605038.610234 read 0x00bb54 thru 0x00f50b (0x39b8 bytes) 000964 1787605038.611498 trunc from 0x040000 to 0x014c84 (0x2b37b bytes) 000965 1787605038.636439 trunc from 0x014c85 to 0x036599 (0x21915 bytes) 000966 1787605038.652502 mapread 0x02ae13 thru 0x036599 (0xb787 bytes) 000968 1787605038.671320 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3659a): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3659a): Mirror 2: 0x591c5fd5 (stale) CRC-32 checksum value for chunk [0, 0x3659a): Mirror 1: 0x1bd1fefd Mirror 3: 0x1bd1fefd lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3659a: skipped 000969 1787605040.129479 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x3659a): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3659a): Mirror 2: 0x591c5fd5 (stale) CRC-32 checksum value for chunk [0, 0x3659a): Mirror 1: 0x1bd1fefd Mirror 3: 0x1bd1fefd Mirror 4: 0x1bd1fefd lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3659a: skipped 000970 1787605041.005669 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3659a): Mirror 2: 0x591c5fd5 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3659a): Mirror 2: 0x591c5fd5 (stale) CRC-32 checksum value for chunk [0, 0x3659a): Mirror 1: 0x1bd1fefd Mirror 3: 0x1bd1fefd lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3659a: skipped 000972 1787605041.682857 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1cc:0x0]: Invalid argument (22) 000974 1787605041.977930 write 0x02a21c thru 0x037868 (0xd64d bytes) 000976 1787605042.061978 mapread 0x011eaa thru 0x01bf49 (0xa0a0 bytes) 000978 1787605042.080411 mapwrite 0x03e794 thru 0x03ffff (0x186c bytes) 000979 1787605042.153441 trunc from 0x040000 to 0x00b6e5 (0x3491a bytes) 000981 1787605042.174274 write 0x01c0a4 thru 0x01c2e1 (0x23e bytes) 000982 1787605042.175190 trunc from 0x01c2e2 to 0x039812 (0x1d531 bytes) 000983 1787605042.210855 mapwrite 0x020cca thru 0x026214 (0x554b bytes) 000985 1787605042.274178 read 0x00b5e0 thru 0x01579c (0xa1bd bytes) 000986 1787605042.292293 write 0x0367be thru 0x03e904 (0x8147 bytes) 000987 1787605042.299571 read 0x0103a6 thru 0x01d7ae (0xd409 bytes) 000989 1787605042.316835 read 0x01ce15 thru 0x020764 (0x3950 bytes) 000991 1787605042.324989 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3e905): Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e905): Mirror 3: 0xc60c4509 (stale) CRC-32 checksum value for chunk [0, 0x3e905): Mirror 1: 0x72e13360 Mirror 4: 0x72e13360 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e905: skipped 000994 1787605043.481225 read 0x01740b thru 0x018082 (0xc78 bytes) 000997 1787605043.487403 trunc from 0x03e905 to 0x036ff8 (0x790c bytes) 000999 1787605043.564108 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x36ff9): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x36ff9): Mirror 3: 0x6958f401 (stale) Mirror 4: 0x5d9cf700 (stale) CRC-32 checksum value for chunk [0, 0x36ff9): Mirror 1: 0x5d9cf700 Mirror 5: 0x5d9cf700 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x36ff9: skipped 001000 1787605044.593942 read 0x028c2a thru 0x036cba (0xe091 bytes) All operations completed A-OK!