== sanity-flr test 70a: flr mode fsx test ================ 10:55:37 (1786114537) Seed set to 4537 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read skipping zero size read 000011 1786114538.080294 MIRROR_EXTEND to 2 mirrors skipping zero size read truncating to largest ever: 0x24c99 000013 1786114538.337233 trunc from 00000000 to 0x024c98 (0x24c99 bytes) 000014 1786114538.378695 write 0x03bc19 thru 0x03ffff (0x43e7 bytes) truncating to largest ever: 0x258e2 000017 1786114538.382238 trunc from 0x040000 to 0x0258e1 (0x1a71e bytes) 000018 1786114538.398437 mapwrite 0x032c3b thru 0x03ffff (0xd3c5 bytes) 000019 1786114538.474676 write 0x03ccb6 thru 0x03ffff (0x334a bytes) truncating to largest ever: 0x2edc8 000020 1786114538.476481 trunc from 0x040000 to 0x02edc7 (0x11238 bytes) 000022 1786114538.491912 trunc from 0x02edc8 to 0x003991 (0x2b436 bytes) 000023 1786114538.512658 mapread 0x000645 thru 0x003991 (0x334d bytes) 000024 1786114538.522892 mapwrite 0x0122e0 thru 0x0125f8 (0x319 bytes) 000025 1786114538.595163 read 0x009668 thru 0x0125f8 (0x8f91 bytes) 000028 1786114538.603301 read 0x0083e9 thru 0x0125f8 (0xa210 bytes) 000029 1786114538.619261 mapread 0x0120e4 thru 0x0125f8 (0x515 bytes) 000030 1786114538.620304 read 0x003585 thru 0x012571 (0xefed bytes) 000031 1786114538.620907 read 0x00f774 thru 0x0125f8 (0x2e85 bytes) 000032 1786114538.622018 mapread 0x001e04 thru 0x00778b (0x5988 bytes) 000034 1786114538.624473 mapwrite 0x00b692 thru 0x016c9e (0xb60d bytes) 000035 1786114538.679663 read 0x001352 thru 0x00ca9c (0xb74b bytes) truncating to largest ever: 0x397b7 000037 1786114538.680894 trunc from 0x016c9f to 0x0397b6 (0x22b18 bytes) 000038 1786114538.690843 read 0x00c2b1 thru 0x01679f (0xa4ef bytes) 000039 1786114538.691395 trunc from 0x0397b7 to 0x0178db (0x21edb bytes) 000040 1786114538.703653 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000042 1786114538.787666 MIRROR_EXTEND to 2 mirrors 000043 1786114539.157192 read 0x011a10 thru 0x0123ee (0x9df bytes) 000044 1786114539.166801 read 0x005d52 thru 0x009bb0 (0x3e5f bytes) 000046 1786114539.183855 MIRROR_EXTEND to 3 mirrors 000047 1786114539.647276 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000049 1786114539.829428 mapwrite 0x0066c4 thru 0x01335a (0xcc97 bytes) 000051 1786114539.974222 mapread 0x00df05 thru 0x0178db (0x99d7 bytes) 000052 1786114539.975927 mapread 0x0060a2 thru 0x007377 (0x12d6 bytes) 000054 1786114539.978580 mapread 0x00569b thru 0x007912 (0x2278 bytes) 000057 1786114539.981466 mapwrite 0x03c3b1 thru 0x03f250 (0x2ea0 bytes) 000058 1786114540.089712 MIRROR_RESYNC 2 mirrors 000059 1786114540.435796 read 0x038ec5 thru 0x03ef2a (0x6066 bytes) 000061 1786114540.446485 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000062 1786114540.536111 mapwrite 0x0091f3 thru 0x011c58 (0x8a66 bytes) 000063 1786114540.610156 trunc from 0x03f251 to 0x02f1fb (0x10055 bytes) 000068 1786114540.635759 mapread 0x00cc5e thru 0x010521 (0x38c4 bytes) 000069 1786114540.636874 trunc from 0x02f1fc to 0x017ae5 (0x17716 bytes) 000070 1786114540.650639 trunc from 0x017ae6 to 0x001831 (0x162b4 bytes) 000071 1786114540.663808 trunc from 0x001832 to 0x02b110 (0x298df bytes) 000072 1786114540.676805 trunc from 0x02b111 to 0x02b9b6 (0x8a6 bytes) 000074 1786114540.689518 mapread 0x01be28 thru 0x020205 (0x43de bytes) 000076 1786114540.697304 mapread 0x0215f6 thru 0x02b9b6 (0xa3c1 bytes) 000078 1786114540.703206 mapwrite 0x003ba0 thru 0x006483 (0x28e4 bytes) 000079 1786114540.728158 mapread 0x0108f2 thru 0x01fa82 (0xf191 bytes) 000083 1786114540.733644 trunc from 0x02b9b7 to 0x0037a2 (0x28214 bytes) 000084 1786114540.748263 read 0x0004ca thru 0x0037a2 (0x32d9 bytes) 000086 1786114540.755380 read 0x002352 thru 0x0037a2 (0x1451 bytes) 000087 1786114540.756050 mapwrite 0x02bc0f thru 0x0342e3 (0x86d5 bytes) 000089 1786114540.810387 mapwrite 0x033e7d thru 0x03ffff (0xc183 bytes) 000090 1786114540.873282 mapwrite 0x03da0d thru 0x03ffff (0x25f3 bytes) 000091 1786114540.902598 mapread 0x029698 thru 0x037bc4 (0xe52d bytes) 000092 1786114540.904017 read 0x018e64 thru 0x02672f (0xd8cc bytes) 000094 1786114540.905557 trunc from 0x040000 to 0x0268c5 (0x1973a bytes) 000095 1786114540.916546 mapread 0x009efd thru 0x019219 (0xf31d bytes) 000096 1786114540.918137 trunc from 0x0268c6 to 0x0238ff (0x2fc6 bytes) 000097 1786114540.930674 mapread 0x00473b thru 0x01179f (0xd065 bytes) 000098 1786114540.932676 write 0x00d311 thru 0x00e708 (0x13f8 bytes) 000099 1786114540.936642 mapread 0x0012cb thru 0x01029b (0xefd1 bytes) 000102 1786114540.940725 mapread 0x021e2c thru 0x0238ff (0x1ad4 bytes) 000103 1786114540.941916 mapread 0x020ec0 thru 0x0238ff (0x2a40 bytes) 000104 1786114540.942931 write 0x03b5ac thru 0x03d19c (0x1bf1 bytes) 000107 1786114540.948589 MIRROR_EXTEND to 2 mirrors 000108 1786114541.355604 trunc from 0x03d19d to 0x023d03 (0x19499 bytes) 000109 1786114541.397493 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x23d04): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x23d04): Mirror 2: 0x507b2435 (stale) CRC-32 checksum value for chunk [0, 0x23d04): Mirror 1: 0x507b2435 Mirror 3: 0x507b2435 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x23d04: skipped 000110 1786114541.887844 mapwrite 0x03b628 thru 0x03ffff (0x49d8 bytes) 000111 1786114542.051036 read 0x0218ce thru 0x029345 (0x7a78 bytes) 000112 1786114542.057504 read 0x01a9e3 thru 0x020e21 (0x643f bytes) 000114 1786114542.068752 read 0x01e50f thru 0x029070 (0xab62 bytes) 000117 1786114542.073031 write 0x014cbb thru 0x020785 (0xbacb bytes) 000120 1786114542.077514 read 0x026719 thru 0x035362 (0xec4a bytes) 000121 1786114542.078496 read 0x00a6af thru 0x00e5f6 (0x3f48 bytes) 000122 1786114542.082737 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xdc602441 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xdc602441 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000127 1786114542.295249 read 0x02c877 thru 0x038e60 (0xc5ea bytes) 000128 1786114542.308580 write 0x0363bf thru 0x03b69c (0x52de bytes) 000129 1786114542.313790 mapread 0x00fb97 thru 0x0149a3 (0x4e0d bytes) 000130 1786114542.336551 trunc from 0x040000 to 0x026b08 (0x194f7 bytes) 000134 1786114542.354795 mapread 0x003f3b thru 0x00ff71 (0xc037 bytes) 000136 1786114542.364902 mapwrite 0x002f8b thru 0x009e42 (0x6eb8 bytes) 000138 1786114542.413824 mapwrite 0x020717 thru 0x0288c6 (0x81b0 bytes) 000139 1786114542.493471 mapwrite 0x0268e0 thru 0x031afe (0xb21f bytes) 000140 1786114542.560283 trunc from 0x031aff to 0x014948 (0x1d1b6 bytes) 000144 1786114542.571516 write 0x015c6f thru 0x015fbb (0x34d bytes) truncating to largest ever: 0x3f768 000145 1786114542.572959 trunc from 0x015fbc to 0x03f767 (0x297ac bytes) 000146 1786114542.589428 mapread 0x00e979 thru 0x01d5b5 (0xec3d bytes) 000148 1786114542.598633 trunc from 0x03f768 to 0x00af8d (0x347da bytes) 000150 1786114542.617152 mapread 0x009c28 thru 0x00af8d (0x1366 bytes) 000152 1786114542.619728 mapwrite 0x03187a thru 0x0324d9 (0xc60 bytes) 000153 1786114542.674673 mapread 0x01be37 thru 0x026c12 (0xaddc bytes) 000154 1786114542.687394 mapread 0x019604 thru 0x01f579 (0x5f76 bytes) 000155 1786114542.693635 MIRROR_RESYNC 2 mirrors 000157 1786114543.075368 MIRROR_RESYNC 2 mirrors 000158 1786114543.279737 mapwrite 0x02dbd8 thru 0x03c733 (0xeb5c bytes) 000160 1786114543.410611 mapwrite 0x01cb48 thru 0x023735 (0x6bee bytes) 000162 1786114543.463102 read 0x021590 thru 0x02653f (0x4fb0 bytes) 000163 1786114543.468531 mapread 0x01dc89 thru 0x01fedf (0x2257 bytes) 000165 1786114543.470368 mapread 0x03a038 thru 0x03c733 (0x26fc bytes) 000166 1786114543.471590 write 0x01e1f5 thru 0x02b3c4 (0xd1d0 bytes) 000168 1786114543.479931 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3c734): Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3c734): Mirror 3: 0x921ee7f5 (stale) CRC-32 checksum value for chunk [0, 0x3c734): Mirror 1: 0xb96edea1 Mirror 4: 0xb96edea1 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3c734: skipped 000171 1786114544.061676 trunc from 0x03c734 to 0x020e2c (0x1b907 bytes) 000172 1786114544.132135 mapwrite 0x009e7e thru 0x00c2d3 (0x2456 bytes) 000173 1786114544.183891 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x20e2d): Mirror 4: 0xdf02c060 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x20e2d): Mirror 4: 0xdf02c060 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x20e2d: skipped 000177 1786114544.420651 mapwrite 0x036685 thru 0x03ffff (0x997b bytes) 000178 1786114544.492991 mapread 0x034e93 thru 0x0385e3 (0x3751 bytes) 000179 1786114544.494054 trunc from 0x040000 to 0x01828a (0x27d75 bytes) 000180 1786114544.503551 read 0x00430a thru 0x009d09 (0x5a00 bytes) 000181 1786114544.518667 write 0x00abda thru 0x019f5b (0xf382 bytes) 000182 1786114544.526912 mapwrite 0x039990 thru 0x03c89f (0x2f10 bytes) 000183 1786114544.598569 write 0x01714c thru 0x01c7ef (0x56a4 bytes) 000184 1786114544.602104 mapread 0x0199ab thru 0x028f70 (0xf5c6 bytes) 000185 1786114544.616675 read 0x0049f8 thru 0x011436 (0xca3f bytes) 000187 1786114544.617549 write 0x0213e6 thru 0x02c9c3 (0xb5de bytes) 000188 1786114544.618500 mapread 0x0055e4 thru 0x01307e (0xda9b bytes) 000189 1786114544.620564 trunc from 0x03c8a0 to 0x0011cd (0x3b6d2 bytes) 000190 1786114544.632449 mapwrite 0x00983b thru 0x00d37f (0x3b45 bytes) 000191 1786114544.689272 mapwrite 0x024bd4 thru 0x02775f (0x2b8c bytes) 000192 1786114544.742485 write 0x007cb7 thru 0x010f99 (0x92e3 bytes) 000193 1786114544.743748 trunc from 0x027760 to 0x001bbc (0x25ba3 bytes) 000195 1786114544.775999 MIRROR_RESYNC 2 mirrors 000200 1786114545.068856 read 0x0016d6 thru 0x001bbc (0x4e7 bytes) 000201 1786114545.071556 write 0x015233 thru 0x019931 (0x46ff bytes) 000203 1786114545.100535 mapread 0x015866 thru 0x019931 (0x40cc bytes) 000207 1786114545.103151 mapread 0x018b47 thru 0x019931 (0xdeb bytes) 000208 1786114545.104236 trunc from 0x019932 to 0x01d09e (0x376d bytes) 000211 1786114545.138576 write 0x0163b8 thru 0x01ec76 (0x88bf bytes) 000213 1786114545.143495 mapwrite 0x00a064 thru 0x014f6a (0xaf07 bytes) 000214 1786114545.214970 trunc from 0x01ec77 to 0x01dee8 (0xd8e bytes) 000216 1786114545.233967 write 0x02e277 thru 0x02e6e6 (0x470 bytes) truncating to largest ever: 0x3fea8 000218 1786114545.236000 trunc from 0x02e6e7 to 0x03fea7 (0x117c1 bytes) 000219 1786114545.253953 mapwrite 0x028095 thru 0x037564 (0xf4d0 bytes) 000220 1786114545.308988 read 0x00df96 thru 0x012616 (0x4681 bytes) 000221 1786114545.309826 read 0x001709 thru 0x003205 (0x1afd bytes) 000222 1786114545.323635 mapwrite 0x03a8df thru 0x03c6d1 (0x1df3 bytes) 000226 1786114545.361697 read 0x017e14 thru 0x01d648 (0x5835 bytes) 000227 1786114545.368011 mapwrite 0x034c54 thru 0x03772f (0x2adc bytes) 000229 1786114545.395681 trunc from 0x03fea8 to 0x000560 (0x3f947 bytes) 000231 1786114545.416491 mapwrite 0x0325bf thru 0x03b225 (0x8c67 bytes) 000233 1786114545.483531 mapwrite 0x0296c7 thru 0x02fbc2 (0x64fc bytes) 000234 1786114545.517337 trunc from 0x03b226 to 0x022326 (0x18eff bytes) 000237 1786114545.527305 trunc from 0x022327 to 0x01b846 (0x6ae0 bytes) 000240 1786114545.538668 write 0x00ba87 thru 0x01abda (0xf154 bytes) 000241 1786114545.539926 write 0x01085c thru 0x01e92d (0xe0d2 bytes) 000243 1786114545.541950 read 0x00a040 thru 0x0115b6 (0x7577 bytes) 000246 1786114545.544990 read 0x011974 thru 0x01e92d (0xcfba bytes) 000250 1786114545.557351 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000254 1786114545.668027 read 0x008b37 thru 0x00fdfd (0x72c7 bytes) 000255 1786114545.678684 trunc from 0x01e92e to 0x014a62 (0x9ecb bytes) 000259 1786114545.692010 mapread 0x0035c3 thru 0x00a051 (0x6a8f bytes) 000262 1786114545.702388 write 0x022de1 thru 0x024243 (0x1463 bytes) 000263 1786114545.703710 write 0x0083f6 thru 0x012f16 (0xab21 bytes) 000265 1786114545.709125 read 0x021dba thru 0x024243 (0x248a bytes) 000266 1786114545.714651 mapwrite 0x02f721 thru 0x03eb79 (0xf459 bytes) 000268 1786114545.773309 trunc from 0x03eb7a to 0x02ac27 (0x13f52 bytes) 000270 1786114545.786813 mapread 0x010f23 thru 0x01b103 (0xa1e1 bytes) 000274 1786114545.790133 mapread 0x01bfeb thru 0x02ac27 (0xec3d bytes) 000275 1786114545.791530 trunc from 0x02ac28 to 0x017d2a (0x12efd bytes) 000276 1786114545.801838 mapread 0x003843 thru 0x00e1af (0xa96d bytes) 000277 1786114545.802889 trunc from 0x017d2b to 0x03f603 (0x278d9 bytes) 000280 1786114545.809630 trunc from 0x03f604 to 0x0028cb (0x3cd38 bytes) 000281 1786114545.815359 write 0x01a655 thru 0x01d75b (0x3107 bytes) 000282 1786114545.816363 write 0x03ac35 thru 0x03ffff (0x53cb bytes) 000283 1786114545.817213 trunc from 0x040000 to 0x036d6b (0x9294 bytes) 000286 1786114545.825395 mapwrite 0x02c2d1 thru 0x038116 (0xbe46 bytes) 000287 1786114545.948218 mapwrite 0x01d696 thru 0x01d938 (0x2a3 bytes) 000289 1786114545.989881 trunc from 0x038117 to 0x011549 (0x26bcd bytes) 000291 1786114546.003658 trunc from 0x01154a to 0x031f59 (0x20a10 bytes) 000295 1786114546.017457 trunc from 0x031f5a to 0x0015d4 (0x30985 bytes) 000297 1786114546.033089 read 0x000ae5 thru 0x0015d4 (0xaf0 bytes) 000298 1786114546.033767 mapread 0x000d0e thru 0x0015d4 (0x8c7 bytes) 000300 1786114546.036648 read 0x0013b3 thru 0x0015d4 (0x222 bytes) 000301 1786114546.037809 mapread 0x00013f thru 0x0015d4 (0x1496 bytes) 000305 1786114546.042674 mapread 0x000e55 thru 0x0015d4 (0x780 bytes) 000307 1786114546.045229 mapwrite 0x028e9a thru 0x02ffed (0x7154 bytes) 000309 1786114546.103480 mapwrite 0x01ef99 thru 0x02e84a (0xf8b2 bytes) 000310 1786114546.141310 mapwrite 0x01eb73 thru 0x0257d5 (0x6c63 bytes) 000311 1786114546.182304 read 0x027ab9 thru 0x02ffed (0x8535 bytes) 000312 1786114546.182837 write 0x019bea thru 0x028d57 (0xf16e bytes) 000313 1786114546.183835 mapwrite 0x0203c2 thru 0x023cf7 (0x3936 bytes) 000314 1786114546.217960 trunc from 0x02ffee to 0x0387da (0x87ed bytes) 000316 1786114546.232361 read 0x01928b thru 0x01bd0d (0x2a83 bytes) 000320 1786114546.235734 mapread 0x0370c7 thru 0x0387da (0x1714 bytes) 000321 1786114546.245318 trunc from 0x0387db to 0x019506 (0x1f2d4 bytes) 000324 1786114546.268580 write 0x0111f3 thru 0x020a66 (0xf874 bytes) 000326 1786114546.273364 trunc from 0x020a67 to 0x022d7e (0x2318 bytes) 000335 1786114546.288988 MIRROR_EXTEND to 2 mirrors 000336 1786114546.734752 trunc from 0x022d7f to 0x024d7e (0x2000 bytes) 000340 1786114546.771381 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x24d7f): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x24d7f): Mirror 2: 0x7b25eb11 (stale) CRC-32 checksum value for chunk [0, 0x24d7f): Mirror 1: 0x7b25eb11 Mirror 3: 0x7b25eb11 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x24d7f: skipped 000341 1786114547.240566 read 0x01bd10 thru 0x024aa3 (0x8d94 bytes) 000343 1786114547.246718 mapwrite 0x02bd64 thru 0x02c3da (0x677 bytes) 000345 1786114547.345929 write 0x0051b8 thru 0x009d95 (0x4bde bytes) 000346 1786114547.351246 trunc from 0x02c3db to 0x01d048 (0xf392 bytes) 000350 1786114547.369342 write 0x001849 thru 0x00b0a3 (0x985b bytes) 000351 1786114547.376136 write 0x00a797 thru 0x00d139 (0x29a3 bytes) 000353 1786114547.381023 read 0x01133a thru 0x01d048 (0xbd0f bytes) 000355 1786114547.386028 trunc from 0x01d049 to 0x0081d6 (0x14e72 bytes) 000356 1786114547.395719 trunc from 0x0081d7 to 0x03cf7a (0x34da4 bytes) 000357 1786114547.408691 read 0x034938 thru 0x038b19 (0x41e2 bytes) 000358 1786114547.422388 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3cf7b): Mirror 3: 0x86addb1 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3cf7b): Mirror 3: 0x86addb1 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3cf7b: skipped 000359 1786114547.609937 mapread 0x025322 thru 0x025fac (0xc8b bytes) 000360 1786114547.618375 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000361 1786114547.712603 MIRROR_EXTEND to 2 mirrors 000365 1786114548.125326 write 0x02e078 thru 0x033258 (0x51e1 bytes) 000368 1786114548.200652 mapread 0x00ea68 thru 0x01d182 (0xe71b bytes) 000372 1786114548.239789 read 0x01050b thru 0x01b2bb (0xadb1 bytes) 000376 1786114548.241733 mapread 0x01364a thru 0x01b36c (0x7d23 bytes) 000379 1786114548.243497 trunc from 0x03cf7b to 0x0088ef (0x3468b bytes) 000380 1786114548.268396 mapwrite 0x02279e thru 0x02e687 (0xbeea bytes) 000381 1786114548.343908 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000383 1786114548.471540 read 0x00fbf5 thru 0x01a843 (0xac4f bytes) 000384 1786114548.481401 trunc from 0x02e688 to 0x0155b0 (0x190d7 bytes) 000386 1786114548.501095 trunc from 0x0155b1 to 0x00dfd7 (0x75d9 bytes) 000387 1786114548.520120 write 0x03b6ac thru 0x03ffff (0x4954 bytes) 000388 1786114548.522425 read 0x02a372 thru 0x03877e (0xe40d bytes) 000392 1786114548.536962 write 0x027df5 thru 0x028856 (0xa62 bytes) 000394 1786114548.538138 trunc from 0x040000 to 0x01cbcf (0x23430 bytes) 000396 1786114548.561038 trunc from 0x01cbd0 to 0x023565 (0x6996 bytes) 000397 1786114548.577706 mapread 0x00d610 thru 0x012373 (0x4d64 bytes) 000399 1786114548.580329 mapwrite 0x0135f4 thru 0x015236 (0x1c43 bytes) 000400 1786114548.619706 MIRROR_EXTEND to 2 mirrors 000401 1786114549.002533 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000404 1786114549.123923 read 0x0159a1 thru 0x01f9c4 (0xa024 bytes) 000407 1786114549.135531 mapwrite 0x02939d thru 0x02bdd2 (0x2a36 bytes) 000409 1786114549.269912 mapread 0x027026 thru 0x02bdd2 (0x4dad bytes) 000410 1786114549.272243 mapwrite 0x030218 thru 0x035e27 (0x5c10 bytes) 000411 1786114549.328611 mapwrite 0x000b85 thru 0x006b78 (0x5ff4 bytes) 000412 1786114549.375575 mapread 0x0343da thru 0x035e27 (0x1a4e bytes) 000413 1786114549.376873 trunc from 0x035e28 to 0x00dc93 (0x28194 bytes) 000414 1786114549.391872 mapread 0x005717 thru 0x00dc93 (0x857d bytes) 000418 1786114549.397806 trunc from 0x00dc94 to 0x027085 (0x193f2 bytes) 000420 1786114549.414505 trunc from 0x027086 to 0x02ccb7 (0x5c32 bytes) 000421 1786114549.433607 mapread 0x01bc7b thru 0x02356b (0x78f1 bytes) 000423 1786114549.443692 read 0x026f40 thru 0x02a295 (0x3356 bytes) 000426 1786114549.452531 write 0x017790 thru 0x024695 (0xcf06 bytes) 000427 1786114549.454997 write 0x007cef thru 0x01172c (0x9a3e bytes) 000428 1786114549.456828 mapwrite 0x03258d thru 0x033734 (0x11a8 bytes) 000429 1786114549.520983 trunc from 0x033735 to 0x003855 (0x2fedf bytes) 000431 1786114549.536946 write 0x02ef7b thru 0x039aa1 (0xab27 bytes) 000433 1786114549.541484 write 0x02fad4 thru 0x039278 (0x97a5 bytes) 000434 1786114549.551780 MIRROR_EXTEND to 2 mirrors 000435 1786114549.952716 read 0x0069cf thru 0x016843 (0xfe75 bytes) 000436 1786114549.966398 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000439 1786114550.074510 mapread 0x03198b thru 0x03897a (0x6ff0 bytes) 000441 1786114550.097254 mapwrite 0x006cb0 thru 0x008a56 (0x1da7 bytes) 000442 1786114550.160979 read 0x009c45 thru 0x015722 (0xbade bytes) 000444 1786114550.169186 mapread 0x0024d9 thru 0x009734 (0x725c bytes) 000445 1786114550.170042 mapwrite 0x0139a2 thru 0x01b419 (0x7a78 bytes) 000446 1786114550.229951 trunc from 0x039aa2 to 0x023e96 (0x15c0b bytes) 000447 1786114550.260454 mapread 0x008d3f thru 0x0111b4 (0x8476 bytes) 000448 1786114550.261339 write 0x013ef7 thru 0x020e97 (0xcfa1 bytes) 000450 1786114550.273771 mapread 0x0121a6 thru 0x01583b (0x3696 bytes) 000451 1786114550.274536 read 0x01c3fd thru 0x023e96 (0x7a9a bytes) 000453 1786114550.287234 MIRROR_EXTEND to 2 mirrors 000454 1786114550.627817 mapread 0x0116e9 thru 0x012b36 (0x144e bytes) 000455 1786114550.651193 read 0x0043a8 thru 0x0120b5 (0xdd0e bytes) 000456 1786114550.661671 trunc from 0x023e97 to 0x00175d (0x22739 bytes) 000458 1786114550.741114 read 0x000b7c thru 0x00175d (0xbe2 bytes) 000460 1786114550.756168 mapread 0x0013f9 thru 0x00175d (0x365 bytes) 000461 1786114550.757264 read 0x0013e6 thru 0x00175d (0x378 bytes) 000462 1786114550.758594 write 0x0025ce thru 0x00b989 (0x93bc bytes) 000465 1786114550.761453 mapread 0x007990 thru 0x00b989 (0x3ffa bytes) 000467 1786114550.763258 write 0x03c449 thru 0x03ffff (0x3bb7 bytes) 000468 1786114550.764456 read 0x00b125 thru 0x012db3 (0x7c8f bytes) 000469 1786114550.777706 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xb76d2e8e (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xea15a47b Mirror 3: 0xea15a47b lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000471 1786114551.359426 mapread 0x03775d thru 0x03ffff (0x88a3 bytes) 000472 1786114551.378050 MIRROR_RESYNC 3 mirrors 000473 1786114551.855460 mapread 0x03dee5 thru 0x03ffff (0x211b bytes) 000474 1786114551.865340 mapwrite 0x02eef9 thru 0x03e435 (0xf53d bytes) 000475 1786114551.989414 MIRROR_RESYNC 3 mirrors 000476 1786114552.463475 mapread 0x014d15 thru 0x01892a (0x3c16 bytes) 000477 1786114552.486879 write 0x022033 thru 0x02a811 (0x87df bytes) 000478 1786114552.555418 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xd26ce70c (stale) Mirror 3: 0xd26ce70c (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x872d3cd Mirror 4: 0x872d3cd lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000481 1786114553.115476 write 0x035eaa thru 0x03639a (0x4f1 bytes) 000482 1786114553.180319 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xd26ce70c (stale) Mirror 4: 0x872d3cd (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xd26ce70c (stale) Mirror 4: 0x872d3cd (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000484 1786114553.390653 mapwrite 0x00c9c6 thru 0x015352 (0x898d bytes) 000485 1786114553.438312 trunc from 0x040000 to 0x03ad48 (0x52b7 bytes) 000486 1786114553.453971 mapread 0x001101 thru 0x001355 (0x255 bytes) 000487 1786114553.458125 mapread 0x024fb8 thru 0x030b3b (0xbb84 bytes) 000491 1786114553.477396 mapread 0x026bd1 thru 0x028fe1 (0x2411 bytes) 000494 1786114553.479683 mapread 0x031dec thru 0x034d0c (0x2f21 bytes) 000497 1786114553.483215 write 0x0322c0 thru 0x03273e (0x47f bytes) 000498 1786114553.489479 MIRROR_RESYNC 3 mirrors 000499 1786114554.051325 write 0x000828 thru 0x0107c2 (0xff9b bytes) 000500 1786114554.130447 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3ad49): Mirror 3: 0x9c351caa (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3ad49): Mirror 3: 0x9c351caa (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3ad49: skipped 000501 1786114554.326554 write 0x002b0a thru 0x00eb5f (0xc056 bytes) 000503 1786114554.346303 trunc from 0x03ad49 to 0x018b64 (0x221e4 bytes) 000504 1786114554.384890 read 0x01021d thru 0x013df0 (0x3bd4 bytes) 000505 1786114554.390807 trunc from 0x018b65 to 0x011e6c (0x6cf8 bytes) 000506 1786114554.407665 write 0x0023a3 thru 0x00ca03 (0xa661 bytes) truncating to largest ever: 0x3fefd 000507 1786114554.409585 trunc from 0x011e6d to 0x03fefc (0x2e090 bytes) 000510 1786114554.428148 trunc from 0x03fefd to 0x011d8f (0x2e16d bytes) 000511 1786114554.454245 write 0x015806 thru 0x020735 (0xaf30 bytes) 000515 1786114554.464598 read 0x017b2f thru 0x01bcd4 (0x41a6 bytes) 000516 1786114554.466586 read 0x0075b8 thru 0x00ac46 (0x368f bytes) 000517 1786114554.468536 trunc from 0x020736 to 0x03e2cc (0x1db97 bytes) 000518 1786114554.490528 write 0x023791 thru 0x024d9f (0x160f bytes) 000519 1786114554.499815 mapread 0x0022af thru 0x009a32 (0x7784 bytes) 000521 1786114554.501268 read 0x029975 thru 0x02f72e (0x5dba bytes) 000527 1786114554.503408 read 0x034a25 thru 0x03e2cc (0x98a8 bytes) 000528 1786114554.503893 read 0x02f1eb thru 0x03b55d (0xc373 bytes) 000531 1786114554.505583 trunc from 0x03e2cd to 0x0002a7 (0x3e025 bytes) 000533 1786114554.521596 mapwrite 0x022866 thru 0x02e101 (0xb89c bytes) 000535 1786114554.618435 mapwrite 0x031b9d thru 0x033415 (0x1879 bytes) 000536 1786114554.691485 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000538 1786114554.817772 write 0x02fab6 thru 0x033603 (0x3b4e bytes) 000539 1786114554.825612 mapread 0x021b81 thru 0x0230e0 (0x1560 bytes) 000541 1786114554.843909 trunc from 0x033604 to 0x038be2 (0x55df bytes) 000542 1786114554.856845 mapread 0x01aa3e thru 0x01ac57 (0x21a bytes) 000543 1786114554.863123 mapread 0x02811e thru 0x034281 (0xc164 bytes) 000544 1786114554.868619 mapread 0x009786 thru 0x00e735 (0x4fb0 bytes) 000545 1786114554.874706 mapread 0x014b1c thru 0x015e00 (0x12e5 bytes) 000551 1786114554.877506 mapread 0x0252a9 thru 0x0344aa (0xf202 bytes) 000552 1786114554.878656 mapread 0x02093f thru 0x021dd3 (0x1495 bytes) 000553 1786114554.879375 mapread 0x03293d thru 0x038be2 (0x62a6 bytes) 000555 1786114554.884263 mapwrite 0x0199b6 thru 0x0200a8 (0x66f3 bytes) 000559 1786114554.922558 mapread 0x004a2c thru 0x00adf0 (0x63c5 bytes) 000561 1786114554.924333 mapwrite 0x01e6af thru 0x02287c (0x41ce bytes) 000562 1786114554.974501 mapwrite 0x02cfbf thru 0x02f28e (0x22d0 bytes) 000564 1786114555.028957 mapwrite 0x025522 thru 0x0315aa (0xc089 bytes) 000568 1786114555.071717 mapwrite 0x019a12 thru 0x027a22 (0xe011 bytes) 000569 1786114555.099483 mapwrite 0x021597 thru 0x027c88 (0x66f2 bytes) 000570 1786114555.124244 read 0x00aed0 thru 0x00b03e (0x16f bytes) 000573 1786114555.126632 read 0x02dfb0 thru 0x02f2e7 (0x1338 bytes) 000575 1786114555.128343 read 0x00a55b thru 0x011c41 (0x76e7 bytes) 000576 1786114555.129631 mapwrite 0x01fe98 thru 0x02ebfc (0xed65 bytes) 000577 1786114555.175695 write 0x03783e thru 0x03993f (0x2102 bytes) 000578 1786114555.179181 write 0x0284c4 thru 0x02fdef (0x792c bytes) 000579 1786114555.181176 write 0x026591 thru 0x036481 (0xfef1 bytes) 000581 1786114555.183903 write 0x026606 thru 0x028c40 (0x263b bytes) 000583 1786114555.186952 trunc from 0x039940 to 0x03e5e1 (0x4ca2 bytes) 000584 1786114555.198147 mapread 0x02eb2e thru 0x0303ed (0x18c0 bytes) 000585 1786114555.199040 mapread 0x037c17 thru 0x03e5e1 (0x69cb bytes) 000586 1786114555.204760 trunc from 0x03e5e2 to 0x03cd34 (0x18ad bytes) 000587 1786114555.222896 trunc from 0x03cd35 to 0x00f409 (0x2d92b bytes) 000588 1786114555.245198 mapwrite 0x002afa thru 0x00f7fa (0xcd01 bytes) 000590 1786114555.296435 mapwrite 0x019198 thru 0x025b4f (0xc9b8 bytes) 000591 1786114555.392022 mapread 0x0128b0 thru 0x015855 (0x2fa6 bytes) 000592 1786114555.419036 mapread 0x00dc48 thru 0x018be8 (0xafa1 bytes) 000593 1786114555.442369 mapread 0x02250b thru 0x025b4f (0x3645 bytes) 000594 1786114555.443481 mapread 0x00b78b thru 0x016541 (0xadb7 bytes) 000596 1786114555.445328 mapread 0x001d7a thru 0x00aa13 (0x8c9a bytes) 000600 1786114555.448263 read 0x00ac03 thru 0x015a7b (0xae79 bytes) 000603 1786114555.450577 write 0x008af5 thru 0x009217 (0x723 bytes) 000604 1786114555.451834 write 0x01e626 thru 0x02b756 (0xd131 bytes) 000605 1786114555.453199 mapwrite 0x00e73e thru 0x014dce (0x6691 bytes) 000606 1786114555.501010 mapread 0x02929d thru 0x02b756 (0x24ba bytes) 000607 1786114555.502209 trunc from 0x02b757 to 0x039ed8 (0xe782 bytes) 000608 1786114555.516793 write 0x029a5e thru 0x02e91b (0x4ebe bytes) 000609 1786114555.525834 read 0x005c5d thru 0x00bbac (0x5f50 bytes) 000610 1786114555.526691 mapwrite 0x01758a thru 0x023d75 (0xc7ec bytes) 000613 1786114555.591840 MIRROR_EXTEND to 2 mirrors 000616 1786114556.037942 mapwrite 0x00d256 thru 0x01489c (0x7647 bytes) 000617 1786114556.156239 mapread 0x021b5d thru 0x0304d5 (0xe979 bytes) 000619 1786114556.174976 read 0x020838 thru 0x02fa90 (0xf259 bytes) 000620 1786114556.176012 write 0x03a61e thru 0x03ffff (0x59e2 bytes) 000623 1786114556.179788 mapwrite 0x031be3 thru 0x03a407 (0x8825 bytes) 000625 1786114556.231482 mapwrite 0x0228f6 thru 0x02cff7 (0xa702 bytes) 000626 1786114556.271196 mapread 0x018266 thru 0x0222bc (0xa057 bytes) 000627 1786114556.282176 mapread 0x00d0ee thru 0x016c28 (0x9b3b bytes) 000628 1786114556.284410 read 0x00d46d thru 0x01be44 (0xe9d8 bytes) 000629 1786114556.285393 read 0x02c706 thru 0x034043 (0x793e bytes) 000630 1786114556.286215 read 0x014211 thru 0x01e75f (0xa54f bytes) 000631 1786114556.287313 trunc from 0x040000 to 0x0384f4 (0x7b0b bytes) 000633 1786114556.314038 read 0x031935 thru 0x0384f4 (0x6bc0 bytes) 000634 1786114556.314946 mapwrite 0x00cd61 thru 0x01c281 (0xf521 bytes) 000635 1786114556.343201 read 0x00fc55 thru 0x01b120 (0xb4cc bytes) 000636 1786114556.344174 mapwrite 0x03b778 thru 0x03ffff (0x4888 bytes) 000637 1786114556.394868 mapread 0x026562 thru 0x028db7 (0x2856 bytes) 000638 1786114556.397040 mapread 0x014be4 thru 0x01f407 (0xa824 bytes) 000642 1786114556.400738 mapread 0x028251 thru 0x029ff8 (0x1da8 bytes) 000644 1786114556.402899 write 0x00bc05 thru 0x01554d (0x9949 bytes) 000645 1786114556.405114 read 0x02eb15 thru 0x0394ea (0xa9d6 bytes) 000647 1786114556.409875 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000650 1786114556.513469 mapwrite 0x0018e1 thru 0x00800c (0x672c bytes) 000653 1786114556.562397 trunc from 0x040000 to 0x03ceb6 (0x3149 bytes) 000654 1786114556.572386 MIRROR_EXTEND to 2 mirrors 000655 1786114556.961967 mapread 0x01d556 thru 0x023f85 (0x6a30 bytes) 000657 1786114556.996526 mapwrite 0x008d08 thru 0x00ddfa (0x50f3 bytes) 000659 1786114557.111413 read 0x0277e2 thru 0x035968 (0xe187 bytes) 000660 1786114557.123898 trunc from 0x03ceb7 to 0x00a6a7 (0x3280f bytes) 000661 1786114557.144861 write 0x0211df thru 0x02274e (0x1570 bytes) 000663 1786114557.147691 mapwrite 0x006d1d thru 0x016a8a (0xfd6e bytes) 000665 1786114557.227181 write 0x03df1a thru 0x03ffff (0x20e6 bytes) 000666 1786114557.228035 mapwrite 0x0361f0 thru 0x03e3b9 (0x81ca bytes) 000667 1786114557.268390 mapwrite 0x00fc15 thru 0x01ded5 (0xe2c1 bytes) 000670 1786114557.310437 read 0x034fa3 thru 0x03f64b (0xa6a9 bytes) 000671 1786114557.315209 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x8ba8807a (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xaa834d7e Mirror 3: 0xaa834d7e lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000672 1786114557.790348 MIRROR_RESYNC 3 mirrors 000673 1786114558.124349 trunc from 0x040000 to 0x0377c0 (0x883f bytes) 000676 1786114558.218238 read 0x009366 thru 0x01566f (0xc30a bytes) 000678 1786114558.227181 read 0x0273f4 thru 0x027ba5 (0x7b2 bytes) 000679 1786114558.238224 mapread 0x003c74 thru 0x005b11 (0x1e9e bytes) 000682 1786114558.239881 trunc from 0x0377c1 to 0x0059d2 (0x31dee bytes) 000683 1786114558.259832 mapread 0x00029a thru 0x0059d2 (0x5739 bytes) 000685 1786114558.261956 write 0x0357d4 thru 0x0360a7 (0x8d4 bytes) 000688 1786114558.264117 write 0x035acc thru 0x037503 (0x1a38 bytes) 000690 1786114558.265273 mapread 0x0148ab thru 0x020d56 (0xc4ac bytes) 000691 1786114558.324106 trunc from 0x037504 to 0x03e76d (0x726a bytes) 000692 1786114558.350995 MIRROR_RESYNC 3 mirrors 000694 1786114558.781623 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000695 1786114559.043626 write 0x039b24 thru 0x03e28c (0x4769 bytes) 000697 1786114559.099565 trunc from 0x03e76e to 0x01ad62 (0x23a0b bytes) 000698 1786114559.134514 mapwrite 0x03f946 thru 0x03ffff (0x6ba bytes) 000699 1786114559.169892 MIRROR_RESYNC 2 mirrors 000701 1786114559.592600 write 0x017568 thru 0x02324d (0xbce6 bytes) 000702 1786114559.638509 write 0x00863e thru 0x00e279 (0x5c3c bytes) 000705 1786114559.655416 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000706 1786114559.759175 MIRROR_EXTEND to 2 mirrors 000707 1786114560.138981 write 0x032edf thru 0x037c67 (0x4d89 bytes) 000709 1786114560.194645 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000710 1786114560.318799 write 0x006a1b thru 0x00f28e (0x8874 bytes) 000711 1786114560.329465 mapread 0x01fd24 thru 0x02ca68 (0xcd45 bytes) 000717 1786114560.359432 trunc from 0x040000 to 0x02b829 (0x147d6 bytes) 000718 1786114560.386359 mapwrite 0x01ed48 thru 0x029866 (0xab1f bytes) 000722 1786114560.440706 trunc from 0x02b82a to 0x029fe8 (0x1841 bytes) 000723 1786114560.459604 MIRROR_EXTEND to 2 mirrors 000724 1786114560.837562 write 0x0176c8 thru 0x01cf1e (0x5857 bytes) 000725 1786114560.890645 mapwrite 0x02a18c thru 0x030b5c (0x69d1 bytes) 000726 1786114560.953195 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000731 1786114561.059906 mapwrite 0x000074 thru 0x0008d6 (0x863 bytes) 000735 1786114561.093485 trunc from 0x030b5d to 0x0133dc (0x1d780 bytes) 000737 1786114561.104212 mapread 0x00dd74 thru 0x010fdf (0x326c bytes) 000738 1786114561.123216 mapread 0x0067f7 thru 0x00933b (0x2b45 bytes) 000739 1786114561.135831 read 0x00fb1d thru 0x0133dc (0x38c0 bytes) 000740 1786114561.136424 mapwrite 0x030347 thru 0x034d08 (0x49c2 bytes) 000741 1786114561.191160 mapwrite 0x021d2b thru 0x02d913 (0xbbe9 bytes) 000742 1786114561.230842 trunc from 0x034d09 to 0x03b3a9 (0x66a1 bytes) 000743 1786114561.241847 mapwrite 0x01fbeb thru 0x0247f6 (0x4c0c bytes) 000747 1786114561.277906 read 0x00ab3c thru 0x013edc (0x93a1 bytes) 000749 1786114561.279684 read 0x033694 thru 0x037b30 (0x449d bytes) 000750 1786114561.293603 mapwrite 0x01b5c4 thru 0x0223f7 (0x6e34 bytes) 000752 1786114561.318488 mapwrite 0x00c41d thru 0x00f9d8 (0x35bc bytes) 000754 1786114561.348782 trunc from 0x03b3aa to 0x036ed0 (0x44d9 bytes) 000757 1786114561.363623 trunc from 0x036ed1 to 0x031939 (0x5597 bytes) 000759 1786114561.377034 mapread 0x01ea96 thru 0x024095 (0x5600 bytes) 000760 1786114561.377872 mapread 0x01b410 thru 0x0286ba (0xd2ab bytes) 000761 1786114561.379567 mapwrite 0x03d248 thru 0x03d30a (0xc3 bytes) 000762 1786114561.445923 trunc from 0x03d30b to 0x00e59f (0x2ed6b bytes) 000763 1786114561.463625 mapwrite 0x02a7f3 thru 0x032f8b (0x8799 bytes) 000765 1786114561.554670 write 0x02a6c1 thru 0x032f89 (0x88c9 bytes) 000767 1786114561.568142 MIRROR_EXTEND to 2 mirrors 000768 1786114562.019993 mapwrite 0x03f5c9 thru 0x03ffff (0xa37 bytes) 000769 1786114562.151192 write 0x02b3b3 thru 0x02d90d (0x255b bytes) 000771 1786114562.165215 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000775 1786114562.258878 mapwrite 0x003c8b thru 0x00d001 (0x9377 bytes) 000776 1786114562.352588 write 0x0038d3 thru 0x012290 (0xe9be bytes) 000777 1786114562.357463 read 0x0214de thru 0x02401a (0x2b3d bytes) 000779 1786114562.362295 mapwrite 0x02cca9 thru 0x032272 (0x55ca bytes) 000780 1786114562.407679 trunc from 0x040000 to 0x0377cd (0x8832 bytes) 000781 1786114562.425394 mapwrite 0x00851f thru 0x00c7c8 (0x42aa bytes) 000783 1786114562.487808 write 0x02ea62 thru 0x0307f3 (0x1d92 bytes) 000784 1786114562.491674 MIRROR_EXTEND to 2 mirrors 000787 1786114562.893442 mapwrite 0x002a3a thru 0x00e220 (0xb7e7 bytes) 000788 1786114563.050738 read 0x01a586 thru 0x0272b4 (0xcd2f bytes) 000789 1786114563.057567 read 0x00cb85 thru 0x01b49f (0xe91b bytes) 000790 1786114563.066233 mapwrite 0x00b8a2 thru 0x00c4eb (0xc4a bytes) 000792 1786114563.100311 MIRROR_RESYNC 2 mirrors 000794 1786114563.414947 mapread 0x014da4 thru 0x020f8b (0xc1e8 bytes) 000796 1786114563.439679 write 0x0182d3 thru 0x020231 (0x7f5f bytes) 000797 1786114563.481846 write 0x01ff72 thru 0x024cbf (0x4d4e bytes) 000798 1786114563.487512 read 0x00b53b thru 0x019c4a (0xe710 bytes) 000800 1786114563.496039 mapread 0x017072 thru 0x01b0f8 (0x4087 bytes) 000802 1786114563.497457 trunc from 0x0377ce to 0x03618e (0x163f bytes) 000803 1786114563.537444 trunc from 0x03618f to 0x00269a (0x33af4 bytes) 000804 1786114563.551506 mapwrite 0x02d432 thru 0x038b2f (0xb6fe bytes) 000806 1786114563.605719 mapwrite 0x0381f6 thru 0x03ffff (0x7e0a bytes) 000807 1786114563.651965 write 0x012b9f thru 0x022b33 (0xff95 bytes) 000809 1786114563.660584 trunc from 0x040000 to 0x010f64 (0x2f09b bytes) 000810 1786114563.670886 write 0x020e4a thru 0x025621 (0x47d8 bytes) 000813 1786114563.673194 mapwrite 0x004691 thru 0x00bd9c (0x770c bytes) 000814 1786114563.725643 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000816 1786114563.842715 write 0x015f04 thru 0x01bc2a (0x5d27 bytes) 000819 1786114563.851901 trunc from 0x025622 to 0x039f98 (0x14977 bytes) 000829 1786114563.866193 read 0x03230c thru 0x039f98 (0x7c8d bytes) 000830 1786114563.872762 read 0x02328b thru 0x02c63b (0x93b1 bytes) 000833 1786114563.883384 mapread 0x024485 thru 0x02f580 (0xb0fc bytes) 000834 1786114563.884583 read 0x006cad thru 0x010596 (0x98ea bytes) 000837 1786114563.886785 trunc from 0x039f99 to 0x016549 (0x23a4f bytes) 000839 1786114563.902495 mapwrite 0x03c1ac thru 0x03ffff (0x3e54 bytes) 000843 1786114563.949632 mapread 0x005607 thru 0x00b9c6 (0x63c0 bytes) 000844 1786114563.950496 mapwrite 0x026506 thru 0x027017 (0xb12 bytes) 000848 1786114563.985318 trunc from 0x040000 to 0x02d260 (0x12d9f bytes) 000849 1786114564.002097 trunc from 0x02d261 to 0x03a840 (0xd5e0 bytes) 000853 1786114564.020174 mapread 0x008e80 thru 0x00edab (0x5f2c bytes) 000854 1786114564.021502 mapwrite 0x020c8d thru 0x023f4b (0x32bf bytes) 000855 1786114564.055429 write 0x0387ba thru 0x03ffff (0x7846 bytes) 000856 1786114564.060650 mapread 0x023caf thru 0x025044 (0x1396 bytes) 000858 1786114564.063194 read 0x004323 thru 0x00b145 (0x6e23 bytes) 000859 1786114564.064146 mapwrite 0x02b919 thru 0x03a3ba (0xeaa2 bytes) 000860 1786114564.125631 mapwrite 0x016f52 thru 0x019be9 (0x2c98 bytes) 000861 1786114564.162830 MIRROR_EXTEND to 2 mirrors 000863 1786114564.571559 mapwrite 0x030dc9 thru 0x031f6a (0x11a2 bytes) 000865 1786114564.714462 trunc from 0x040000 to 0x02ec2c (0x113d3 bytes) 000866 1786114564.746725 trunc from 0x02ec2d to 0x03fe59 (0x1122d bytes) 000868 1786114564.755754 trunc from 0x03fe5a to 0x024e6d (0x1afec bytes) 000869 1786114564.764697 MIRROR_RESYNC 2 mirrors 000870 1786114565.200305 read 0x006298 thru 0x00d29a (0x7003 bytes) 000871 1786114565.207767 MIRROR_EXTEND to 3 mirrors 000872 1786114565.594113 mapwrite 0x02ff43 thru 0x03fcdb (0xfd99 bytes) 000875 1786114565.750035 mapread 0x023533 thru 0x02e376 (0xae44 bytes) 000876 1786114565.763357 mapwrite 0x0021c7 thru 0x010786 (0xe5c0 bytes) 000877 1786114565.819392 write 0x029770 thru 0x031463 (0x7cf4 bytes) 000879 1786114565.821314 read 0x01c430 thru 0x026a14 (0xa5e5 bytes) 000880 1786114565.826560 MIRROR_RESYNC 3 mirrors 000882 1786114566.440676 read 0x001de1 thru 0x002b3d (0xd5d bytes) 000883 1786114566.444351 mapread 0x027680 thru 0x034fd0 (0xd951 bytes) 000884 1786114566.464923 read 0x0043d2 thru 0x00c51a (0x8149 bytes) 000886 1786114566.470773 write 0x00e953 thru 0x01c62c (0xdcda bytes) 000888 1786114566.523275 trunc from 0x03fcdc to 0x01e3f5 (0x218e6 bytes) 000892 1786114566.542312 trunc from 0x01e3f6 to 0x034e4f (0x16a5a bytes) 000894 1786114566.552741 write 0x02904b thru 0x029ec3 (0xe79 bytes) 000895 1786114566.557441 read 0x01687b thru 0x01abb2 (0x4338 bytes) 000898 1786114566.562243 MIRROR_RESYNC 3 mirrors 000902 1786114567.074710 trunc from 0x034e50 to 0x00d702 (0x2774d bytes) 000903 1786114567.152238 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0xd703): Mirror 3: 0xd9201c0a (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xd703): Mirror 3: 0xd9201c0a (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xd703: skipped 000904 1786114567.350779 trunc from 0x00d703 to 0x01c2d9 (0xebd7 bytes) 000905 1786114567.360320 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) 000907 1786114567.439310 mapwrite 0x00bb8d thru 0x00bcae (0x122 bytes) 000917 1786114567.475614 trunc from 0x01c2da to 0x034f5c (0x18c83 bytes) 000921 1786114567.495033 mapwrite 0x00a306 thru 0x018fff (0xecfa bytes) 000922 1786114567.573305 mapread 0x023fe1 thru 0x02d9b9 (0x99d9 bytes) 000928 1786114567.597189 mapwrite 0x038650 thru 0x03ffff (0x79b0 bytes) 000931 1786114567.677731 read 0x01f47a thru 0x0213ee (0x1f75 bytes) 000933 1786114567.679679 read 0x014577 thru 0x019a0e (0x5498 bytes) 000934 1786114567.681206 trunc from 0x040000 to 0x0194e4 (0x26b1b bytes) 000938 1786114567.710368 mapread 0x00e26f thru 0x00fc62 (0x19f4 bytes) 000941 1786114567.716431 trunc from 0x0194e5 to 0x0195d5 (0xf1 bytes) 000944 1786114567.736878 write 0x03f11f thru 0x03ffff (0xee1 bytes) 000946 1786114567.740457 trunc from 0x040000 to 0x006d46 (0x392b9 bytes) 000948 1786114567.750759 mapwrite 0x01befb thru 0x01d264 (0x136a bytes) 000949 1786114567.821355 read 0x011aea thru 0x01bb33 (0xa04a bytes) 000950 1786114567.822104 read 0x00957b thru 0x015506 (0xbf8c bytes) 000951 1786114567.822773 read 0x00a32b thru 0x00bdea (0x1ac0 bytes) 000954 1786114567.825684 mapread 0x009abe thru 0x013620 (0x9b63 bytes) 000956 1786114567.828024 trunc from 0x01d265 to 0x0014be (0x1bda6 bytes) 000960 1786114567.850285 trunc from 0x0014bf to 0x01c975 (0x1b4b7 bytes) 000965 1786114567.880543 MIRROR_EXTEND to 2 mirrors 000967 1786114568.340338 read 0x0187c3 thru 0x01c975 (0x41b3 bytes) 000970 1786114568.376796 read 0x009470 thru 0x017ee0 (0xea71 bytes) 000971 1786114568.390252 mapwrite 0x007932 thru 0x00dc95 (0x6364 bytes) 000974 1786114568.521242 read 0x018701 thru 0x01c975 (0x4275 bytes) 000975 1786114568.533204 read 0x010f1a thru 0x01a6d1 (0x97b8 bytes) 000976 1786114568.533881 write 0x00d443 thru 0x01a66c (0xd22a bytes) 000980 1786114568.540327 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x1c976): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1c976): Mirror 2: 0x9b92ef4d (stale) CRC-32 checksum value for chunk [0, 0x1c976): Mirror 1: 0x8396615f Mirror 3: 0x8396615f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1c976: skipped 000983 1786114569.028246 trunc from 0x01c976 to 0x001a42 (0x1af33 bytes) 000984 1786114569.109901 read 0x00136b thru 0x001a42 (0x6d8 bytes) 000985 1786114569.114478 mapwrite 0x0013ee thru 0x00587b (0x448e bytes) 000987 1786114569.173738 mapread 0x0044be thru 0x00587b (0x13be bytes) 000988 1786114569.174558 read 0x001156 thru 0x003ea9 (0x2d54 bytes) 000989 1786114569.175264 write 0x005021 thru 0x00dcb8 (0x8c98 bytes) 000990 1786114569.176276 mapread 0x005afe thru 0x00853e (0x2a41 bytes) 000991 1786114569.177487 read 0x00a9f1 thru 0x00dcb8 (0x32c8 bytes) 000993 1786114569.178523 mapread 0x007edc thru 0x00dcb8 (0x5ddd bytes) 000996 1786114569.180771 write 0x0196b4 thru 0x01e53d (0x4e8a bytes) 000997 1786114569.182230 read 0x003800 thru 0x004e38 (0x1639 bytes) 000998 1786114569.185398 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x2e0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1e53e): Mirror 3: 0x5b34e1f5 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1e53e): Mirror 3: 0x5b34e1f5 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1e53e: skipped 000999 1786114569.431406 MIRROR_RESYNC 2 mirrors 001000 1786114569.910226 write 0x0223c0 thru 0x031557 (0xf198 bytes) All operations completed A-OK!