== sanity-flr test 70a: flr mode fsx test ================ 12:31:41 (1774888301) Seed set to 8303 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x2a724 000001 1774888303.895794 trunc from 00000000 to 0x02a723 (0x2a724 bytes) 000004 1774888303.910711 trunc from 0x02a724 to 0x00e1b1 (0x1c572 bytes) 000005 1774888303.931378 trunc from 0x00e1b2 to 0x0037da (0xa9d7 bytes) 000006 1774888303.948964 read 0x0014d4 thru 0x0037da (0x2307 bytes) 000007 1774888303.955933 mapwrite 0x018113 thru 0x01bf1a (0x3e08 bytes) truncating to largest ever: 0x2d224 000008 1774888304.086060 trunc from 0x01bf1b to 0x02d223 (0x11309 bytes) 000012 1774888304.125467 mapwrite 0x0092de thru 0x017ba1 (0xe8c4 bytes) truncating to largest ever: 0x370b4 000015 1774888304.279622 trunc from 0x02d224 to 0x0370b3 (0x9e90 bytes) 000018 1774888304.320717 mapread 0x0310c7 thru 0x0370b3 (0x5fed bytes) 000019 1774888304.335935 read 0x013c8a thru 0x022d6e (0xf0e5 bytes) 000021 1774888304.351349 trunc from 0x0370b4 to 0x00dd3a (0x29379 bytes) 000022 1774888304.369651 trunc from 0x00dd3b to 0x025c8a (0x17f50 bytes) 000023 1774888304.390909 mapwrite 0x023c4a thru 0x02f1fc (0xb5b3 bytes) 000024 1774888304.532313 read 0x008a84 thru 0x009f06 (0x1483 bytes) 000026 1774888304.535854 mapread 0x02d474 thru 0x02f1fc (0x1d89 bytes) 000027 1774888304.539370 trunc from 0x02f1fd to 0x0179b1 (0x1784b bytes) truncating to largest ever: 0x3e67e 000029 1774888304.562408 trunc from 0x0179b2 to 0x03e67d (0x26ccc bytes) 000031 1774888304.596558 mapread 0x02b06c thru 0x035bf0 (0xab85 bytes) 000034 1774888304.611613 mapwrite 0x00becc thru 0x00de18 (0x1f4d bytes) 000037 1774888304.652387 mapwrite 0x01c8f9 thru 0x02ad69 (0xe471 bytes) 000038 1774888304.702863 read 0x0034aa thru 0x01146b (0xdfc2 bytes) 000039 1774888304.703867 write 0x023412 thru 0x02700e (0x3bfd bytes) 000041 1774888304.706710 mapwrite 0x03cb82 thru 0x03dbcf (0x104e bytes) 000044 1774888304.772482 mapwrite 0x008136 thru 0x00ef32 (0x6dfd bytes) 000045 1774888304.832128 trunc from 0x03e67e to 0x030101 (0xe57c bytes) 000046 1774888304.856219 write 0x022022 thru 0x02fbb6 (0xdb95 bytes) 000047 1774888304.857643 trunc from 0x030102 to 0x03459f (0x449e bytes) 000058 1774888304.910741 MIRROR_EXTEND to 2 mirrors 000059 1774888305.721511 MIRROR_EXTEND to 3 mirrors 000060 1774888306.749696 MIRROR_RESYNC 3 mirrors 000061 1774888307.482382 mapwrite 0x01bec3 thru 0x01d86a (0x19a8 bytes) 000064 1774888307.797855 read 0x030581 thru 0x03459f (0x401f bytes) 000065 1774888307.813006 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x345a0): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x345a0): Mirror 2: 0x76d6ec74 (stale) Mirror 3: 0x76d6ec74 (stale) CRC-32 checksum value for chunk [0, 0x345a0): Mirror 1: 0x1432b917 Mirror 4: 0x1432b917 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x345a0: skipped 000066 1774888309.505782 mapread 0x002581 thru 0x00d372 (0xadf2 bytes) 000068 1774888309.547590 mapwrite 0x03e766 thru 0x03ffff (0x189a bytes) 000072 1774888309.713672 mapread 0x0169fb thru 0x01a61e (0x3c24 bytes) 000073 1774888309.728719 trunc from 0x040000 to 0x02c6d6 (0x13929 bytes) 000074 1774888309.763096 mapread 0x00ac59 thru 0x01457f (0x9927 bytes) truncating to largest ever: 0x3f150 000075 1774888309.775511 trunc from 0x02c6d7 to 0x03f14f (0x12a79 bytes) 000076 1774888309.812513 mapwrite 0x004f46 thru 0x007149 (0x2204 bytes) 000077 1774888309.863391 mapwrite 0x00211c thru 0x005a83 (0x3968 bytes) 000080 1774888309.942198 mapwrite 0x0203ee thru 0x02f26a (0xee7d bytes) 000081 1774888310.061688 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x3f150): Mirror 2: (stale - read error) Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3f150): Mirror 2: 0x42317e76 (stale) Mirror 3: 0x42317e76 (stale) Mirror 4: 0x1203b6a5 (stale) CRC-32 checksum value for chunk [0, 0x3f150): Mirror 1: 0x59bbb9d7 Mirror 5: 0x59bbb9d7 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3f150: skipped 000082 1774888311.628622 mapwrite 0x023887 thru 0x03129a (0xda14 bytes) 000083 1774888312.022307 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x3f150): Mirror 2: (stale - read error) Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3f150): Mirror 2: 0x42317e76 (stale) Mirror 3: 0x42317e76 (stale) Mirror 4: 0x1203b6a5 (stale) Mirror 5: 0x59bbb9d7 (stale) CRC-32 checksum value for chunk [0, 0x3f150): Mirror 1: 0xd4f7e7f8 Mirror 6: 0xd4f7e7f8 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3f150: skipped 000084 1774888313.394746 MIRROR_RESYNC 6 mirrors 000085 1774888314.517813 trunc from 0x03f150 to 0x026217 (0x18f38 bytes) 000086 1774888314.591028 trunc from 0x026218 to 0x0148e4 (0x11933 bytes) truncating to largest ever: 0x3ff16 000088 1774888314.615501 trunc from 0x0148e5 to 0x03ff15 (0x2b631 bytes) 000089 1774888314.634780 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x3ff16): Mirror 2: (stale - read error) Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3ff16): Mirror 2: 0xcc9320d3 (stale) Mirror 3: 0xcc9320d3 (stale) Mirror 4: 0xcc9320d3 (stale) Mirror 5: 0xcc9320d3 (stale) Mirror 6: 0xcc9320d3 (stale) CRC-32 checksum value for chunk [0, 0x3ff16): Mirror 1: 0x6897f63c Mirror 7: 0x6897f63c lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3ff16: skipped 000090 1774888315.751793 write 0x0140d8 thru 0x0170fa (0x3023 bytes) 000091 1774888315.953949 mapwrite 0x012ddb thru 0x0219a0 (0xebc6 bytes) 000092 1774888316.149157 MIRROR_SPLIT mirror 4 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3ff16): Mirror 2: 0xcc9320d3 (stale) Mirror 3: 0xcc9320d3 (stale) Mirror 4: 0xcc9320d3 (stale) Mirror 6: 0xcc9320d3 (stale) Mirror 7: 0x6897f63c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3ff16): Mirror 2: 0xcc9320d3 (stale) Mirror 3: 0xcc9320d3 (stale) Mirror 4: 0xcc9320d3 (stale) Mirror 6: 0xcc9320d3 (stale) Mirror 7: 0x6897f63c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3ff16: skipped 000095 1774888316.509159 write 0x02d8b2 thru 0x0309ac (0x30fb bytes) 000096 1774888316.520776 trunc from 0x03ff16 to 0x0052f0 (0x3ac25 bytes) 000097 1774888316.551359 write 0x0114b1 thru 0x016178 (0x4cc8 bytes) 000098 1774888316.552805 write 0x01c0f1 thru 0x026ca7 (0xabb7 bytes) 000101 1774888316.558534 write 0x014764 thru 0x01e629 (0x9ec6 bytes) 000102 1774888316.561490 read 0x0251f1 thru 0x02554c (0x35c bytes) 000104 1774888316.563542 read 0x00115f thru 0x0073d8 (0x627a bytes) 000105 1774888316.580005 MIRROR_RESYNC 6 mirrors 000106 1774888318.332189 read 0x021a2f thru 0x026ca7 (0x5279 bytes) 000108 1774888318.340492 trunc from 0x026ca8 to 0x024663 (0x2644 bytes) 000109 1774888318.406724 write 0x03843a thru 0x03d583 (0x514a bytes) 000110 1774888318.413518 MIRROR_SPLIT mirror 4 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3d584): Mirror 2: 0xf4b2358d (stale) Mirror 3: 0xf4b2358d (stale) Mirror 4: 0xf4b2358d (stale) Mirror 7: 0xf4b2358d (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3d584): Mirror 2: 0xf4b2358d (stale) Mirror 3: 0xf4b2358d (stale) Mirror 4: 0xf4b2358d (stale) Mirror 7: 0xf4b2358d (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3d584: skipped 000111 1774888318.744425 mapwrite 0x01d43f thru 0x021f3a (0x4afc bytes) 000112 1774888318.857642 read 0x031a50 thru 0x034ab4 (0x3065 bytes) 000116 1774888318.868224 mapwrite 0x03a046 thru 0x03ffff (0x5fba bytes) 000117 1774888318.989009 mapwrite 0x026076 thru 0x02e962 (0x88ed bytes) 000119 1774888319.074701 write 0x028b8c thru 0x038831 (0xfca6 bytes) 000121 1774888319.082209 MIRROR_SPLIT mirror 3 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xf19b0c14 (stale) Mirror 3: 0xf19b0c14 (stale) Mirror 7: 0xf19b0c14 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xf19b0c14 (stale) Mirror 3: 0xf19b0c14 (stale) Mirror 7: 0xf19b0c14 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000124 1774888319.446253 read 0x009a32 thru 0x00b8fd (0x1ecc bytes) 000125 1774888319.452496 mapwrite 0x026255 thru 0x031383 (0xb12f bytes) 000127 1774888319.550422 mapread 0x0221c4 thru 0x02c2d9 (0xa116 bytes) 000128 1774888319.551841 mapread 0x0349ec thru 0x03ffff (0xb614 bytes) 000129 1774888319.582425 MIRROR_RESYNC 4 mirrors 000131 1774888320.938551 trunc from 0x040000 to 0x016280 (0x29d7f bytes) 000133 1774888321.059988 mapread 0x0122f9 thru 0x016280 (0x3f88 bytes) 000135 1774888321.084846 trunc from 0x016281 to 0x028c9b (0x12a1b bytes) 000136 1774888321.126468 mapread 0x006786 thru 0x014898 (0xe113 bytes) 000137 1774888321.144969 read 0x008656 thru 0x00e7aa (0x6155 bytes) 000140 1774888321.147913 trunc from 0x028c9c to 0x026b65 (0x2136 bytes) 000141 1774888321.189721 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x26b66): Mirror 3: 0x22c43cd3 (stale) Mirror 7: 0x22c43cd3 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x26b66): Mirror 3: 0x22c43cd3 (stale) Mirror 7: 0x22c43cd3 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x26b66: skipped 000142 1774888321.730023 trunc from 0x026b66 to 0x017c84 (0xeee1 bytes) 000146 1774888321.754026 mapread 0x00dbdd thru 0x010cfb (0x311f bytes) 000148 1774888321.781513 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x17c85): Mirror 3: 0x63614287 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x17c85): Mirror 3: 0x63614287 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x17c85: skipped 000149 1774888322.133684 read 0x002fb0 thru 0x007feb (0x503c bytes) 000150 1774888322.146533 read 0x016cde thru 0x017c84 (0xfa7 bytes) 000151 1774888322.159380 mapread 0x00d77f thru 0x015350 (0x7bd2 bytes) 000153 1774888322.163556 mapwrite 0x003ee7 thru 0x012742 (0xe85c bytes) 000154 1774888322.265224 read 0x001854 thru 0x00d4b1 (0xbc5e bytes) 000155 1774888322.267991 mapwrite 0x0338e5 thru 0x0355b4 (0x1cd0 bytes) 000156 1774888322.474959 trunc from 0x0355b5 to 0x0245f8 (0x10fbc bytes) 000157 1774888322.500000 trunc from 0x0245f9 to 0x02cf3c (0x8944 bytes) 000158 1774888322.523353 trunc from 0x02cf3d to 0x004ad4 (0x28468 bytes) 000160 1774888322.553121 read 0x004258 thru 0x004ad4 (0x87d bytes) 000161 1774888322.556356 read 0x0001c9 thru 0x004ad4 (0x490c bytes) 000163 1774888322.563793 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) 000164 1774888322.795837 MIRROR_EXTEND to 2 mirrors 000167 1774888323.616567 read 0x003683 thru 0x004ad4 (0x1452 bytes) 000168 1774888323.635176 write 0x034ae5 thru 0x03ffff (0xb51b bytes) 000169 1774888323.707612 trunc from 0x040000 to 0x033fb0 (0xc04f bytes) 000170 1774888323.740080 mapwrite 0x004ee1 thru 0x007643 (0x2763 bytes) 000171 1774888323.914548 mapread 0x01a119 thru 0x025245 (0xb12d bytes) 000172 1774888323.946654 mapwrite 0x01727c thru 0x025a8b (0xe810 bytes) 000173 1774888324.051572 read 0x00ae16 thru 0x00d901 (0x2aec bytes) 000174 1774888324.060558 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x33fb1): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x33fb1): Mirror 2: 0xae931f4c (stale) CRC-32 checksum value for chunk [0, 0x33fb1): Mirror 1: 0xbeebf80f Mirror 3: 0xbeebf80f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x33fb1: skipped 000175 1774888325.401466 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x33fb1): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x33fb1): Mirror 2: 0xae931f4c (stale) CRC-32 checksum value for chunk [0, 0x33fb1): Mirror 1: 0xbeebf80f Mirror 3: 0xbeebf80f Mirror 4: 0xbeebf80f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x33fb1: skipped 000176 1774888326.267635 trunc from 0x033fb1 to 0x01e6b8 (0x158f8 bytes) 000177 1774888326.356898 mapread 0x00f580 thru 0x015e07 (0x6888 bytes) 000178 1774888326.375008 mapread 0x002d38 thru 0x005b35 (0x2dfe bytes) 000179 1774888326.384793 read 0x003109 thru 0x005a13 (0x290b bytes) 000182 1774888326.387815 read 0x00f0ae thru 0x018dbf (0x9d12 bytes) 000183 1774888326.389428 mapwrite 0x025d2e thru 0x0296cd (0x39a0 bytes) 000185 1774888326.476007 mapread 0x0217f3 thru 0x0296cd (0x7edb bytes) 000186 1774888326.479005 read 0x0031f5 thru 0x011278 (0xe084 bytes) 000187 1774888326.480847 mapread 0x00f885 thru 0x01a5d9 (0xad55 bytes) 000189 1774888326.484913 read 0x002dc9 thru 0x007475 (0x46ad bytes) 000190 1774888326.486804 write 0x01575a thru 0x01c141 (0x69e8 bytes) 000191 1774888326.488136 mapwrite 0x034157 thru 0x0345c3 (0x46d bytes) 000193 1774888326.591253 mapwrite 0x00db22 thru 0x01d167 (0xf646 bytes) 000195 1774888326.694790 mapwrite 0x0083e6 thru 0x00a8ab (0x24c6 bytes) 000198 1774888326.762975 mapwrite 0x033f54 thru 0x038694 (0x4741 bytes) 000199 1774888326.891922 trunc from 0x038695 to 0x000c45 (0x37a4f bytes) 000200 1774888326.915849 write 0x01f473 thru 0x02a86e (0xb3fc bytes) 000202 1774888326.918671 read 0x025482 thru 0x027b5a (0x26d9 bytes) 000203 1774888326.919800 mapwrite 0x00998b thru 0x01829b (0xe911 bytes) 000207 1774888326.991428 mapread 0x029778 thru 0x02a86e (0x10f7 bytes) 000209 1774888326.993970 write 0x02ef61 thru 0x03647a (0x751a bytes) 000210 1774888326.995969 trunc from 0x03647b to 0x019543 (0x1cf37 bytes) 000211 1774888327.011590 mapread 0x00aa8f thru 0x00cf8b (0x24fd bytes) 000213 1774888327.014880 mapwrite 0x022b27 thru 0x02ee15 (0xc2ef bytes) 000216 1774888327.125890 write 0x01c110 thru 0x0257fb (0x96ec bytes) 000217 1774888327.131567 write 0x01524b thru 0x01d6c0 (0x8476 bytes) 000218 1774888327.133283 mapread 0x01106d thru 0x0126d7 (0x166b bytes) 000219 1774888327.139579 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x2ee16): Mirror 2: (stale - read error) Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2ee16): Mirror 2: 0x255f3ee9 (stale) Mirror 3: 0x6491e7f (stale) Mirror 4: 0x6491e7f (stale) CRC-32 checksum value for chunk [0, 0x2ee16): Mirror 1: 0xf0de65fa Mirror 5: 0xf0de65fa lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2ee16: skipped 000220 1774888327.982916 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2ee16): Mirror 3: 0x6491e7f (stale) Mirror 4: 0x6491e7f (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2ee16): Mirror 3: 0x6491e7f (stale) Mirror 4: 0x6491e7f (stale) CRC-32 checksum value for chunk [0, 0x2ee16): Mirror 1: 0xf0de65fa Mirror 5: 0xf0de65fa lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2ee16: skipped 000221 1774888328.670453 trunc from 0x02ee16 to 0x029f8e (0x4e87 bytes) 000224 1774888328.878858 write 0x0184ee thru 0x01ce07 (0x491a bytes) 000228 1774888328.907384 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x29f8f): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x29f8f): Mirror 3: 0x94250545 (stale) Mirror 4: 0x94250545 (stale) Mirror 5: 0xf059274b (stale) CRC-32 checksum value for chunk [0, 0x29f8f): Mirror 1: 0x55e1aeef Mirror 6: 0x55e1aeef lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x29f8f: skipped 000229 1774888330.435601 MIRROR_RESYNC 5 mirrors 000230 1774888331.312490 mapread 0x01e0c5 thru 0x020cb3 (0x2bef bytes) 000231 1774888331.358255 trunc from 0x029f8f to 0x00257d (0x27a11 bytes) 000232 1774888331.561209 write 0x0156ef thru 0x019938 (0x424a bytes) 000233 1774888331.562515 read 0x000da9 thru 0x003dda (0x3032 bytes) 000234 1774888331.585408 mapwrite 0x027934 thru 0x02dd88 (0x6455 bytes) 000235 1774888331.781432 read 0x010e49 thru 0x01c6b0 (0xb868 bytes) 000236 1774888331.806000 trunc from 0x02dd89 to 0x02ab3c (0x324c bytes) 000238 1774888331.844791 mapread 0x003c4d thru 0x00f8a9 (0xbc5d bytes) 000239 1774888331.848494 mapread 0x017a74 thru 0x01a682 (0x2c0f bytes) 000242 1774888331.853444 mapread 0x003990 thru 0x009c3f (0x62b0 bytes) 000243 1774888331.855720 mapwrite 0x012876 thru 0x018a82 (0x620d bytes) 000245 1774888331.938237 MIRROR_RESYNC 5 mirrors 000248 1774888333.328225 mapread 0x01d59c thru 0x022151 (0x4bb6 bytes) 000249 1774888333.361292 mapwrite 0x03f20a thru 0x03ffff (0xdf6 bytes) 000252 1774888333.492325 read 0x0132a6 thru 0x013fa8 (0xd03 bytes) 000254 1774888333.499528 read 0x01fb61 thru 0x026d8d (0x722d bytes) 000255 1774888333.518292 MIRROR_RESYNC 5 mirrors 000256 1774888334.724936 trunc from 0x040000 to 0x03cf6a (0x3095 bytes) 000257 1774888334.795406 trunc from 0x03cf6b to 0x017874 (0x256f6 bytes) 000258 1774888334.822554 read 0x016403 thru 0x017874 (0x1472 bytes) 000262 1774888334.833283 mapwrite 0x0193d0 thru 0x020817 (0x7448 bytes) 000263 1774888334.924416 mapread 0x003ede thru 0x00805c (0x417f bytes) 000264 1774888334.934268 trunc from 0x020818 to 0x0275df (0x6dc8 bytes) 000265 1774888334.958765 mapwrite 0x02fdf9 thru 0x03b138 (0xb340 bytes) 000267 1774888335.060489 mapwrite 0x02d696 thru 0x03d25a (0xfbc5 bytes) 000269 1774888335.162848 write 0x034d95 thru 0x036df2 (0x205e bytes) 000273 1774888335.173028 mapread 0x018c4f thru 0x01be3e (0x31f0 bytes) 000274 1774888335.175840 trunc from 0x03d25b to 0x0333f5 (0x9e65 bytes) 000275 1774888335.197762 mapread 0x015dbc thru 0x017b36 (0x1d7b bytes) 000277 1774888335.203456 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x333f6): Mirror 4: 0xc841a3bd (stale) Mirror 5: 0xc841a3bd (stale) Mirror 6: 0xc841a3bd (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x333f6): Mirror 4: 0xc841a3bd (stale) Mirror 5: 0xc841a3bd (stale) Mirror 6: 0xc841a3bd (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x333f6: skipped 000279 1774888335.643831 read 0x02a507 thru 0x02c2f6 (0x1df0 bytes) 000280 1774888335.649792 mapread 0x012902 thru 0x01569c (0x2d9b bytes) 000285 1774888335.680992 read 0x020b4c thru 0x02d6ce (0xcb83 bytes) 000286 1774888335.693319 mapwrite 0x025067 thru 0x02f0e6 (0xa080 bytes) 000287 1774888335.771078 mapwrite 0x0352ba thru 0x03ffff (0xad46 bytes) 000289 1774888335.868965 mapread 0x00d98b thru 0x01cf7f (0xf5f5 bytes) 000290 1774888335.870234 write 0x0188a3 thru 0x02576c (0xceca bytes) 000293 1774888335.875881 trunc from 0x040000 to 0x009591 (0x36a6e bytes) 000299 1774888335.929934 read 0x0062b6 thru 0x009591 (0x32dc bytes) 000300 1774888335.930718 trunc from 0x009592 to 0x0201e4 (0x16c53 bytes) 000301 1774888335.954424 mapwrite 0x03230a thru 0x0356da (0x33d1 bytes) 000302 1774888336.040793 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x356db): Mirror 5: 0xc855a725 (stale) Mirror 6: 0xc855a725 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x356db): Mirror 5: 0xc855a725 (stale) Mirror 6: 0xc855a725 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x356db: skipped 000305 1774888336.370117 read 0x00b173 thru 0x011ca4 (0x6b32 bytes) 000306 1774888336.380340 trunc from 0x0356db to 0x00c6cf (0x2900b bytes) 000308 1774888336.416827 write 0x03718d thru 0x03ffff (0x8e73 bytes) 000309 1774888336.419404 write 0x010a82 thru 0x0150bd (0x463c bytes) 000310 1774888336.431236 read 0x004273 thru 0x0102a6 (0xc034 bytes) 000311 1774888336.443807 read 0x031440 thru 0x032ea8 (0x1a69 bytes) 000313 1774888336.445749 mapwrite 0x0224c5 thru 0x029182 (0x6cbe bytes) 000318 1774888336.519105 mapwrite 0x01d4e8 thru 0x023889 (0x63a2 bytes) 000322 1774888336.592314 mapread 0x0126bd thru 0x01de4e (0xb792 bytes) 000323 1774888336.596073 trunc from 0x040000 to 0x03bee8 (0x4117 bytes) 000324 1774888336.621128 write 0x029bd2 thru 0x0323cf (0x87fe bytes) 000325 1774888336.624198 mapwrite 0x014b37 thru 0x01f1c9 (0xa693 bytes) 000327 1774888336.699464 trunc from 0x03bee9 to 0x01dddd (0x1e10b bytes) 000328 1774888336.718762 write 0x01b116 thru 0x01c2ac (0x1197 bytes) 000330 1774888336.722376 mapwrite 0x02e63d thru 0x02fbd9 (0x159d bytes) 000332 1774888336.815904 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x2fbda): Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2fbda): Mirror 5: 0x69487f45 (stale) Mirror 6: 0x69487f45 (stale) CRC-32 checksum value for chunk [0, 0x2fbda): Mirror 1: 0x6b1a63c3 Mirror 7: 0x6b1a63c3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2fbda: skipped 000333 1774888337.611734 write 0x0244f8 thru 0x03056f (0xc078 bytes) 000335 1774888337.701641 trunc from 0x030570 to 0x034234 (0x3cc5 bytes) 000339 1774888337.728618 read 0x0022a2 thru 0x00d743 (0xb4a2 bytes) 000342 1774888337.738778 trunc from 0x034235 to 0x0325ba (0x1c7a bytes) 000345 1774888337.771529 mapread 0x00672d thru 0x014967 (0xe23b bytes) 000346 1774888337.787642 read 0x01d482 thru 0x024ce9 (0x7868 bytes) 000350 1774888337.802010 mapread 0x0047d1 thru 0x007117 (0x2947 bytes) 000351 1774888337.804705 read 0x019081 thru 0x01b1aa (0x212a bytes) 000352 1774888337.811787 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x325bb): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x325bb): Mirror 5: 0x2da4d68b (stale) Mirror 6: 0x2da4d68b (stale) Mirror 7: 0x98ea095e (stale) CRC-32 checksum value for chunk [0, 0x325bb): Mirror 1: 0x4c32c481 Mirror 8: 0x4c32c481 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x325bb: skipped 000355 1774888338.848969 write 0x007025 thru 0x013204 (0xc1e0 bytes) 000357 1774888338.910001 read 0x02ae6a thru 0x0325ba (0x7751 bytes) 000359 1774888338.918085 mapwrite 0x016e56 thru 0x0245a4 (0xd74f bytes) 000361 1774888339.008788 mapwrite 0x038636 thru 0x03ffff (0x79ca bytes) 000365 1774888339.124384 write 0x03bd90 thru 0x03dc4c (0x1ebd bytes) 000366 1774888339.125731 write 0x032373 thru 0x03566a (0x32f8 bytes) 000367 1774888339.128258 read 0x001dd7 thru 0x00b764 (0x998e bytes) 000368 1774888339.139133 write 0x028870 thru 0x0337da (0xaf6b bytes) 000369 1774888339.140620 trunc from 0x040000 to 0x027236 (0x18dc9 bytes) 000370 1774888339.166143 write 0x0333d5 thru 0x0337a0 (0x3cc bytes) 000371 1774888339.167718 read 0x00bb24 thru 0x0185ca (0xcaa7 bytes) 000372 1774888339.175536 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x337a1): Mirror 6: 0x529452e1 (stale) Mirror 7: 0x974ef33d (stale) Mirror 8: 0x8c8723ab (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x337a1): Mirror 6: 0x529452e1 (stale) Mirror 7: 0x974ef33d (stale) Mirror 8: 0x8c8723ab (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x337a1: skipped 000375 1774888339.571209 mapwrite 0x007b04 thru 0x017667 (0xfb64 bytes) 000379 1774888339.669908 trunc from 0x0337a1 to 0x03d5c0 (0x9e20 bytes) 000382 1774888339.696532 mapwrite 0x0069de thru 0x00afec (0x460f bytes) 000383 1774888339.783115 trunc from 0x03d5c1 to 0x035a08 (0x7bb8 bytes) 000385 1774888339.807929 mapwrite 0x007227 thru 0x00cab9 (0x5893 bytes) 000387 1774888339.876720 mapwrite 0x0233d3 thru 0x02bec9 (0x8af7 bytes) 000389 1774888339.978260 mapread 0x006130 thru 0x014f8f (0xee60 bytes) 000390 1774888339.982876 trunc from 0x035a09 to 0x03d133 (0x772b bytes) 000392 1774888340.009319 mapwrite 0x003d8a thru 0x00f138 (0xb3af bytes) 000394 1774888340.113910 trunc from 0x03d134 to 0x0080b3 (0x35080 bytes) 000395 1774888340.138261 write 0x003d87 thru 0x0090ea (0x5364 bytes) 000396 1774888340.141212 mapread 0x0088bb thru 0x0090ea (0x830 bytes) 000398 1774888340.144566 write 0x02cc36 thru 0x030cf0 (0x40bb bytes) 000399 1774888340.151255 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x30cf1): Mirror 7: 0x7bacfda5 (stale) Mirror 8: 0x70284740 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x30cf1): Mirror 7: 0x7bacfda5 (stale) Mirror 8: 0x70284740 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x30cf1: skipped 000400 1774888340.435983 mapwrite 0x036223 thru 0x03ffff (0x9ddd bytes) 000402 1774888340.546993 MIRROR_RESYNC 3 mirrors 000403 1774888341.614199 mapwrite 0x001c26 thru 0x008d92 (0x716d bytes) 000404 1774888341.998003 mapread 0x00be90 thru 0x00c84d (0x9be bytes) 000405 1774888342.020853 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 7: 0xdbf40c85 (stale) Mirror 8: 0xdbf40c85 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xb906e2c3 Mirror 9: 0xb906e2c3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000406 1774888342.959302 trunc from 0x040000 to 0x025378 (0x1ac87 bytes) 000408 1774888343.035709 mapwrite 0x02691a thru 0x03094f (0xa036 bytes) 000409 1774888343.229306 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x30950): Mirror 7: 0xb987429e (stale) Mirror 9: 0x8d1282bb (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x30950): Mirror 7: 0xb987429e (stale) Mirror 9: 0x8d1282bb (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x30950: skipped 000412 1774888343.751798 mapwrite 0x02567d thru 0x02a11e (0x4aa2 bytes) 000414 1774888343.813142 mapread 0x009ec6 thru 0x013bb6 (0x9cf1 bytes) 000417 1774888343.839589 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x30950): Mirror 7: (stale - read error) Mirror 9: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x30950): Mirror 7: 0xb987429e (stale) Mirror 9: 0x8d1282bb (stale) CRC-32 checksum value for chunk [0, 0x30950): Mirror 1: 0x6aa6fff3 Mirror 10: 0x6aa6fff3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x30950: skipped 000418 1774888345.073650 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x30950): Mirror 7: (stale - read error) Mirror 9: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x30950): Mirror 7: 0xb987429e (stale) Mirror 9: 0x8d1282bb (stale) CRC-32 checksum value for chunk [0, 0x30950): Mirror 1: 0x6aa6fff3 Mirror 10: 0x6aa6fff3 Mirror 11: 0x6aa6fff3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x30950: skipped 000419 1774888345.968922 write 0x03a889 thru 0x03f983 (0x50fb bytes) 000421 1774888346.061017 MIRROR_RESYNC 5 mirrors 000422 1774888347.377619 MIRROR_SPLIT mirror 4 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) 000424 1774888347.682733 write 0x02f567 thru 0x0359ba (0x6454 bytes) 000425 1774888347.769845 mapwrite 0x017572 thru 0x02317c (0xbc0b bytes) 000427 1774888347.878135 trunc from 0x03f984 to 0x03ef6a (0xa19 bytes) 000428 1774888347.910562 mapread 0x037d5f thru 0x03ef6a (0x720c bytes) 000432 1774888347.938613 write 0x03fd9f thru 0x03ffff (0x261 bytes) 000433 1774888347.943574 mapwrite 0x012aaf thru 0x0177fd (0x4d4f bytes) 000434 1774888348.153661 read 0x01c894 thru 0x028d1d (0xc48a bytes) 000439 1774888348.190629 MIRROR_RESYNC 4 mirrors 000440 1774888349.472535 mapread 0x0334e3 thru 0x03ffff (0xcb1d bytes) 000441 1774888349.500440 read 0x0080eb thru 0x015f41 (0xde57 bytes) 000442 1774888349.518018 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) 000443 1774888349.880636 read 0x02e315 thru 0x03302c (0x4d18 bytes) 000445 1774888349.900113 MIRROR_EXTEND to 4 mirrors 000446 1774888350.600370 write 0x0294ab thru 0x031da1 (0x88f7 bytes) 000447 1774888350.731334 mapread 0x01576f thru 0x016556 (0xde8 bytes) 000448 1774888350.743023 write 0x035582 thru 0x037d13 (0x2792 bytes) 000453 1774888350.769377 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 9: 0x2c51ceb (stale) Mirror 10: 0x2c51ceb (stale) Mirror 11: 0x2c51ceb (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xf758c7b4 Mirror 12: 0xf758c7b4 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000455 1774888351.960177 read 0x03a304 thru 0x03ffff (0x5cfc bytes) 000457 1774888351.998593 trunc from 0x040000 to 0x00c3a2 (0x33c5d bytes) 000458 1774888352.171897 mapread 0x0092e3 thru 0x00c3a2 (0x30c0 bytes) 000460 1774888352.198953 MIRROR_RESYNC 5 mirrors 000463 1774888353.649994 MIRROR_RESYNC 5 mirrors 000464 1774888354.327977 read 0x005398 thru 0x00c3a2 (0x700b bytes) 000465 1774888354.346995 mapread 0x006fb8 thru 0x00bbb3 (0x4bfc bytes) 000466 1774888354.348492 mapwrite 0x028751 thru 0x0370b2 (0xe962 bytes) 000470 1774888354.644176 mapwrite 0x00171d thru 0x008e84 (0x7768 bytes) 000471 1774888354.806882 read 0x033b7e thru 0x0370b2 (0x3535 bytes) 000473 1774888354.817320 mapread 0x00f7b7 thru 0x01d3e3 (0xdc2d bytes) 000474 1774888354.847198 mapwrite 0x007d11 thru 0x00c508 (0x47f8 bytes) 000475 1774888354.915391 mapread 0x00ff60 thru 0x01942b (0x94cc bytes) 000477 1774888354.924530 MIRROR_SPLIT mirror 4 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x370b3): Mirror 9: 0x33187ee4 (stale) Mirror 10: 0x33187ee4 (stale) Mirror 11: 0x33187ee4 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x370b3): Mirror 9: 0x33187ee4 (stale) Mirror 10: 0x33187ee4 (stale) Mirror 11: 0x33187ee4 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x370b3: skipped 000478 1774888355.292800 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x370b3): Mirror 10: 0x33187ee4 (stale) Mirror 11: 0x33187ee4 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x370b3): Mirror 10: 0x33187ee4 (stale) Mirror 11: 0x33187ee4 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x370b3: skipped 000481 1774888355.613386 write 0x00ba9b thru 0x00e652 (0x2bb8 bytes) 000483 1774888355.631366 MIRROR_RESYNC 3 mirrors 000485 1774888356.329411 read 0x008cd2 thru 0x011021 (0x8350 bytes) 000488 1774888356.340826 mapwrite 0x025151 thru 0x025f24 (0xdd4 bytes) 000491 1774888356.560447 mapwrite 0x01563c thru 0x022884 (0xd249 bytes) 000492 1774888356.720790 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x370b3): Mirror 11: 0xaebd215a (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x370b3): Mirror 11: 0xaebd215a (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x370b3: skipped 000493 1774888357.136655 mapread 0x0013b3 thru 0x00ce64 (0xbab2 bytes) 000494 1774888357.162763 read 0x0137df thru 0x0210b9 (0xd8db bytes) 000495 1774888357.178265 mapwrite 0x01389c thru 0x01f4b6 (0xbc1b bytes) 000496 1774888357.293905 mapread 0x027650 thru 0x02c6df (0x5090 bytes) 000498 1774888357.306804 trunc from 0x0370b3 to 0x027d6c (0xf346 bytes) 000499 1774888357.325930 mapread 0x012f9b thru 0x018aef (0x5b55 bytes) 000500 1774888357.339574 trunc from 0x027d6d to 0x03cff3 (0x15287 bytes) 000501 1774888357.368448 mapwrite 0x028417 thru 0x0304e0 (0x80ca bytes) 000502 1774888357.545997 trunc from 0x03cff4 to 0x022ce9 (0x1a30a bytes) 000504 1774888357.599827 write 0x011891 thru 0x015338 (0x3aa8 bytes) 000508 1774888357.611835 trunc from 0x022cea to 0x01478c (0xe55d bytes) 000509 1774888357.635163 read 0x010dde thru 0x012b4d (0x1d70 bytes) 000513 1774888357.652215 read 0x00c2bb thru 0x01127e (0x4fc4 bytes) 000514 1774888357.653969 write 0x025dbd thru 0x02c758 (0x699c bytes) 000515 1774888357.656825 write 0x03e3d4 thru 0x03ffff (0x1c2c bytes) 000516 1774888357.664350 trunc from 0x040000 to 0x00e684 (0x3197b bytes) 000518 1774888357.700357 read 0x001774 thru 0x00e684 (0xcf11 bytes) 000520 1774888357.704954 read 0x00c20e thru 0x00e684 (0x2477 bytes) 000522 1774888357.715201 read 0x00229b thru 0x003330 (0x1096 bytes) 000524 1774888357.721008 trunc from 0x00e685 to 0x0220c6 (0x13a42 bytes) 000526 1774888357.773472 mapwrite 0x027638 thru 0x030efb (0x98c4 bytes) 000527 1774888357.975402 read 0x02ede0 thru 0x030efb (0x211c bytes) 000528 1774888357.979610 read 0x00149d thru 0x00fab5 (0xe619 bytes) 000529 1774888357.980570 trunc from 0x030efc to 0x039723 (0x8828 bytes) 000530 1774888358.017377 read 0x0379c4 thru 0x039723 (0x1d60 bytes) 000532 1774888358.034229 write 0x00d1cf thru 0x016b8d (0x99bf bytes) 000533 1774888358.035487 read 0x0040c5 thru 0x010d3c (0xcc78 bytes) 000534 1774888358.038921 trunc from 0x039724 to 0x016544 (0x231df bytes) 000536 1774888358.065466 mapread 0x00482f thru 0x008631 (0x3e03 bytes) 000537 1774888358.066540 mapwrite 0x00874d thru 0x008df2 (0x6a6 bytes) 000538 1774888358.199318 trunc from 0x016545 to 0x02d18c (0x16c48 bytes) 000539 1774888358.243258 write 0x00a6f7 thru 0x00c761 (0x206b bytes) 000541 1774888358.244855 read 0x01c39e thru 0x01d2d9 (0xf3c bytes) 000542 1774888358.259757 mapread 0x015941 thru 0x01ccb4 (0x7374 bytes) 000546 1774888358.273122 mapread 0x00ff59 thru 0x01d389 (0xd431 bytes) 000547 1774888358.278935 write 0x01babf thru 0x01d68a (0x1bcc bytes) 000548 1774888358.282067 mapread 0x01b936 thru 0x020140 (0x480b bytes) 000549 1774888358.283753 mapwrite 0x025972 thru 0x02c4b2 (0x6b41 bytes) 000550 1774888358.546267 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x2d18d): Mirror 11: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2d18d): Mirror 11: 0x9629bc4d (stale) CRC-32 checksum value for chunk [0, 0x2d18d): Mirror 1: 0x6744cb6f Mirror 12: 0x6744cb6f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2d18d: skipped 000551 1774888359.987417 write 0x00b6b9 thru 0x010da4 (0x56ec bytes) 000552 1774888360.317168 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2d18d): Mirror 12: 0x6744cb6f (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2d18d): Mirror 12: 0x6744cb6f (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2d18d: skipped 000553 1774888361.068513 write 0x0002d3 thru 0x00aa69 (0xa797 bytes) 000556 1774888361.098603 mapread 0x00d204 thru 0x00d3c1 (0x1be bytes) 000557 1774888361.109881 mapread 0x01b58a thru 0x02060e (0x5085 bytes) 000558 1774888361.140705 mapread 0x01aec5 thru 0x0232c9 (0x8405 bytes) 000559 1774888361.146791 mapwrite 0x0016e9 thru 0x01145d (0xfd75 bytes) 000561 1774888361.366353 trunc from 0x02d18d to 0x03fe5f (0x12cd3 bytes) 000563 1774888361.431399 read 0x009fd9 thru 0x0139e0 (0x9a08 bytes) 000564 1774888361.439606 mapread 0x03e80c thru 0x03fe5f (0x1654 bytes) 000565 1774888361.459987 mapread 0x025ff2 thru 0x03452f (0xe53e bytes) 000567 1774888361.510868 mapwrite 0x0137e2 thru 0x01a501 (0x6d20 bytes) 000569 1774888361.693181 mapwrite 0x036a38 thru 0x037dd9 (0x13a2 bytes) 000570 1774888361.837274 mapread 0x018b2d thru 0x01f388 (0x685c bytes) 000571 1774888361.841316 read 0x0115d0 thru 0x01e612 (0xd043 bytes) 000572 1774888361.846724 read 0x014f61 thru 0x0205b4 (0xb654 bytes) 000573 1774888361.855634 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) 000577 1774888362.043326 mapwrite 0x01e5dd thru 0x0240c1 (0x5ae5 bytes) 000580 1774888362.217865 mapwrite 0x0202c7 thru 0x02efb9 (0xecf3 bytes) 000583 1774888362.422024 mapread 0x02cedd thru 0x02e0fe (0x1222 bytes) 000584 1774888362.423884 write 0x022f18 thru 0x031f94 (0xf07d bytes) 000585 1774888362.443525 MIRROR_EXTEND to 2 mirrors 000586 1774888363.433793 write 0x034b63 thru 0x037233 (0x26d1 bytes) 000587 1774888363.624160 mapwrite 0x036ec5 thru 0x038301 (0x143d bytes) 000588 1774888363.807790 read 0x016bbb thru 0x01884f (0x1c95 bytes) 000589 1774888363.826569 MIRROR_RESYNC 2 mirrors 000591 1774888364.878870 mapwrite 0x027859 thru 0x02a4b0 (0x2c58 bytes) 000592 1774888365.103197 trunc from 0x03fe60 to 0x032107 (0xdd58 bytes) 000593 1774888365.128621 read 0x018fa8 thru 0x0288e7 (0xf940 bytes) 000594 1774888365.149689 MIRROR_RESYNC 2 mirrors 000595 1774888366.297265 trunc from 0x032108 to 0x01ac17 (0x174f0 bytes) 000597 1774888366.558187 mapread 0x007268 thru 0x016e6e (0xfc07 bytes) 000601 1774888366.678542 mapread 0x018cf3 thru 0x0194d3 (0x7e1 bytes) 000602 1774888366.684466 trunc from 0x01ac18 to 0x0137be (0x7459 bytes) 000603 1774888366.735730 write 0x01b463 thru 0x024598 (0x9136 bytes) 000604 1774888366.740669 mapwrite 0x006132 thru 0x0128cf (0xc79e bytes) 000608 1774888367.020900 mapwrite 0x02c45a thru 0x03a03d (0xdbe4 bytes) 000611 1774888367.367688 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) 000613 1774888367.555354 trunc from 0x03a03e to 0x01be42 (0x1e1fb bytes) 000614 1774888367.599572 write 0x02113b thru 0x021aff (0x9c5 bytes) 000615 1774888367.602788 read 0x00d750 thru 0x0199b2 (0xc263 bytes) 000618 1774888367.625718 mapread 0x002e16 thru 0x003e43 (0x102e bytes) 000620 1774888367.647796 mapwrite 0x004952 thru 0x00d8f8 (0x8fa7 bytes) 000621 1774888367.767954 MIRROR_EXTEND to 2 mirrors 000623 1774888368.908373 mapwrite 0x03864b thru 0x03ffff (0x79b5 bytes) 000625 1774888369.311835 mapread 0x0035fe thru 0x00b5a9 (0x7fac bytes) 000626 1774888369.359243 trunc from 0x040000 to 0x020b85 (0x1f47a bytes) 000628 1774888369.446406 write 0x03d464 thru 0x03ffff (0x2b9c bytes) 000629 1774888369.474816 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x67862ad1 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x733a96dc Mirror 3: 0x733a96dc lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000631 1774888371.089332 write 0x032f96 thru 0x036966 (0x39d1 bytes) 000632 1774888371.280550 trunc from 0x040000 to 0x00bcfd (0x34302 bytes) 000636 1774888371.343691 read 0x007693 thru 0x00bcfd (0x466b bytes) 000637 1774888371.386913 MIRROR_RESYNC 3 mirrors 000638 1774888372.974594 MIRROR_RESYNC 3 mirrors 000640 1774888373.338128 mapread 0x004bab thru 0x00bcfd (0x7153 bytes) 000643 1774888373.365610 mapwrite 0x00e633 thru 0x0156dc (0x70aa bytes) 000644 1774888373.521064 mapread 0x014e74 thru 0x0156dc (0x869 bytes) 000645 1774888373.522116 trunc from 0x0156dd to 0x01b29b (0x5bbf bytes) 000646 1774888373.545507 write 0x004f79 thru 0x010b32 (0xbbba bytes) 000647 1774888373.559607 write 0x008200 thru 0x015ec5 (0xdcc6 bytes) 000648 1774888373.567543 mapwrite 0x007568 thru 0x00f9c7 (0x8460 bytes) 000649 1774888373.640669 mapread 0x0175dd thru 0x01b29b (0x3cbf bytes) 000651 1774888373.651247 trunc from 0x01b29c to 0x03aee0 (0x1fc45 bytes) 000652 1774888373.667523 mapread 0x032bfd thru 0x0388c6 (0x5cca bytes) 000653 1774888373.678637 write 0x027f36 thru 0x0312be (0x9389 bytes) 000654 1774888373.688878 mapwrite 0x0159ca thru 0x015f2f (0x566 bytes) 000655 1774888373.739940 read 0x021209 thru 0x027c94 (0x6a8c bytes) 000656 1774888373.741972 trunc from 0x03aee1 to 0x02511a (0x15dc6 bytes) 000657 1774888373.770548 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2511b): Mirror 2: 0x20b5b2c0 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2511b): Mirror 2: 0x20b5b2c0 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2511b: skipped 000658 1774888374.057886 write 0x0306aa thru 0x03542a (0x4d81 bytes) 000660 1774888374.059705 trunc from 0x03542b to 0x01f142 (0x162e8 bytes) 000663 1774888374.105582 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) 000667 1774888374.318646 write 0x03ee37 thru 0x03ffff (0x11c9 bytes) 000668 1774888374.324972 write 0x02869f thru 0x02c36b (0x3ccd bytes) 000669 1774888374.338584 read 0x010140 thru 0x018198 (0x8059 bytes) 000670 1774888374.360615 mapwrite 0x01cb6e thru 0x024619 (0x7aac bytes) 000672 1774888374.433559 trunc from 0x040000 to 0x013313 (0x2ccec bytes) 000673 1774888374.461820 read 0x00ba84 thru 0x013313 (0x7890 bytes) 000675 1774888374.475525 trunc from 0x013314 to 0x000034 (0x132df bytes) 000677 1774888374.496609 mapread 0x000020 thru 0x000034 (0x15 bytes) 000679 1774888374.498676 write 0x007166 thru 0x00f26b (0x8106 bytes) 000680 1774888374.502629 trunc from 0x00f26c to 0x0130cb (0x3e60 bytes) 000682 1774888374.524587 read 0x004846 thru 0x00c0b9 (0x7874 bytes) 000684 1774888374.537970 mapread 0x005ebb thru 0x00babe (0x5c04 bytes) 000685 1774888374.540595 write 0x01ea04 thru 0x02b1b4 (0xc7b1 bytes) 000686 1774888374.543556 mapwrite 0x012fa0 thru 0x01716f (0x41d0 bytes) 000688 1774888374.641003 read 0x01140c thru 0x01d978 (0xc56d bytes) 000690 1774888374.643397 write 0x0332da thru 0x03ffff (0xcd26 bytes) 000692 1774888374.648424 mapwrite 0x033518 thru 0x033785 (0x26e bytes) 000693 1774888374.700262 mapwrite 0x03c03a thru 0x03ffff (0x3fc6 bytes) 000696 1774888374.763449 mapwrite 0x0023b2 thru 0x0035a3 (0x11f2 bytes) 000697 1774888374.828165 trunc from 0x040000 to 0x0035de (0x3ca21 bytes) 000699 1774888374.853567 mapwrite 0x00969a thru 0x0134ad (0x9e14 bytes) 000701 1774888374.930619 mapread 0x00fa31 thru 0x0134ad (0x3a7d bytes) 000704 1774888374.933623 trunc from 0x0134ae to 0x0106de (0x2dcf bytes) 000705 1774888374.970442 mapwrite 0x010c8a thru 0x01f8a9 (0xec20 bytes) 000706 1774888375.210016 mapwrite 0x028bcc thru 0x0302a9 (0x76de bytes) 000707 1774888375.410900 mapwrite 0x0175ef thru 0x01da0b (0x641d bytes) 000708 1774888375.557803 mapread 0x025368 thru 0x02b983 (0x661c bytes) 000709 1774888375.558890 mapwrite 0x032556 thru 0x03a2a3 (0x7d4e bytes) 000713 1774888375.668325 mapwrite 0x010d42 thru 0x01a345 (0x9604 bytes) 000714 1774888375.730027 mapwrite 0x00d389 thru 0x016634 (0x92ac bytes) 000715 1774888375.789393 write 0x011388 thru 0x01ea19 (0xd692 bytes) 000716 1774888375.792374 mapread 0x00b01f thru 0x013fb2 (0x8f94 bytes) 000719 1774888375.796352 trunc from 0x03a2a4 to 0x017e56 (0x2244d bytes) 000721 1774888375.825816 write 0x0188d8 thru 0x0244aa (0xbbd3 bytes) 000722 1774888375.829467 write 0x0000be thru 0x00fb99 (0xfadc bytes) 000728 1774888375.836842 mapwrite 0x03fabf thru 0x03ffff (0x541 bytes) 000730 1774888375.945347 trunc from 0x040000 to 0x02e002 (0x11ffd bytes) 000731 1774888375.958713 read 0x011346 thru 0x01cc49 (0xb904 bytes) 000736 1774888375.961630 read 0x004003 thru 0x00b5f1 (0x75ef bytes) 000737 1774888375.963360 mapread 0x0204e1 thru 0x02a1dc (0x9cfc bytes) 000738 1774888375.972529 write 0x0279af thru 0x02bbef (0x4241 bytes) 000739 1774888375.973492 trunc from 0x02e003 to 0x014ed5 (0x1912d bytes) 000741 1774888375.989379 read 0x006ab6 thru 0x0116aa (0xabf5 bytes) 000743 1774888375.990477 read 0x00ffe5 thru 0x012b81 (0x2b9d bytes) 000745 1774888375.992371 read 0x00d570 thru 0x014ed5 (0x7966 bytes) 000746 1774888375.993273 write 0x01022d thru 0x01f031 (0xee05 bytes) 000747 1774888375.994499 trunc from 0x01f032 to 0x02dc0a (0xebd9 bytes) 000749 1774888376.015286 write 0x01a67f thru 0x01c8fc (0x227e bytes) 000750 1774888376.017352 mapread 0x010de9 thru 0x020087 (0xf29f bytes) 000751 1774888376.028330 read 0x0240b0 thru 0x02dc0a (0x9b5b bytes) 000752 1774888376.029002 trunc from 0x02dc0b to 0x03e9ca (0x10dc0 bytes) 000758 1774888376.058648 mapwrite 0x02dec2 thru 0x0313d0 (0x350f bytes) 000759 1774888376.127099 MIRROR_EXTEND to 2 mirrors 000760 1774888376.868948 MIRROR_RESYNC 2 mirrors 000762 1774888377.362441 mapwrite 0x0231ba thru 0x02925a (0x60a1 bytes) 000765 1774888377.637978 mapread 0x00db42 thru 0x01bd73 (0xe232 bytes) 000766 1774888377.663471 read 0x033f73 thru 0x03b596 (0x7624 bytes) 000767 1774888377.670965 mapread 0x00990b thru 0x00c8de (0x2fd4 bytes) 000768 1774888377.672621 read 0x00744d thru 0x00c607 (0x51bb bytes) 000770 1774888377.674964 write 0x023de4 thru 0x02a18b (0x63a8 bytes) 000771 1774888377.676988 read 0x025ad5 thru 0x029622 (0x3b4e bytes) 000773 1774888377.678504 write 0x01ffa6 thru 0x02fef7 (0xff52 bytes) 000774 1774888377.689420 read 0x006abd thru 0x009953 (0x2e97 bytes) 000775 1774888377.690532 trunc from 0x03e9cb to 0x00846d (0x3655d bytes) 000776 1774888377.741741 write 0x02d321 thru 0x031217 (0x3ef7 bytes) 000777 1774888377.742995 trunc from 0x031218 to 0x03be82 (0xac6b bytes) 000778 1774888377.768438 MIRROR_RESYNC 2 mirrors 000779 1774888378.569485 trunc from 0x03be83 to 0x007bdd (0x342a5 bytes) 000781 1774888378.664462 write 0x02c2e8 thru 0x034ed8 (0x8bf1 bytes) 000782 1774888378.673266 mapread 0x01f47b thru 0x02e014 (0xeb9a bytes) 000784 1774888378.734885 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x34ed9): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x34ed9): Mirror 2: 0xccbb2c02 (stale) CRC-32 checksum value for chunk [0, 0x34ed9): Mirror 1: 0x3a6e167e Mirror 3: 0x3a6e167e lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x34ed9: skipped 000785 1774888380.054329 mapread 0x00c2cc thru 0x01b6bb (0xf3f0 bytes) 000786 1774888380.142654 mapread 0x032012 thru 0x033443 (0x1432 bytes) 000787 1774888380.209613 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) 000791 1774888380.655417 MIRROR_RESYNC 2 mirrors 000793 1774888381.352174 trunc from 0x034ed9 to 0x03cdd7 (0x7eff bytes) 000794 1774888381.489335 trunc from 0x03cdd8 to 0x017e36 (0x24fa1 bytes) 000796 1774888381.513679 trunc from 0x017e37 to 0x01fdf0 (0x7fba bytes) 000797 1774888381.535427 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x1fdf1): Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1fdf1): Mirror 3: 0x90c5a035 (stale) CRC-32 checksum value for chunk [0, 0x1fdf1): Mirror 1: 0x90c5a035 Mirror 4: 0x90c5a035 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1fdf1: skipped 000800 1774888382.830345 mapread 0x00e86b thru 0x0106e8 (0x1e7e bytes) 000803 1774888382.963687 write 0x02f80f thru 0x0323bb (0x2bad bytes) 000805 1774888383.116228 write 0x03f4a4 thru 0x03ffff (0xb5c bytes) 000807 1774888383.124557 read 0x02604c thru 0x034cc2 (0xec77 bytes) 000808 1774888383.135353 read 0x02c6e5 thru 0x03c33a (0xfc56 bytes) 000809 1774888383.162575 mapread 0x016186 thru 0x016426 (0x2a1 bytes) 000813 1774888383.197403 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xabd1b5e8 (stale) Mirror 4: 0x64faa751 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x4135f94f Mirror 5: 0x4135f94f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000814 1774888384.227831 write 0x01b64c thru 0x0282fc (0xccb1 bytes) 000815 1774888384.431965 trunc from 0x040000 to 0x00d47c (0x32b83 bytes) 000816 1774888384.537291 trunc from 0x00d47d to 0x00c2c9 (0x11b3 bytes) 000817 1774888384.571377 MIRROR_RESYNC 4 mirrors 000818 1774888386.109661 trunc from 0x00c2ca to 0x001beb (0xa6de bytes) 000820 1774888386.210751 mapread 0x001080 thru 0x001beb (0xb6c bytes) 000822 1774888386.219168 mapread 0x0013ea thru 0x001beb (0x802 bytes) 000823 1774888386.224563 mapread 0x000b3c thru 0x001beb (0x10b0 bytes) 000826 1774888386.235981 read 0x001a7a thru 0x001beb (0x172 bytes) 000827 1774888386.245087 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1bec): Mirror 4: 0x5f59d7de (stale) Mirror 5: 0x5f59d7de (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1bec): Mirror 4: 0x5f59d7de (stale) Mirror 5: 0x5f59d7de (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1bec: skipped 000828 1774888386.831731 read 0x0002bf thru 0x001beb (0x192d bytes) 000829 1774888386.861202 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1bec): Mirror 5: 0x5f59d7de (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1bec): Mirror 5: 0x5f59d7de (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1bec: skipped 000830 1774888387.204686 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) 000831 1774888387.322431 trunc from 0x001bec to 0x002d7f (0x1194 bytes) 000833 1774888387.351734 mapwrite 0x037678 thru 0x037a15 (0x39e bytes) 000834 1774888387.495761 mapwrite 0x03caf3 thru 0x03ffff (0x350d bytes) 000835 1774888387.597681 mapread 0x021445 thru 0x02c6ce (0xb28a bytes) 000837 1774888387.629520 mapread 0x034b88 thru 0x03f910 (0xad89 bytes) 000838 1774888387.638779 read 0x001daa thru 0x00b366 (0x95bd bytes) 000839 1774888387.646987 mapwrite 0x02536c thru 0x032e4a (0xdadf bytes) 000841 1774888387.707644 write 0x003403 thru 0x0055cd (0x21cb bytes) 000843 1774888387.713343 mapread 0x026f57 thru 0x02fa72 (0x8b1c bytes) 000844 1774888387.715173 mapread 0x036ec4 thru 0x03ffff (0x913c bytes) 000845 1774888387.718358 trunc from 0x040000 to 0x01f982 (0x2067d bytes) 000847 1774888387.752172 MIRROR_EXTEND to 2 mirrors 000849 1774888388.687552 mapread 0x013cf1 thru 0x01c97a (0x8c8a bytes) 000850 1774888388.723827 mapread 0x0072c6 thru 0x00bc08 (0x4943 bytes) 000851 1774888388.754147 mapwrite 0x0343cb thru 0x03a94e (0x6584 bytes) 000852 1774888388.950888 MIRROR_RESYNC 2 mirrors 000853 1774888390.017602 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) 000856 1774888390.231609 trunc from 0x03a94f to 0x026019 (0x14935 bytes) 000858 1774888390.283264 mapread 0x011c8b thru 0x016274 (0x45ea bytes) 000859 1774888390.308594 mapread 0x015abb thru 0x023f83 (0xe4c9 bytes) 000860 1774888390.321599 write 0x00bc6e thru 0x014099 (0x842c bytes) 000862 1774888390.336368 trunc from 0x02601a to 0x00da01 (0x18618 bytes) 000864 1774888390.367508 read 0x00b85d thru 0x00da01 (0x21a5 bytes) 000865 1774888390.371886 mapwrite 0x03e82c thru 0x03ffff (0x17d4 bytes) 000866 1774888390.499333 MIRROR_EXTEND to 2 mirrors 000867 1774888391.041281 trunc from 0x040000 to 0x03abbe (0x5441 bytes) 000868 1774888391.147611 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3abbf): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3abbf): Mirror 2: 0x555f07d6 (stale) CRC-32 checksum value for chunk [0, 0x3abbf): Mirror 1: 0x555f07d6 Mirror 3: 0x555f07d6 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3abbf: skipped 000875 1774888392.475298 mapread 0x0195e2 thru 0x0252bf (0xbcde bytes) 000876 1774888392.534739 read 0x015152 thru 0x023481 (0xe330 bytes) 000877 1774888392.545099 write 0x036f3d thru 0x03ffff (0x90c3 bytes) 000879 1774888392.669337 trunc from 0x040000 to 0x03f973 (0x68c bytes) 000881 1774888392.697269 mapread 0x01e7d7 thru 0x029266 (0xaa90 bytes) 000890 1774888392.720685 read 0x026e73 thru 0x031af7 (0xac85 bytes) 000892 1774888392.722761 read 0x009a86 thru 0x00c54b (0x2ac6 bytes) 000893 1774888392.734367 mapread 0x03a747 thru 0x03f973 (0x522d bytes) 000895 1774888392.735981 write 0x0065e2 thru 0x010dfd (0xa81c bytes) 000898 1774888392.744190 MIRROR_RESYNC 3 mirrors 000899 1774888393.633721 trunc from 0x03f974 to 0x026293 (0x196e0 bytes) 000900 1774888393.736475 trunc from 0x026294 to 0x03f9c1 (0x1972e bytes) 000901 1774888393.784178 write 0x0248c4 thru 0x0333cd (0xeb0a bytes) 000902 1774888393.797391 mapwrite 0x0290a7 thru 0x038b1e (0xfa78 bytes) 000903 1774888393.904274 MIRROR_RESYNC 3 mirrors 000904 1774888394.800240 mapwrite 0x003250 thru 0x00f178 (0xbf29 bytes) 000907 1774888395.033372 mapread 0x032ee4 thru 0x03c2c8 (0x93e5 bytes) 000908 1774888395.067021 read 0x02d6c3 thru 0x0347b9 (0x70f7 bytes) 000911 1774888395.072981 write 0x03284e thru 0x03ba51 (0x9204 bytes) 000912 1774888395.076792 mapread 0x0191bb thru 0x026668 (0xd4ae bytes) 000914 1774888395.111078 mapread 0x013ff7 thru 0x01c0aa (0x80b4 bytes) 000915 1774888395.116332 write 0x01058d thru 0x011e5a (0x18ce bytes) 000916 1774888395.124184 trunc from 0x03f9c2 to 0x036984 (0x903d bytes) 000917 1774888395.170737 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x36985): Mirror 2: 0x3f770cbc (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x36985): Mirror 2: 0x3f770cbc (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x36985: skipped 000918 1774888395.492365 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) 000919 1774888395.654554 read 0x007b4c thru 0x014f57 (0xd40c bytes) 000921 1774888395.673809 MIRROR_EXTEND to 2 mirrors 000922 1774888396.660402 mapwrite 0x001954 thru 0x010a30 (0xf0dd bytes) 000923 1774888396.912084 read 0x006904 thru 0x00e384 (0x7a81 bytes) 000924 1774888396.914129 trunc from 0x036985 to 0x00dfbb (0x289c9 bytes) 000925 1774888396.939596 read 0x00cad5 thru 0x00dfbb (0x14e7 bytes) 000928 1774888396.943741 read 0x000f71 thru 0x008a6e (0x7afe bytes) 000929 1774888396.950704 write 0x03191e thru 0x03a778 (0x8e5b bytes) 000930 1774888396.953228 mapread 0x00a5e3 thru 0x00ca6a (0x2488 bytes) 000933 1774888396.961372 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3a779): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3a779): Mirror 2: 0xfcaac99f (stale) CRC-32 checksum value for chunk [0, 0x3a779): Mirror 1: 0xfe3aadc6 Mirror 3: 0xfe3aadc6 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3a779: skipped 000934 1774888397.912791 trunc from 0x03a779 to 0x012d57 (0x27a21 bytes) 000935 1774888398.084078 mapwrite 0x00e3e7 thru 0x00f2c7 (0xee1 bytes) 000937 1774888398.278343 trunc from 0x012d58 to 0x00e8be (0x4499 bytes) 000938 1774888398.327753 trunc from 0x00e8bf to 0x00ac1c (0x3ca2 bytes) 000939 1774888398.358265 read 0x0095e8 thru 0x00ac1c (0x1635 bytes) 000940 1774888398.366592 trunc from 0x00ac1d to 0x037d2b (0x2d10f bytes) 000941 1774888398.397121 write 0x02fa55 thru 0x02fccb (0x277 bytes) 000942 1774888398.419314 trunc from 0x037d2c to 0x01c268 (0x1bac3 bytes) 000943 1774888398.440595 mapwrite 0x01885e thru 0x019919 (0x10bc bytes) 000944 1774888398.604995 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1c269): Mirror 2: 0xc2c814da (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1c269): Mirror 2: 0xc2c814da (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1c269: skipped 000946 1774888398.932589 write 0x022ac9 thru 0x0232ea (0x822 bytes) 000948 1774888398.936688 read 0x00f689 thru 0x01ca48 (0xd3c0 bytes) 000951 1774888398.950766 write 0x033c9d thru 0x03edd7 (0xb13b bytes) 000952 1774888398.953662 write 0x009e46 thru 0x018b63 (0xed1e bytes) 000953 1774888398.960608 trunc from 0x03edd8 to 0x00e64f (0x30788 bytes) 000956 1774888398.981915 trunc from 0x00e650 to 0x00e667 (0x18 bytes) 000957 1774888399.000899 mapwrite 0x016269 thru 0x016c38 (0x9d0 bytes) 000963 1774888399.081746 mapwrite 0x014e2c thru 0x015abe (0xc93 bytes) 000964 1774888399.130498 mapwrite 0x03ad60 thru 0x03ffff (0x52a0 bytes) 000965 1774888399.220502 read 0x02ce91 thru 0x03ac6f (0xdddf bytes) 000967 1774888399.231715 mapread 0x03eef1 thru 0x03ffff (0x110f bytes) 000969 1774888399.237855 MIRROR_RESYNC 2 mirrors 000971 1774888400.105894 write 0x00b7d0 thru 0x01403a (0x886b bytes) 000975 1774888400.253988 mapread 0x00e60a thru 0x01dee3 (0xf8da bytes) 000978 1774888400.324023 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x5f3c2f66 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x1ae61319 Mirror 3: 0x1ae61319 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000980 1774888401.254298 mapread 0x036435 thru 0x03950b (0x30d7 bytes) 000981 1774888401.298163 read 0x01e442 thru 0x02d6db (0xf29a bytes) 000982 1774888401.333620 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x191:0x0]: Invalid argument (22) 000985 1774888401.977085 mapread 0x039ab6 thru 0x03eea8 (0x53f3 bytes) 000987 1774888402.050700 read 0x006738 thru 0x00efb0 (0x8879 bytes) 000988 1774888402.093669 trunc from 0x040000 to 0x002610 (0x3d9ef bytes) 000989 1774888402.364274 write 0x015837 thru 0x01b444 (0x5c0e bytes) 000990 1774888402.366893 write 0x0158b9 thru 0x01f18a (0x98d2 bytes) 000994 1774888402.372589 trunc from 0x01f18b to 0x01a715 (0x4a75 bytes) 000995 1774888402.421875 write 0x02eaf0 thru 0x032f73 (0x4484 bytes) 000997 1774888402.435153 MIRROR_RESYNC 2 mirrors 000998 1774888403.169401 mapwrite 0x02a2d6 thru 0x02b4ed (0x1218 bytes) 000999 1774888403.417976 trunc from 0x032f74 to 0x00eaaf (0x244c4 bytes) 001000 1774888403.449896 mapwrite 0x0347c9 thru 0x03de63 (0x969b bytes) All operations completed A-OK!