== sanity-flr test 70a: flr mode fsx test ================ 10:24:43 (1783952683) Seed set to 2685 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000005 1783952686.289299 mapwrite 0x02425b thru 0x0324d2 (0xe278 bytes) truncating to largest ever: 0x269a8 000006 1783952686.504147 trunc from 0x0324d3 to 0x0269a7 (0xbb2b bytes) 000007 1783952686.545453 mapwrite 0x00606d thru 0x0087a2 (0x2736 bytes) 000008 1783952686.665205 trunc from 0x0269a8 to 0x01d6f0 (0x92b7 bytes) 000009 1783952686.710759 MIRROR_EXTEND to 2 mirrors truncating to largest ever: 0x3bbf3 000011 1783952687.947350 trunc from 0x01d6f1 to 0x03bbf2 (0x1e502 bytes) 000016 1783952688.138959 mapread 0x003ede thru 0x00a93b (0x6a5e bytes) 000017 1783952688.195239 read 0x01a4a1 thru 0x01e658 (0x41b8 bytes) 000018 1783952688.215434 trunc from 0x03bbf3 to 0x002d8e (0x38e64 bytes) 000019 1783952688.266343 read 0x000555 thru 0x002d8e (0x283a bytes) 000020 1783952688.278271 mapwrite 0x03c9d5 thru 0x03eac2 (0x20ee bytes) 000023 1783952688.430502 trunc from 0x03eac3 to 0x0170ef (0x279d3 bytes) 000024 1783952688.505434 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x170f0): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x170f0): Mirror 2: 0x6db7beb2 (stale) CRC-32 checksum value for chunk [0, 0x170f0): Mirror 1: 0xd4a55393 Mirror 3: 0xd4a55393 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x170f0: skipped 000025 1783952690.015657 read 0x00a4c3 thru 0x010d00 (0x683e bytes) 000028 1783952690.030524 write 0x00ef42 thru 0x00f87e (0x93d bytes) 000034 1783952690.144198 mapread 0x0114fd thru 0x016091 (0x4b95 bytes) 000035 1783952690.168404 mapwrite 0x0019a5 thru 0x0082bf (0x691b bytes) 000036 1783952690.316118 MIRROR_RESYNC 3 mirrors 000039 1783952691.388212 read 0x0144a3 thru 0x0170ef (0x2c4d bytes) 000040 1783952691.400111 read 0x00f227 thru 0x0170ef (0x7ec9 bytes) 000044 1783952691.430457 read 0x013585 thru 0x016624 (0x30a0 bytes) 000047 1783952691.435582 read 0x00842c thru 0x00f37b (0x6f50 bytes) 000049 1783952691.441945 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) 000051 1783952691.990660 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) 000052 1783952692.221512 read 0x002b3b thru 0x005b9a (0x3060 bytes) 000054 1783952692.237664 trunc from 0x0170f0 to 0x03963d (0x2254e bytes) 000055 1783952692.268868 mapwrite 0x004574 thru 0x00688f (0x231c bytes) 000058 1783952692.346536 mapread 0x0292af thru 0x02c9ac (0x36fe bytes) 000059 1783952692.356833 write 0x03f2a3 thru 0x03ffff (0xd5d bytes) 000060 1783952692.358209 trunc from 0x040000 to 0x024610 (0x1b9ef bytes) 000062 1783952692.385068 trunc from 0x024611 to 0x014b17 (0xfaf9 bytes) 000063 1783952692.414473 mapread 0x0021b7 thru 0x00c1a8 (0x9ff2 bytes) 000064 1783952692.417056 trunc from 0x014b18 to 0x013cc9 (0xe4e bytes) 000065 1783952692.441274 trunc from 0x013cca to 0x003f72 (0xfd57 bytes) 000066 1783952692.466435 trunc from 0x003f73 to 0x0294b9 (0x25547 bytes) 000067 1783952692.488897 trunc from 0x0294ba to 0x02db93 (0x46da bytes) 000072 1783952692.526601 write 0x000836 thru 0x00c3b6 (0xbb81 bytes) 000075 1783952692.547091 mapread 0x02c454 thru 0x02db93 (0x1740 bytes) 000077 1783952692.568884 mapwrite 0x02706e thru 0x035edf (0xee72 bytes) 000080 1783952692.733866 read 0x035894 thru 0x035edf (0x64c bytes) 000081 1783952692.738765 mapwrite 0x01fd6f thru 0x0239e4 (0x3c76 bytes) 000082 1783952692.932780 read 0x029dda thru 0x02a012 (0x239 bytes) 000084 1783952692.944649 read 0x003c02 thru 0x007994 (0x3d93 bytes) 000085 1783952692.952436 write 0x00eda9 thru 0x00ef5c (0x1b4 bytes) 000088 1783952692.966252 mapread 0x003944 thru 0x00823d (0x48fa bytes) 000091 1783952692.970240 mapread 0x02e199 thru 0x033597 (0x53ff bytes) 000092 1783952692.972255 trunc from 0x035ee0 to 0x033279 (0x2c66 bytes) 000093 1783952693.003405 mapread 0x014d52 thru 0x01e9a6 (0x9c55 bytes) 000096 1783952693.016936 read 0x01e12e thru 0x020b9a (0x2a6d bytes) 000097 1783952693.017980 trunc from 0x03327a to 0x019bbd (0x196bc bytes) 000098 1783952693.046746 mapread 0x006815 thru 0x00b304 (0x4af0 bytes) 000101 1783952693.069595 MIRROR_EXTEND to 2 mirrors 000102 1783952694.132779 mapread 0x017b90 thru 0x019168 (0x15d9 bytes) 000103 1783952694.170736 mapwrite 0x009657 thru 0x00a126 (0xad0 bytes) 000105 1783952694.342181 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) 000106 1783952694.504132 write 0x004cb8 thru 0x012547 (0xd890 bytes) 000107 1783952694.529957 mapread 0x006548 thru 0x009909 (0x33c2 bytes) 000108 1783952694.531598 write 0x00d77c thru 0x01d707 (0xff8c bytes) 000109 1783952694.535038 trunc from 0x01d708 to 0x00de31 (0xf8d6 bytes) 000110 1783952694.604537 trunc from 0x00de32 to 0x029de9 (0x1bfb8 bytes) truncating to largest ever: 0x3cb0c 000112 1783952694.638800 trunc from 0x029dea to 0x03cb0b (0x12d22 bytes) 000114 1783952694.683380 trunc from 0x03cb0c to 0x000f14 (0x3bbf7 bytes) 000117 1783952694.718462 write 0x006edf thru 0x00ae21 (0x3f43 bytes) 000122 1783952694.721420 write 0x02cbc2 thru 0x03039b (0x37da bytes) 000124 1783952694.723085 mapread 0x00e83d thru 0x0110b5 (0x2879 bytes) 000125 1783952694.752037 mapread 0x01522a thru 0x022a3f (0xd816 bytes) 000127 1783952694.794538 write 0x036d4a thru 0x0391a7 (0x245e bytes) 000129 1783952694.798492 trunc from 0x0391a8 to 0x019834 (0x1f973 bytes) 000131 1783952694.851990 read 0x008a56 thru 0x017b0c (0xf0b7 bytes) 000132 1783952694.853482 read 0x012675 thru 0x01339d (0xd29 bytes) 000134 1783952694.859475 mapwrite 0x00c145 thru 0x013bc9 (0x7a85 bytes) 000135 1783952695.006526 trunc from 0x019835 to 0x00f520 (0xa314 bytes) 000137 1783952695.061268 read 0x004e4f thru 0x00d90f (0x8ac1 bytes) 000138 1783952695.086536 trunc from 0x00f521 to 0x03be63 (0x2c943 bytes) 000139 1783952695.110479 read 0x00cc86 thru 0x015c2e (0x8fa9 bytes) 000140 1783952695.161185 read 0x014b87 thru 0x01d5a6 (0x8a20 bytes) 000141 1783952695.162101 trunc from 0x03be64 to 0x02d705 (0xe75e bytes) 000142 1783952695.197512 read 0x0251d7 thru 0x02d705 (0x852f bytes) 000143 1783952695.198481 write 0x01f9e5 thru 0x022aaf (0x30cb bytes) 000145 1783952695.207018 mapread 0x00f93c thru 0x0158a1 (0x5f66 bytes) 000149 1783952695.221914 trunc from 0x02d706 to 0x01d5f1 (0x10114 bytes) 000150 1783952695.273998 trunc from 0x01d5f2 to 0x00819f (0x15452 bytes) 000151 1783952695.326616 mapread 0x001b31 thru 0x00819f (0x666f bytes) 000153 1783952695.335625 trunc from 0x0081a0 to 0x02b640 (0x234a1 bytes) 000154 1783952695.380420 read 0x004fde thru 0x014907 (0xf92a bytes) 000157 1783952695.420436 write 0x01db89 thru 0x02cb3a (0xefb2 bytes) 000160 1783952695.459130 mapwrite 0x0072f0 thru 0x00e2af (0x6fc0 bytes) 000162 1783952695.619920 write 0x03a5a8 thru 0x03ffff (0x5a58 bytes) 000163 1783952695.621752 trunc from 0x040000 to 0x039318 (0x6ce7 bytes) 000165 1783952695.652992 trunc from 0x039319 to 0x0120cd (0x2724b bytes) 000169 1783952695.707159 mapread 0x0094f4 thru 0x00d744 (0x4251 bytes) 000170 1783952695.719341 read 0x011060 thru 0x0120cd (0x106e bytes) 000171 1783952695.723427 mapread 0x00e2d7 thru 0x0120cd (0x3df7 bytes) 000174 1783952695.730463 write 0x017888 thru 0x022f17 (0xb690 bytes) 000178 1783952695.738319 trunc from 0x022f18 to 0x0344ae (0x11597 bytes) 000180 1783952695.796280 read 0x0072fe thru 0x00fa83 (0x8786 bytes) 000181 1783952695.798099 mapwrite 0x004d8f thru 0x00511d (0x38f bytes) 000182 1783952696.003394 mapwrite 0x00c82d thru 0x0172cc (0xaaa0 bytes) 000184 1783952696.134155 mapwrite 0x00af8b thru 0x00b7dd (0x853 bytes) 000185 1783952696.179295 write 0x02ff13 thru 0x031059 (0x1147 bytes) 000186 1783952696.180563 trunc from 0x0344af to 0x032df5 (0x16b9 bytes) 000187 1783952696.210453 mapwrite 0x036965 thru 0x03d223 (0x68bf bytes) 000188 1783952696.359537 trunc from 0x03d224 to 0x0369cb (0x6858 bytes) 000191 1783952696.417580 read 0x00b963 thru 0x00f177 (0x3815 bytes) 000197 1783952696.425262 trunc from 0x0369cc to 0x016d23 (0x1fca8 bytes) 000198 1783952696.460252 read 0x00b20b thru 0x01227f (0x7075 bytes) 000200 1783952696.463340 read 0x00de7a thru 0x016d23 (0x8eaa bytes) 000202 1783952696.468492 mapwrite 0x025fda thru 0x0346ef (0xe716 bytes) truncating to largest ever: 0x3f367 000203 1783952696.622128 trunc from 0x0346f0 to 0x03f366 (0xac77 bytes) 000205 1783952696.668369 mapread 0x037d65 thru 0x03e6a5 (0x6941 bytes) 000207 1783952696.702908 read 0x020d8d thru 0x02db1b (0xcd8f bytes) 000208 1783952696.704146 write 0x00fab1 thru 0x0126d9 (0x2c29 bytes) 000209 1783952696.706419 mapwrite 0x03feb8 thru 0x03ffff (0x148 bytes) 000212 1783952696.812957 mapread 0x03f65f thru 0x03ffff (0x9a1 bytes) 000213 1783952696.814956 mapwrite 0x033f8b thru 0x03ffff (0xc075 bytes) 000214 1783952696.923427 mapwrite 0x01f19e thru 0x02d5b2 (0xe415 bytes) 000215 1783952697.020635 read 0x00ceca thru 0x017961 (0xaa98 bytes) 000217 1783952697.024135 read 0x005814 thru 0x00a64d (0x4e3a bytes) 000219 1783952697.030525 write 0x0217eb thru 0x026037 (0x484d bytes) 000220 1783952697.046100 MIRROR_EXTEND to 2 mirrors 000221 1783952697.803349 write 0x005952 thru 0x011100 (0xb7af bytes) 000222 1783952697.892474 write 0x03b73a thru 0x03ffff (0x48c6 bytes) 000225 1783952697.917136 read 0x001320 thru 0x0057bf (0x44a0 bytes) 000226 1783952697.937171 trunc from 0x040000 to 0x0281aa (0x17e55 bytes) 000228 1783952698.059905 write 0x02145c thru 0x030e76 (0xfa1b bytes) 000232 1783952698.076130 read 0x025bf0 thru 0x030e76 (0xb287 bytes) 000234 1783952698.089010 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) 000236 1783952698.394437 trunc from 0x030e77 to 0x03f32c (0xe4b6 bytes) 000237 1783952698.429660 read 0x03a6ba thru 0x03e8ee (0x4235 bytes) 000238 1783952698.446260 mapwrite 0x00db0b thru 0x015a56 (0x7f4c bytes) 000239 1783952698.644961 write 0x039208 thru 0x03ffff (0x6df8 bytes) 000241 1783952698.657789 mapread 0x01e0cd thru 0x02248a (0x43be bytes) 000242 1783952698.672002 MIRROR_EXTEND to 2 mirrors 000243 1783952699.825975 mapread 0x01fb92 thru 0x0214d2 (0x1941 bytes) 000244 1783952699.843491 write 0x016f69 thru 0x022514 (0xb5ac bytes) 000246 1783952699.921319 MIRROR_RESYNC 2 mirrors 000248 1783952701.041505 write 0x015df3 thru 0x025c42 (0xfe50 bytes) 000252 1783952701.185635 trunc from 0x040000 to 0x013c56 (0x2c3a9 bytes) 000253 1783952701.223798 trunc from 0x013c57 to 0x02071c (0xcac6 bytes) 000255 1783952701.255078 mapread 0x009309 thru 0x016553 (0xd24b bytes) 000256 1783952701.289974 read 0x017e42 thru 0x02071c (0x88db bytes) 000257 1783952701.297952 write 0x00fe2d thru 0x01d3cb (0xd59f bytes) 000258 1783952701.309772 mapread 0x011126 thru 0x01ce35 (0xbd10 bytes) 000259 1783952701.339762 MIRROR_RESYNC 2 mirrors 000260 1783952702.393620 mapwrite 0x0182aa thru 0x026746 (0xe49d bytes) 000262 1783952702.736216 mapwrite 0x007e34 thru 0x017117 (0xf2e4 bytes) 000263 1783952702.948801 mapwrite 0x0086f8 thru 0x011004 (0x890d bytes) 000264 1783952703.016484 mapwrite 0x0158e9 thru 0x022ddb (0xd4f3 bytes) 000265 1783952703.097379 trunc from 0x026747 to 0x031de5 (0xb69f bytes) 000266 1783952703.143768 MIRROR_RESYNC 2 mirrors 000268 1783952703.931494 mapread 0x00a974 thru 0x00df68 (0x35f5 bytes) 000269 1783952703.978181 read 0x00686f thru 0x00f6c4 (0x8e56 bytes) 000272 1783952703.981583 mapwrite 0x020a22 thru 0x0274ae (0x6a8d bytes) 000273 1783952704.122096 mapwrite 0x001ae2 thru 0x0107b7 (0xecd6 bytes) 000274 1783952704.245165 mapread 0x027b51 thru 0x02ce87 (0x5337 bytes) 000275 1783952704.261042 mapwrite 0x02270c thru 0x024cc4 (0x25b9 bytes) 000276 1783952704.330427 read 0x016b41 thru 0x0250e5 (0xe5a5 bytes) 000277 1783952704.331818 mapread 0x0207e3 thru 0x02120f (0xa2d bytes) 000278 1783952704.335090 read 0x01c059 thru 0x01f900 (0x38a8 bytes) 000279 1783952704.335962 trunc from 0x031de6 to 0x01f488 (0x1295d bytes) 000281 1783952704.369371 write 0x012790 thru 0x02068b (0xdefc bytes) 000282 1783952704.380781 MIRROR_RESYNC 2 mirrors 000285 1783952705.151442 read 0x015697 thru 0x02068b (0xaff5 bytes) 000286 1783952705.166598 trunc from 0x02068c to 0x00d8e5 (0x12da6 bytes) 000288 1783952705.294996 mapread 0x00aa12 thru 0x00d8e5 (0x2ed4 bytes) 000290 1783952705.349500 trunc from 0x00d8e6 to 0x021865 (0x13f80 bytes) 000291 1783952705.392019 read 0x015b87 thru 0x017be0 (0x205a bytes) 000292 1783952705.408286 write 0x00e810 thru 0x014b89 (0x637a bytes) 000293 1783952705.409715 mapwrite 0x01ce5f thru 0x01f58c (0x272e bytes) 000294 1783952705.531586 mapread 0x00ff83 thru 0x01890d (0x898b bytes) 000295 1783952705.546096 MIRROR_RESYNC 2 mirrors 000297 1783952706.257429 MIRROR_RESYNC 2 mirrors 000299 1783952706.689796 write 0x02e3a0 thru 0x03b82c (0xd48d bytes) 000300 1783952706.803990 write 0x03000b thru 0x03e062 (0xe058 bytes) 000303 1783952706.809257 trunc from 0x03e063 to 0x009616 (0x34a4c bytes) 000305 1783952706.887898 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) 000306 1783952707.072051 MIRROR_EXTEND to 2 mirrors 000309 1783952707.990231 mapwrite 0x003d78 thru 0x011ecc (0xe155 bytes) 000310 1783952708.378325 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x11ecd): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x11ecd): Mirror 2: 0xe7809097 (stale) CRC-32 checksum value for chunk [0, 0x11ecd): Mirror 1: 0xd9d9c62f Mirror 3: 0xd9d9c62f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x11ecd: skipped 000312 1783952709.289961 write 0x009b85 thru 0x00ba57 (0x1ed3 bytes) 000313 1783952709.369652 write 0x00ee80 thru 0x018903 (0x9a84 bytes) 000314 1783952709.378317 write 0x02acb5 thru 0x032fe6 (0x8332 bytes) 000315 1783952709.380064 read 0x013484 thru 0x017743 (0x42c0 bytes) 000319 1783952709.382877 mapwrite 0x03f67f thru 0x03ffff (0x981 bytes) 000321 1783952709.472119 read 0x038bac thru 0x03ffff (0x7454 bytes) 000323 1783952709.484417 trunc from 0x040000 to 0x02d898 (0x12767 bytes) 000324 1783952709.505832 trunc from 0x02d899 to 0x026d9c (0x6afc bytes) 000325 1783952709.541291 mapread 0x02626e thru 0x026d9c (0xb2f bytes) 000327 1783952709.543421 trunc from 0x026d9d to 0x0256a4 (0x16f8 bytes) 000328 1783952709.572125 mapwrite 0x018d22 thru 0x01f7d1 (0x6ab0 bytes) 000330 1783952709.647870 read 0x00fc13 thru 0x01d511 (0xd8ff bytes) 000331 1783952709.648959 write 0x0302fe thru 0x035edd (0x5be0 bytes) 000332 1783952709.656481 mapread 0x00e0d3 thru 0x00ee10 (0xd3e bytes) 000336 1783952709.687660 read 0x025a39 thru 0x02a7a6 (0x4d6e bytes) 000337 1783952709.717878 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x35ede): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x35ede): Mirror 2: 0xeb7445e7 (stale) Mirror 3: 0x82f07273 (stale) CRC-32 checksum value for chunk [0, 0x35ede): Mirror 1: 0xf74d3ebb Mirror 4: 0xf74d3ebb lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x35ede: skipped 000338 1783952710.802002 MIRROR_RESYNC 4 mirrors 000339 1783952712.154187 trunc from 0x035ede to 0x01ba3e (0x1a49f bytes) 000340 1783952712.288510 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1ba3f): Mirror 3: 0xf5178bfc (stale) Mirror 4: 0xf5178bfc (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1ba3f): Mirror 3: 0xf5178bfc (stale) Mirror 4: 0xf5178bfc (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1ba3f: skipped 000341 1783952712.727000 read 0x00706e thru 0x0104f3 (0x9486 bytes) 000342 1783952712.753490 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x1ba3f): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1ba3f): Mirror 3: 0xf5178bfc (stale) Mirror 4: 0xf5178bfc (stale) CRC-32 checksum value for chunk [0, 0x1ba3f): Mirror 1: 0xf5178bfc Mirror 5: 0xf5178bfc lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1ba3f: skipped 000343 1783952713.978086 mapwrite 0x015861 thru 0x018ba3 (0x3343 bytes) 000344 1783952714.184077 write 0x0115f1 thru 0x0178a9 (0x62b9 bytes) 000345 1783952714.189045 write 0x02b859 thru 0x0348b7 (0x905f bytes) 000346 1783952714.191328 trunc from 0x0348b8 to 0x012b24 (0x21d93 bytes) 000347 1783952714.242299 mapwrite 0x0075e1 thru 0x00c5f1 (0x5011 bytes) 000348 1783952714.359609 write 0x02906b thru 0x02b632 (0x25c8 bytes) 000349 1783952714.362000 mapread 0x020f3f thru 0x02b632 (0xa6f4 bytes) 000352 1783952714.414532 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2b633): Mirror 4: 0x473c4f24 (stale) Mirror 5: 0x8b7a57f1 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2b633): Mirror 4: 0x473c4f24 (stale) Mirror 5: 0x8b7a57f1 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2b633: skipped 000353 1783952714.831317 mapwrite 0x015d8c thru 0x01a7e4 (0x4a59 bytes) 000354 1783952714.919482 trunc from 0x02b633 to 0x0361c0 (0xab8e bytes) 000355 1783952714.964160 trunc from 0x0361c1 to 0x00afb7 (0x2b209 bytes) 000356 1783952715.012078 trunc from 0x00afb8 to 0x012afa (0x7b43 bytes) 000357 1783952715.044485 mapread 0x008dff thru 0x00fc6c (0x6e6e bytes) 000359 1783952715.065354 trunc from 0x012afb to 0x0380d1 (0x255d7 bytes) 000360 1783952715.083603 write 0x024ecc thru 0x03337e (0xe4b3 bytes) 000361 1783952715.100930 read 0x030870 thru 0x030a67 (0x1f8 bytes) 000362 1783952715.102307 write 0x000162 thru 0x00344e (0x32ed bytes) 000367 1783952715.115036 mapwrite 0x00d60c thru 0x01a167 (0xcb5c bytes) 000370 1783952715.228037 write 0x03e75d thru 0x03ffff (0x18a3 bytes) 000371 1783952715.239237 MIRROR_RESYNC 3 mirrors 000373 1783952716.116604 read 0x00d379 thru 0x011c9c (0x4924 bytes) 000375 1783952716.135484 mapwrite 0x0280f2 thru 0x037eae (0xfdbd bytes) 000383 1783952716.440943 mapread 0x00627a thru 0x007053 (0xdda bytes) 000384 1783952716.458882 trunc from 0x040000 to 0x038fd9 (0x7026 bytes) 000385 1783952716.517384 write 0x02c618 thru 0x02ee0e (0x27f7 bytes) 000386 1783952716.518546 read 0x025034 thru 0x02f87a (0xa847 bytes) 000387 1783952716.523221 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x38fda): Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x38fda): Mirror 4: 0x503c4ea4 (stale) Mirror 5: 0x503c4ea4 (stale) CRC-32 checksum value for chunk [0, 0x38fda): Mirror 1: 0xcdfba4c9 Mirror 6: 0xcdfba4c9 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x38fda: skipped 000389 1783952717.650058 trunc from 0x038fda to 0x03012c (0x8ead bytes) 000390 1783952717.857357 read 0x00af2d thru 0x014e54 (0x9f28 bytes) 000391 1783952717.874362 trunc from 0x03012d to 0x01d5a3 (0x12b89 bytes) 000394 1783952717.927872 mapwrite 0x010673 thru 0x0180dc (0x7a6a bytes) 000395 1783952718.094056 write 0x0266e0 thru 0x028fac (0x28cd bytes) 000396 1783952718.094959 write 0x00f514 thru 0x012532 (0x301f bytes) 000400 1783952718.100000 mapwrite 0x0361ae thru 0x03ffff (0x9e52 bytes) 000401 1783952718.276058 mapwrite 0x021766 thru 0x028d6a (0x7605 bytes) 000403 1783952718.390082 read 0x01be89 thru 0x02bc08 (0xfd80 bytes) 000404 1783952718.401157 write 0x036100 thru 0x03dbc7 (0x7ac8 bytes) 000407 1783952718.405807 read 0x02b7db thru 0x0308a2 (0x50c8 bytes) 000409 1783952718.407925 read 0x0077d8 thru 0x015cc2 (0xe4eb bytes) 000410 1783952718.408847 trunc from 0x040000 to 0x0023f2 (0x3dc0d bytes) 000412 1783952718.448105 write 0x006425 thru 0x007052 (0xc2e bytes) 000413 1783952718.449209 write 0x02a00c thru 0x02a149 (0x13e bytes) 000414 1783952718.462094 trunc from 0x02a14a to 0x01fb43 (0xa606 bytes) 000417 1783952718.509580 trunc from 0x01fb44 to 0x029e12 (0xa2cf bytes) 000421 1783952718.563533 mapwrite 0x00f04f thru 0x015d25 (0x6cd7 bytes) 000422 1783952718.705103 MIRROR_RESYNC 4 mirrors 000423 1783952719.841016 mapwrite 0x02bece thru 0x02e380 (0x24b3 bytes) 000425 1783952720.064983 read 0x0135e4 thru 0x0151a8 (0x1bc5 bytes) 000429 1783952720.076025 write 0x01e29f thru 0x020cae (0x2a10 bytes) 000431 1783952720.087634 mapwrite 0x009334 thru 0x018b5c (0xf829 bytes) 000435 1783952720.173289 MIRROR_RESYNC 4 mirrors 000436 1783952721.207116 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) 000438 1783952721.701336 mapread 0x0017ee thru 0x003a02 (0x2215 bytes) 000441 1783952721.724221 write 0x02c3c7 thru 0x039e26 (0xda60 bytes) 000442 1783952721.871205 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x39e27): Mirror 5: 0x6f7d7259 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x39e27): Mirror 5: 0x6f7d7259 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x39e27: skipped 000447 1783952722.326271 read 0x01ba5f thru 0x022246 (0x67e8 bytes) 000448 1783952722.348436 MIRROR_RESYNC 2 mirrors 000450 1783952723.601292 mapread 0x02cbc0 thru 0x036704 (0x9b45 bytes) 000452 1783952723.642577 mapwrite 0x021036 thru 0x02f260 (0xe22b bytes) 000453 1783952724.011389 write 0x014be7 thru 0x023297 (0xe6b1 bytes) 000454 1783952724.021246 mapwrite 0x033500 thru 0x03c083 (0x8b84 bytes) 000457 1783952724.353860 write 0x00259b thru 0x009889 (0x72ef bytes) 000458 1783952724.372333 trunc from 0x03c084 to 0x02b7f8 (0x1088b bytes) 000459 1783952724.442553 mapwrite 0x018e86 thru 0x021404 (0x857f bytes) 000460 1783952724.625967 read 0x0177c4 thru 0x01a41f (0x2c5c bytes) 000461 1783952724.626734 write 0x0052d7 thru 0x00bf79 (0x6ca3 bytes) 000462 1783952724.627887 trunc from 0x02b7f9 to 0x01d1cb (0xe62d bytes) 000463 1783952724.668237 trunc from 0x01d1cc to 0x0014ed (0x1bcde bytes) 000465 1783952724.716375 read 0x00120e thru 0x0014ed (0x2e0 bytes) 000466 1783952724.726397 trunc from 0x0014ee to 0x00bcff (0xa812 bytes) 000467 1783952724.749984 mapread 0x000c6f thru 0x00ad43 (0xa0d5 bytes) 000469 1783952724.775988 mapread 0x00b707 thru 0x00bcff (0x5f9 bytes) 000471 1783952724.780795 read 0x004b2c thru 0x007008 (0x24dd bytes) 000472 1783952724.796185 MIRROR_RESYNC 2 mirrors 000473 1783952725.686143 read 0x00a435 thru 0x00bcff (0x18cb bytes) 000474 1783952725.692545 mapread 0x006e71 thru 0x009354 (0x24e4 bytes) 000476 1783952725.715815 trunc from 0x00bd00 to 0x01b16a (0xf46b bytes) 000477 1783952725.804013 trunc from 0x01b16b to 0x001f0d (0x1925d bytes) 000478 1783952725.832966 write 0x02fcd1 thru 0x039f66 (0xa296 bytes) 000479 1783952725.846578 mapwrite 0x033166 thru 0x03995c (0x67f7 bytes) 000480 1783952726.009076 mapwrite 0x02e598 thru 0x0315f7 (0x3060 bytes) 000484 1783952726.244340 trunc from 0x039f67 to 0x03eedf (0x4f79 bytes) 000485 1783952726.293176 mapread 0x03dc8c thru 0x03eedf (0x1254 bytes) 000486 1783952726.347188 MIRROR_RESYNC 2 mirrors 000487 1783952727.419604 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) 000490 1783952727.675170 mapread 0x014e93 thru 0x021bbb (0xcd29 bytes) 000491 1783952727.759987 trunc from 0x03eee0 to 0x011857 (0x2d688 bytes) 000493 1783952727.879952 trunc from 0x011858 to 0x019b73 (0x831c bytes) 000495 1783952727.923885 read 0x001d12 thru 0x006228 (0x4517 bytes) 000496 1783952727.938065 mapread 0x0139ea thru 0x019b73 (0x618a bytes) 000497 1783952727.952604 mapwrite 0x0080fa thru 0x00ac93 (0x2b9a bytes) 000502 1783952728.036923 write 0x01edb3 thru 0x023b98 (0x4de6 bytes) 000503 1783952728.038432 mapwrite 0x029078 thru 0x034ebb (0xbe44 bytes) 000504 1783952728.143019 mapread 0x00d3f2 thru 0x018588 (0xb197 bytes) 000505 1783952728.146067 read 0x01620b thru 0x01aadd (0x48d3 bytes) 000506 1783952728.157497 write 0x01de93 thru 0x02cb5c (0xecca bytes) 000507 1783952728.160034 mapread 0x027c74 thru 0x02c190 (0x451d bytes) 000508 1783952728.161942 mapread 0x031bf4 thru 0x034ebb (0x32c8 bytes) 000509 1783952728.163535 mapwrite 0x014af1 thru 0x016047 (0x1557 bytes) 000510 1783952728.232766 trunc from 0x034ebc to 0x012360 (0x22b5b bytes) 000511 1783952728.265978 read 0x003c7a thru 0x00e704 (0xaa8b bytes) 000512 1783952728.268141 mapread 0x00e143 thru 0x012360 (0x421e bytes) 000513 1783952728.269853 mapread 0x00bf9f thru 0x012360 (0x63c2 bytes) 000517 1783952728.276109 read 0x00c5ab thru 0x00c890 (0x2e6 bytes) 000518 1783952728.277250 mapread 0x00c62b thru 0x012360 (0x5d36 bytes) 000519 1783952728.279721 mapread 0x00bc2b thru 0x012360 (0x6736 bytes) 000521 1783952728.281616 mapread 0x01166b thru 0x012360 (0xcf6 bytes) 000522 1783952728.284541 read 0x00cce5 thru 0x012360 (0x567c bytes) 000523 1783952728.285890 mapwrite 0x01defc thru 0x020bd3 (0x2cd8 bytes) 000524 1783952728.378210 mapread 0x005f88 thru 0x00c915 (0x698e bytes) 000525 1783952728.380975 read 0x001cfc thru 0x00cbe3 (0xaee8 bytes) 000527 1783952728.383963 read 0x00ab9f thru 0x00c4c2 (0x1924 bytes) 000530 1783952728.387528 read 0x0073f4 thru 0x009b81 (0x278e bytes) 000535 1783952728.390933 mapread 0x01226e thru 0x01fd78 (0xdb0b bytes) 000539 1783952728.396818 mapwrite 0x017379 thru 0x0241c6 (0xce4e bytes) 000542 1783952728.506890 read 0x0087a5 thru 0x0174b5 (0xed11 bytes) 000545 1783952728.512639 write 0x004d41 thru 0x006ca6 (0x1f66 bytes) 000546 1783952728.514235 trunc from 0x0241c7 to 0x00321d (0x20fa9 bytes) 000547 1783952728.539079 read 0x000e88 thru 0x00321d (0x2396 bytes) 000548 1783952728.540511 mapwrite 0x021429 thru 0x022acb (0x16a3 bytes) 000549 1783952728.643961 trunc from 0x022acc to 0x004d97 (0x1dd34 bytes) 000551 1783952728.668036 read 0x0041f4 thru 0x004d97 (0xba4 bytes) 000554 1783952728.675081 read 0x004652 thru 0x004d97 (0x746 bytes) 000555 1783952728.676233 write 0x01dfc7 thru 0x024ef6 (0x6f30 bytes) 000557 1783952728.686556 MIRROR_EXTEND to 2 mirrors 000558 1783952729.640350 read 0x0216cd thru 0x022af8 (0x142c bytes) 000560 1783952729.655037 write 0x03c47e thru 0x03ffff (0x3b82 bytes) 000561 1783952729.748982 write 0x03738f thru 0x03ffff (0x8c71 bytes) 000562 1783952729.756614 write 0x01a8e7 thru 0x01c459 (0x1b73 bytes) 000563 1783952729.767551 read 0x028133 thru 0x035897 (0xd765 bytes) 000567 1783952729.782810 mapread 0x03b3d4 thru 0x03ffff (0x4c2c bytes) 000570 1783952729.786862 trunc from 0x040000 to 0x03447f (0xbb80 bytes) 000573 1783952729.822016 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x34480): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x34480): Mirror 2: 0x3ca5e1b8 (stale) CRC-32 checksum value for chunk [0, 0x34480): Mirror 1: 0x407f2cf3 Mirror 3: 0x407f2cf3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x34480: skipped 000575 1783952730.993262 mapread 0x027422 thru 0x02f3ec (0x7fcb bytes) 000576 1783952731.039572 read 0x022659 thru 0x026f2a (0x48d2 bytes) 000578 1783952731.044478 write 0x0343dc thru 0x03ffff (0xbc24 bytes) 000579 1783952731.139579 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x4c942ce (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x4c942ce (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000580 1783952731.574674 read 0x011c91 thru 0x020200 (0xe570 bytes) 000581 1783952731.595052 write 0x00eb30 thru 0x01593c (0x6e0d bytes) 000582 1783952731.610054 read 0x033f70 thru 0x03f7c0 (0xb851 bytes) 000592 1783952731.651090 read 0x02ca7d thru 0x02f590 (0x2b14 bytes) 000593 1783952731.652791 write 0x009a68 thru 0x0196ff (0xfc98 bytes) 000594 1783952731.662631 trunc from 0x040000 to 0x0250e9 (0x1af16 bytes) 000595 1783952731.695090 mapwrite 0x01a411 thru 0x029e1f (0xfa0f bytes) 000596 1783952731.875980 mapread 0x013362 thru 0x01df74 (0xac13 bytes) 000597 1783952731.877771 trunc from 0x029e20 to 0x01cd19 (0xd106 bytes) 000598 1783952731.896403 mapread 0x01190e thru 0x012b89 (0x127c bytes) 000602 1783952731.901946 write 0x01eded thru 0x01fc45 (0xe59 bytes) 000604 1783952731.905020 mapread 0x00e6bb thru 0x0133d6 (0x4d1c bytes) 000605 1783952731.907071 write 0x00fc8c thru 0x013ad3 (0x3e48 bytes) 000606 1783952731.908762 write 0x0209e2 thru 0x02a62c (0x9c4b bytes) 000608 1783952731.911530 trunc from 0x02a62d to 0x03046e (0x5e42 bytes) 000609 1783952731.943819 mapwrite 0x0308b6 thru 0x0381dd (0x7928 bytes) 000610 1783952732.183892 write 0x0008c0 thru 0x005a3f (0x5180 bytes) 000612 1783952732.197895 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x381de): Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x381de): Mirror 3: 0x4f8ba110 (stale) CRC-32 checksum value for chunk [0, 0x381de): Mirror 1: 0x61cea03d Mirror 4: 0x61cea03d lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x381de: skipped 000613 1783952733.831976 mapread 0x00d7b9 thru 0x017c68 (0xa4b0 bytes) 000615 1783952733.880930 mapwrite 0x0221ea thru 0x02443f (0x2256 bytes) 000616 1783952734.156160 trunc from 0x0381de to 0x01c9d8 (0x1b805 bytes) 000621 1783952734.214576 read 0x009051 thru 0x0147b3 (0xb763 bytes) 000622 1783952734.227331 write 0x019e99 thru 0x0250a9 (0xb211 bytes) 000623 1783952734.236584 mapwrite 0x01af31 thru 0x0280c0 (0xd190 bytes) 000624 1783952734.351435 write 0x02b519 thru 0x0396a3 (0xe18b bytes) 000625 1783952734.356502 mapread 0x030f90 thru 0x0368af (0x5920 bytes) 000628 1783952734.370825 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x396a4): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x396a4): Mirror 3: 0x7e731f5c (stale) Mirror 4: 0xad4f2326 (stale) CRC-32 checksum value for chunk [0, 0x396a4): Mirror 1: 0xe8f62c95 Mirror 5: 0xe8f62c95 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x396a4: skipped 000629 1783952735.695198 write 0x0143c9 thru 0x020f58 (0xcb90 bytes) 000630 1783952735.785281 write 0x0385d5 thru 0x03bdfd (0x3829 bytes) 000632 1783952735.795923 read 0x01ab0a thru 0x01d702 (0x2bf9 bytes) 000633 1783952735.797100 read 0x013b98 thru 0x01a3c2 (0x682b bytes) 000634 1783952735.825205 trunc from 0x03bdfe to 0x02dc73 (0xe18a bytes) 000635 1783952735.871402 mapwrite 0x017c37 thru 0x01ea2a (0x6df4 bytes) 000636 1783952736.001426 mapwrite 0x02235e thru 0x028777 (0x641a bytes) 000637 1783952736.156118 trunc from 0x02dc74 to 0x014a19 (0x1925a bytes) 000638 1783952736.208382 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x14a1a): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x14a1a): Mirror 3: 0x1dc7f0af (stale) Mirror 4: 0xdb085f4e (stale) Mirror 5: 0xdb085f4e (stale) CRC-32 checksum value for chunk [0, 0x14a1a): Mirror 1: 0x336896b0 Mirror 6: 0x336896b0 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x14a1a: skipped 000640 1783952737.687422 read 0x0119c5 thru 0x014a19 (0x3055 bytes) 000642 1783952737.720252 write 0x02b304 thru 0x035cd7 (0xa9d4 bytes) 000643 1783952737.888861 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x35cd8): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x35cd8): Mirror 3: 0x93c37cb3 (stale) Mirror 4: 0x1458bcf0 (stale) Mirror 5: 0xa7842e99 (stale) Mirror 6: 0xce439ba4 (stale) CRC-32 checksum value for chunk [0, 0x35cd8): Mirror 1: 0x731db906 Mirror 7: 0x731db906 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x35cd8: skipped 000644 1783952739.097291 trunc from 0x035cd8 to 0x030dff (0x4ed8 bytes) 000645 1783952739.256854 trunc from 0x030e00 to 0x03f10b (0xe30c bytes) 000646 1783952739.290630 mapwrite 0x01c46d thru 0x029685 (0xd219 bytes) 000647 1783952739.443547 read 0x001059 thru 0x00be20 (0xadc8 bytes) 000648 1783952739.471999 write 0x025f76 thru 0x02960c (0x3697 bytes) 000649 1783952739.506185 MIRROR_SPLIT mirror 2 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3f10c): Mirror 3: 0xd23ccfeb (stale) Mirror 5: 0x7ec0722f (stale) Mirror 6: 0x5a9fce43 (stale) Mirror 7: 0x30332fbf (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3f10c): Mirror 3: 0xd23ccfeb (stale) Mirror 5: 0x7ec0722f (stale) Mirror 6: 0x5a9fce43 (stale) Mirror 7: 0x30332fbf (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3f10c: skipped 000650 1783952740.142911 write 0x023207 thru 0x02a8a3 (0x769d bytes) 000651 1783952740.156754 read 0x00fa4e thru 0x01e08c (0xe63f bytes) 000653 1783952740.172534 mapwrite 0x01d9a1 thru 0x02bfea (0xe64a bytes) 000654 1783952740.401455 write 0x0280b8 thru 0x03650b (0xe454 bytes) 000655 1783952740.413378 mapread 0x01f6e5 thru 0x02cb5a (0xd476 bytes) 000660 1783952740.420243 mapread 0x0265c4 thru 0x031717 (0xb154 bytes) 000661 1783952740.425904 trunc from 0x03f10c to 0x0343b3 (0xad58 bytes) 000662 1783952740.469010 MIRROR_SPLIT mirror 4 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x343b4): Mirror 3: 0xc3420e0d (stale) Mirror 5: 0xbf15e768 (stale) Mirror 6: 0xcbadfa90 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x343b4): Mirror 3: 0xc3420e0d (stale) Mirror 5: 0xbf15e768 (stale) Mirror 6: 0xcbadfa90 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x343b4: skipped 000667 1783952741.113381 read 0x00ae62 thru 0x013a7d (0x8c1c bytes) 000669 1783952741.131650 trunc from 0x0343b4 to 0x021293 (0x13120 bytes) 000670 1783952741.158279 MIRROR_RESYNC 4 mirrors 000671 1783952742.672859 mapread 0x006573 thru 0x008045 (0x1ad3 bytes) 000673 1783952742.712070 mapwrite 0x03d91f thru 0x03ffff (0x26e1 bytes) 000677 1783952742.967183 mapwrite 0x01a333 thru 0x022ac5 (0x8793 bytes) 000679 1783952743.025786 write 0x019257 thru 0x01e32d (0x50d7 bytes) 000680 1783952743.028548 mapwrite 0x025a33 thru 0x0274b2 (0x1a80 bytes) 000681 1783952743.130153 trunc from 0x040000 to 0x03bab3 (0x454c bytes) 000682 1783952743.153590 mapwrite 0x034920 thru 0x03b776 (0x6e57 bytes) 000687 1783952743.310431 read 0x03750c thru 0x037acc (0x5c1 bytes) 000689 1783952743.338314 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3bab4): Mirror 3: 0xf19e49cd (stale) Mirror 6: 0xf19e49cd (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3bab4): Mirror 3: 0xf19e49cd (stale) Mirror 6: 0xf19e49cd (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3bab4: skipped 000690 1783952743.940197 mapwrite 0x005ed0 thru 0x011ae6 (0xbc17 bytes) 000691 1783952744.136019 trunc from 0x03bab4 to 0x016911 (0x251a2 bytes) 000692 1783952744.248736 write 0x037986 thru 0x03dd65 (0x63e0 bytes) 000693 1783952744.260476 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3dd66): Mirror 3: 0xba33a0cf (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3dd66): Mirror 3: 0xba33a0cf (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3dd66: skipped 000695 1783952744.781791 mapwrite 0x037330 thru 0x03ffff (0x8cd0 bytes) 000696 1783952745.002149 mapread 0x0225bd thru 0x029c76 (0x76ba bytes) 000698 1783952745.024627 mapread 0x035e87 thru 0x03ffff (0xa179 bytes) 000699 1783952745.034749 read 0x00c507 thru 0x01663d (0xa137 bytes) 000701 1783952745.060448 mapwrite 0x00b1e8 thru 0x0185e0 (0xd3f9 bytes) 000704 1783952745.246798 write 0x009eb7 thru 0x019143 (0xf28d bytes) 000706 1783952745.259184 mapread 0x031c7a thru 0x035a4a (0x3dd1 bytes) 000707 1783952745.290726 write 0x022084 thru 0x030032 (0xdfaf bytes) 000708 1783952745.298709 mapread 0x029af3 thru 0x038645 (0xeb53 bytes) 000709 1783952745.310715 write 0x003004 thru 0x004cba (0x1cb7 bytes) 000710 1783952745.327157 mapwrite 0x0020a9 thru 0x00d20d (0xb165 bytes) 000711 1783952745.556400 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) 000713 1783952745.791477 mapread 0x0127a2 thru 0x01b3d1 (0x8c30 bytes) 000714 1783952745.819675 mapwrite 0x003d19 thru 0x00d37a (0x9662 bytes) 000715 1783952745.904746 trunc from 0x040000 to 0x017d1b (0x282e4 bytes) 000719 1783952745.942085 mapread 0x01221e thru 0x016513 (0x42f6 bytes) 000720 1783952745.943885 write 0x0259da thru 0x026479 (0xaa0 bytes) 000722 1783952745.953866 trunc from 0x02647a to 0x02e766 (0x82ed bytes) 000723 1783952746.004861 trunc from 0x02e767 to 0x0017c4 (0x2cfa2 bytes) 000729 1783952746.056447 read 0x0014d5 thru 0x0017c4 (0x2f0 bytes) 000730 1783952746.062498 read 0x000d6a thru 0x0017c4 (0xa5b bytes) 000733 1783952746.083638 write 0x02f6d5 thru 0x03c550 (0xce7c bytes) 000734 1783952746.087391 read 0x03711e thru 0x0395d8 (0x24bb bytes) 000735 1783952746.088579 mapwrite 0x028007 thru 0x0288db (0x8d5 bytes) 000736 1783952746.231519 trunc from 0x03c551 to 0x03444a (0x8106 bytes) 000739 1783952746.263378 mapread 0x021d8a thru 0x0295da (0x7851 bytes) 000747 1783952746.270866 read 0x032873 thru 0x03444a (0x1bd8 bytes) 000752 1783952746.274207 mapwrite 0x008617 thru 0x00d594 (0x4f7e bytes) 000753 1783952746.323671 write 0x00f436 thru 0x01d9c1 (0xe58c bytes) 000754 1783952746.325300 trunc from 0x03444b to 0x00cad7 (0x27973 bytes) 000755 1783952746.342163 mapwrite 0x004dca thru 0x00c66c (0x78a3 bytes) 000756 1783952746.404340 write 0x011bde thru 0x018963 (0x6d86 bytes) 000757 1783952746.408466 mapread 0x00048e thru 0x0006fa (0x26d bytes) 000758 1783952746.423367 MIRROR_EXTEND to 2 mirrors 000759 1783952747.202063 read 0x00b003 thru 0x0136d1 (0x86cf bytes) 000762 1783952747.217398 trunc from 0x018964 to 0x01b71a (0x2db7 bytes) 000763 1783952747.400117 mapwrite 0x02f47f thru 0x035850 (0x63d2 bytes) 000764 1783952747.560561 read 0x0262f3 thru 0x02f680 (0x938e bytes) 000765 1783952747.594405 trunc from 0x035851 to 0x001855 (0x33ffb bytes) 000766 1783952747.634311 mapwrite 0x0311c5 thru 0x03642f (0x526b bytes) 000767 1783952747.886819 mapread 0x023fdb thru 0x0258e0 (0x1906 bytes) 000770 1783952747.939073 read 0x02b106 thru 0x03642f (0xb32a bytes) 000771 1783952747.962003 write 0x026f38 thru 0x027269 (0x332 bytes) 000775 1783952747.981596 mapwrite 0x028726 thru 0x02ac37 (0x2512 bytes) 000776 1783952748.087849 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) 000778 1783952748.311057 trunc from 0x036430 to 0x015e28 (0x20607 bytes) 000779 1783952748.346225 trunc from 0x015e29 to 0x024883 (0xea5b bytes) 000780 1783952748.376749 read 0x00ae8a thru 0x01113b (0x62b2 bytes) 000785 1783952748.401761 mapwrite 0x02c30a thru 0x035967 (0x965e bytes) 000786 1783952748.604750 mapwrite 0x017eaf thru 0x019ded (0x1f3f bytes) 000788 1783952748.757029 write 0x011213 thru 0x01e70c (0xd4fa bytes) 000790 1783952748.762527 read 0x0159f8 thru 0x02014f (0xa758 bytes) 000792 1783952748.787025 read 0x01f51c thru 0x02c96e (0xd453 bytes) 000793 1783952748.799355 MIRROR_EXTEND to 2 mirrors 000795 1783952749.908191 read 0x005c9a thru 0x006240 (0x5a7 bytes) 000796 1783952749.915623 mapread 0x0296a7 thru 0x035967 (0xc2c1 bytes) 000799 1783952749.946174 write 0x01da5a thru 0x01e74f (0xcf6 bytes) 000801 1783952750.057848 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) 000802 1783952750.256579 trunc from 0x035968 to 0x02e92e (0x7039 bytes) 000804 1783952750.275144 mapwrite 0x021fde thru 0x02fc85 (0xdca8 bytes) 000807 1783952750.397527 mapread 0x003fa5 thru 0x010c6d (0xccc9 bytes) 000808 1783952750.407980 mapwrite 0x021b79 thru 0x02ee3c (0xd2c4 bytes) 000812 1783952750.471337 mapread 0x01ada3 thru 0x0201cc (0x542a bytes) 000813 1783952750.503319 trunc from 0x02fc86 to 0x00d097 (0x22bee bytes) 000815 1783952750.527179 trunc from 0x00d098 to 0x00c87b (0x81c bytes) 000816 1783952750.542487 mapwrite 0x005d71 thru 0x006df7 (0x1087 bytes) 000817 1783952750.588530 trunc from 0x00c87c to 0x02d89d (0x21022 bytes) 000818 1783952750.619307 read 0x02a7ae thru 0x02d89d (0x30f0 bytes) 000819 1783952750.658476 read 0x01d04f thru 0x0268e3 (0x9895 bytes) 000822 1783952750.695432 trunc from 0x02d89e to 0x00d283 (0x2061a bytes) 000823 1783952750.739759 trunc from 0x00d284 to 0x01fbcc (0x12949 bytes) 000826 1783952750.784479 mapread 0x017c62 thru 0x01fbcc (0x7f6b bytes) 000827 1783952750.926354 mapread 0x01c8ad thru 0x01fbcc (0x3320 bytes) 000831 1783952750.943852 write 0x01569f thru 0x017ba4 (0x2506 bytes) 000832 1783952750.973054 mapwrite 0x003c5d thru 0x00c03b (0x83df bytes) 000833 1783952751.181663 trunc from 0x01fbcd to 0x03dc64 (0x1e098 bytes) 000835 1783952751.243624 mapread 0x002315 thru 0x00ae72 (0x8b5e bytes) 000837 1783952751.261024 write 0x023cc5 thru 0x02fbe3 (0xbf1f bytes) 000838 1783952751.297518 mapwrite 0x027c53 thru 0x0356d9 (0xda87 bytes) 000839 1783952751.523532 MIRROR_EXTEND to 2 mirrors 000840 1783952752.823269 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) 000847 1783952753.046811 read 0x034f9a thru 0x03d101 (0x8168 bytes) 000848 1783952753.058639 read 0x02f260 thru 0x02fa30 (0x7d1 bytes) 000849 1783952753.078506 trunc from 0x03dc65 to 0x002d9c (0x3aec8 bytes) 000850 1783952753.117533 mapread 0x0007f4 thru 0x002d9c (0x25a9 bytes) 000851 1783952753.136043 mapread 0x002914 thru 0x002d9c (0x489 bytes) 000852 1783952753.137527 read 0x0014d5 thru 0x002d9c (0x18c8 bytes) 000853 1783952753.138287 mapwrite 0x0109d5 thru 0x01c2e3 (0xb90f bytes) 000855 1783952753.231644 mapwrite 0x031adc thru 0x03eeb3 (0xd3d8 bytes) 000856 1783952753.433487 mapwrite 0x03c2e1 thru 0x03ffff (0x3d1f bytes) 000857 1783952753.573511 mapwrite 0x00538a thru 0x00fb30 (0xa7a7 bytes) 000860 1783952753.696130 trunc from 0x040000 to 0x013b32 (0x2c4cd bytes) 000862 1783952753.736363 MIRROR_EXTEND to 2 mirrors 000864 1783952754.572229 MIRROR_EXTEND to 3 mirrors 000865 1783952755.299015 mapwrite 0x027846 thru 0x032001 (0xa7bc bytes) 000866 1783952755.685521 mapwrite 0x00aefc thru 0x0184a7 (0xd5ac bytes) 000867 1783952755.874063 trunc from 0x032002 to 0x018a73 (0x1958e bytes) 000868 1783952755.947449 trunc from 0x018a74 to 0x0003a4 (0x186cf bytes) 000869 1783952756.023410 mapread 0x000032 thru 0x0003a4 (0x373 bytes) 000870 1783952756.040372 mapwrite 0x0373a6 thru 0x03ffff (0x8c5a bytes) 000871 1783952756.228000 trunc from 0x040000 to 0x002a9a (0x3d565 bytes) 000873 1783952756.263719 write 0x011571 thru 0x01e3ac (0xce3c bytes) 000874 1783952756.296287 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x1e3ad): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1e3ad): Mirror 2: 0xd66a4c90 (stale) Mirror 3: 0xd66a4c90 (stale) CRC-32 checksum value for chunk [0, 0x1e3ad): Mirror 1: 0xd5de47a0 Mirror 4: 0xd5de47a0 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1e3ad: skipped 000875 1783952757.698474 mapread 0x0172d1 thru 0x01b149 (0x3e79 bytes) 000876 1783952757.721544 mapread 0x011afd thru 0x01bfbf (0xa4c3 bytes) 000877 1783952757.731138 mapwrite 0x030c0d thru 0x033c99 (0x308d bytes) 000880 1783952757.883443 read 0x013915 thru 0x01cfd1 (0x96bd bytes) 000882 1783952757.920416 write 0x00e3af thru 0x00fc1d (0x186f bytes) 000883 1783952757.943783 write 0x0205ee thru 0x02be68 (0xb87b bytes) 000884 1783952757.953072 mapread 0x0278aa thru 0x02a049 (0x27a0 bytes) 000885 1783952757.955368 trunc from 0x033c9a to 0x030927 (0x3372 bytes) 000887 1783952758.020373 read 0x00c97a thru 0x015fc5 (0x964c bytes) 000889 1783952758.046277 write 0x019525 thru 0x0232ed (0x9dc9 bytes) 000890 1783952758.053849 write 0x00ccfe thru 0x015316 (0x8619 bytes) 000895 1783952758.069999 write 0x02c2eb thru 0x02efd1 (0x2ce7 bytes) 000896 1783952758.082973 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x30928): Mirror 2: (stale - read error) Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x30928): Mirror 2: 0xc27ee11b (stale) Mirror 3: 0xc27ee11b (stale) Mirror 4: 0xeeb0db00 (stale) CRC-32 checksum value for chunk [0, 0x30928): Mirror 1: 0x505a121a Mirror 5: 0x505a121a lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x30928: skipped 000897 1783952759.406874 trunc from 0x030928 to 0x0079c8 (0x28f5f bytes) 000898 1783952759.542474 mapwrite 0x01aec6 thru 0x01f618 (0x4753 bytes) 000899 1783952759.653919 mapread 0x0174ba thru 0x01f618 (0x815f bytes) 000902 1783952759.660287 write 0x02a850 thru 0x034741 (0x9ef2 bytes) 000903 1783952759.662321 read 0x02064a thru 0x022f5c (0x2913 bytes) 000907 1783952759.675882 write 0x01c36d thru 0x02bc7a (0xf90e bytes) 000908 1783952759.679166 trunc from 0x034742 to 0x03b524 (0x6de3 bytes) 000909 1783952759.703637 mapwrite 0x00251a thru 0x00920d (0x6cf4 bytes) 000912 1783952759.789286 mapwrite 0x0136f8 thru 0x01b973 (0x827c bytes) 000913 1783952759.848014 mapwrite 0x02be8c thru 0x03b3a1 (0xf516 bytes) 000917 1783952759.935005 mapread 0x01bb7f thru 0x02572e (0x9bb0 bytes) 000918 1783952759.937245 read 0x0054bc thru 0x0074f9 (0x203e bytes) 000920 1783952759.943949 read 0x004b36 thru 0x010e19 (0xc2e4 bytes) 000921 1783952759.945181 mapread 0x001626 thru 0x011444 (0xfe1f bytes) 000922 1783952759.947442 read 0x011a35 thru 0x01acb7 (0x9283 bytes) 000923 1783952759.948948 mapwrite 0x038374 thru 0x03ffff (0x7c8c bytes) 000924 1783952760.049953 write 0x0191db thru 0x023287 (0xa0ad bytes) 000926 1783952760.058129 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xd353b517 (stale) Mirror 4: 0x761a492a (stale) Mirror 5: 0x2429b18c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xd353b517 (stale) Mirror 4: 0x761a492a (stale) Mirror 5: 0x2429b18c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000927 1783952760.627724 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xd353b517 (stale) Mirror 4: 0x761a492a (stale) Mirror 5: 0x2429b18c (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x1a44ccbb Mirror 6: 0x1a44ccbb lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000930 1783952761.927254 write 0x01d159 thru 0x020347 (0x31ef bytes) 000932 1783952762.005482 trunc from 0x040000 to 0x0029ee (0x3d611 bytes) 000933 1783952762.046121 mapwrite 0x001081 thru 0x00edb5 (0xdd35 bytes) 000934 1783952762.199338 write 0x0016c6 thru 0x007500 (0x5e3b bytes) 000935 1783952762.204426 trunc from 0x00edb6 to 0x033c7a (0x24ec5 bytes) 000936 1783952762.242322 write 0x005e4b thru 0x00603a (0x1f0 bytes) 000937 1783952762.243640 read 0x000167 thru 0x00ba84 (0xb91e bytes) 000939 1783952762.269953 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x33c7b): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x33c7b): Mirror 3: 0xa8994adb (stale) Mirror 4: 0x5a2545fa (stale) Mirror 5: 0x1711d660 (stale) Mirror 6: 0x53955464 (stale) CRC-32 checksum value for chunk [0, 0x33c7b): Mirror 1: 0x44bcb4b2 Mirror 7: 0x44bcb4b2 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x33c7b: skipped 000940 1783952763.465305 mapread 0x00944c thru 0x010c00 (0x77b5 bytes) 000941 1783952763.516778 mapread 0x015c3b thru 0x020b60 (0xaf26 bytes) 000943 1783952763.584349 trunc from 0x033c7b to 0x0249ae (0xf2cc bytes) 000944 1783952763.780238 mapwrite 0x036e22 thru 0x03ffff (0x91de bytes) 000945 1783952763.955027 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xd353b517 (stale) Mirror 4: 0x761a492a (stale) Mirror 5: 0x2429b18c (stale) Mirror 6: 0x1a44ccbb (stale) Mirror 7: 0x90a62635 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x393e85c9 Mirror 8: 0x393e85c9 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000947 1783952765.353770 trunc from 0x040000 to 0x018cc4 (0x2733b bytes) 000948 1783952765.481988 trunc from 0x018cc5 to 0x038822 (0x1fb5e bytes) 000950 1783952765.527001 trunc from 0x038823 to 0x0060ab (0x32777 bytes) 000953 1783952765.566436 read 0x001cb5 thru 0x0060ab (0x43f7 bytes) 000954 1783952765.604913 MIRROR_RESYNC 7 mirrors 000956 1783952767.375300 mapread 0x0046c4 thru 0x0060ab (0x19e8 bytes) 000957 1783952767.437844 trunc from 0x0060ac to 0x01fa12 (0x19967 bytes) 000959 1783952767.548039 MIRROR_EXTEND to 8 mirrors CRC-32 checksum value for chunk [0, 0x1fa13): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1fa13): Mirror 3: 0xf890579a (stale) Mirror 4: 0xf890579a (stale) Mirror 5: 0xf890579a (stale) Mirror 6: 0xf890579a (stale) Mirror 7: 0xf890579a (stale) Mirror 8: 0xf890579a (stale) CRC-32 checksum value for chunk [0, 0x1fa13): Mirror 1: 0xf890579a Mirror 9: 0xf890579a lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1fa13: skipped 000960 1783952768.689152 MIRROR_SPLIT mirror 1 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1fa13): Mirror 4: 0xf890579a (stale) Mirror 5: 0xf890579a (stale) Mirror 6: 0xf890579a (stale) Mirror 7: 0xf890579a (stale) Mirror 8: 0xf890579a (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1fa13): Mirror 4: 0xf890579a (stale) Mirror 5: 0xf890579a (stale) Mirror 6: 0xf890579a (stale) Mirror 7: 0xf890579a (stale) Mirror 8: 0xf890579a (stale) CRC-32 checksum value for chunk [0, 0x1fa13): Mirror 1: 0xf890579a Mirror 9: 0xf890579a lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1fa13: skipped 000961 1783952769.961592 mapwrite 0x01fc5f thru 0x02e7c5 (0xeb67 bytes) 000963 1783952770.348178 write 0x0209f8 thru 0x02d634 (0xcc3d bytes) 000964 1783952770.381328 MIRROR_SPLIT mirror 5 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2e7c6): Mirror 4: 0x40bd8f45 (stale) Mirror 5: 0x40bd8f45 (stale) Mirror 6: 0x40bd8f45 (stale) Mirror 7: 0x40bd8f45 (stale) Mirror 9: 0x40bd8f45 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2e7c6): Mirror 4: 0x40bd8f45 (stale) Mirror 5: 0x40bd8f45 (stale) Mirror 6: 0x40bd8f45 (stale) Mirror 7: 0x40bd8f45 (stale) Mirror 9: 0x40bd8f45 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2e7c6: skipped 000969 1783952771.249007 read 0x01bbf4 thru 0x021919 (0x5d26 bytes) 000971 1783952771.288231 read 0x012882 thru 0x01d3fd (0xab7c bytes) 000975 1783952771.345958 mapwrite 0x0040ef thru 0x0099af (0x58c1 bytes) 000977 1783952771.605566 read 0x01c633 thru 0x02a0c2 (0xda90 bytes) 000979 1783952771.620925 MIRROR_SPLIT mirror 1 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x166:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2e7c6): Mirror 5: 0x40bd8f45 (stale) Mirror 6: 0x40bd8f45 (stale) Mirror 7: 0x40bd8f45 (stale) Mirror 9: 0x40bd8f45 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2e7c6): Mirror 5: 0x40bd8f45 (stale) Mirror 6: 0x40bd8f45 (stale) Mirror 7: 0x40bd8f45 (stale) Mirror 9: 0x40bd8f45 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2e7c6: skipped 000980 1783952772.296974 mapread 0x012731 thru 0x01a12f (0x79ff bytes) 000982 1783952772.319672 mapread 0x005bdb thru 0x009607 (0x3a2d bytes) 000985 1783952772.333082 trunc from 0x02e7c6 to 0x01d1f0 (0x115d5 bytes) 000987 1783952772.366028 mapread 0x01afa7 thru 0x01d1f0 (0x224a bytes) 000989 1783952772.396400 write 0x03b32d thru 0x03b99b (0x66f bytes) 000990 1783952772.399883 trunc from 0x03b99c to 0x031cc8 (0x9cd3 bytes) 000993 1783952772.459578 MIRROR_RESYNC 5 mirrors 000994 1783952774.170284 read 0x029e03 thru 0x030ddb (0x6fd9 bytes) 000995 1783952774.189208 MIRROR_RESYNC 5 mirrors 000996 1783952774.932219 write 0x010ebc thru 0x011cf5 (0xe3a bytes) 000998 1783952775.103004 mapwrite 0x00b924 thru 0x00c280 (0x95d bytes) 000999 1783952775.257651 mapread 0x025c43 thru 0x031cc8 (0xc086 bytes) All operations completed A-OK!