== sanity-flr test 70a: flr mode fsx test ================ 18:43:17 (1786401797) Seed set to 1799 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0xce28 000002 1786401800.204087 trunc from 00000000 to 0x00ce27 (0xce28 bytes) 000004 1786401800.223571 mapread 0x0032e9 thru 0x00ce27 (0x9b3f bytes) 000005 1786401800.264928 write 0x02576d thru 0x0264ad (0xd41 bytes) 000008 1786401800.268357 mapread 0x00490e thru 0x008c42 (0x4335 bytes) 000010 1786401800.270714 mapread 0x021cbd thru 0x02250e (0x852 bytes) 000011 1786401800.284912 write 0x022350 thru 0x025505 (0x31b6 bytes) truncating to largest ever: 0x19667 000012 1786401800.286789 trunc from 0x0264ae to 0x019666 (0xce47 bytes) 000013 1786401800.318368 write 0x011a0a thru 0x015e7a (0x4471 bytes) 000014 1786401800.344580 read 0x0022d5 thru 0x006376 (0x40a2 bytes) 000015 1786401800.347561 mapread 0x00560a thru 0x0066fd (0x10f4 bytes) truncating to largest ever: 0x36d50 000019 1786401800.353827 trunc from 0x019667 to 0x036d4f (0x1d6e9 bytes) 000020 1786401800.385311 read 0x02d268 thru 0x03354a (0x62e3 bytes) 000022 1786401800.413893 mapread 0x027cdd thru 0x031ae2 (0x9e06 bytes) 000026 1786401800.420430 mapread 0x00ac8b thru 0x011a5a (0x6dd0 bytes) 000027 1786401800.422924 mapread 0x000b12 thru 0x005382 (0x4871 bytes) 000028 1786401800.425146 write 0x02f9a3 thru 0x03a8e3 (0xaf41 bytes) 000029 1786401800.428324 read 0x016b1a thru 0x01db6c (0x7053 bytes) 000030 1786401800.429715 mapwrite 0x00e446 thru 0x010405 (0x1fc0 bytes) 000031 1786401800.557113 mapread 0x01b6e1 thru 0x02a026 (0xe946 bytes) 000032 1786401800.560805 mapread 0x01ea63 thru 0x02bfc5 (0xd563 bytes) 000034 1786401800.566772 mapread 0x004424 thru 0x00ebb2 (0xa78f bytes) 000035 1786401800.570222 write 0x020ecd thru 0x02602d (0x5161 bytes) 000037 1786401800.578106 mapread 0x014d67 thru 0x019c39 (0x4ed3 bytes) 000038 1786401800.580575 mapwrite 0x03eee8 thru 0x03ffff (0x1118 bytes) 000042 1786401800.678751 trunc from 0x040000 to 0x008693 (0x3796c bytes) 000044 1786401800.704943 mapwrite 0x00bf86 thru 0x00fe26 (0x3ea1 bytes) 000046 1786401800.797430 trunc from 0x00fe27 to 0x010585 (0x75f bytes) 000048 1786401800.814909 trunc from 0x010586 to 0x0198bc (0x9337 bytes) 000049 1786401800.831179 write 0x0311f1 thru 0x03989a (0x86aa bytes) 000051 1786401800.834564 mapwrite 0x02d1a6 thru 0x02e8b5 (0x1710 bytes) truncating to largest ever: 0x3f413 000053 1786401800.897652 trunc from 0x03989b to 0x03f412 (0x5b78 bytes) 000054 1786401800.917717 trunc from 0x03f413 to 0x009b2c (0x358e6 bytes) 000055 1786401800.939038 mapread 0x007e04 thru 0x009b2c (0x1d29 bytes) 000056 1786401800.945969 write 0x01b9ef thru 0x028021 (0xc633 bytes) 000057 1786401800.948248 read 0x013a85 thru 0x022931 (0xeead bytes) 000058 1786401800.957702 trunc from 0x028022 to 0x02207f (0x5fa2 bytes) 000059 1786401800.983738 trunc from 0x022080 to 0x005bf9 (0x1c486 bytes) 000060 1786401801.004395 mapwrite 0x0130ab thru 0x016c7d (0x3bd3 bytes) 000063 1786401801.099807 trunc from 0x016c7e to 0x023a39 (0xcdbc bytes) 000065 1786401801.120995 mapwrite 0x02640e thru 0x02c73a (0x632d bytes) 000067 1786401801.224809 write 0x024352 thru 0x031ef5 (0xdba4 bytes) 000068 1786401801.233463 mapwrite 0x01bebc thru 0x028881 (0xc9c6 bytes) 000069 1786401801.351235 read 0x0195a5 thru 0x01cc06 (0x3662 bytes) 000070 1786401801.367119 mapread 0x013a33 thru 0x01dd36 (0xa304 bytes) 000076 1786401801.374604 mapread 0x02a073 thru 0x031ef5 (0x7e83 bytes) 000077 1786401801.376620 trunc from 0x031ef6 to 0x010bf7 (0x212fe bytes) 000079 1786401801.401097 mapread 0x00443e thru 0x00bbef (0x77b2 bytes) 000081 1786401801.406499 mapwrite 0x001c28 thru 0x006553 (0x492c bytes) 000087 1786401801.463981 mapwrite 0x018c23 thru 0x01f2d7 (0x66b5 bytes) 000091 1786401801.609167 MIRROR_EXTEND to 2 mirrors 000096 1786401802.233337 write 0x025adc thru 0x02e4a3 (0x89c8 bytes) 000097 1786401802.315058 mapwrite 0x02ce4d thru 0x034ff6 (0x81aa bytes) 000099 1786401802.441740 trunc from 0x034ff7 to 0x0206c4 (0x14932 bytes) 000100 1786401802.480904 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) 000101 1786401802.650988 mapread 0x01b1fe thru 0x01bc0f (0xa12 bytes) 000103 1786401802.660082 mapwrite 0x01a37a thru 0x029f55 (0xfbdc bytes) 000104 1786401802.757853 write 0x01921c thru 0x01b7eb (0x25d0 bytes) 000105 1786401802.759415 mapread 0x01329c thru 0x014126 (0xe8b bytes) 000108 1786401802.770782 write 0x0300a6 thru 0x03a777 (0xa6d2 bytes) 000112 1786401802.776379 trunc from 0x03a778 to 0x020888 (0x19eef bytes) 000113 1786401802.799114 trunc from 0x020889 to 0x005ed6 (0x1a9b2 bytes) 000114 1786401802.822067 mapwrite 0x017eea thru 0x023b6e (0xbc85 bytes) 000116 1786401802.924666 write 0x025f28 thru 0x0269be (0xa97 bytes) 000117 1786401802.926172 mapread 0x01299d thru 0x013805 (0xe69 bytes) 000119 1786401802.929572 mapwrite 0x009714 thru 0x00bf0a (0x27f7 bytes) 000120 1786401802.982380 write 0x026875 thru 0x02b0f4 (0x4880 bytes) 000122 1786401802.985864 mapread 0x01cdb3 thru 0x028e4b (0xc099 bytes) 000126 1786401802.994446 read 0x01377b thru 0x01ab2c (0x73b2 bytes) 000127 1786401802.995589 mapwrite 0x00499a thru 0x010d3f (0xc3a6 bytes) 000129 1786401803.046167 MIRROR_EXTEND to 2 mirrors 000130 1786401803.600658 mapwrite 0x0185d8 thru 0x02132b (0x8d54 bytes) 000131 1786401803.797447 trunc from 0x02b0f5 to 0x015e1d (0x152d7 bytes) 000132 1786401803.839065 mapread 0x00095a thru 0x000d75 (0x41c bytes) 000133 1786401803.860712 mapwrite 0x019b61 thru 0x026218 (0xc6b8 bytes) 000134 1786401803.989369 read 0x00c307 thru 0x017a02 (0xb6fc bytes) 000135 1786401803.996295 mapread 0x005869 thru 0x00bc9f (0x6437 bytes) 000137 1786401803.998325 read 0x025fbd thru 0x026218 (0x25c bytes) 000138 1786401803.999620 trunc from 0x026219 to 0x00c0da (0x1a13e bytes) 000139 1786401804.038919 read 0x003653 thru 0x00c0da (0x8a88 bytes) 000140 1786401804.041113 mapread 0x00046e thru 0x00603f (0x5bd2 bytes) 000141 1786401804.047853 read 0x007a9c thru 0x00b60a (0x3b6f bytes) 000143 1786401804.058934 mapwrite 0x000135 thru 0x00692f (0x67fb bytes) 000144 1786401804.152903 trunc from 0x00c0db to 0x03dede (0x31e04 bytes) 000146 1786401804.169832 mapread 0x024570 thru 0x031556 (0xcfe7 bytes) 000147 1786401804.179727 mapwrite 0x0350e1 thru 0x03ffff (0xaf1f bytes) 000148 1786401804.264855 read 0x032ef6 thru 0x038857 (0x5962 bytes) 000150 1786401804.267051 mapread 0x00e30a thru 0x01148a (0x3181 bytes) 000151 1786401804.268655 mapwrite 0x01e454 thru 0x025d50 (0x78fd bytes) 000154 1786401804.337526 MIRROR_RESYNC 2 mirrors 000156 1786401805.077434 trunc from 0x040000 to 0x001c7a (0x3e385 bytes) 000158 1786401805.219830 mapwrite 0x03a828 thru 0x03ffff (0x57d8 bytes) 000160 1786401805.335583 trunc from 0x040000 to 0x00d3a6 (0x32c59 bytes) 000161 1786401805.362097 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) 000163 1786401805.499050 read 0x00b613 thru 0x00d3a6 (0x1d94 bytes) 000164 1786401805.506402 trunc from 0x00d3a7 to 0x02b02e (0x1dc88 bytes) 000169 1786401805.534502 mapwrite 0x01a762 thru 0x022278 (0x7b17 bytes) 000170 1786401805.620947 mapread 0x006bd8 thru 0x00a7d6 (0x3bff bytes) 000171 1786401805.638388 trunc from 0x02b02f to 0x002bea (0x28444 bytes) 000172 1786401805.657353 mapread 0x0001cf thru 0x002bea (0x2a1c bytes) 000174 1786401805.666739 read 0x0017ef thru 0x002bea (0x13fc bytes) 000175 1786401805.668027 mapread 0x00123c thru 0x002bea (0x19af bytes) 000176 1786401805.669782 write 0x013489 thru 0x01e281 (0xadf9 bytes) 000180 1786401805.674162 mapread 0x007ee3 thru 0x009b66 (0x1c84 bytes) 000182 1786401805.686101 mapread 0x01d67b thru 0x01e281 (0xc07 bytes) 000184 1786401805.688743 read 0x017795 thru 0x01d3cf (0x5c3b bytes) 000185 1786401805.690001 mapread 0x0170d3 thru 0x019c2e (0x2b5c bytes) 000186 1786401805.691639 mapwrite 0x002388 thru 0x00c1da (0x9e53 bytes) 000187 1786401805.742105 trunc from 0x01e282 to 0x002f45 (0x1b33c bytes) 000189 1786401805.769147 read 0x001232 thru 0x002f45 (0x1d14 bytes) 000190 1786401805.770121 write 0x020b12 thru 0x02af00 (0xa3ef bytes) 000191 1786401805.772621 mapwrite 0x006b7f thru 0x008311 (0x1793 bytes) 000193 1786401805.833714 read 0x001d6b thru 0x00e0fd (0xc393 bytes) 000196 1786401805.836010 trunc from 0x02af01 to 0x02b880 (0x980 bytes) 000197 1786401805.854354 read 0x0006cd thru 0x00d380 (0xccb4 bytes) 000198 1786401805.855990 read 0x01a101 thru 0x01ab71 (0xa71 bytes) 000199 1786401805.857007 trunc from 0x02b881 to 0x0216ed (0xa193 bytes) 000202 1786401805.881332 write 0x032e4c thru 0x036384 (0x3539 bytes) 000203 1786401805.882806 write 0x02ce77 thru 0x03ca15 (0xfb9f bytes) 000204 1786401805.891266 trunc from 0x03ca16 to 0x028bd3 (0x13e42 bytes) 000206 1786401805.918723 MIRROR_EXTEND to 2 mirrors 000207 1786401806.639937 write 0x02367b thru 0x02729f (0x3c25 bytes) 000208 1786401806.738615 mapwrite 0x0310e1 thru 0x039a4e (0x896e bytes) 000213 1786401806.967143 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) 000216 1786401807.152794 write 0x0338fa thru 0x039a4f (0x6156 bytes) 000219 1786401807.164398 mapread 0x02454c thru 0x0310c2 (0xcb77 bytes) 000221 1786401807.198785 trunc from 0x039a50 to 0x0223b2 (0x1769d bytes) 000222 1786401807.221582 trunc from 0x0223b3 to 0x03c54c (0x1a19a bytes) 000223 1786401807.246130 trunc from 0x03c54d to 0x00c213 (0x30339 bytes) 000225 1786401807.268742 read 0x003eca thru 0x00c213 (0x834a bytes) 000226 1786401807.283152 mapwrite 0x01f300 thru 0x02ab87 (0xb888 bytes) 000227 1786401807.385248 write 0x02aad0 thru 0x0354ef (0xaa20 bytes) 000228 1786401807.389947 trunc from 0x0354f0 to 0x027b79 (0xd976 bytes) 000231 1786401807.422310 mapwrite 0x01648c thru 0x01b1a3 (0x4d18 bytes) 000232 1786401807.487933 mapread 0x004750 thru 0x00eb19 (0xa3ca bytes) 000233 1786401807.497835 trunc from 0x027b7a to 0x02048c (0x76ed bytes) 000234 1786401807.512970 mapread 0x0027b8 thru 0x0060e6 (0x392f bytes) 000235 1786401807.521590 mapread 0x01b3b8 thru 0x01c900 (0x1549 bytes) 000236 1786401807.522827 read 0x00ef95 thru 0x011934 (0x29a0 bytes) 000237 1786401807.523774 write 0x01c639 thru 0x0274b3 (0xae7b bytes) 000240 1786401807.526668 read 0x00728b thru 0x015cc5 (0xea3b bytes) 000241 1786401807.527415 read 0x01b24b thru 0x0206b7 (0x546d bytes) 000243 1786401807.528694 write 0x0188ca thru 0x026bd3 (0xe30a bytes) 000246 1786401807.532077 write 0x01b7f0 thru 0x021393 (0x5ba4 bytes) 000247 1786401807.533497 write 0x019e35 thru 0x02690e (0xcada bytes) 000248 1786401807.534882 trunc from 0x0274b4 to 0x031e11 (0xa95e bytes) 000249 1786401807.553088 mapread 0x012b23 thru 0x0138d7 (0xdb5 bytes) 000250 1786401807.555031 trunc from 0x031e12 to 0x037244 (0x5433 bytes) 000251 1786401807.572461 mapwrite 0x030451 thru 0x03713b (0x6ceb bytes) 000252 1786401807.652703 read 0x016b40 thru 0x01c16a (0x562b bytes) 000256 1786401807.656243 mapwrite 0x013150 thru 0x0136c1 (0x572 bytes) 000258 1786401807.717354 write 0x005e4d thru 0x008c0d (0x2dc1 bytes) 000259 1786401807.718436 trunc from 0x037245 to 0x0393df (0x219b bytes) 000264 1786401807.746055 trunc from 0x0393e0 to 0x01d335 (0x1c0aa bytes) 000265 1786401807.773681 MIRROR_EXTEND to 2 mirrors 000266 1786401808.390951 write 0x00d892 thru 0x00f42a (0x1b99 bytes) 000270 1786401808.468774 write 0x03ea60 thru 0x03ffff (0x15a0 bytes) 000272 1786401808.470784 read 0x0085d9 thru 0x0114f1 (0x8f19 bytes) 000275 1786401808.481832 read 0x01e0e9 thru 0x01f849 (0x1761 bytes) 000277 1786401808.492565 write 0x031dae thru 0x033769 (0x19bc bytes) 000279 1786401808.494740 trunc from 0x040000 to 0x032da3 (0xd25c bytes) 000280 1786401808.518575 mapread 0x008fe6 thru 0x015354 (0xc36f bytes) 000281 1786401808.520793 trunc from 0x032da4 to 0x01d61d (0x15786 bytes) 000282 1786401808.543341 MIRROR_RESYNC 2 mirrors 000283 1786401809.091093 read 0x01abf4 thru 0x01d61d (0x2a2a bytes) 000286 1786401809.100737 write 0x02a036 thru 0x034b9a (0xab65 bytes) 000288 1786401809.168571 MIRROR_RESYNC 2 mirrors 000289 1786401809.753446 read 0x023438 thru 0x027854 (0x441d bytes) 000290 1786401809.763091 read 0x01e235 thru 0x02bca4 (0xda70 bytes) 000291 1786401809.788488 mapread 0x013c16 thru 0x019449 (0x5834 bytes) 000292 1786401809.789892 mapread 0x009ec5 thru 0x00bd0d (0x1e49 bytes) 000294 1786401809.795047 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) 000295 1786401809.932496 read 0x007755 thru 0x00fd7d (0x8629 bytes) 000297 1786401809.943502 mapread 0x00fad4 thru 0x014755 (0x4c82 bytes) 000299 1786401809.961790 write 0x003b2d thru 0x007b0b (0x3fdf bytes) 000300 1786401809.987587 mapread 0x02439e thru 0x025a22 (0x1685 bytes) 000304 1786401810.008689 read 0x01b80b thru 0x0228a4 (0x709a bytes) 000306 1786401810.024450 mapwrite 0x0124e7 thru 0x01b11b (0x8c35 bytes) 000309 1786401810.139900 mapwrite 0x03de41 thru 0x03ffff (0x21bf bytes) 000314 1786401810.277880 mapread 0x027813 thru 0x02fe18 (0x8606 bytes) 000315 1786401810.281030 mapwrite 0x0055cf thru 0x00660b (0x103d bytes) 000317 1786401810.381777 mapwrite 0x02e8ee thru 0x0398ce (0xafe1 bytes) 000319 1786401810.461116 trunc from 0x040000 to 0x00822d (0x37dd2 bytes) 000320 1786401810.486136 trunc from 0x00822e to 0x00c3ea (0x41bd bytes) 000322 1786401810.514438 read 0x00982a thru 0x00c3ea (0x2bc1 bytes) 000325 1786401810.526766 trunc from 0x00c3eb to 0x01438b (0x7fa1 bytes) 000326 1786401810.550064 write 0x02765c thru 0x0288e2 (0x1287 bytes) 000328 1786401810.552045 trunc from 0x0288e3 to 0x030af9 (0x8217 bytes) 000330 1786401810.573834 write 0x01c269 thru 0x027983 (0xb71b bytes) 000331 1786401810.585596 read 0x02e498 thru 0x030af9 (0x2662 bytes) 000332 1786401810.586550 write 0x0102ef thru 0x015c19 (0x592b bytes) 000333 1786401810.588316 read 0x00befb thru 0x0129f2 (0x6af8 bytes) 000334 1786401810.589116 trunc from 0x030afa to 0x03c3b5 (0xb8bc bytes) 000335 1786401810.608427 mapwrite 0x00d8aa thru 0x01bd13 (0xe46a bytes) 000336 1786401810.664302 read 0x039279 thru 0x03c3b5 (0x313d bytes) 000337 1786401810.678938 mapwrite 0x022081 thru 0x0267b0 (0x4730 bytes) 000339 1786401810.729878 write 0x01ce07 thru 0x0203c5 (0x35bf bytes) 000341 1786401810.732606 read 0x0237f4 thru 0x024ca1 (0x14ae bytes) 000342 1786401810.733794 mapread 0x019c9f thru 0x021b9c (0x7efe bytes) 000343 1786401810.740242 MIRROR_EXTEND to 2 mirrors 000344 1786401811.533784 write 0x0262f4 thru 0x02fc2c (0x9939 bytes) 000347 1786401811.608867 mapread 0x02224b thru 0x02d0ab (0xae61 bytes) 000350 1786401811.627733 mapwrite 0x009492 thru 0x00b2be (0x1e2d bytes) 000351 1786401811.681400 read 0x002758 thru 0x007430 (0x4cd9 bytes) 000352 1786401811.682269 mapread 0x03a093 thru 0x03c3b5 (0x2323 bytes) 000353 1786401811.693646 write 0x003141 thru 0x00e6cc (0xb58c bytes) 000354 1786401811.694986 mapwrite 0x0286f6 thru 0x02f287 (0x6b92 bytes) 000356 1786401811.746489 read 0x02f1b5 thru 0x037f70 (0x8dbc bytes) 000358 1786401811.780242 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3c3b6): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3c3b6): Mirror 2: 0xe6326f12 (stale) CRC-32 checksum value for chunk [0, 0x3c3b6): Mirror 1: 0xd7e12e8b Mirror 3: 0xd7e12e8b lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3c3b6: skipped 000360 1786401812.510862 read 0x00e62e thru 0x019b6b (0xb53e bytes) 000361 1786401812.525035 trunc from 0x03c3b6 to 0x0348fa (0x7abb bytes) 000362 1786401812.639620 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x348fb): Mirror 3: 0x8e8584ac (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x348fb): Mirror 3: 0x8e8584ac (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x348fb: skipped 000364 1786401812.936863 read 0x00a32b thru 0x011b6c (0x7842 bytes) 000365 1786401812.961654 MIRROR_RESYNC 2 mirrors 000366 1786401813.553148 read 0x005394 thru 0x0054f6 (0x163 bytes) 000367 1786401813.559027 mapwrite 0x03148e thru 0x03ffff (0xeb72 bytes) 000369 1786401813.731381 mapwrite 0x027166 thru 0x02f857 (0x86f2 bytes) 000370 1786401813.801056 trunc from 0x040000 to 0x00ddc3 (0x3223c bytes) 000372 1786401813.841639 read 0x003c6e thru 0x0062a4 (0x2637 bytes) 000374 1786401813.856935 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0xddc4): Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xddc4): Mirror 3: 0x5ab961b3 (stale) CRC-32 checksum value for chunk [0, 0xddc4): Mirror 1: 0x5ab961b3 Mirror 4: 0x5ab961b3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xddc4: skipped 000375 1786401814.569430 trunc from 0x00ddc4 to 0x03c134 (0x2e371 bytes) 000376 1786401814.665954 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3c135): Mirror 4: 0x3ee9114 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3c135): Mirror 4: 0x3ee9114 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3c135: skipped 000377 1786401814.982504 write 0x017156 thru 0x019f69 (0x2e14 bytes) 000378 1786401815.002682 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3c135): Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3c135): Mirror 4: 0x3ee9114 (stale) CRC-32 checksum value for chunk [0, 0x3c135): Mirror 1: 0x8a7e5368 Mirror 5: 0x8a7e5368 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3c135: skipped 000379 1786401816.149759 mapwrite 0x020762 thru 0x02347d (0x2d1c bytes) 000380 1786401816.356978 mapread 0x006879 thru 0x00a2f7 (0x3a7f bytes) 000381 1786401816.383890 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3c135): Mirror 4: 0x3ee9114 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3c135): Mirror 4: 0x3ee9114 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3c135: skipped 000382 1786401816.681999 read 0x0154e8 thru 0x01e4ce (0x8fe7 bytes) 000384 1786401816.698807 write 0x03e2cd thru 0x03ffff (0x1d33 bytes) 000385 1786401816.701646 read 0x009824 thru 0x00dce2 (0x44bf bytes) 000387 1786401816.719737 write 0x02d3c6 thru 0x03cad2 (0xf70d bytes) 000388 1786401816.728406 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) 000391 1786401816.874043 trunc from 0x040000 to 0x02c232 (0x13dcd bytes) 000392 1786401816.901388 MIRROR_EXTEND to 2 mirrors 000393 1786401817.538660 MIRROR_RESYNC 2 mirrors 000395 1786401817.859114 mapread 0x00ac1d thru 0x00ea32 (0x3e16 bytes) 000399 1786401817.929387 MIRROR_EXTEND to 3 mirrors 000400 1786401818.630570 write 0x0365c0 thru 0x03818d (0x1bce bytes) 000403 1786401818.731132 read 0x0070b9 thru 0x016a73 (0xf9bb bytes) 000404 1786401818.747318 write 0x037bc5 thru 0x03ffff (0x843b bytes) 000405 1786401818.749108 mapwrite 0x00f60c thru 0x011d20 (0x2715 bytes) 000407 1786401818.807290 MIRROR_RESYNC 3 mirrors 000411 1786401819.613737 mapread 0x01dea3 thru 0x02b6dd (0xd83b bytes) 000412 1786401819.663443 mapread 0x021328 thru 0x02193e (0x617 bytes) 000413 1786401819.664633 trunc from 0x040000 to 0x016eb7 (0x29148 bytes) 000414 1786401819.747118 mapread 0x005335 thru 0x006edb (0x1ba7 bytes) 000415 1786401819.756907 mapwrite 0x00e7d5 thru 0x014606 (0x5e32 bytes) 000416 1786401819.819592 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x16eb8): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x16eb8): Mirror 2: 0x38534970 (stale) Mirror 3: 0x38534970 (stale) CRC-32 checksum value for chunk [0, 0x16eb8): Mirror 1: 0x5900f67c Mirror 4: 0x5900f67c lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x16eb8: skipped 000418 1786401820.712450 mapread 0x0156d5 thru 0x016eb7 (0x17e3 bytes) 000419 1786401820.751862 mapread 0x00c26e thru 0x016eb7 (0xac4a bytes) 000420 1786401820.786344 mapwrite 0x03a64a thru 0x03bacd (0x1484 bytes) 000422 1786401820.959713 MIRROR_RESYNC 4 mirrors 000424 1786401821.789576 write 0x037b66 thru 0x03ffff (0x849a bytes) 000426 1786401821.919987 trunc from 0x040000 to 0x02d724 (0x128db bytes) 000427 1786401821.969714 write 0x008c9b thru 0x00c810 (0x3b76 bytes) 000428 1786401821.982376 read 0x00d077 thru 0x0172c7 (0xa251 bytes) 000429 1786401821.994684 read 0x007438 thru 0x0166ec (0xf2b5 bytes) 000430 1786401822.023905 trunc from 0x02d725 to 0x037ce8 (0xa5c4 bytes) 000432 1786401822.043736 mapwrite 0x038f21 thru 0x03fdc8 (0x6ea8 bytes) 000435 1786401822.202535 write 0x02e2ff thru 0x03b662 (0xd364 bytes) 000436 1786401822.214243 MIRROR_RESYNC 4 mirrors 000437 1786401823.203769 read 0x010ed6 thru 0x011ae9 (0xc14 bytes) 000438 1786401823.209303 read 0x003854 thru 0x003e10 (0x5bd bytes) 000439 1786401823.220888 mapwrite 0x01d1a7 thru 0x02a0a6 (0xcf00 bytes) 000440 1786401823.417233 write 0x01f8eb thru 0x022a31 (0x3147 bytes) 000441 1786401823.418930 trunc from 0x03fdc9 to 0x01a655 (0x25773 bytes) 000442 1786401823.479538 MIRROR_RESYNC 4 mirrors 000444 1786401824.639738 MIRROR_RESYNC 4 mirrors 000446 1786401825.048340 write 0x0188a0 thru 0x026892 (0xdff3 bytes) 000447 1786401825.127625 MIRROR_RESYNC 4 mirrors 000448 1786401826.130176 write 0x01fafb thru 0x02dd9c (0xe2a2 bytes) 000449 1786401826.186287 mapread 0x025fcf thru 0x02a45d (0x448f bytes) 000452 1786401826.190366 mapread 0x02bc98 thru 0x02dd9c (0x2105 bytes) 000453 1786401826.192112 mapread 0x0151eb thru 0x023242 (0xe058 bytes) 000455 1786401826.218154 trunc from 0x02dd9d to 0x0232ac (0xaaf0 bytes) 000457 1786401826.246961 mapread 0x01d256 thru 0x0232ac (0x6057 bytes) 000458 1786401826.250177 write 0x000ec8 thru 0x00dabe (0xcbf7 bytes) 000459 1786401826.261638 write 0x01ab1c thru 0x01e9ff (0x3ee4 bytes) 000460 1786401826.263065 read 0x0000ae thru 0x00c32e (0xc281 bytes) 000462 1786401826.271638 MIRROR_RESYNC 4 mirrors 000463 1786401826.962600 trunc from 0x0232ad to 0x03672c (0x13480 bytes) 000464 1786401827.072725 trunc from 0x03672d to 0x01873c (0x1dff0 bytes) 000466 1786401827.112279 mapread 0x01445e thru 0x017e09 (0x39ac bytes) 000467 1786401827.146363 write 0x022c24 thru 0x023daa (0x1187 bytes) 000468 1786401827.147982 mapread 0x008d13 thru 0x0150ac (0xc39a bytes) 000472 1786401827.162146 mapwrite 0x0126f6 thru 0x012cb7 (0x5c2 bytes) 000474 1786401827.213623 mapread 0x009ef6 thru 0x012b38 (0x8c43 bytes) 000475 1786401827.215603 mapwrite 0x018a1d thru 0x0244c0 (0xbaa4 bytes) 000476 1786401827.301680 mapread 0x01f0b7 thru 0x0244c0 (0x540a bytes) 000477 1786401827.309091 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x244c1): Mirror 2: (stale - read error) Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x244c1): Mirror 2: 0xbe86109b (stale) Mirror 3: 0xbe86109b (stale) Mirror 4: 0xbe86109b (stale) CRC-32 checksum value for chunk [0, 0x244c1): Mirror 1: 0xbe3fbcd9 Mirror 5: 0xbe3fbcd9 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x244c1: skipped 000478 1786401827.986365 write 0x01f0af thru 0x02c919 (0xd86b bytes) 000479 1786401828.064996 read 0x022d57 thru 0x02c919 (0x9bc3 bytes) 000481 1786401828.067657 mapwrite 0x0195dd thru 0x01e940 (0x5364 bytes) 000483 1786401828.146798 write 0x001273 thru 0x00f491 (0xe21f bytes) 000484 1786401828.159593 mapread 0x00d763 thru 0x01c965 (0xf203 bytes) 000485 1786401828.169905 write 0x016d0c thru 0x01b29a (0x458f bytes) 000486 1786401828.171952 write 0x023e2a thru 0x032ccd (0xeea4 bytes) 000487 1786401828.174029 mapread 0x01010a thru 0x0153e9 (0x52e0 bytes) 000493 1786401828.185820 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x32cce): Mirror 3: 0xd5b44073 (stale) Mirror 4: 0xd5b44073 (stale) Mirror 5: 0x300e5ce (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x32cce): Mirror 3: 0xd5b44073 (stale) Mirror 4: 0xd5b44073 (stale) Mirror 5: 0x300e5ce (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x32cce: skipped 000494 1786401828.561838 mapwrite 0x037703 thru 0x038d78 (0x1676 bytes) 000495 1786401828.681647 mapwrite 0x001562 thru 0x00d927 (0xc3c6 bytes) 000496 1786401828.764672 MIRROR_RESYNC 4 mirrors 000497 1786401829.652906 MIRROR_EXTEND to 5 mirrors 000499 1786401830.251313 mapread 0x012b1d thru 0x01fa12 (0xcef6 bytes) 000500 1786401830.278837 trunc from 0x038d79 to 0x0338ab (0x54cd bytes) 000501 1786401830.389087 write 0x01b1fc thru 0x026132 (0xaf37 bytes) 000502 1786401830.403113 mapread 0x02f5f7 thru 0x03294f (0x3359 bytes) 000503 1786401830.425570 trunc from 0x0338ac to 0x0375b9 (0x3d0e bytes) 000505 1786401830.461714 write 0x02eae5 thru 0x03aa82 (0xbf9e bytes) 000507 1786401830.471877 MIRROR_RESYNC 5 mirrors 000508 1786401831.400675 trunc from 0x03aa83 to 0x02c4a9 (0xe5d9 bytes) 000509 1786401831.491420 mapread 0x01ceae thru 0x01f16d (0x22c0 bytes) 000510 1786401831.526583 read 0x00bc2d thru 0x0156fb (0x9acf bytes) 000511 1786401831.540835 mapread 0x018793 thru 0x01e99e (0x620c bytes) 000513 1786401831.543389 read 0x008b16 thru 0x0119a0 (0x8e8b bytes) 000514 1786401831.558791 mapread 0x01c283 thru 0x027280 (0xaffe bytes) 000515 1786401831.560626 mapread 0x01fb46 thru 0x02ad1e (0xb1d9 bytes) 000516 1786401831.562768 trunc from 0x02c4aa to 0x014849 (0x17c60 bytes) 000517 1786401831.594011 mapread 0x0053a2 thru 0x00ba51 (0x66b0 bytes) 000518 1786401831.596070 mapwrite 0x0045bf thru 0x00a0a9 (0x5aeb bytes) 000519 1786401831.664114 MIRROR_RESYNC 5 mirrors 000520 1786401832.786677 mapwrite 0x02172a thru 0x02745c (0x5d33 bytes) 000521 1786401832.941051 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x2745d): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2745d): Mirror 3: 0x2e8519b5 (stale) Mirror 4: 0x2e8519b5 (stale) Mirror 5: 0x2e8519b5 (stale) Mirror 6: 0x2e8519b5 (stale) CRC-32 checksum value for chunk [0, 0x2745d): Mirror 1: 0xb169099 Mirror 7: 0xb169099 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2745d: skipped 000525 1786401833.986774 mapwrite 0x02ee27 thru 0x03e4df (0xf6b9 bytes) 000526 1786401834.215508 mapwrite 0x00e642 thru 0x019cc0 (0xb67f bytes) 000528 1786401834.360158 write 0x023e80 thru 0x0332d6 (0xf457 bytes) 000530 1786401834.373787 write 0x02303f thru 0x02e366 (0xb328 bytes) 000531 1786401834.379315 read 0x00c315 thru 0x012ccd (0x69b9 bytes) 000534 1786401834.393426 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x3e4e0): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e4e0): Mirror 3: 0x5c260655 (stale) Mirror 4: 0x5c260655 (stale) Mirror 5: 0x5c260655 (stale) Mirror 6: 0x5c260655 (stale) Mirror 7: 0xfa9bc996 (stale) CRC-32 checksum value for chunk [0, 0x3e4e0): Mirror 1: 0xd98b958f Mirror 8: 0xd98b958f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e4e0: skipped 000535 1786401835.285525 write 0x014234 thru 0x01e530 (0xa2fd bytes) 000536 1786401835.401097 MIRROR_RESYNC 7 mirrors 000537 1786401836.692711 trunc from 0x03e4e0 to 0x03d61b (0xec4 bytes) 000540 1786401836.776240 mapread 0x01f23c thru 0x02b839 (0xc5fe bytes) 000541 1786401836.816564 read 0x02c467 thru 0x03a0a7 (0xdc41 bytes) 000543 1786401836.828315 read 0x0159ed thru 0x01b2c4 (0x58d8 bytes) 000548 1786401836.843419 mapread 0x0044c6 thru 0x007776 (0x32b1 bytes) 000550 1786401836.846032 write 0x036d86 thru 0x03b60b (0x4886 bytes) 000553 1786401836.850096 mapread 0x0218df thru 0x0228c6 (0xfe8 bytes) 000555 1786401836.853778 read 0x034104 thru 0x035b9e (0x1a9b bytes) 000557 1786401836.868306 MIRROR_EXTEND to 8 mirrors CRC-32 checksum value for chunk [0, 0x3d61c): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3d61c): Mirror 3: 0x5e7a856 (stale) Mirror 4: 0x5e7a856 (stale) Mirror 5: 0x5e7a856 (stale) Mirror 6: 0x5e7a856 (stale) Mirror 7: 0x5e7a856 (stale) Mirror 8: 0x5e7a856 (stale) CRC-32 checksum value for chunk [0, 0x3d61c): Mirror 1: 0x85738515 Mirror 9: 0x85738515 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3d61c: skipped 000559 1786401837.827246 trunc from 0x03d61c to 0x0219b5 (0x1bc66 bytes) 000563 1786401837.932711 read 0x01271d thru 0x0219b5 (0xf299 bytes) 000565 1786401837.942408 trunc from 0x0219b6 to 0x006079 (0x1b93c bytes) 000566 1786401837.975981 mapread 0x005c82 thru 0x006079 (0x3f8 bytes) 000567 1786401837.999724 MIRROR_SPLIT mirror 2 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x607a): Mirror 3: 0xdeb7a99a (stale) Mirror 5: 0xdeb7a99a (stale) Mirror 6: 0xdeb7a99a (stale) Mirror 7: 0xdeb7a99a (stale) Mirror 8: 0xdeb7a99a (stale) Mirror 9: 0xdeb7a99a (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x607a): Mirror 3: 0xdeb7a99a (stale) Mirror 5: 0xdeb7a99a (stale) Mirror 6: 0xdeb7a99a (stale) Mirror 7: 0xdeb7a99a (stale) Mirror 8: 0xdeb7a99a (stale) Mirror 9: 0xdeb7a99a (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x607a: skipped 000568 1786401838.574164 mapread 0x005a7a thru 0x006079 (0x600 bytes) 000569 1786401838.592636 mapread 0x000851 thru 0x006079 (0x5829 bytes) 000570 1786401838.605942 mapwrite 0x0370d1 thru 0x0381d7 (0x1107 bytes) 000571 1786401838.692251 mapwrite 0x03a7b3 thru 0x03fac2 (0x5310 bytes) 000572 1786401838.783461 MIRROR_EXTEND to 8 mirrors CRC-32 checksum value for chunk [0, 0x3fac3): Mirror 3: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3fac3): Mirror 3: 0xb601de4c (stale) Mirror 5: 0xb601de4c (stale) Mirror 6: 0xb601de4c (stale) Mirror 7: 0xb601de4c (stale) Mirror 8: 0xb601de4c (stale) Mirror 9: 0x26248fd9 (stale) CRC-32 checksum value for chunk [0, 0x3fac3): Mirror 1: 0x18b1d278 Mirror 10: 0x18b1d278 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3fac3: skipped 000574 1786401839.969499 write 0x0239cb thru 0x0277e2 (0x3e18 bytes) 000576 1786401840.150417 mapread 0x03f0fe thru 0x03fac2 (0x9c5 bytes) 000577 1786401840.177587 MIRROR_SPLIT mirror 4 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3fac3): Mirror 3: 0xb601de4c (stale) Mirror 5: 0xb601de4c (stale) Mirror 6: 0xb601de4c (stale) Mirror 8: 0xb601de4c (stale) Mirror 9: 0x26248fd9 (stale) Mirror 10: 0x18b1d278 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3fac3): Mirror 3: 0xb601de4c (stale) Mirror 5: 0xb601de4c (stale) Mirror 6: 0xb601de4c (stale) Mirror 8: 0xb601de4c (stale) Mirror 9: 0x26248fd9 (stale) Mirror 10: 0x18b1d278 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3fac3: skipped 000578 1786401840.714958 read 0x0350c0 thru 0x03c250 (0x7191 bytes) 000582 1786401840.726897 write 0x028d9c thru 0x0339a9 (0xac0e bytes) 000586 1786401840.744661 write 0x0381e4 thru 0x03f3f2 (0x720f bytes) 000587 1786401840.752027 mapread 0x02a170 thru 0x0328fe (0x878f bytes) 000588 1786401840.754030 read 0x03e4ae thru 0x03fac2 (0x1615 bytes) 000589 1786401840.759913 MIRROR_SPLIT mirror 1 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3fac3): Mirror 5: 0xb601de4c (stale) Mirror 6: 0xb601de4c (stale) Mirror 8: 0xb601de4c (stale) Mirror 9: 0x26248fd9 (stale) Mirror 10: 0x18b1d278 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3fac3): Mirror 5: 0xb601de4c (stale) Mirror 6: 0xb601de4c (stale) Mirror 8: 0xb601de4c (stale) Mirror 9: 0x26248fd9 (stale) Mirror 10: 0x18b1d278 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3fac3: skipped 000592 1786401841.273169 write 0x00f60e thru 0x01b791 (0xc184 bytes) 000593 1786401841.282894 write 0x00fb38 thru 0x0119c3 (0x1e8c bytes) 000594 1786401841.284390 write 0x00eace thru 0x01669e (0x7bd1 bytes) 000595 1786401841.294423 MIRROR_SPLIT mirror 4 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3fac3): Mirror 5: 0xb601de4c (stale) Mirror 6: 0xb601de4c (stale) Mirror 8: 0xb601de4c (stale) Mirror 10: 0x18b1d278 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3fac3): Mirror 5: 0xb601de4c (stale) Mirror 6: 0xb601de4c (stale) Mirror 8: 0xb601de4c (stale) Mirror 10: 0x18b1d278 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3fac3: skipped 000597 1786401841.676721 trunc from 0x03fac3 to 0x036fe4 (0x8ade bytes) 000601 1786401841.699695 write 0x00e6ee thru 0x00ff28 (0x183b bytes) 000602 1786401841.709543 read 0x001cea thru 0x0061d3 (0x44ea bytes) 000603 1786401841.719570 mapread 0x017399 thru 0x01ca58 (0x56c0 bytes) 000604 1786401841.737692 mapread 0x0347e4 thru 0x036772 (0x1f8f bytes) 000605 1786401841.746584 mapwrite 0x00c8c2 thru 0x01c541 (0xfc80 bytes) 000606 1786401841.816278 write 0x006ab9 thru 0x011c33 (0xb17b bytes) 000608 1786401841.821320 mapread 0x0280aa thru 0x02a700 (0x2657 bytes) 000614 1786401841.851328 MIRROR_RESYNC 5 mirrors 000617 1786401843.015378 mapwrite 0x014cd5 thru 0x020fb9 (0xc2e5 bytes) 000618 1786401843.195415 read 0x00b86b thru 0x00e84b (0x2fe1 bytes) 000619 1786401843.204922 trunc from 0x036fe5 to 0x01b480 (0x1bb64 bytes) 000620 1786401843.232307 read 0x00e89a thru 0x00fafe (0x1265 bytes) 000622 1786401843.234266 mapwrite 0x000850 thru 0x000d2c (0x4dd bytes) 000623 1786401843.307588 trunc from 0x01b481 to 0x0357f3 (0x1a373 bytes) 000624 1786401843.330566 mapread 0x0290a0 thru 0x0357f3 (0xc754 bytes) 000625 1786401843.344852 trunc from 0x0357f4 to 0x0068a9 (0x2ef4a bytes) 000626 1786401843.375869 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x68aa): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 8: (stale - read error) Mirror 10: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x68aa): Mirror 5: 0x29b4fc6e (stale) Mirror 6: 0x29b4fc6e (stale) Mirror 8: 0x29b4fc6e (stale) Mirror 10: 0x29b4fc6e (stale) CRC-32 checksum value for chunk [0, 0x68aa): Mirror 1: 0x928fef53 Mirror 11: 0x928fef53 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x68aa: skipped 000628 1786401844.150002 mapread 0x000904 thru 0x0068a9 (0x5fa6 bytes) 000631 1786401844.205351 read 0x0002a5 thru 0x0068a9 (0x6605 bytes) 000634 1786401844.220297 mapread 0x00344c thru 0x0059d2 (0x2587 bytes) 000635 1786401844.227839 read 0x000e73 thru 0x0068a9 (0x5a37 bytes) 000637 1786401844.235940 read 0x000b3c thru 0x0068a9 (0x5d6e bytes) 000638 1786401844.239619 read 0x0016d4 thru 0x0032fe (0x1c2b bytes) 000640 1786401844.247822 read 0x002350 thru 0x0068a9 (0x455a bytes) 000641 1786401844.253651 MIRROR_SPLIT mirror 4 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x68aa): Mirror 5: 0x29b4fc6e (stale) Mirror 6: 0x29b4fc6e (stale) Mirror 8: 0x29b4fc6e (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x68aa): Mirror 5: 0x29b4fc6e (stale) Mirror 6: 0x29b4fc6e (stale) Mirror 8: 0x29b4fc6e (stale) CRC-32 checksum value for chunk [0, 0x68aa): Mirror 1: 0x928fef53 Mirror 11: 0x928fef53 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x68aa: skipped 000642 1786401844.800840 mapread 0x0030ef thru 0x0068a9 (0x37bb bytes) 000644 1786401844.838163 write 0x019038 thru 0x0260ff (0xd0c8 bytes) 000648 1786401844.956812 trunc from 0x026100 to 0x02ed16 (0x8c17 bytes) 000649 1786401845.007501 MIRROR_RESYNC 5 mirrors 000651 1786401845.935466 trunc from 0x02ed17 to 0x00df0d (0x20e09 bytes) 000652 1786401846.008812 read 0x00b914 thru 0x00df0d (0x25fa bytes) 000653 1786401846.015867 mapread 0x00c296 thru 0x00df0d (0x1c78 bytes) 000654 1786401846.017318 mapwrite 0x00ce1f thru 0x00f880 (0x2a62 bytes) 000657 1786401846.104247 read 0x008d0c thru 0x00f880 (0x6b75 bytes) 000659 1786401846.119740 MIRROR_SPLIT mirror 2 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0xf881): Mirror 5: 0x3a8a575b (stale) Mirror 8: 0x3a8a575b (stale) Mirror 11: 0x3a8a575b (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xf881): Mirror 5: 0x3a8a575b (stale) Mirror 8: 0x3a8a575b (stale) Mirror 11: 0x3a8a575b (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xf881: skipped 000660 1786401846.463481 read 0x003873 thru 0x00f880 (0xc00e bytes) 000661 1786401846.477756 trunc from 0x00f881 to 0x026e52 (0x175d2 bytes) 000662 1786401846.512142 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x26e53): Mirror 8: 0xb9ea18db (stale) Mirror 11: 0xb9ea18db (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x26e53): Mirror 8: 0xb9ea18db (stale) Mirror 11: 0xb9ea18db (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x26e53: skipped 000664 1786401846.857223 trunc from 0x026e53 to 0x035974 (0xeb22 bytes) 000665 1786401846.874717 read 0x013949 thru 0x01bd58 (0x8410 bytes) 000666 1786401846.886838 mapwrite 0x0164c7 thru 0x021657 (0xb191 bytes) 000667 1786401846.963287 write 0x033e56 thru 0x03db1c (0x9cc7 bytes) 000668 1786401846.974349 mapread 0x026c42 thru 0x029684 (0x2a43 bytes) 000669 1786401846.989334 read 0x01f415 thru 0x02cc26 (0xd812 bytes) 000671 1786401846.992404 read 0x006b13 thru 0x00c4bb (0x59a9 bytes) 000672 1786401847.014849 trunc from 0x03db1d to 0x03b6b4 (0x2468 bytes) 000673 1786401847.036141 mapread 0x028db6 thru 0x037436 (0xe681 bytes) 000674 1786401847.038916 read 0x03749f thru 0x03b6b4 (0x4216 bytes) 000675 1786401847.044155 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3b6b5): Mirror 11: 0x8a7ef89c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3b6b5): Mirror 11: 0x8a7ef89c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3b6b5: skipped 000676 1786401847.349290 write 0x028e51 thru 0x02e30c (0x54bc bytes) 000677 1786401847.360665 mapread 0x01edfb thru 0x0226b9 (0x38bf bytes) 000678 1786401847.376116 trunc from 0x03b6b5 to 0x014bcc (0x26ae8 bytes) 000680 1786401847.394834 write 0x032cc6 thru 0x035936 (0x2c71 bytes) 000681 1786401847.397225 read 0x02b55c thru 0x035936 (0xa3db bytes) 000682 1786401847.408484 MIRROR_RESYNC 2 mirrors 000684 1786401848.091779 mapread 0x005844 thru 0x0078a8 (0x2065 bytes) 000685 1786401848.110972 mapread 0x0050f4 thru 0x00e245 (0x9152 bytes) 000687 1786401848.112150 read 0x003416 thru 0x010ae2 (0xd6cd bytes) 000688 1786401848.112751 mapwrite 0x014547 thru 0x01efce (0xaa88 bytes) 000689 1786401848.308271 write 0x019663 thru 0x0269a7 (0xd345 bytes) 000690 1786401848.321686 mapwrite 0x005d09 thru 0x013859 (0xdb51 bytes) 000693 1786401848.393032 write 0x00b8cd thru 0x01397b (0x80af bytes) 000698 1786401848.397716 trunc from 0x035937 to 0x02e837 (0x70ff bytes) 000699 1786401848.445738 write 0x017e7a thru 0x021bd5 (0x9d5c bytes) 000700 1786401848.452974 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) 000703 1786401848.641157 mapwrite 0x0007a4 thru 0x00f7cc (0xf029 bytes) 000707 1786401848.761176 read 0x001ae5 thru 0x009985 (0x7ea1 bytes) 000708 1786401848.762166 read 0x016d73 thru 0x01ecdb (0x7f69 bytes) 000709 1786401848.774527 write 0x002f52 thru 0x0073a7 (0x4456 bytes) 000710 1786401848.775994 mapread 0x027c79 thru 0x02e837 (0x6bbf bytes) 000712 1786401848.778287 mapread 0x017ef0 thru 0x027a02 (0xfb13 bytes) 000714 1786401848.781067 write 0x0369b9 thru 0x03ffff (0x9647 bytes) 000715 1786401848.784025 mapread 0x00f02b thru 0x011eb5 (0x2e8b bytes) 000716 1786401848.788705 trunc from 0x040000 to 0x00878c (0x37873 bytes) 000717 1786401848.818810 mapwrite 0x000af6 thru 0x00f9fc (0xef07 bytes) 000718 1786401848.936510 write 0x03fea1 thru 0x03ffff (0x15f bytes) 000719 1786401848.940976 trunc from 0x040000 to 0x02d7b2 (0x1284d bytes) 000721 1786401848.961887 read 0x001327 thru 0x00e084 (0xcd5e bytes) 000722 1786401848.964546 trunc from 0x02d7b3 to 0x01063f (0x1d173 bytes) 000723 1786401848.990046 write 0x00e304 thru 0x01ca6b (0xe768 bytes) 000728 1786401849.002915 write 0x0043fa thru 0x010fe5 (0xcbec bytes) 000729 1786401849.007886 read 0x0161b9 thru 0x01650c (0x354 bytes) 000730 1786401849.011817 write 0x02351b thru 0x032226 (0xed0c bytes) 000732 1786401849.019016 mapwrite 0x016e53 thru 0x0241da (0xd388 bytes) 000733 1786401849.134503 MIRROR_EXTEND to 2 mirrors 000735 1786401849.745531 mapread 0x02554c thru 0x02ab9e (0x5653 bytes) 000736 1786401849.773627 write 0x029390 thru 0x0353b6 (0xc027 bytes) 000739 1786401849.882087 MIRROR_RESYNC 2 mirrors 000741 1786401850.565665 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) 000743 1786401850.707896 trunc from 0x0353b7 to 0x002bd7 (0x327df bytes) 000745 1786401850.754832 MIRROR_EXTEND to 2 mirrors 000746 1786401851.509635 MIRROR_EXTEND to 3 mirrors 000748 1786401852.107920 read 0x000b7c thru 0x002bd7 (0x205c bytes) 000749 1786401852.116988 mapread 0x001585 thru 0x002bd7 (0x1653 bytes) 000750 1786401852.118025 mapread 0x00120b thru 0x002bd7 (0x19cd bytes) 000751 1786401852.119711 write 0x00ee51 thru 0x01b82f (0xc9df bytes) 000753 1786401852.197756 mapread 0x01b0f1 thru 0x01b82f (0x73f bytes) 000756 1786401852.201230 write 0x036788 thru 0x03fae3 (0x935c bytes) 000757 1786401852.208295 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3fae4): Mirror 2: 0x359c0189 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3fae4): Mirror 2: 0x359c0189 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3fae4: skipped 000758 1786401852.509012 mapwrite 0x031893 thru 0x03dfc8 (0xc736 bytes) 000759 1786401852.591003 read 0x02a09f thru 0x034aec (0xaa4e bytes) 000761 1786401852.597378 mapread 0x00f327 thru 0x00fe0b (0xae5 bytes) 000762 1786401852.606960 read 0x02da15 thru 0x02db87 (0x173 bytes) 000763 1786401852.608021 mapread 0x01c838 thru 0x022d14 (0x64dd bytes) 000764 1786401852.620225 read 0x03d100 thru 0x03fae3 (0x29e4 bytes) 000766 1786401852.622178 write 0x00b445 thru 0x0113d0 (0x5f8c bytes) 000769 1786401852.625793 mapwrite 0x0044ee thru 0x00e3ae (0x9ec1 bytes) 000771 1786401852.671739 read 0x00562e thru 0x014461 (0xee34 bytes) 000772 1786401852.672975 mapwrite 0x02864f thru 0x0379ca (0xf37c bytes) 000774 1786401852.722730 read 0x032999 thru 0x0348bf (0x1f27 bytes) 000775 1786401852.723429 read 0x00db08 thru 0x01864e (0xab47 bytes) 000776 1786401852.724857 mapwrite 0x01bb7b thru 0x02a071 (0xe4f7 bytes) 000778 1786401852.779132 mapread 0x024e50 thru 0x0345e2 (0xf793 bytes) 000779 1786401852.783137 read 0x0222b3 thru 0x02257b (0x2c9 bytes) 000780 1786401852.784210 mapwrite 0x02e5bb thru 0x03ab74 (0xc5ba bytes) 000781 1786401852.825917 mapread 0x03dc04 thru 0x03fae3 (0x1ee0 bytes) 000782 1786401852.829701 mapwrite 0x039a7a thru 0x03ffff (0x6586 bytes) 000786 1786401852.895700 mapread 0x01e3c9 thru 0x01fa71 (0x16a9 bytes) 000787 1786401852.897165 write 0x01aa98 thru 0x01b4e9 (0xa52 bytes) 000788 1786401852.898732 read 0x00fb55 thru 0x016954 (0x6e00 bytes) 000790 1786401852.903312 read 0x03f70e thru 0x03ffff (0x8f2 bytes) 000792 1786401852.905086 mapread 0x03e9ea thru 0x03ffff (0x1616 bytes) 000793 1786401852.907031 read 0x007967 thru 0x010064 (0x86fe bytes) 000794 1786401852.908087 read 0x011dd4 thru 0x013d0a (0x1f37 bytes) 000795 1786401852.915680 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xc27183f8 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x3d3a2967 Mirror 3: 0x3d3a2967 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000797 1786401853.696964 read 0x021d2b thru 0x024194 (0x246a bytes) 000800 1786401853.711483 MIRROR_RESYNC 3 mirrors 000804 1786401854.381173 MIRROR_EXTEND to 4 mirrors 000805 1786401855.065142 mapread 0x00187c thru 0x00fc57 (0xe3dc bytes) 000806 1786401855.109842 MIRROR_EXTEND to 5 mirrors 000807 1786401855.694256 mapwrite 0x02ec66 thru 0x03e4c8 (0xf863 bytes) 000808 1786401855.932177 trunc from 0x040000 to 0x036e7b (0x9184 bytes) 000809 1786401855.963375 MIRROR_RESYNC 5 mirrors 000811 1786401856.913948 write 0x035300 thru 0x037781 (0x2482 bytes) 000812 1786401856.989628 mapread 0x00780b thru 0x00acf0 (0x34e6 bytes) 000813 1786401857.011755 write 0x02d943 thru 0x030da1 (0x345f bytes) 000814 1786401857.022058 write 0x02047a thru 0x026236 (0x5dbd bytes) 000816 1786401857.034569 trunc from 0x037782 to 0x00d8f3 (0x29e8e bytes) 000818 1786401857.066120 read 0x003754 thru 0x007a43 (0x42f0 bytes) 000823 1786401857.074907 MIRROR_RESYNC 5 mirrors 000826 1786401857.993728 trunc from 0x00d8f4 to 0x00c1cd (0x1726 bytes) 000827 1786401858.105991 mapwrite 0x03d602 thru 0x03f7c2 (0x21c1 bytes) 000828 1786401858.201724 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3f7c3): Mirror 3: 0x2acc73b0 (stale) Mirror 4: 0x2acc73b0 (stale) Mirror 5: 0x2acc73b0 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3f7c3): Mirror 3: 0x2acc73b0 (stale) Mirror 4: 0x2acc73b0 (stale) Mirror 5: 0x2acc73b0 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3f7c3: skipped 000829 1786401858.569060 mapwrite 0x0012d4 thru 0x010325 (0xf052 bytes) 000831 1786401858.683161 mapwrite 0x020cb7 thru 0x029b27 (0x8e71 bytes) 000834 1786401858.806275 read 0x025fa2 thru 0x02726f (0x12ce bytes) 000838 1786401858.813803 MIRROR_RESYNC 4 mirrors 000839 1786401859.637723 read 0x00780f thru 0x00b422 (0x3c14 bytes) 000840 1786401859.647513 trunc from 0x03f7c3 to 0x0003c0 (0x3f402 bytes) 000841 1786401859.707089 read 0x0000a2 thru 0x0003c0 (0x31f bytes) 000843 1786401859.716954 write 0x015931 thru 0x01af7b (0x564b bytes) 000844 1786401859.719718 write 0x011957 thru 0x01deff (0xc5a9 bytes) 000845 1786401859.735006 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x1df00): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1df00): Mirror 3: 0xf0ebdec9 (stale) Mirror 4: 0xf0ebdec9 (stale) Mirror 5: 0xf0ebdec9 (stale) CRC-32 checksum value for chunk [0, 0x1df00): Mirror 1: 0xbef7fb0e Mirror 6: 0xbef7fb0e lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1df00: skipped 000848 1786401860.576383 trunc from 0x01df00 to 0x0096b9 (0x14846 bytes) 000854 1786401860.685107 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x96ba): Mirror 4: 0xd51da616 (stale) Mirror 5: 0xd51da616 (stale) Mirror 6: 0xc8f8870e (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x96ba): Mirror 4: 0xd51da616 (stale) Mirror 5: 0xd51da616 (stale) Mirror 6: 0xc8f8870e (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x96ba: skipped 000855 1786401861.088025 trunc from 0x0096ba to 0x0004d9 (0x91e0 bytes) 000856 1786401861.109077 mapwrite 0x03a6b6 thru 0x03e935 (0x4280 bytes) 000857 1786401861.190723 read 0x0064d3 thru 0x0095f2 (0x3120 bytes) 000858 1786401861.196855 read 0x015770 thru 0x024c37 (0xf4c8 bytes) 000859 1786401861.223558 write 0x01c2d6 thru 0x024e86 (0x8bb1 bytes) 000860 1786401861.225451 read 0x010f86 thru 0x01759a (0x6615 bytes) 000861 1786401861.226283 trunc from 0x03e936 to 0x01e1b2 (0x20783 bytes) 000864 1786401861.248477 mapwrite 0x005484 thru 0x011368 (0xbee5 bytes) 000866 1786401861.323137 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1e1b3): Mirror 4: 0xea91c77d (stale) Mirror 6: 0x24c6c2a5 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1e1b3): Mirror 4: 0xea91c77d (stale) Mirror 6: 0x24c6c2a5 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1e1b3: skipped 000868 1786401861.693941 trunc from 0x01e1b3 to 0x034636 (0x16484 bytes) 000869 1786401861.723899 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x34637): Mirror 4: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x34637): Mirror 4: 0xe72f00cd (stale) Mirror 6: 0xf8dbd6d1 (stale) CRC-32 checksum value for chunk [0, 0x34637): Mirror 1: 0xefac8811 Mirror 7: 0xefac8811 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x34637: skipped 000870 1786401862.519169 trunc from 0x034637 to 0x000f0a (0x3372c bytes) 000871 1786401862.608663 mapread 0x000bc3 thru 0x000f0a (0x348 bytes) 000872 1786401862.615727 write 0x0285ee thru 0x03758e (0xefa1 bytes) 000873 1786401862.619233 trunc from 0x03758f to 0x00658a (0x31004 bytes) 000875 1786401862.654765 mapread 0x001a97 thru 0x002943 (0xead bytes) 000878 1786401862.671512 read 0x000860 thru 0x00658a (0x5d2b bytes) 000879 1786401862.677938 read 0x001493 thru 0x00658a (0x50f8 bytes) 000882 1786401862.689559 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x658b): Mirror 4: 0x3bfe20a0 (stale) Mirror 6: 0x18e1dc73 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x658b): Mirror 4: 0x3bfe20a0 (stale) Mirror 6: 0x18e1dc73 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x658b: skipped 000884 1786401863.092171 trunc from 0x00658b to 0x03122b (0x2aca1 bytes) 000885 1786401863.119731 trunc from 0x03122c to 0x0375af (0x6384 bytes) 000889 1786401863.141139 mapread 0x02fa8a thru 0x0375af (0x7b26 bytes) 000890 1786401863.170013 trunc from 0x0375b0 to 0x020690 (0x16f1f bytes) 000891 1786401863.187320 trunc from 0x020691 to 0x01cb8d (0x3b03 bytes) 000892 1786401863.209835 mapread 0x0109f6 thru 0x0152b6 (0x48c1 bytes) 000894 1786401863.234132 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x1cb8e): Mirror 4: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1cb8e): Mirror 4: 0x6b2d9e91 (stale) Mirror 6: 0x3b985c86 (stale) CRC-32 checksum value for chunk [0, 0x1cb8e): Mirror 1: 0x93e27dc2 Mirror 7: 0x93e27dc2 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1cb8e: skipped 000897 1786401864.142851 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1cb8e): Mirror 4: 0x6b2d9e91 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1cb8e): Mirror 4: 0x6b2d9e91 (stale) CRC-32 checksum value for chunk [0, 0x1cb8e): Mirror 1: 0x93e27dc2 Mirror 7: 0x93e27dc2 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1cb8e: skipped 000898 1786401864.561482 read 0x004290 thru 0x00a576 (0x62e7 bytes) 000899 1786401864.574850 mapread 0x01b99c thru 0x01cb8d (0x11f2 bytes) 000900 1786401864.590656 write 0x033872 thru 0x03e65b (0xadea bytes) 000901 1786401864.668885 write 0x003bca thru 0x00d2d3 (0x970a bytes) 000902 1786401864.690588 MIRROR_RESYNC 3 mirrors 000903 1786401865.489720 mapwrite 0x02b05f thru 0x031c10 (0x6bb2 bytes) 000905 1786401865.643331 mapread 0x031a09 thru 0x031ee1 (0x4d9 bytes) 000906 1786401865.645614 trunc from 0x03e65c to 0x012216 (0x2c445 bytes) 000907 1786401865.676916 mapwrite 0x017f7f thru 0x020dfb (0x8e7d bytes) 000908 1786401865.760376 write 0x01c860 thru 0x01eff4 (0x2795 bytes) 000910 1786401865.762785 mapread 0x00d2be thru 0x01c1bd (0xef00 bytes) 000918 1786401865.778672 mapread 0x00aedc thru 0x00e158 (0x327d bytes) 000919 1786401865.780400 write 0x021a51 thru 0x02d29f (0xb84f bytes) 000922 1786401865.783816 read 0x024767 thru 0x02c989 (0x8223 bytes) 000923 1786401865.784883 read 0x00cc13 thru 0x019c74 (0xd062 bytes) 000924 1786401865.786402 write 0x007927 thru 0x00a6a8 (0x2d82 bytes) 000926 1786401865.788573 mapwrite 0x02fc65 thru 0x03953e (0x98da bytes) 000929 1786401865.867682 trunc from 0x03953f to 0x003107 (0x36437 bytes) 000930 1786401865.882540 write 0x0343c7 thru 0x034ca8 (0x8e2 bytes) 000932 1786401865.890379 MIRROR_RESYNC 3 mirrors 000935 1786401866.606798 MIRROR_RESYNC 3 mirrors 000938 1786401866.997456 read 0x011cbc thru 0x016660 (0x49a5 bytes) 000942 1786401867.014803 MIRROR_RESYNC 3 mirrors 000943 1786401867.394351 MIRROR_RESYNC 3 mirrors 000945 1786401867.737676 mapwrite 0x03f526 thru 0x03ffff (0xada bytes) 000948 1786401867.845996 trunc from 0x040000 to 0x005862 (0x3a79d bytes) 000949 1786401867.884676 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x5863): Mirror 7: 0xb8d45a00 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x5863): Mirror 7: 0xb8d45a00 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x5863: skipped 000951 1786401868.232716 mapwrite 0x01f4b4 thru 0x022a69 (0x35b6 bytes) 000953 1786401868.313813 mapread 0x01b89e thru 0x0203cc (0x4b2f bytes) 000955 1786401868.320577 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x22a6a): Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x22a6a): Mirror 7: 0x42f1dd07 (stale) CRC-32 checksum value for chunk [0, 0x22a6a): Mirror 1: 0x892758af Mirror 8: 0x892758af lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x22a6a: skipped 000959 1786401868.977832 write 0x01e562 thru 0x01fcf4 (0x1793 bytes) 000961 1786401869.065106 trunc from 0x022a6a to 0x018289 (0xa7e0 bytes) 000962 1786401869.113347 write 0x01e6d6 thru 0x02d95a (0xf285 bytes) 000964 1786401869.121054 write 0x03ecc2 thru 0x03ffff (0x133e bytes) 000965 1786401869.122620 mapwrite 0x03b63c thru 0x03ffff (0x49c4 bytes) 000968 1786401869.216362 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 8: 0xcf6138bc (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 8: 0xcf6138bc (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000969 1786401869.480899 read 0x036320 thru 0x03ffff (0x9ce0 bytes) 000970 1786401869.491576 mapwrite 0x01061a thru 0x01b24b (0xac32 bytes) 000971 1786401869.559328 trunc from 0x040000 to 0x00734b (0x38cb4 bytes) 000972 1786401869.582479 mapread 0x002256 thru 0x00734b (0x50f6 bytes) 000973 1786401869.593856 read 0x00015a thru 0x00734b (0x71f2 bytes) 000974 1786401869.599854 trunc from 0x00734c to 0x0278de (0x20593 bytes) 000975 1786401869.624091 read 0x00fb98 thru 0x017ebd (0x8326 bytes) 000978 1786401869.636032 trunc from 0x0278df to 0x023735 (0x41a9 bytes) 000982 1786401869.662711 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x23736): Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x23736): Mirror 8: 0x49b0c4b2 (stale) CRC-32 checksum value for chunk [0, 0x23736): Mirror 1: 0x139a6ff5 Mirror 9: 0x139a6ff5 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x23736: skipped 000983 1786401870.506541 read 0x012833 thru 0x016f38 (0x4706 bytes) 000985 1786401870.520320 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x23736): Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x23736): Mirror 8: 0x49b0c4b2 (stale) CRC-32 checksum value for chunk [0, 0x23736): Mirror 1: 0x139a6ff5 Mirror 9: 0x139a6ff5 Mirror 10: 0x139a6ff5 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x23736: skipped 000987 1786401871.318856 trunc from 0x023736 to 0x019966 (0x9dcf bytes) 000989 1786401871.398925 mapwrite 0x03c396 thru 0x03ffff (0x3c6a bytes) 000990 1786401871.487472 trunc from 0x040000 to 0x036746 (0x98b9 bytes) 000993 1786401871.520641 mapread 0x02ae18 thru 0x033978 (0x8b61 bytes) 000994 1786401871.536516 mapread 0x030aee thru 0x0353d6 (0x48e9 bytes) 000996 1786401871.540656 trunc from 0x036747 to 0x02a97e (0xbdc8 bytes) 000997 1786401871.565063 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x168:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2a97f): Mirror 8: 0x32f547d2 (stale) Mirror 9: 0x77a577ed (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2a97f): Mirror 8: 0x32f547d2 (stale) Mirror 9: 0x77a577ed (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2a97f: skipped 000998 1786401871.890842 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x2a97f): Mirror 8: (stale - read error) Mirror 9: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2a97f): Mirror 8: 0x32f547d2 (stale) Mirror 9: 0x77a577ed (stale) CRC-32 checksum value for chunk [0, 0x2a97f): Mirror 1: 0x77a577ed Mirror 10: 0x77a577ed lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2a97f: skipped 000999 1786401872.832962 MIRROR_RESYNC 4 mirrors All operations completed A-OK!