== sanity-flr test 70a: flr mode fsx test ================ 08:12:14 (1787659934) striped dir -i0 -c2 -H crush2 /mnt/lustre/d70a.sanity-flr Seed set to 9936 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000002 1787659936.526413 mapwrite 0x002856 thru 0x003138 (0x8e3 bytes) 000004 1787659936.667730 mapread 0x001f1e thru 0x003138 (0x121b bytes) 000005 1787659936.681056 read 0x0015c2 thru 0x003138 (0x1b77 bytes) truncating to largest ever: 0x3a859 000006 1787659936.682875 trunc from 0x003139 to 0x03a858 (0x37720 bytes) 000007 1787659936.720584 mapread 0x024f89 thru 0x02620f (0x1287 bytes) 000009 1787659936.741769 mapread 0x01e503 thru 0x02b2d3 (0xcdd1 bytes) 000011 1787659936.745987 mapread 0x011067 thru 0x01f12d (0xe0c7 bytes) 000014 1787659936.791126 MIRROR_EXTEND to 2 mirrors 000016 1787659937.999338 mapread 0x031931 thru 0x033a5a (0x212a bytes) 000017 1787659938.069888 mapread 0x030520 thru 0x03a858 (0xa339 bytes) truncating to largest ever: 0x3e355 000018 1787659938.108054 trunc from 0x03a859 to 0x03e354 (0x3afc bytes) 000020 1787659938.238333 write 0x0135a7 thru 0x01f8a3 (0xc2fd bytes) 000021 1787659938.256565 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) 000022 1787659938.446802 write 0x02ea13 thru 0x030480 (0x1a6e bytes) 000025 1787659938.464393 write 0x03027e thru 0x03ba66 (0xb7e9 bytes) 000027 1787659938.481264 trunc from 0x03e355 to 0x0268a5 (0x17aaf bytes) 000028 1787659938.508945 write 0x03b3f7 thru 0x03ffff (0x4c09 bytes) 000029 1787659938.511479 read 0x02d92f thru 0x0387c7 (0xae99 bytes) truncating to largest ever: 0x3e955 000030 1787659938.524182 trunc from 0x040000 to 0x03e954 (0x16ab bytes) 000031 1787659938.548184 mapread 0x02d82c thru 0x0386d5 (0xaeaa bytes) 000033 1787659938.553039 mapwrite 0x022091 thru 0x0279fa (0x596a bytes) 000038 1787659938.673592 mapread 0x01468e thru 0x01dce6 (0x9659 bytes) 000041 1787659938.689030 mapread 0x01a69a thru 0x01f7bf (0x5126 bytes) 000042 1787659938.690977 write 0x02d79c thru 0x03b2dd (0xdb42 bytes) 000043 1787659938.693921 write 0x01863b thru 0x01c992 (0x4358 bytes) 000044 1787659938.695153 read 0x02272e thru 0x02c96f (0xa242 bytes) 000045 1787659938.697339 trunc from 0x03e955 to 0x00add7 (0x33b7d bytes) 000046 1787659938.724297 write 0x00ee04 thru 0x019bdb (0xadd8 bytes) 000048 1787659938.731510 mapread 0x00fe9b thru 0x014cd9 (0x4e3f bytes) 000049 1787659938.734298 mapwrite 0x002448 thru 0x0038cc (0x1485 bytes) 000050 1787659938.875148 mapread 0x0186f0 thru 0x019bdb (0x14ec bytes) 000052 1787659938.881499 mapwrite 0x00fc23 thru 0x01f6e9 (0xfac7 bytes) 000059 1787659939.057853 trunc from 0x01f6ea to 0x015279 (0xa470 bytes) 000061 1787659939.094097 read 0x00ae65 thru 0x00e204 (0x33a0 bytes) 000064 1787659939.099656 write 0x00e02f thru 0x0175fa (0x95cc bytes) 000066 1787659939.106048 mapread 0x0060b9 thru 0x00a74e (0x4696 bytes) 000067 1787659939.111828 write 0x0032f5 thru 0x006958 (0x3664 bytes) 000070 1787659939.122906 write 0x001d07 thru 0x004547 (0x2841 bytes) 000071 1787659939.127659 mapread 0x00fc86 thru 0x015094 (0x540f bytes) 000072 1787659939.132558 mapwrite 0x02096e thru 0x0306c1 (0xfd54 bytes) 000074 1787659939.278030 mapwrite 0x01e343 thru 0x02b37e (0xd03c bytes) 000075 1787659939.340021 mapread 0x002b5b thru 0x00b00e (0x84b4 bytes) 000076 1787659939.343303 mapread 0x005ba6 thru 0x006728 (0xb83 bytes) 000077 1787659939.345462 write 0x004eaf thru 0x0076cb (0x281d bytes) 000081 1787659939.353620 trunc from 0x0306c2 to 0x00fc81 (0x20a40 bytes) 000082 1787659939.375923 write 0x0294dc thru 0x031183 (0x7ca8 bytes) 000083 1787659939.379851 read 0x00f4e3 thru 0x01b617 (0xc135 bytes) 000084 1787659939.391777 write 0x00555e thru 0x007bf2 (0x2695 bytes) 000086 1787659939.394867 mapread 0x009e5c thru 0x015728 (0xb8cd bytes) 000087 1787659939.398642 mapread 0x00e27b thru 0x010992 (0x2718 bytes) 000088 1787659939.400237 mapwrite 0x008487 thru 0x014fc5 (0xcb3f bytes) 000089 1787659939.484595 write 0x024fc9 thru 0x031c56 (0xcc8e bytes) 000090 1787659939.494546 mapwrite 0x01a85e thru 0x0229e2 (0x8185 bytes) 000095 1787659939.600085 mapwrite 0x0215c2 thru 0x02c0f5 (0xab34 bytes) 000097 1787659939.714540 mapwrite 0x00af43 thru 0x01a311 (0xf3cf bytes) 000101 1787659939.817152 mapwrite 0x02d9ff thru 0x03703c (0x963e bytes) 000102 1787659939.920271 read 0x0229ce thru 0x02ba78 (0x90ab bytes) 000103 1787659939.923423 mapwrite 0x024158 thru 0x024872 (0x71b bytes) 000104 1787659939.966087 mapwrite 0x0397ea thru 0x03ffff (0x6816 bytes) 000105 1787659940.089843 trunc from 0x040000 to 0x00163a (0x3e9c5 bytes) 000107 1787659940.133065 mapread 0x0007e2 thru 0x00163a (0xe59 bytes) 000110 1787659940.144295 mapwrite 0x00bde7 thru 0x0179d6 (0xbbf0 bytes) 000111 1787659940.427087 mapwrite 0x034173 thru 0x03447a (0x308 bytes) 000112 1787659940.504173 mapread 0x00c5c9 thru 0x00d691 (0x10c9 bytes) 000113 1787659940.505233 mapread 0x00ea41 thru 0x01be76 (0xd436 bytes) 000114 1787659940.534129 mapwrite 0x003e07 thru 0x00b9ee (0x7be8 bytes) 000118 1787659940.657911 mapwrite 0x020c9a thru 0x023341 (0x26a8 bytes) 000119 1787659940.725970 trunc from 0x03447b to 0x02e81b (0x5c5f bytes) 000120 1787659940.747281 mapread 0x020685 thru 0x029dfb (0x9777 bytes) 000121 1787659940.779264 mapread 0x006fe7 thru 0x014e58 (0xde72 bytes) 000123 1787659940.781821 write 0x0003ea thru 0x00f6ff (0xf316 bytes) 000124 1787659940.785430 write 0x0294ce thru 0x0384ac (0xefdf bytes) 000125 1787659940.787985 trunc from 0x0384ad to 0x03c2b8 (0x3e0c bytes) truncating to largest ever: 0x3fcde 000127 1787659940.814102 trunc from 0x03c2b9 to 0x03fcdd (0x3a25 bytes) 000130 1787659940.841231 read 0x02be91 thru 0x03b694 (0xf804 bytes) 000132 1787659940.852733 mapread 0x001b7a thru 0x004435 (0x28bc bytes) 000133 1787659940.853795 trunc from 0x03fcde to 0x00dece (0x31e0f bytes) 000134 1787659940.871859 mapread 0x0024f0 thru 0x00ab2f (0x8640 bytes) 000135 1787659940.873034 read 0x003175 thru 0x00879a (0x5626 bytes) 000136 1787659940.877974 mapread 0x00c1ad thru 0x00dece (0x1d22 bytes) 000139 1787659940.881505 mapwrite 0x033b98 thru 0x03b706 (0x7b6f bytes) 000140 1787659940.978985 write 0x00bf61 thru 0x01bef3 (0xff93 bytes) 000145 1787659940.985136 mapread 0x030253 thru 0x036d52 (0x6b00 bytes) 000151 1787659940.991436 mapread 0x024a7d thru 0x026a28 (0x1fac bytes) 000152 1787659940.992597 trunc from 0x03b707 to 0x032119 (0x95ed bytes) 000153 1787659941.009787 mapwrite 0x000029 thru 0x00aa4d (0xaa25 bytes) 000155 1787659941.075563 trunc from 0x03211a to 0x01c723 (0x159f6 bytes) 000157 1787659941.092631 mapwrite 0x022c6c thru 0x028325 (0x56ba bytes) 000158 1787659941.196030 mapwrite 0x011b29 thru 0x0206f8 (0xebd0 bytes) 000160 1787659941.273109 read 0x000de3 thru 0x00ef14 (0xe132 bytes) 000162 1787659941.274386 write 0x026b06 thru 0x02e15b (0x7656 bytes) 000163 1787659941.276659 trunc from 0x02e15c to 0x030c9f (0x2b44 bytes) 000164 1787659941.297201 trunc from 0x030ca0 to 0x027aeb (0x91b4 bytes) 000169 1787659941.340033 mapread 0x008c7f thru 0x010a6e (0x7df0 bytes) 000171 1787659941.344317 mapwrite 0x035bf6 thru 0x03ae62 (0x526d bytes) 000172 1787659941.447408 write 0x02c984 thru 0x035de4 (0x9461 bytes) 000173 1787659941.462752 MIRROR_EXTEND to 2 mirrors 000174 1787659942.698801 write 0x00e471 thru 0x01483d (0x63cd bytes) 000175 1787659942.878959 mapwrite 0x0331b1 thru 0x03ffff (0xce4f bytes) 000176 1787659943.096350 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) 000177 1787659943.307245 trunc from 0x040000 to 0x02c1ff (0x13e00 bytes) 000178 1787659943.345379 write 0x03629a thru 0x03ffff (0x9d66 bytes) 000179 1787659943.346858 mapwrite 0x027afd thru 0x02f180 (0x7684 bytes) 000180 1787659943.476543 read 0x01c530 thru 0x0296ff (0xd1d0 bytes) 000186 1787659943.494566 trunc from 0x040000 to 0x01c900 (0x236ff bytes) 000188 1787659943.526905 trunc from 0x01c901 to 0x014e08 (0x7af8 bytes) 000189 1787659943.555606 read 0x013918 thru 0x014e08 (0x14f1 bytes) 000192 1787659943.569570 mapread 0x009944 thru 0x014e08 (0xb4c5 bytes) 000195 1787659943.573027 write 0x014717 thru 0x0245af (0xfe99 bytes) 000196 1787659943.575538 mapread 0x014972 thru 0x0245af (0xfc3e bytes) 000198 1787659943.595828 write 0x01ee08 thru 0x02bb06 (0xccff bytes) 000199 1787659943.597526 write 0x026056 thru 0x02d77d (0x7728 bytes) 000201 1787659943.600595 mapwrite 0x01fa58 thru 0x02dd41 (0xe2ea bytes) 000202 1787659943.699011 write 0x00063c thru 0x004ec9 (0x488e bytes) 000203 1787659943.701938 write 0x024147 thru 0x02de5b (0x9d15 bytes) 000204 1787659943.704517 read 0x01e144 thru 0x02cadd (0xe99a bytes) 000205 1787659943.705553 mapwrite 0x030749 thru 0x03a412 (0x9cca bytes) 000206 1787659943.848125 mapread 0x024e7b thru 0x02da37 (0x8bbd bytes) 000207 1787659943.849370 read 0x00439e thru 0x007d68 (0x39cb bytes) 000213 1787659943.862373 trunc from 0x03a413 to 0x0162ee (0x24124 bytes) 000214 1787659943.883720 mapread 0x0158ab thru 0x0162ee (0xa44 bytes) 000216 1787659943.886149 trunc from 0x0162ef to 0x02b860 (0x15572 bytes) 000217 1787659943.900846 mapwrite 0x00827f thru 0x00d5b7 (0x5339 bytes) 000219 1787659943.958195 write 0x01eb56 thru 0x026da7 (0x8252 bytes) 000220 1787659943.971164 mapwrite 0x01d204 thru 0x0244e2 (0x72df bytes) 000221 1787659944.032500 mapread 0x01478f thru 0x01a964 (0x61d6 bytes) 000226 1787659944.042453 mapread 0x002b1d thru 0x00ae75 (0x8359 bytes) 000229 1787659944.047404 write 0x030cc5 thru 0x03ee61 (0xe19d bytes) 000230 1787659944.055138 MIRROR_EXTEND to 2 mirrors 000232 1787659945.248498 write 0x01f8c0 thru 0x0215d4 (0x1d15 bytes) 000235 1787659945.351198 read 0x0324dc thru 0x03ee61 (0xc986 bytes) 000238 1787659945.375155 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3ee62): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3ee62): Mirror 2: 0x55d123b4 (stale) CRC-32 checksum value for chunk [0, 0x3ee62): Mirror 1: 0xe55de6c7 Mirror 3: 0xe55de6c7 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3ee62: skipped 000240 1787659947.110490 mapwrite 0x022580 thru 0x02b85b (0x92dc bytes) 000242 1787659947.340584 mapread 0x023491 thru 0x02666d (0x31dd bytes) 000244 1787659947.348497 MIRROR_RESYNC 3 mirrors 000247 1787659948.819050 mapread 0x02d1ce thru 0x03ac12 (0xda45 bytes) 000250 1787659948.881086 write 0x02c3ff thru 0x032976 (0x6578 bytes) 000251 1787659949.043806 read 0x029949 thru 0x038bbb (0xf273 bytes) 000252 1787659949.068553 mapread 0x021e27 thru 0x02b0ec (0x92c6 bytes) 000256 1787659949.101160 read 0x0229c5 thru 0x0252d3 (0x290f bytes) 000257 1787659949.105398 write 0x02c831 thru 0x0359d7 (0x91a7 bytes) 000258 1787659949.108314 trunc from 0x03ee62 to 0x004945 (0x3a51c bytes) 000259 1787659949.137838 trunc from 0x004946 to 0x0237d2 (0x1ee8d bytes) 000261 1787659949.183884 write 0x00797a thru 0x009ae4 (0x216b bytes) 000263 1787659949.206830 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x237d3): Mirror 3: 0xa033a5ee (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x237d3): Mirror 3: 0xa033a5ee (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x237d3: skipped 000266 1787659949.698863 mapwrite 0x01e859 thru 0x02ac84 (0xc42c bytes) 000267 1787659949.884128 mapread 0x01b947 thru 0x0264dc (0xab96 bytes) 000268 1787659949.886574 mapread 0x011845 thru 0x01e0ec (0xc8a8 bytes) 000269 1787659949.900409 trunc from 0x02ac85 to 0x02f952 (0x4cce bytes) 000270 1787659949.928415 write 0x011c73 thru 0x01757d (0x590b bytes) 000271 1787659949.933181 mapwrite 0x00343f thru 0x00d124 (0x9ce6 bytes) 000273 1787659950.003882 mapwrite 0x02e7eb thru 0x03b426 (0xcc3c bytes) 000274 1787659950.102282 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) 000275 1787659950.254093 mapwrite 0x0365ac thru 0x03b858 (0x52ad bytes) 000277 1787659950.345018 write 0x00a9d6 thru 0x0123e5 (0x7a10 bytes) 000278 1787659950.359246 read 0x00648d thru 0x00ad0c (0x4880 bytes) 000279 1787659950.370832 write 0x008b56 thru 0x00b386 (0x2831 bytes) 000284 1787659950.376920 trunc from 0x03b859 to 0x024738 (0x17120 bytes) 000289 1787659950.419075 trunc from 0x024739 to 0x028c19 (0x44e1 bytes) 000290 1787659950.456962 write 0x01c330 thru 0x023bbb (0x788c bytes) 000292 1787659950.494777 mapread 0x019c7d thru 0x026650 (0xc9d4 bytes) 000293 1787659950.529544 write 0x03530c thru 0x03ffff (0xacf4 bytes) 000294 1787659950.555594 MIRROR_EXTEND to 2 mirrors 000296 1787659952.272606 mapread 0x039d30 thru 0x03ced7 (0x31a8 bytes) 000297 1787659952.309418 read 0x00bece thru 0x018327 (0xc45a bytes) 000299 1787659952.340344 MIRROR_RESYNC 2 mirrors 000300 1787659952.880806 mapread 0x024261 thru 0x02d368 (0x9108 bytes) 000301 1787659952.943476 mapwrite 0x023ac0 thru 0x02879d (0x4cde bytes) 000303 1787659953.220743 mapread 0x0122bd thru 0x01aa72 (0x87b6 bytes) 000306 1787659953.232423 mapread 0x02518a thru 0x033df2 (0xec69 bytes) 000309 1787659953.242263 trunc from 0x040000 to 0x031e24 (0xe1db bytes) 000311 1787659953.263070 mapwrite 0x00d62f thru 0x01a45d (0xce2f bytes) 000312 1787659953.323889 mapwrite 0x00cb51 thru 0x010b58 (0x4008 bytes) 000313 1787659953.366988 write 0x016215 thru 0x019309 (0x30f5 bytes) 000315 1787659953.368752 read 0x01131e thru 0x013c9f (0x2982 bytes) 000316 1787659953.370985 mapread 0x02330d thru 0x026f60 (0x3c54 bytes) 000318 1787659953.373562 mapwrite 0x035a0c thru 0x03a9b8 (0x4fad bytes) 000319 1787659953.453970 mapread 0x01fe0d thru 0x02d1ef (0xd3e3 bytes) 000320 1787659953.455598 mapread 0x02fe1e thru 0x0366a4 (0x6887 bytes) 000321 1787659953.457765 write 0x007dee thru 0x01593e (0xdb51 bytes) 000324 1787659953.466901 read 0x02c52d thru 0x0316f7 (0x51cb bytes) 000326 1787659953.469028 read 0x03133f thru 0x03a9b8 (0x967a bytes) 000328 1787659953.471268 trunc from 0x03a9b9 to 0x014ff5 (0x259c3 bytes) 000332 1787659953.497022 mapwrite 0x02d5bf thru 0x02fa8e (0x24d0 bytes) 000333 1787659953.594064 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x2fa8f): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2fa8f): Mirror 2: 0xf5d852bc (stale) CRC-32 checksum value for chunk [0, 0x2fa8f): Mirror 1: 0xd5ad4cd3 Mirror 3: 0xd5ad4cd3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2fa8f: skipped 000334 1787659955.023176 trunc from 0x02fa8f to 0x01c050 (0x13a3e bytes) 000335 1787659955.152774 write 0x030557 thru 0x0396fc (0x91a6 bytes) 000336 1787659955.155777 write 0x02240c thru 0x027721 (0x5316 bytes) 000337 1787659955.170590 write 0x00fcf0 thru 0x01d49e (0xd7af bytes) 000338 1787659955.236749 trunc from 0x0396fd to 0x029d5c (0xf9a0 bytes) 000341 1787659955.299216 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x29d5d): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x29d5d): Mirror 2: 0xe8919a94 (stale) Mirror 3: 0xf7c658c7 (stale) CRC-32 checksum value for chunk [0, 0x29d5d): Mirror 1: 0x9a7bbc83 Mirror 4: 0x9a7bbc83 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x29d5d: skipped 000344 1787659957.136643 mapwrite 0x0243ff thru 0x02e069 (0x9c6b bytes) 000346 1787659957.389236 mapwrite 0x0117a9 thru 0x01258d (0xde5 bytes) 000347 1787659957.444784 mapwrite 0x017ba9 thru 0x026658 (0xeab0 bytes) 000348 1787659957.510992 write 0x031773 thru 0x03b86f (0xa0fd bytes) 000351 1787659957.518573 trunc from 0x03b870 to 0x00a3c6 (0x314a9 bytes) 000352 1787659957.545953 write 0x028385 thru 0x02cfd3 (0x4c4f bytes) 000354 1787659957.549586 trunc from 0x02cfd4 to 0x010188 (0x1ce4b bytes) 000355 1787659957.572933 read 0x0044cd thru 0x00f08d (0xabc1 bytes) 000356 1787659957.583467 mapwrite 0x01afc1 thru 0x02866a (0xd6aa bytes) 000358 1787659957.681357 mapread 0x00e630 thru 0x01c9b2 (0xe383 bytes) 000359 1787659957.691927 read 0x000ed1 thru 0x004b55 (0x3c85 bytes) 000361 1787659957.700124 read 0x008864 thru 0x00db39 (0x52d6 bytes) 000364 1787659957.716037 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2866b): Mirror 3: 0x4a8de3a2 (stale) Mirror 4: 0x99db4431 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2866b): Mirror 3: 0x4a8de3a2 (stale) Mirror 4: 0x99db4431 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2866b: skipped 000365 1787659958.241759 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x2866b): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2866b): Mirror 3: 0x4a8de3a2 (stale) Mirror 4: 0x99db4431 (stale) CRC-32 checksum value for chunk [0, 0x2866b): Mirror 1: 0x3ee9d648 Mirror 5: 0x3ee9d648 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2866b: skipped 000366 1787659959.608153 mapread 0x00468a thru 0x006ae0 (0x2457 bytes) 000367 1787659959.664852 trunc from 0x02866b to 0x03e92d (0x162c3 bytes) 000368 1787659959.836135 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x3e92e): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e92e): Mirror 3: 0x8f920a09 (stale) Mirror 4: 0xe41b2035 (stale) Mirror 5: 0x6510c460 (stale) CRC-32 checksum value for chunk [0, 0x3e92e): Mirror 1: 0x6510c460 Mirror 6: 0x6510c460 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e92e: skipped 000371 1787659961.719587 mapwrite 0x00b0f2 thru 0x01a41e (0xf32d bytes) 000372 1787659961.947413 read 0x0276f1 thru 0x03474b (0xd05b bytes) 000373 1787659961.958185 mapread 0x02f2f1 thru 0x037b20 (0x8830 bytes) 000374 1787659961.973786 trunc from 0x03e92e to 0x016c76 (0x27cb7 bytes) 000376 1787659962.002223 write 0x02ddc6 thru 0x03c629 (0xe864 bytes) 000377 1787659962.006631 read 0x032a98 thru 0x03c629 (0x9b92 bytes) 000379 1787659962.009190 trunc from 0x03c62a to 0x02a962 (0x11cc7 bytes) 000380 1787659962.055228 mapread 0x01a64d thru 0x020a0d (0x63c1 bytes) 000384 1787659962.072502 mapread 0x00c0c2 thru 0x00f756 (0x3695 bytes) 000387 1787659962.077178 trunc from 0x02a963 to 0x02ff59 (0x55f7 bytes) 000391 1787659962.103111 trunc from 0x02ff5a to 0x00fb5d (0x203fc bytes) 000392 1787659962.123097 trunc from 0x00fb5e to 0x01cc7f (0xd122 bytes) 000393 1787659962.139067 write 0x022c30 thru 0x02ffee (0xd3bf bytes) 000394 1787659962.141829 write 0x00ae0f thru 0x00f998 (0x4b8a bytes) 000396 1787659962.144820 mapwrite 0x027051 thru 0x034b74 (0xdb24 bytes) 000398 1787659962.246714 mapwrite 0x03718a thru 0x0372ff (0x176 bytes) 000400 1787659962.345841 mapwrite 0x01f865 thru 0x0276c0 (0x7e5c bytes) 000401 1787659962.456567 MIRROR_RESYNC 5 mirrors 000402 1787659965.188281 trunc from 0x037300 to 0x013271 (0x2408e bytes) 000403 1787659965.390177 read 0x00aad6 thru 0x013271 (0x879c bytes) 000405 1787659965.437523 read 0x003684 thru 0x008870 (0x51ed bytes) 000406 1787659965.467046 read 0x010b52 thru 0x013271 (0x2720 bytes) 000408 1787659965.478521 write 0x035c6f thru 0x03ffff (0xa391 bytes) 000409 1787659965.487319 mapwrite 0x02479a thru 0x025c02 (0x1469 bytes) 000419 1787659965.716970 mapwrite 0x00737a thru 0x016d8e (0xfa15 bytes) 000422 1787659965.863549 read 0x02de17 thru 0x02e0a7 (0x291 bytes) 000424 1787659965.866718 mapwrite 0x009725 thru 0x00c58c (0x2e68 bytes) 000425 1787659965.942197 write 0x03af68 thru 0x03d6c2 (0x275b bytes) 000428 1787659965.950115 write 0x033ef9 thru 0x03725f (0x3367 bytes) 000429 1787659965.955711 MIRROR_SPLIT mirror 4 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xfa6926c4 (stale) Mirror 4: 0xfa6926c4 (stale) Mirror 5: 0xfa6926c4 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xfa6926c4 (stale) Mirror 4: 0xfa6926c4 (stale) Mirror 5: 0xfa6926c4 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000430 1787659966.459167 read 0x02d7c0 thru 0x039c06 (0xc447 bytes) 000431 1787659966.471371 trunc from 0x040000 to 0x037135 (0x8eca bytes) 000432 1787659966.528131 MIRROR_RESYNC 4 mirrors 000434 1787659969.186265 MIRROR_EXTEND to 5 mirrors 000436 1787659970.670083 trunc from 0x037136 to 0x0111e3 (0x25f52 bytes) 000437 1787659970.770559 trunc from 0x0111e4 to 0x024090 (0x12ead bytes) 000438 1787659970.793283 mapwrite 0x01d9b7 thru 0x01f543 (0x1b8d bytes) 000440 1787659970.859540 read 0x0108cf thru 0x0150db (0x480d bytes) 000441 1787659970.870508 read 0x00e6dc thru 0x01a3e7 (0xbd0c bytes) 000443 1787659970.882547 trunc from 0x024091 to 0x015807 (0xe889 bytes) 000446 1787659970.905573 MIRROR_RESYNC 5 mirrors 000447 1787659972.948936 read 0x0000fc thru 0x008fa9 (0x8eae bytes) 000448 1787659972.963595 mapwrite 0x03ff84 thru 0x03ffff (0x7c bytes) 000452 1787659973.126515 write 0x009f0b thru 0x00c15d (0x2253 bytes) 000453 1787659973.187581 trunc from 0x040000 to 0x027492 (0x18b6d bytes) 000456 1787659973.263662 trunc from 0x027493 to 0x02353e (0x3f54 bytes) 000457 1787659973.296932 write 0x01658f thru 0x025635 (0xf0a7 bytes) 000458 1787659973.306575 write 0x004e49 thru 0x01204b (0xd203 bytes) 000462 1787659973.333733 trunc from 0x025636 to 0x00d259 (0x183dc bytes) 000463 1787659973.387163 read 0x0046be thru 0x00c05d (0x79a0 bytes) 000464 1787659973.388280 read 0x001a5e thru 0x00a6bc (0x8c5f bytes) 000465 1787659973.397227 mapwrite 0x0256a2 thru 0x03067e (0xafdd bytes) 000466 1787659973.577469 write 0x038492 thru 0x03ffff (0x7b6e bytes) 000467 1787659973.579951 write 0x01e15f thru 0x02ac35 (0xcad7 bytes) 000471 1787659973.585816 mapwrite 0x029b0f thru 0x02b940 (0x1e32 bytes) 000474 1787659973.688774 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x67505301 (stale) Mirror 4: 0x67505301 (stale) Mirror 5: 0x67505301 (stale) Mirror 6: 0x67505301 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x5bd13fa1 Mirror 7: 0x5bd13fa1 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000475 1787659975.545646 mapwrite 0x02c46d thru 0x02d1a8 (0xd3c bytes) 000477 1787659976.091390 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x67505301 (stale) Mirror 4: 0x67505301 (stale) Mirror 5: 0x67505301 (stale) Mirror 6: 0x67505301 (stale) Mirror 7: 0x5bd13fa1 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x8080e251 Mirror 8: 0x8080e251 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000480 1787659978.447442 mapread 0x016be1 thru 0x019d59 (0x3179 bytes) 000481 1787659978.493845 read 0x024076 thru 0x02b56e (0x74f9 bytes) 000482 1787659978.515715 mapwrite 0x0046f9 thru 0x0092cc (0x4bd4 bytes) 000483 1787659978.700362 write 0x00c059 thru 0x00d52f (0x14d7 bytes) 000484 1787659978.708253 MIRROR_SPLIT mirror 3 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x67505301 (stale) Mirror 4: 0x67505301 (stale) Mirror 6: 0x67505301 (stale) Mirror 7: 0x5bd13fa1 (stale) Mirror 8: 0x8080e251 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x67505301 (stale) Mirror 4: 0x67505301 (stale) Mirror 6: 0x67505301 (stale) Mirror 7: 0x5bd13fa1 (stale) Mirror 8: 0x8080e251 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000486 1787659979.383736 mapwrite 0x013253 thru 0x019366 (0x6114 bytes) 000488 1787659979.605342 mapread 0x010aeb thru 0x01dbab (0xd0c1 bytes) 000490 1787659979.614002 MIRROR_RESYNC 6 mirrors 000492 1787659983.247322 write 0x00da9c thru 0x00e92c (0xe91 bytes) 000493 1787659983.383200 mapread 0x000b04 thru 0x00503f (0x453c bytes) 000497 1787659983.435009 mapread 0x002c5f thru 0x00e076 (0xb418 bytes) 000498 1787659983.439075 mapwrite 0x014f55 thru 0x023180 (0xe22c bytes) 000499 1787659983.620623 read 0x003b5f thru 0x00ee74 (0xb316 bytes) 000501 1787659983.630159 MIRROR_RESYNC 6 mirrors 000504 1787659986.851306 write 0x005fa1 thru 0x00efe5 (0x9045 bytes) 000506 1787659986.947500 write 0x03063b thru 0x032450 (0x1e16 bytes) 000507 1787659986.962268 mapread 0x03994a thru 0x039dc4 (0x47b bytes) 000512 1787659986.973085 read 0x015808 thru 0x020acd (0xb2c6 bytes) 000513 1787659986.994564 MIRROR_RESYNC 6 mirrors 000515 1787659990.132550 mapread 0x02f2be thru 0x034425 (0x5168 bytes) 000517 1787659990.153923 trunc from 0x040000 to 0x034eca (0xb135 bytes) 000519 1787659990.242281 MIRROR_RESYNC 6 mirrors 000522 1787659992.982936 write 0x0342bf thru 0x03ffff (0xbd41 bytes) 000523 1787659993.088323 mapwrite 0x006362 thru 0x0150be (0xed5d bytes) 000524 1787659993.266605 MIRROR_SPLIT mirror 4 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x9c4cd9f1 (stale) Mirror 4: 0x9c4cd9f1 (stale) Mirror 6: 0x9c4cd9f1 (stale) Mirror 8: 0x9c4cd9f1 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x9c4cd9f1 (stale) Mirror 4: 0x9c4cd9f1 (stale) Mirror 6: 0x9c4cd9f1 (stale) Mirror 8: 0x9c4cd9f1 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000525 1787659994.129438 read 0x02ca16 thru 0x039721 (0xcd0c bytes) 000527 1787659994.156923 read 0x03665a thru 0x03e0a5 (0x7a4c bytes) 000528 1787659994.202353 trunc from 0x040000 to 0x03383b (0xc7c4 bytes) 000529 1787659994.247328 MIRROR_SPLIT mirror 4 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3383c): Mirror 3: 0x6ba4ae86 (stale) Mirror 4: 0x6ba4ae86 (stale) Mirror 6: 0x6ba4ae86 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3383c): Mirror 3: 0x6ba4ae86 (stale) Mirror 4: 0x6ba4ae86 (stale) Mirror 6: 0x6ba4ae86 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3383c: skipped 000530 1787659994.723817 trunc from 0x03383c to 0x015f9c (0x1d89f bytes) 000531 1787659994.757880 write 0x0222e4 thru 0x02a013 (0x7d30 bytes) 000532 1787659994.759410 mapread 0x00ac7b thru 0x00d15c (0x24e2 bytes) 000534 1787659994.808459 MIRROR_RESYNC 4 mirrors 000535 1787659997.102736 mapwrite 0x023895 thru 0x028c80 (0x53ec bytes) 000537 1787659997.347801 mapread 0x02266d thru 0x025452 (0x2de6 bytes) 000540 1787659997.353749 write 0x000a87 thru 0x004d82 (0x42fc bytes) 000541 1787659997.363942 mapwrite 0x001824 thru 0x01113d (0xf91a bytes) 000542 1787659997.538394 write 0x00918f thru 0x015f19 (0xcd8b bytes) 000543 1787659997.539833 mapwrite 0x038bd5 thru 0x03ffff (0x742b bytes) 000545 1787659997.724348 trunc from 0x040000 to 0x0267e9 (0x19816 bytes) 000546 1787659997.781575 read 0x01e3d2 thru 0x01f0a8 (0xcd7 bytes) 000550 1787659997.786631 mapread 0x001e77 thru 0x00a7a1 (0x892b bytes) 000552 1787659997.791006 write 0x02b9d1 thru 0x03a686 (0xecb6 bytes) 000553 1787659997.799400 read 0x01f05e thru 0x0207a0 (0x1743 bytes) 000556 1787659997.807320 trunc from 0x03a687 to 0x008961 (0x31d25 bytes) 000560 1787659997.854605 MIRROR_RESYNC 4 mirrors 000561 1787659999.066169 mapread 0x0011d4 thru 0x008961 (0x778e bytes) 000562 1787659999.097486 write 0x020d79 thru 0x02fffe (0xf286 bytes) 000563 1787659999.177602 mapwrite 0x007201 thru 0x009655 (0x2455 bytes) 000564 1787659999.253994 trunc from 0x02ffff to 0x0154a1 (0x1ab5d bytes) 000565 1787659999.280547 mapwrite 0x03ee8f thru 0x03ffff (0x1171 bytes) 000566 1787659999.374194 trunc from 0x040000 to 0x02593c (0x1a6c3 bytes) 000567 1787659999.391888 read 0x011dd4 thru 0x01b278 (0x94a5 bytes) 000571 1787659999.405074 mapwrite 0x012511 thru 0x01902e (0x6b1e bytes) 000573 1787659999.469171 read 0x02398c thru 0x02593c (0x1fb1 bytes) 000576 1787659999.488379 write 0x01eff2 thru 0x021d4c (0x2d5b bytes) 000577 1787659999.511035 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2593d): Mirror 3: 0x7dec2d34 (stale) Mirror 4: 0x7dec2d34 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2593d): Mirror 3: 0x7dec2d34 (stale) Mirror 4: 0x7dec2d34 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2593d: skipped 000579 1787659999.979951 mapread 0x023888 thru 0x02593c (0x20b5 bytes) 000581 1787660000.006785 mapwrite 0x01bdb3 thru 0x02aaf8 (0xed46 bytes) 000584 1787660000.121243 mapread 0x0029d6 thru 0x0118c3 (0xeeee bytes) 000585 1787660000.155100 MIRROR_RESYNC 3 mirrors 000586 1787660001.596203 read 0x004d0d thru 0x007d4a (0x303e bytes) 000588 1787660001.606554 MIRROR_EXTEND to 4 mirrors 000589 1787660003.184091 mapread 0x028f29 thru 0x02aaf8 (0x1bd0 bytes) 000590 1787660003.217104 read 0x01937e thru 0x025ea9 (0xcb2c bytes) 000592 1787660003.229381 mapwrite 0x001b94 thru 0x00d14c (0xb5b9 bytes) 000593 1787660003.730866 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x2aaf9): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2aaf9): Mirror 3: 0xf09b6899 (stale) Mirror 4: 0xf09b6899 (stale) Mirror 5: 0xf09b6899 (stale) CRC-32 checksum value for chunk [0, 0x2aaf9): Mirror 1: 0xc0fa2ca0 Mirror 6: 0xc0fa2ca0 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2aaf9: skipped 000594 1787660005.148155 write 0x02e8eb thru 0x039ee1 (0xb5f7 bytes) 000596 1787660005.367166 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x39ee2): Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x39ee2): Mirror 3: 0x5a3255f5 (stale) Mirror 4: 0x5a3255f5 (stale) Mirror 5: 0x5a3255f5 (stale) Mirror 6: 0x7f7e0161 (stale) CRC-32 checksum value for chunk [0, 0x39ee2): Mirror 1: 0x8ceede6d Mirror 7: 0x8ceede6d lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x39ee2: skipped 000597 1787660007.571007 write 0x01d66c thru 0x01db0e (0x4a3 bytes) 000600 1787660007.728512 read 0x011c57 thru 0x01682f (0x4bd9 bytes) 000601 1787660007.748641 MIRROR_RESYNC 6 mirrors 000602 1787660011.027094 write 0x0310cb thru 0x03ffff (0xef35 bytes) 000603 1787660011.100988 trunc from 0x040000 to 0x028193 (0x17e6c bytes) 000604 1787660011.133891 MIRROR_RESYNC 6 mirrors 000605 1787660013.901093 MIRROR_RESYNC 6 mirrors 000606 1787660015.056062 mapwrite 0x0091ad thru 0x00f6e6 (0x653a bytes) 000610 1787660015.326741 mapwrite 0x008fe5 thru 0x00a6e0 (0x16fc bytes) 000614 1787660015.446701 read 0x00887a thru 0x0130f6 (0xa87d bytes) 000616 1787660015.466821 trunc from 0x028194 to 0x0217b3 (0x69e0 bytes) 000617 1787660015.510971 write 0x006641 thru 0x00d7f1 (0x71b1 bytes) 000619 1787660015.525200 MIRROR_RESYNC 6 mirrors 000621 1787660019.477709 trunc from 0x0217b4 to 0x01ae79 (0x693a bytes) 000622 1787660019.636489 MIRROR_SPLIT mirror 5 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1ae7a): Mirror 3: 0xa9bcd7e9 (stale) Mirror 4: 0xa9bcd7e9 (stale) Mirror 5: 0xa9bcd7e9 (stale) Mirror 6: 0xa9bcd7e9 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1ae7a): Mirror 3: 0xa9bcd7e9 (stale) Mirror 4: 0xa9bcd7e9 (stale) Mirror 5: 0xa9bcd7e9 (stale) Mirror 6: 0xa9bcd7e9 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1ae7a: skipped 000624 1787660020.203085 write 0x005cc9 thru 0x009987 (0x3cbf bytes) 000625 1787660020.219421 read 0x0049cf thru 0x014034 (0xf666 bytes) 000626 1787660020.238967 write 0x024431 thru 0x033863 (0xf433 bytes) 000627 1787660020.242717 trunc from 0x033864 to 0x03e434 (0xabd1 bytes) 000630 1787660020.274791 write 0x0295c1 thru 0x037578 (0xdfb8 bytes) 000632 1787660020.291172 write 0x019532 thru 0x026fc9 (0xda98 bytes) 000633 1787660020.307757 read 0x01795a thru 0x019098 (0x173f bytes) 000634 1787660020.330774 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3e435): Mirror 4: 0xa16e04ed (stale) Mirror 5: 0xa16e04ed (stale) Mirror 6: 0xa16e04ed (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e435): Mirror 4: 0xa16e04ed (stale) Mirror 5: 0xa16e04ed (stale) Mirror 6: 0xa16e04ed (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e435: skipped 000635 1787660020.883404 mapread 0x00382d thru 0x0049b9 (0x118d bytes) 000637 1787660020.896458 mapread 0x01b60f thru 0x02a60d (0xefff bytes) 000638 1787660020.960477 MIRROR_RESYNC 4 mirrors 000639 1787660023.750373 trunc from 0x03e435 to 0x0207fa (0x1dc3a bytes) 000640 1787660023.854770 read 0x0037fa thru 0x0099e7 (0x61ee bytes) 000642 1787660023.868861 trunc from 0x0207fb to 0x000453 (0x203a7 bytes) 000644 1787660023.891814 mapwrite 0x012778 thru 0x016bff (0x4488 bytes) 000646 1787660023.992337 read 0x008fc1 thru 0x00c120 (0x3160 bytes) 000647 1787660024.005342 read 0x016aaa thru 0x016bff (0x156 bytes) 000651 1787660024.008401 trunc from 0x016c00 to 0x0073e9 (0xf816 bytes) 000653 1787660024.028178 MIRROR_RESYNC 4 mirrors 000654 1787660025.669618 MIRROR_EXTEND to 5 mirrors 000655 1787660026.647113 MIRROR_EXTEND to 6 mirrors 000656 1787660027.938358 mapread 0x0035ed thru 0x006c15 (0x3629 bytes) 000657 1787660028.019278 read 0x0031e6 thru 0x0073e9 (0x4204 bytes) 000658 1787660028.019995 trunc from 0x0073ea to 0x0014d1 (0x5f18 bytes) 000661 1787660028.397243 mapread 0x0007de thru 0x0014d1 (0xcf4 bytes) 000664 1787660028.428925 mapwrite 0x00f071 thru 0x01d81f (0xe7af bytes) 000665 1787660028.732372 trunc from 0x01d820 to 0x01e3a6 (0xb87 bytes) 000666 1787660028.784281 mapwrite 0x00e53c thru 0x00ea37 (0x4fc bytes) 000670 1787660028.937545 write 0x014119 thru 0x017d4f (0x3c37 bytes) 000674 1787660028.949488 write 0x01949d thru 0x01c407 (0x2f6b bytes) 000675 1787660028.953009 write 0x013bc8 thru 0x01e307 (0xa740 bytes) 000676 1787660028.979449 mapwrite 0x01fdcf thru 0x02f529 (0xf75b bytes) 000677 1787660029.165350 mapwrite 0x039fb9 thru 0x03ffff (0x6047 bytes) 000678 1787660029.392557 trunc from 0x040000 to 0x0143df (0x2bc20 bytes) 000681 1787660029.452639 mapread 0x013dc8 thru 0x0143df (0x618 bytes) 000683 1787660029.485852 mapwrite 0x018e6b thru 0x02402e (0xb1c4 bytes) 000684 1787660029.670239 trunc from 0x02402f to 0x007abf (0x1c56f bytes) 000685 1787660029.710946 mapread 0x00277e thru 0x00549d (0x2d20 bytes) 000687 1787660029.750961 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x7ac0): Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x7ac0): Mirror 4: 0x4d845ea8 (stale) Mirror 5: 0x4d845ea8 (stale) Mirror 6: 0x4d845ea8 (stale) Mirror 7: 0x4d845ea8 (stale) Mirror 8: 0x4d845ea8 (stale) CRC-32 checksum value for chunk [0, 0x7ac0): Mirror 1: 0x4d845ea8 Mirror 9: 0x4d845ea8 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x7ac0: skipped 000688 1787660031.877168 mapwrite 0x02a0bc thru 0x02cbde (0x2b23 bytes) 000689 1787660032.181302 write 0x01e4f5 thru 0x024c6a (0x6776 bytes) 000690 1787660032.207736 mapwrite 0x018dc8 thru 0x0245ad (0xb7e6 bytes) 000692 1787660032.434272 trunc from 0x02cbdf to 0x02cded (0x20f bytes) 000693 1787660032.502909 mapwrite 0x02ee47 thru 0x0386d4 (0x988e bytes) 000696 1787660032.662916 trunc from 0x0386d5 to 0x000dc6 (0x3790e bytes) 000697 1787660032.692942 trunc from 0x000dc7 to 0x01986f (0x18aa9 bytes) 000698 1787660032.725297 trunc from 0x019870 to 0x032c14 (0x193a5 bytes) 000699 1787660032.746030 mapwrite 0x0259e5 thru 0x02de90 (0x84ac bytes) 000701 1787660032.854089 write 0x00825f thru 0x016299 (0xe03b bytes) 000702 1787660032.874840 MIRROR_EXTEND to 8 mirrors CRC-32 checksum value for chunk [0, 0x32c15): Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x32c15): Mirror 4: 0x44a2bd18 (stale) Mirror 5: 0x44a2bd18 (stale) Mirror 6: 0x44a2bd18 (stale) Mirror 7: 0x44a2bd18 (stale) Mirror 8: 0x44a2bd18 (stale) Mirror 9: 0x44a2bd18 (stale) CRC-32 checksum value for chunk [0, 0x32c15): Mirror 1: 0x588e16b4 Mirror 10: 0x588e16b4 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x32c15: skipped 000703 1787660034.866906 mapwrite 0x02055f thru 0x0205fb (0x9d bytes) 000705 1787660035.064744 mapread 0x011954 thru 0x015aa5 (0x4152 bytes) 000707 1787660035.076346 trunc from 0x032c15 to 0x024e75 (0xdd9f bytes) 000709 1787660035.105248 mapwrite 0x010a1d thru 0x02055e (0xfb42 bytes) 000711 1787660035.192845 trunc from 0x024e76 to 0x01a65e (0xa817 bytes) 000712 1787660035.237236 mapread 0x010cc3 thru 0x012412 (0x1750 bytes) 000713 1787660035.257157 MIRROR_SPLIT mirror 3 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1a65f): Mirror 4: 0xcb40eb5d (stale) Mirror 5: 0xcb40eb5d (stale) Mirror 7: 0xcb40eb5d (stale) Mirror 8: 0xcb40eb5d (stale) Mirror 9: 0xcb40eb5d (stale) Mirror 10: 0xfde8b13a (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1a65f): Mirror 4: 0xcb40eb5d (stale) Mirror 5: 0xcb40eb5d (stale) Mirror 7: 0xcb40eb5d (stale) Mirror 8: 0xcb40eb5d (stale) Mirror 9: 0xcb40eb5d (stale) Mirror 10: 0xfde8b13a (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1a65f: skipped 000714 1787660035.965186 MIRROR_RESYNC 7 mirrors 000715 1787660038.357371 mapwrite 0x034145 thru 0x03b331 (0x71ed bytes) 000716 1787660038.539762 write 0x03e7bf thru 0x03ffff (0x1841 bytes) 000718 1787660038.543320 write 0x025e85 thru 0x02fc13 (0x9d8f bytes) 000720 1787660038.553933 write 0x03de2e thru 0x03ffff (0x21d2 bytes) 000721 1787660038.561146 trunc from 0x040000 to 0x030851 (0xf7ae bytes) 000722 1787660038.586358 mapwrite 0x02ed6e thru 0x03e892 (0xfb25 bytes) 000724 1787660038.712282 mapread 0x006bfb thru 0x00af7d (0x4383 bytes) 000725 1787660038.728189 MIRROR_EXTEND to 8 mirrors CRC-32 checksum value for chunk [0, 0x3e893): Mirror 4: (stale - read error) Mirror 5: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e893): Mirror 4: 0xe51c61fb (stale) Mirror 5: 0xe51c61fb (stale) Mirror 7: 0xe51c61fb (stale) Mirror 8: 0xe51c61fb (stale) Mirror 9: 0xe51c61fb (stale) Mirror 10: 0xe51c61fb (stale) CRC-32 checksum value for chunk [0, 0x3e893): Mirror 1: 0x502fe3a7 Mirror 11: 0x502fe3a7 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e893: skipped 000726 1787660040.357881 write 0x03d01c thru 0x03d55f (0x544 bytes) 000727 1787660040.633224 mapwrite 0x0132ad thru 0x013690 (0x3e4 bytes) 000728 1787660040.766472 mapwrite 0x00e8c7 thru 0x0172f6 (0x8a30 bytes) 000729 1787660040.896947 mapread 0x010ccf thru 0x01fc90 (0xefc2 bytes) 000730 1787660040.932195 MIRROR_SPLIT mirror 1 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3e893): Mirror 5: 0xe51c61fb (stale) Mirror 7: 0xe51c61fb (stale) Mirror 8: 0xe51c61fb (stale) Mirror 9: 0xe51c61fb (stale) Mirror 10: 0xe51c61fb (stale) Mirror 11: 0x502fe3a7 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e893): Mirror 5: 0xe51c61fb (stale) Mirror 7: 0xe51c61fb (stale) Mirror 8: 0xe51c61fb (stale) Mirror 9: 0xe51c61fb (stale) Mirror 10: 0xe51c61fb (stale) Mirror 11: 0x502fe3a7 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e893: skipped 000731 1787660041.514217 write 0x0166fd thru 0x025205 (0xeb09 bytes) 000733 1787660041.528937 mapread 0x01ece1 thru 0x028b5a (0x9e7a bytes) 000734 1787660041.552415 MIRROR_EXTEND to 8 mirrors CRC-32 checksum value for chunk [0, 0x3e893): Mirror 5: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e893): Mirror 5: 0xe51c61fb (stale) Mirror 7: 0xe51c61fb (stale) Mirror 8: 0xe51c61fb (stale) Mirror 9: 0xe51c61fb (stale) Mirror 10: 0xe51c61fb (stale) Mirror 11: 0x502fe3a7 (stale) CRC-32 checksum value for chunk [0, 0x3e893): Mirror 1: 0xaa3dc02c Mirror 12: 0xaa3dc02c lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e893: skipped 000736 1787660042.964086 read 0x0331b3 thru 0x03d934 (0xa782 bytes) 000737 1787660043.002655 MIRROR_RESYNC 8 mirrors 000738 1787660046.179000 write 0x023e6d thru 0x025ff5 (0x2189 bytes) 000739 1787660046.359459 mapwrite 0x027cb2 thru 0x0293c1 (0x1710 bytes) 000740 1787660046.527842 mapread 0x034e9d thru 0x036fd8 (0x213c bytes) 000742 1787660046.574376 write 0x0307b6 thru 0x0364be (0x5d09 bytes) 000743 1787660046.623991 MIRROR_EXTEND to 9 mirrors CRC-32 checksum value for chunk [0, 0x3e893): Mirror 5: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3e893): Mirror 5: 0xaa3dc02c (stale) Mirror 7: 0xaa3dc02c (stale) Mirror 8: 0xaa3dc02c (stale) Mirror 9: 0xaa3dc02c (stale) Mirror 10: 0xaa3dc02c (stale) Mirror 11: 0xaa3dc02c (stale) Mirror 12: 0xaa3dc02c (stale) CRC-32 checksum value for chunk [0, 0x3e893): Mirror 1: 0x6ad508bc Mirror 13: 0x6ad508bc lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3e893: skipped 000744 1787660048.503856 trunc from 0x03e893 to 0x032423 (0xc46f bytes) 000745 1787660048.615919 trunc from 0x032424 to 0x01dfc8 (0x1445b bytes) 000747 1787660048.642090 mapwrite 0x003aba thru 0x012cf2 (0xf239 bytes) 000748 1787660048.756365 mapread 0x01b113 thru 0x01dfc8 (0x2eb6 bytes) 000749 1787660048.775184 mapread 0x014108 thru 0x014247 (0x140 bytes) 000751 1787660048.779774 mapwrite 0x0042d6 thru 0x00c864 (0x858f bytes) 000753 1787660048.879910 trunc from 0x01dfc9 to 0x00cd00 (0x112c8 bytes) 000755 1787660048.929088 mapread 0x0037c6 thru 0x00cd00 (0x953b bytes) 000756 1787660048.931168 mapwrite 0x02b1de thru 0x03b1b8 (0xffdb bytes) 000757 1787660049.151825 write 0x00d41f thru 0x01b568 (0xe14a bytes) 000758 1787660049.180711 mapread 0x00076b thru 0x002344 (0x1bda bytes) 000759 1787660049.193438 write 0x0014ae thru 0x00e130 (0xcc83 bytes) 000762 1787660049.212590 mapread 0x0146e7 thru 0x01a083 (0x599d bytes) 000765 1787660049.224997 trunc from 0x03b1b9 to 0x0214ff (0x19cb9 bytes) 000767 1787660049.260799 trunc from 0x021500 to 0x0092bd (0x18242 bytes) 000768 1787660049.306924 MIRROR_RESYNC 9 mirrors 000770 1787660051.888282 mapwrite 0x0009ac thru 0x0057d9 (0x4e2e bytes) 000772 1787660052.120097 read 0x0028f2 thru 0x0092bd (0x69cc bytes) 000774 1787660052.123026 mapwrite 0x028811 thru 0x02fdb9 (0x75a9 bytes) 000777 1787660052.245924 read 0x01a928 thru 0x01c442 (0x1b1b bytes) 000778 1787660052.272080 mapread 0x000032 thru 0x001d81 (0x1d50 bytes) 000779 1787660052.279497 MIRROR_EXTEND to 10 mirrors CRC-32 checksum value for chunk [0, 0x2fdba): Mirror 5: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) Mirror 13: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2fdba): Mirror 5: 0x6ab04e2e (stale) Mirror 7: 0x6ab04e2e (stale) Mirror 8: 0x6ab04e2e (stale) Mirror 9: 0x6ab04e2e (stale) Mirror 10: 0x6ab04e2e (stale) Mirror 11: 0x6ab04e2e (stale) Mirror 12: 0x6ab04e2e (stale) Mirror 13: 0x6ab04e2e (stale) CRC-32 checksum value for chunk [0, 0x2fdba): Mirror 1: 0x6b99bf98 Mirror 14: 0x6b99bf98 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2fdba: skipped 000781 1787660053.501168 mapread 0x022bd0 thru 0x02f129 (0xc55a bytes) 000782 1787660053.562608 read 0x019b76 thru 0x01d57b (0x3a06 bytes) 000783 1787660053.596959 read 0x0271d7 thru 0x02fdb9 (0x8be3 bytes) 000786 1787660053.625967 read 0x003f98 thru 0x00b5a0 (0x7609 bytes) 000787 1787660053.646011 mapwrite 0x02697f thru 0x030345 (0x99c7 bytes) 000789 1787660054.051638 mapread 0x01d5a7 thru 0x026784 (0x91de bytes) 000792 1787660054.151320 mapread 0x0296a4 thru 0x030345 (0x6ca2 bytes) 000793 1787660054.153927 mapwrite 0x01def5 thru 0x02938f (0xb49b bytes) 000794 1787660054.269272 write 0x02f31a thru 0x032e00 (0x3ae7 bytes) 000795 1787660054.271841 write 0x03d106 thru 0x03ffff (0x2efa bytes) 000796 1787660054.273489 mapwrite 0x01d7df thru 0x01e089 (0x8ab bytes) 000800 1787660054.391247 write 0x0363a7 thru 0x03ffff (0x9c59 bytes) 000801 1787660054.414966 MIRROR_SPLIT mirror 4 to 9 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0x5dabbdcd (stale) Mirror 7: 0x5dabbdcd (stale) Mirror 8: 0x5dabbdcd (stale) Mirror 10: 0x5dabbdcd (stale) Mirror 11: 0x5dabbdcd (stale) Mirror 12: 0x5dabbdcd (stale) Mirror 13: 0x5dabbdcd (stale) Mirror 14: 0xc25c1947 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0x5dabbdcd (stale) Mirror 7: 0x5dabbdcd (stale) Mirror 8: 0x5dabbdcd (stale) Mirror 10: 0x5dabbdcd (stale) Mirror 11: 0x5dabbdcd (stale) Mirror 12: 0x5dabbdcd (stale) Mirror 13: 0x5dabbdcd (stale) Mirror 14: 0xc25c1947 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000802 1787660055.028149 read 0x00eb9f thru 0x01653f (0x79a1 bytes) 000803 1787660055.040402 read 0x016eb4 thru 0x01d819 (0x6966 bytes) 000804 1787660055.056999 mapwrite 0x0280c4 thru 0x034793 (0xc6d0 bytes) 000808 1787660055.128094 mapread 0x02d92b thru 0x031984 (0x405a bytes) 000809 1787660055.130586 read 0x01b409 thru 0x021a7d (0x6675 bytes) 000810 1787660055.137465 MIRROR_SPLIT mirror 8 to 8 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0x5dabbdcd (stale) Mirror 7: 0x5dabbdcd (stale) Mirror 8: 0x5dabbdcd (stale) Mirror 10: 0x5dabbdcd (stale) Mirror 11: 0x5dabbdcd (stale) Mirror 12: 0x5dabbdcd (stale) Mirror 13: 0x5dabbdcd (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0x5dabbdcd (stale) Mirror 7: 0x5dabbdcd (stale) Mirror 8: 0x5dabbdcd (stale) Mirror 10: 0x5dabbdcd (stale) Mirror 11: 0x5dabbdcd (stale) Mirror 12: 0x5dabbdcd (stale) Mirror 13: 0x5dabbdcd (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000811 1787660055.898995 trunc from 0x040000 to 0x004731 (0x3b8ce bytes) 000812 1787660055.956298 mapwrite 0x022ba6 thru 0x022ed4 (0x32f bytes) 000813 1787660056.085093 MIRROR_SPLIT mirror 1 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x22ed5): Mirror 7: 0xba58ac22 (stale) Mirror 8: 0xba58ac22 (stale) Mirror 10: 0xba58ac22 (stale) Mirror 11: 0xba58ac22 (stale) Mirror 12: 0xba58ac22 (stale) Mirror 13: 0xba58ac22 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x22ed5): Mirror 7: 0xba58ac22 (stale) Mirror 8: 0xba58ac22 (stale) Mirror 10: 0xba58ac22 (stale) Mirror 11: 0xba58ac22 (stale) Mirror 12: 0xba58ac22 (stale) Mirror 13: 0xba58ac22 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x22ed5: skipped 000814 1787660056.747200 trunc from 0x022ed5 to 0x03e985 (0x1bab1 bytes) 000817 1787660056.775006 trunc from 0x03e986 to 0x004e4c (0x39b39 bytes) 000818 1787660056.814783 trunc from 0x004e4d to 0x01a205 (0x153b9 bytes) 000821 1787660056.834757 read 0x00f97e thru 0x01a205 (0xa888 bytes) 000823 1787660056.848906 trunc from 0x01a206 to 0x01cee6 (0x2ce1 bytes) 000824 1787660056.872520 trunc from 0x01cee7 to 0x0171a6 (0x5d40 bytes) 000825 1787660056.897398 MIRROR_SPLIT mirror 6 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x171a7): Mirror 7: 0xf6d0a6d5 (stale) Mirror 8: 0xf6d0a6d5 (stale) Mirror 10: 0xf6d0a6d5 (stale) Mirror 11: 0xf6d0a6d5 (stale) Mirror 12: 0xf6d0a6d5 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x171a7): Mirror 7: 0xf6d0a6d5 (stale) Mirror 8: 0xf6d0a6d5 (stale) Mirror 10: 0xf6d0a6d5 (stale) Mirror 11: 0xf6d0a6d5 (stale) Mirror 12: 0xf6d0a6d5 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x171a7: skipped 000826 1787660057.489652 mapread 0x016543 thru 0x0171a6 (0xc64 bytes) 000828 1787660057.523555 trunc from 0x0171a7 to 0x037bb9 (0x20a13 bytes) 000830 1787660057.556467 trunc from 0x037bba to 0x03d1b2 (0x55f9 bytes) 000831 1787660057.596708 trunc from 0x03d1b3 to 0x021317 (0x1be9b bytes) 000832 1787660057.613596 read 0x00b422 thru 0x00eb34 (0x3713 bytes) 000833 1787660057.627598 mapwrite 0x02ea2d thru 0x03af10 (0xc4e4 bytes) 000834 1787660057.811997 MIRROR_RESYNC 6 mirrors 000838 1787660060.313163 mapread 0x022dd5 thru 0x024e5c (0x2088 bytes) 000839 1787660060.337391 trunc from 0x03af11 to 0x0200ef (0x1ae21 bytes) 000840 1787660060.417549 read 0x0119ea thru 0x01789f (0x5eb6 bytes) 000841 1787660060.432999 MIRROR_SPLIT mirror 1 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x200f0): Mirror 8: 0xb3ccfe4e (stale) Mirror 10: 0xb3ccfe4e (stale) Mirror 11: 0xb3ccfe4e (stale) Mirror 12: 0xb3ccfe4e (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x200f0): Mirror 8: 0xb3ccfe4e (stale) Mirror 10: 0xb3ccfe4e (stale) Mirror 11: 0xb3ccfe4e (stale) Mirror 12: 0xb3ccfe4e (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x200f0: skipped 000843 1787660060.922770 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x200f0): Mirror 8: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x200f0): Mirror 8: 0xb3ccfe4e (stale) Mirror 10: 0xb3ccfe4e (stale) Mirror 11: 0xb3ccfe4e (stale) Mirror 12: 0xb3ccfe4e (stale) CRC-32 checksum value for chunk [0, 0x200f0): Mirror 1: 0xb3ccfe4e Mirror 13: 0xb3ccfe4e lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x200f0: skipped 000844 1787660062.439608 read 0x011b60 thru 0x0200ef (0xe590 bytes) 000847 1787660062.486812 read 0x005b40 thru 0x00c245 (0x6706 bytes) 000849 1787660062.517556 read 0x003064 thru 0x00ff09 (0xcea6 bytes) 000850 1787660062.535600 MIRROR_RESYNC 6 mirrors 000852 1787660063.952096 read 0x01103b thru 0x0176ac (0x6672 bytes) 000853 1787660063.971451 write 0x022dbb thru 0x02ebb4 (0xbdfa bytes) 000854 1787660064.090307 read 0x01676f thru 0x019960 (0x31f2 bytes) 000855 1787660064.114652 MIRROR_RESYNC 6 mirrors 000857 1787660066.121781 MIRROR_RESYNC 6 mirrors 000859 1787660066.699166 write 0x00e292 thru 0x015da0 (0x7b0f bytes) 000860 1787660066.870593 mapread 0x020276 thru 0x02995a (0x96e5 bytes) 000861 1787660066.915879 MIRROR_RESYNC 6 mirrors 000864 1787660069.409558 mapread 0x00355c thru 0x013505 (0xffaa bytes) 000865 1787660069.458191 mapwrite 0x0309e1 thru 0x03b0d4 (0xa6f4 bytes) 000866 1787660069.684273 mapwrite 0x024269 thru 0x028948 (0x46e0 bytes) 000867 1787660069.742711 mapread 0x024aad thru 0x0317ce (0xcd22 bytes) 000869 1787660069.744571 read 0x01823f thru 0x01a148 (0x1f0a bytes) 000870 1787660069.759263 MIRROR_RESYNC 6 mirrors 000872 1787660072.837044 mapread 0x01b835 thru 0x028069 (0xc835 bytes) 000874 1787660072.906302 mapread 0x030e83 thru 0x03b0d4 (0xa252 bytes) 000876 1787660072.929276 mapread 0x03894a thru 0x03a50c (0x1bc3 bytes) 000877 1787660072.929965 write 0x02468a thru 0x029a9d (0x5414 bytes) 000880 1787660073.046050 mapread 0x0331a5 thru 0x03b0d4 (0x7f30 bytes) 000881 1787660073.057945 read 0x01ba8f thru 0x024de5 (0x9357 bytes) 000886 1787660073.070914 mapwrite 0x009c76 thru 0x00eba1 (0x4f2c bytes) 000887 1787660073.148272 mapread 0x037674 thru 0x03b0d4 (0x3a61 bytes) 000888 1787660073.150505 mapread 0x00b59d thru 0x015dd0 (0xa834 bytes) 000889 1787660073.165853 mapwrite 0x01b845 thru 0x0274d1 (0xbc8d bytes) 000890 1787660073.266195 MIRROR_RESYNC 6 mirrors 000892 1787660075.899018 write 0x02adb3 thru 0x035603 (0xa851 bytes) 000894 1787660075.968698 mapread 0x037b15 thru 0x03b0d4 (0x35c0 bytes) 000895 1787660075.988443 write 0x017429 thru 0x01b50e (0x40e6 bytes) 000898 1787660076.000204 mapread 0x025304 thru 0x031f08 (0xcc05 bytes) 000900 1787660076.018062 trunc from 0x03b0d5 to 0x026702 (0x149d2 bytes) 000901 1787660076.049338 write 0x0243ab thru 0x032419 (0xe06f bytes) 000902 1787660076.054996 mapwrite 0x0007c3 thru 0x00423e (0x3a7c bytes) 000903 1787660076.148957 read 0x0064cf thru 0x0065ac (0xde bytes) 000904 1787660076.149898 read 0x01433d thru 0x01ec94 (0xa958 bytes) 000905 1787660076.160394 mapread 0x029517 thru 0x02e68d (0x5177 bytes) 000906 1787660076.163302 mapread 0x0213d1 thru 0x024be3 (0x3813 bytes) 000907 1787660076.165699 read 0x008881 thru 0x012dde (0xa55e bytes) 000909 1787660076.172328 MIRROR_SPLIT mirror 1 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3241a): Mirror 10: 0xae62b2d7 (stale) Mirror 11: 0xae62b2d7 (stale) Mirror 12: 0xae62b2d7 (stale) Mirror 13: 0xae62b2d7 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3241a): Mirror 10: 0xae62b2d7 (stale) Mirror 11: 0xae62b2d7 (stale) Mirror 12: 0xae62b2d7 (stale) Mirror 13: 0xae62b2d7 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3241a: skipped 000910 1787660076.644919 MIRROR_RESYNC 5 mirrors 000911 1787660078.532313 mapwrite 0x03bbbb thru 0x03ffff (0x4445 bytes) 000912 1787660078.821870 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 11: 0xf3be3d3c (stale) Mirror 12: 0xf3be3d3c (stale) Mirror 13: 0xf3be3d3c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 11: 0xf3be3d3c (stale) Mirror 12: 0xf3be3d3c (stale) Mirror 13: 0xf3be3d3c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000915 1787660079.245140 write 0x02fbce thru 0x037be1 (0x8014 bytes) 000919 1787660079.263226 trunc from 0x040000 to 0x03d038 (0x2fc7 bytes) 000920 1787660079.286039 trunc from 0x03d039 to 0x0296a6 (0x13992 bytes) 000924 1787660079.311262 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x296a7): Mirror 12: 0xf290cc49 (stale) Mirror 13: 0xf290cc49 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x296a7): Mirror 12: 0xf290cc49 (stale) Mirror 13: 0xf290cc49 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x296a7: skipped 000925 1787660079.660046 read 0x026f35 thru 0x0296a6 (0x2772 bytes) 000926 1787660079.675153 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x296a7): Mirror 13: 0xf290cc49 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x296a7): Mirror 13: 0xf290cc49 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x296a7: skipped 000929 1787660080.055510 mapread 0x010396 thru 0x012931 (0x259c bytes) 000930 1787660080.085771 mapwrite 0x010488 thru 0x01466d (0x41e6 bytes) 000932 1787660080.191261 read 0x006893 thru 0x014dc2 (0xe530 bytes) 000934 1787660080.210210 write 0x02f610 thru 0x036570 (0x6f61 bytes) 000941 1787660080.218800 read 0x0092b8 thru 0x012be8 (0x9931 bytes) 000943 1787660080.221952 read 0x00a289 thru 0x00ab96 (0x90e bytes) 000944 1787660080.223285 mapwrite 0x003363 thru 0x004e3a (0x1ad8 bytes) 000945 1787660080.356830 mapwrite 0x00091f thru 0x006bba (0x629c bytes) 000946 1787660080.428527 mapread 0x01c9ea thru 0x023f27 (0x753e bytes) 000948 1787660080.431434 read 0x02805f thru 0x029b2a (0x1acc bytes) 000949 1787660080.432563 mapwrite 0x0036da thru 0x00c3c9 (0x8cf0 bytes) 000950 1787660080.501952 trunc from 0x036571 to 0x035d72 (0x7fe bytes) 000951 1787660080.526203 mapwrite 0x01fb63 thru 0x02ec93 (0xf131 bytes) 000952 1787660080.635583 write 0x029244 thru 0x02b7cf (0x258c bytes) 000953 1787660080.638385 read 0x018f58 thru 0x020430 (0x74d9 bytes) 000954 1787660080.640263 mapwrite 0x000553 thru 0x008fc0 (0x8a6e bytes) 000957 1787660080.723421 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) 000960 1787660080.908999 write 0x01d8f4 thru 0x024246 (0x6953 bytes) 000963 1787660080.926329 trunc from 0x035d73 to 0x01db5e (0x18214 bytes) 000964 1787660080.950011 read 0x009aec thru 0x00b89b (0x1db0 bytes) 000965 1787660080.960487 trunc from 0x01db5f to 0x036b9b (0x1903d bytes) 000966 1787660080.989356 mapread 0x01ccbd thru 0x02bcf0 (0xf034 bytes) 000968 1787660081.043823 trunc from 0x036b9c to 0x021681 (0x1551a bytes) 000969 1787660081.066897 mapread 0x011c9c thru 0x015a90 (0x3df5 bytes) 000971 1787660081.080889 write 0x02ac1f thru 0x02c89a (0x1c7c bytes) 000972 1787660081.082041 mapwrite 0x028c73 thru 0x03579c (0xcb2a bytes) 000977 1787660081.277455 mapwrite 0x032863 thru 0x033d63 (0x1501 bytes) 000981 1787660081.352497 mapwrite 0x03d427 thru 0x03ffff (0x2bd9 bytes) 000982 1787660081.447530 write 0x022d9f thru 0x03027f (0xd4e1 bytes) 000983 1787660081.456539 MIRROR_EXTEND to 2 mirrors 000984 1787660082.646906 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x240000404:0x122:0x0]: Invalid argument (22) 000985 1787660082.870393 mapread 0x0104ce thru 0x01e47f (0xdfb2 bytes) 000987 1787660082.905555 mapwrite 0x01f7c4 thru 0x026e07 (0x7644 bytes) 000988 1787660083.042163 trunc from 0x040000 to 0x008f48 (0x370b7 bytes) 000990 1787660083.071946 trunc from 0x008f49 to 0x007c5c (0x12ec bytes) 000993 1787660083.093883 mapread 0x001643 thru 0x007c5c (0x661a bytes) 000995 1787660083.104938 read 0x000ef1 thru 0x007c5c (0x6d6c bytes) 000996 1787660083.111017 trunc from 0x007c5d to 0x02c196 (0x2453a bytes) 000997 1787660083.132039 write 0x02ea8e thru 0x02f708 (0xc7b bytes) 000999 1787660083.136957 write 0x034b6f thru 0x038df4 (0x4286 bytes) All operations completed A-OK!