== sanity-flr test 70a: flr mode fsx test ================ 09:23:17 (1790774597) Seed set to 4600 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read 000004 1790774601.243174 mapwrite 0x02d9e8 thru 0x0358db (0x7ef4 bytes) 000006 1790774601.579785 punch from 0x017e10 to 0x0238b2 (0xbaa3 bytes) 000007 1790774601.618852 MIRROR_EXTEND to 2 mirrors 000008 1790774602.784923 mapwrite 0x03fddf thru 0x03ffff (0x221 bytes) 000014 1790774603.080976 mapwrite 0x03e6cd thru 0x03ffff (0x1933 bytes) 000015 1790774603.148080 read 0x035972 thru 0x037eaa (0x2539 bytes) 000016 1790774603.162495 read 0x03535d thru 0x03ffff (0xaca3 bytes) 000017 1790774603.184020 mapread 0x014f7e thru 0x01cec0 (0x7f43 bytes) truncating to largest ever: 0x1affb 000019 1790774603.186447 trunc from 0x040000 to 0x01affa (0x25005 bytes) 000020 1790774603.232181 write 0x03989d thru 0x03cb30 (0x3294 bytes) 000021 1790774603.233384 write 0x01e14c thru 0x02192f (0x37e4 bytes) 000022 1790774603.263206 mapread 0x00598e thru 0x00a79b (0x4e0e bytes) 000023 1790774603.264297 write 0x016216 thru 0x01f796 (0x9581 bytes) 000024 1790774603.269896 mapwrite 0x03c2b5 thru 0x03f23d (0x2f89 bytes) 000025 1790774603.428099 punch from 0x03683c to 0x037c53 (0x1418 bytes) 000026 1790774603.433071 write 0x00be72 thru 0x00f2ea (0x3479 bytes) 000028 1790774603.434605 trunc from 0x03f23e to 0x00fa3d (0x2f800 bytes) 000029 1790774603.461187 punch from 0x00c078 to 0x00fa3d (0x39c6 bytes) 000030 1790774603.487876 mapread 0x008b06 thru 0x00fa3d (0x6f38 bytes) 000032 1790774603.545907 trunc from 0x00fa3e to 0x00e3a2 (0x169b bytes) 000033 1790774603.583129 mapwrite 0x01111c thru 0x01dbca (0xcaaf bytes) 000034 1790774603.935606 punch from 0x00b6d0 to 0x017cde (0xc60f bytes) 000037 1790774603.945366 mapread 0x01734f thru 0x01dbca (0x687c bytes) 000038 1790774603.959491 mapwrite 0x00aef5 thru 0x00fcbc (0x4dc8 bytes) 000040 1790774604.065158 read 0x0144d1 thru 0x019675 (0x51a5 bytes) 000041 1790774604.082968 MIRROR_RESYNC 2 mirrors 000043 1790774605.165881 mapwrite 0x029611 thru 0x02c2b3 (0x2ca3 bytes) 000045 1790774605.545244 mapwrite 0x02ffc6 thru 0x03c969 (0xc9a4 bytes) truncating to largest ever: 0x3cd1b 000046 1790774605.784004 trunc from 0x03c96a to 0x03cd1a (0x3b1 bytes) 000048 1790774605.801160 punch from 0x03bd02 to 0x03cd1a (0x1019 bytes) 000049 1790774605.816898 punch from 0x034ece to 0x038079 (0x31ac bytes) 000050 1790774605.842300 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3cd1b): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3cd1b): Mirror 2: 0xf851f917 (stale) CRC-32 checksum value for chunk [0, 0x3cd1b): Mirror 1: 0xab68385a Mirror 3: 0xab68385a lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3cd1b: skipped 000051 1790774608.120837 punch from 0x01d464 to 0x0266b0 (0x924d bytes) 000053 1790774608.288008 read 0x00b388 thru 0x014a79 (0x96f2 bytes) 000055 1790774608.315480 mapwrite 0x01549b thru 0x01ab95 (0x56fb bytes) 000056 1790774608.507481 punch from 0x029751 to 0x03678c (0xd03c bytes) 000057 1790774608.533229 mapwrite 0x025562 thru 0x0317bc (0xc25b bytes) 000058 1790774608.718117 mapread 0x00e7cd thru 0x01bc69 (0xd49d bytes) 000059 1790774608.731680 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3cd1b): Mirror 3: 0xab68385a (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3cd1b): Mirror 3: 0xab68385a (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3cd1b: skipped 000060 1790774609.409335 read 0x00655f thru 0x00f106 (0x8ba8 bytes) 000061 1790774609.430193 read 0x03aa0a thru 0x03cd1a (0x2311 bytes) 000062 1790774609.454961 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) 000065 1790774609.618985 punch from 0x014f38 to 0x01fa27 (0xaaf0 bytes) 000070 1790774609.629162 trunc from 0x03cd1b to 0x00600a (0x36d10 bytes) 000071 1790774609.654434 MIRROR_EXTEND to 2 mirrors 000073 1790774610.922163 trunc from 0x00600b to 0x029359 (0x2334f bytes) 000074 1790774611.119232 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) 000075 1790774611.296783 mapread 0x002c23 thru 0x00eb85 (0xbf63 bytes) 000077 1790774611.324038 mapwrite 0x030190 thru 0x03cee8 (0xcd59 bytes) 000078 1790774611.428353 read 0x007e8c thru 0x014647 (0xc7bc bytes) 000079 1790774611.436676 read 0x0022df thru 0x00f997 (0xd6b9 bytes) 000080 1790774611.437428 mapwrite 0x036646 thru 0x03bafd (0x54b8 bytes) 000081 1790774611.491222 trunc from 0x03cee9 to 0x002b0d (0x3a3db bytes) 000082 1790774611.512108 write 0x005600 thru 0x00f554 (0x9f55 bytes) 000083 1790774611.518127 read 0x00f3b3 thru 0x00f554 (0x1a2 bytes) 000084 1790774611.519847 mapread 0x0072b4 thru 0x00f554 (0x82a1 bytes) 000086 1790774611.522254 mapread 0x006688 thru 0x00dddb (0x7754 bytes) 000087 1790774611.524324 read 0x00318a thru 0x008e90 (0x5d07 bytes) 000088 1790774611.542033 punch from 0x008ac6 to 0x0091a1 (0x6dc bytes) 000089 1790774611.562922 mapwrite 0x01042a thru 0x01564d (0x5224 bytes) 000092 1790774611.648190 punch from 0x00e101 to 0x01564d (0x754d bytes) 000094 1790774611.658630 trunc from 0x01564e to 0x03161e (0x1bfd1 bytes) 000095 1790774611.676267 write 0x008919 thru 0x0097ef (0xed7 bytes) 000096 1790774611.695071 read 0x020577 thru 0x0234b7 (0x2f41 bytes) 000098 1790774611.696399 read 0x01e74b thru 0x026846 (0x80fc bytes) 000101 1790774611.699641 write 0x01b8cd thru 0x0279de (0xc112 bytes) 000102 1790774611.701963 trunc from 0x03161f to 0x013162 (0x1e4bc bytes) 000103 1790774611.727816 MIRROR_EXTEND to 2 mirrors 000104 1790774612.719466 mapread 0x004d32 thru 0x00d20f (0x84de bytes) 000105 1790774612.764624 MIRROR_RESYNC 2 mirrors 000106 1790774613.334110 punch from 0x00e051 to 0x011c32 (0x3be2 bytes) 000107 1790774613.529981 punch from 0x005211 to 0x0067ba (0x15aa bytes) 000108 1790774613.540744 punch from 0x0121d7 to 0x013162 (0xf8c bytes) 000112 1790774613.563250 trunc from 0x013163 to 0x00a7c8 (0x899a bytes) 000113 1790774613.614027 mapwrite 0x0036ff thru 0x011ffa (0xe8fc bytes) 000115 1790774613.832098 write 0x01ade5 thru 0x029bba (0xedd6 bytes) 000120 1790774613.849421 punch from 0x01b0bd to 0x022efe (0x7e42 bytes) 000121 1790774613.875617 write 0x025189 thru 0x02a70f (0x5587 bytes) 000123 1790774613.884016 MIRROR_RESYNC 2 mirrors 000125 1790774615.247029 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) 000126 1790774615.395727 mapwrite 0x02a554 thru 0x02c9c3 (0x2470 bytes) 000127 1790774615.531682 mapwrite 0x02c0ee thru 0x02fc6a (0x3b7d bytes) 000130 1790774615.683078 trunc from 0x02fc6b to 0x007bb3 (0x280b7 bytes) 000131 1790774615.727034 write 0x020290 thru 0x02dcf6 (0xda67 bytes) 000132 1790774615.728838 mapread 0x00b947 thru 0x01a05e (0xe718 bytes) 000133 1790774615.758932 read 0x027f17 thru 0x02dcf6 (0x5de0 bytes) 000135 1790774615.767554 MIRROR_EXTEND to 2 mirrors 000136 1790774616.915045 trunc from 0x02dcf7 to 0x0251b4 (0x8b42 bytes) 000137 1790774617.090636 mapread 0x01fca0 thru 0x0219e2 (0x1d43 bytes) 000138 1790774617.135308 mapwrite 0x009e3b thru 0x013752 (0x9918 bytes) 000139 1790774617.253976 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) 000142 1790774617.432426 mapread 0x007ca5 thru 0x017af6 (0xfe52 bytes) 000143 1790774617.476145 read 0x00647e thru 0x011fad (0xbb30 bytes) 000145 1790774617.479420 punch from 0x003c5c to 0x00c36a (0x870f bytes) 000146 1790774617.486761 mapread 0x013e96 thru 0x016638 (0x27a3 bytes) 000147 1790774617.488838 write 0x016e3f thru 0x01c2dc (0x549e bytes) 000149 1790774617.498644 read 0x012b13 thru 0x01e3ea (0xb8d8 bytes) 000150 1790774617.519128 punch from 0x01af9e to 0x022eef (0x7f52 bytes) 000151 1790774617.540251 trunc from 0x0251b5 to 0x02f026 (0x9e72 bytes) truncating to largest ever: 0x3d400 000152 1790774617.570137 trunc from 0x02f027 to 0x03d3ff (0xe3d9 bytes) 000154 1790774617.616371 punch from 0x038b3f to 0x03d3ff (0x48c1 bytes) 000155 1790774617.630609 read 0x010a10 thru 0x01db9d (0xd18e bytes) 000158 1790774617.671454 mapread 0x030083 thru 0x0316c7 (0x1645 bytes) 000159 1790774617.681396 punch from 0x00488e to 0x00518a (0x8fd bytes) 000160 1790774617.694129 punch from 0x00b259 to 0x00ea5e (0x3806 bytes) 000162 1790774617.717231 mapwrite 0x01f03f thru 0x02203f (0x3001 bytes) 000163 1790774617.791719 read 0x0296d3 thru 0x02b40f (0x1d3d bytes) 000165 1790774617.794135 punch from 0x02fc76 to 0x03d3ff (0xd78a bytes) 000166 1790774617.801050 trunc from 0x03d400 to 0x03374a (0x9cb5 bytes) 000167 1790774617.825217 write 0x024412 thru 0x032f99 (0xeb88 bytes) 000168 1790774617.866515 mapread 0x012a40 thru 0x0161d6 (0x3797 bytes) 000171 1790774617.874920 mapread 0x00c1e0 thru 0x00c95d (0x77e bytes) 000175 1790774617.881077 write 0x020e18 thru 0x030a6b (0xfc54 bytes) 000181 1790774617.894129 write 0x02296f thru 0x02560f (0x2ca1 bytes) 000182 1790774617.897617 trunc from 0x03374b to 0x02a017 (0x9733 bytes) 000183 1790774617.929954 mapread 0x027bbd thru 0x02a017 (0x245b bytes) 000184 1790774617.931993 trunc from 0x02a018 to 0x00c833 (0x1d7e4 bytes) 000185 1790774617.951276 write 0x0263ca thru 0x02d270 (0x6ea7 bytes) 000186 1790774617.955637 trunc from 0x02d271 to 0x003ee3 (0x2938d bytes) 000188 1790774617.979051 trunc from 0x003ee4 to 0x0025fc (0x18e7 bytes) 000190 1790774618.011097 MIRROR_EXTEND to 2 mirrors 000191 1790774618.868189 read 0x00126b thru 0x0025fc (0x1392 bytes) 000192 1790774618.875184 write 0x02425b thru 0x031b71 (0xd917 bytes) 000194 1790774618.946555 trunc from 0x031b72 to 0x01dc40 (0x13f31 bytes) 000195 1790774618.980941 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) 000196 1790774619.129438 mapwrite 0x02d163 thru 0x0363af (0x924d bytes) 000198 1790774619.291085 punch from 0x02b429 to 0x02dbd6 (0x27ae bytes) 000199 1790774619.302433 write 0x013fd4 thru 0x018b0b (0x4b38 bytes) 000202 1790774619.346963 MIRROR_EXTEND to 2 mirrors 000203 1790774620.316223 mapwrite 0x00a39b thru 0x012010 (0x7c76 bytes) 000204 1790774620.567641 mapread 0x02efe4 thru 0x0363af (0x73cc bytes) 000205 1790774620.605640 mapread 0x00a517 thru 0x010dd6 (0x68c0 bytes) 000207 1790774620.612207 write 0x03270d thru 0x03ffff (0xd8f3 bytes) 000208 1790774620.621584 mapread 0x039b2d thru 0x03ffff (0x64d3 bytes) 000209 1790774620.626180 write 0x03f6d2 thru 0x03ffff (0x92e bytes) 000212 1790774620.634118 mapread 0x0167db thru 0x01a734 (0x3f5a bytes) 000213 1790774620.659065 trunc from 0x040000 to 0x00b721 (0x348de bytes) 000214 1790774620.718397 write 0x02e55d thru 0x0341c3 (0x5c67 bytes) 000216 1790774620.723179 read 0x009801 thru 0x0190a7 (0xf8a7 bytes) 000217 1790774620.733090 trunc from 0x0341c4 to 0x02f020 (0x51a3 bytes) 000218 1790774620.753008 read 0x0020e4 thru 0x00c0ce (0x9feb bytes) 000223 1790774620.774069 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x2f021): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2f021): Mirror 2: 0x2b80fc8d (stale) CRC-32 checksum value for chunk [0, 0x2f021): Mirror 1: 0xe6454f1c Mirror 3: 0xe6454f1c lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2f021: skipped 000224 1790774621.894298 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2f021): Mirror 2: 0x2b80fc8d (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2f021): Mirror 2: 0x2b80fc8d (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2f021: skipped 000225 1790774622.454819 write 0x008e5e thru 0x016fb5 (0xe158 bytes) 000233 1790774622.515430 punch from 0x0046e1 to 0x008314 (0x3c34 bytes) 000235 1790774622.540916 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) 000236 1790774622.728266 punch from 0x006fbd to 0x00e0b3 (0x70f7 bytes) 000238 1790774622.739009 write 0x02f8cc thru 0x033dba (0x44ef bytes) 000239 1790774622.749678 read 0x00fea8 thru 0x019fd3 (0xa12c bytes) 000240 1790774622.763846 mapread 0x011ab9 thru 0x01e29f (0xc7e7 bytes) 000241 1790774622.788539 read 0x010468 thru 0x01f877 (0xf410 bytes) 000244 1790774622.797162 write 0x00a2ae thru 0x00b833 (0x1586 bytes) 000249 1790774622.818373 punch from 0x02f757 to 0x0302e3 (0xb8d bytes) 000250 1790774622.867200 punch from 0x023fa2 to 0x028c0e (0x4c6d bytes) 000252 1790774622.881178 trunc from 0x033dbb to 0x00272b (0x3168f bytes) 000255 1790774622.937214 read 0x000452 thru 0x00272b (0x22da bytes) 000257 1790774622.940513 write 0x037aca thru 0x03829c (0x7d3 bytes) 000258 1790774622.942918 write 0x02d76b thru 0x034434 (0x6cca bytes) 000259 1790774622.965785 mapwrite 0x024947 thru 0x02e13a (0x97f4 bytes) 000260 1790774623.098315 mapwrite 0x00bdb7 thru 0x00c78f (0x9d9 bytes) 000262 1790774623.174495 mapwrite 0x020b87 thru 0x0243db (0x3855 bytes) 000263 1790774623.275330 trunc from 0x03829d to 0x01b59a (0x1cd02 bytes) 000264 1790774623.306538 write 0x007997 thru 0x0143d7 (0xca41 bytes) 000265 1790774623.309353 write 0x01e4a6 thru 0x029810 (0xb36b bytes) 000266 1790774623.312513 write 0x03bd8f thru 0x03ffff (0x4271 bytes) 000272 1790774623.347330 mapread 0x03fcc7 thru 0x03ffff (0x339 bytes) 000274 1790774623.349246 mapwrite 0x01e242 thru 0x02aa2e (0xc7ed bytes) 000275 1790774623.459644 read 0x01fe02 thru 0x026233 (0x6432 bytes) 000276 1790774623.460394 mapread 0x039697 thru 0x03ffff (0x6969 bytes) 000277 1790774623.461371 mapwrite 0x015d81 thru 0x01f59f (0x981f bytes) 000278 1790774623.504378 mapread 0x00feb5 thru 0x01a205 (0xa351 bytes) 000279 1790774623.505112 mapwrite 0x020407 thru 0x02290c (0x2506 bytes) 000280 1790774623.533586 punch from 0x01e6ae to 0x0245f8 (0x5f4b bytes) 000281 1790774623.541309 read 0x00f4f3 thru 0x01e75a (0xf268 bytes) 000282 1790774623.556309 write 0x01aefa thru 0x025e78 (0xaf7f bytes) 000285 1790774623.561850 mapwrite 0x0315b6 thru 0x03ffff (0xea4a bytes) 000287 1790774623.617988 read 0x000f00 thru 0x001b06 (0xc07 bytes) 000289 1790774623.628516 MIRROR_EXTEND to 2 mirrors 000290 1790774624.704481 write 0x011bb2 thru 0x020d81 (0xf1d0 bytes) 000291 1790774624.835177 mapwrite 0x027743 thru 0x029b2f (0x23ed bytes) 000292 1790774624.928202 punch from 0x02cb5d to 0x02d3ce (0x872 bytes) 000293 1790774624.942819 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) 000294 1790774625.131661 mapwrite 0x0244e9 thru 0x02caba (0x85d2 bytes) 000296 1790774625.206365 read 0x009291 thru 0x00c2cf (0x303f bytes) 000297 1790774625.213954 read 0x005e25 thru 0x0084e2 (0x26be bytes) 000300 1790774625.228475 punch from 0x00c914 to 0x016a49 (0xa136 bytes) 000301 1790774625.234389 trunc from 0x040000 to 0x00f4e5 (0x30b1a bytes) 000302 1790774625.254801 mapwrite 0x038abe thru 0x03ffff (0x7542 bytes) 000303 1790774625.378621 punch from 0x00b771 to 0x00daa9 (0x2339 bytes) 000304 1790774625.390669 mapread 0x034d07 thru 0x03f544 (0xa83e bytes) 000306 1790774625.418546 mapread 0x009fc9 thru 0x00f4d2 (0x550a bytes) 000307 1790774625.436145 mapread 0x021204 thru 0x02eb9b (0xd998 bytes) 000309 1790774625.562915 punch from 0x024715 to 0x02ce46 (0x8732 bytes) 000310 1790774625.583624 write 0x02928a thru 0x02aec1 (0x1c38 bytes) 000311 1790774625.624957 write 0x02720d thru 0x02d271 (0x6065 bytes) 000315 1790774625.636344 mapread 0x024829 thru 0x0258b0 (0x1088 bytes) 000316 1790774625.674674 trunc from 0x040000 to 0x03383b (0xc7c4 bytes) 000317 1790774625.734560 punch from 0x02c896 to 0x03383b (0x6fa6 bytes) 000318 1790774625.782641 mapwrite 0x03ce73 thru 0x03ffff (0x318d bytes) 000320 1790774626.051429 mapread 0x00fea7 thru 0x010ad3 (0xc2d bytes) 000324 1790774626.082491 read 0x02efaf thru 0x03be70 (0xcec2 bytes) 000327 1790774626.132079 write 0x02da49 thru 0x034f96 (0x754e bytes) 000330 1790774626.139151 mapread 0x031a3b thru 0x03aa1e (0x8fe4 bytes) 000331 1790774626.144091 mapread 0x0271ad thru 0x02e32a (0x717e bytes) 000333 1790774626.153983 write 0x0256fe thru 0x031376 (0xbc79 bytes) 000335 1790774626.169074 mapread 0x027a79 thru 0x0306da (0x8c62 bytes) 000336 1790774626.172316 mapwrite 0x004fa5 thru 0x00ee1c (0x9e78 bytes) 000337 1790774626.325603 trunc from 0x040000 to 0x005a96 (0x3a569 bytes) 000338 1790774626.372234 read 0x0014ff thru 0x004a73 (0x3575 bytes) 000342 1790774626.374251 mapwrite 0x03314f thru 0x03a6f5 (0x75a7 bytes) 000343 1790774626.566505 trunc from 0x03a6f6 to 0x01ac25 (0x1fad0 bytes) 000344 1790774626.592082 punch from 0x00261d to 0x012434 (0xfe18 bytes) 000345 1790774626.596381 trunc from 0x01ac26 to 0x022a2a (0x7e05 bytes) 000347 1790774626.615683 mapread 0x011922 thru 0x01ee38 (0xd517 bytes) 000348 1790774626.650656 mapwrite 0x0214e7 thru 0x02e3fc (0xcf16 bytes) 000351 1790774626.832542 mapread 0x00d9c2 thru 0x00ece8 (0x1327 bytes) 000352 1790774626.846525 write 0x027cc0 thru 0x02fda3 (0x80e4 bytes) 000354 1790774626.850672 punch from 0x023674 to 0x023d27 (0x6b4 bytes) 000356 1790774626.866879 punch from 0x00afd3 to 0x012d47 (0x7d75 bytes) 000357 1790774626.877935 MIRROR_EXTEND to 2 mirrors 000358 1790774628.466743 write 0x01839c thru 0x02474b (0xc3b0 bytes) 000359 1790774628.582457 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x2fda4): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2fda4): Mirror 2: 0xdb0dd014 (stale) CRC-32 checksum value for chunk [0, 0x2fda4): Mirror 1: 0x11e029d6 Mirror 3: 0x11e029d6 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2fda4: skipped 000360 1790774630.195827 read 0x012a87 thru 0x0213b0 (0xe92a bytes) 000361 1790774630.237540 mapwrite 0x0381ba thru 0x03ffff (0x7e46 bytes) 000362 1790774630.559471 trunc from 0x040000 to 0x0125ff (0x2da00 bytes) 000363 1790774630.590302 read 0x0042f5 thru 0x008b8a (0x4896 bytes) 000364 1790774630.602489 write 0x02245b thru 0x02b6f9 (0x929f bytes) 000365 1790774630.605410 write 0x0158ee thru 0x01f1ab (0x98be bytes) 000366 1790774630.619238 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2b6fa): Mirror 2: 0x27b61399 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2b6fa): Mirror 2: 0x27b61399 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2b6fa: skipped 000368 1790774631.078767 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x2b6fa): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2b6fa): Mirror 2: 0x27b61399 (stale) CRC-32 checksum value for chunk [0, 0x2b6fa): Mirror 1: 0x47d48d0d Mirror 3: 0x47d48d0d lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2b6fa: skipped 000371 1790774633.107400 trunc from 0x02b6fa to 0x001247 (0x2a4b2 bytes) 000373 1790774633.211460 punch from 0x000309 to 0x001247 (0xf3f bytes) 000374 1790774633.224579 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x1248): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1248): Mirror 2: 0xc857a5e8 (stale) Mirror 3: 0xc857a5e8 (stale) CRC-32 checksum value for chunk [0, 0x1248): Mirror 1: 0xc857a5e8 Mirror 4: 0xc857a5e8 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1248: skipped 000377 1790774634.889229 mapread 0x0000f1 thru 0x001247 (0x1157 bytes) 000378 1790774634.928237 mapwrite 0x03daf6 thru 0x03ffff (0x250a bytes) 000379 1790774635.101334 punch from 0x00abcd to 0x01298a (0x7dbe bytes) 000380 1790774635.118517 trunc from 0x040000 to 0x00071a (0x3f8e5 bytes) 000381 1790774635.150850 trunc from 0x00071b to 0x0349ce (0x342b4 bytes) 000382 1790774635.170073 read 0x02e50b thru 0x034456 (0x5f4c bytes) 000385 1790774635.180590 write 0x011472 thru 0x015148 (0x3cd7 bytes) 000388 1790774635.211918 read 0x0134f3 thru 0x018222 (0x4d30 bytes) 000389 1790774635.252519 mapwrite 0x01bd50 thru 0x025d88 (0xa039 bytes) 000390 1790774635.354957 read 0x0020c1 thru 0x007108 (0x5048 bytes) 000391 1790774635.374633 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x349cf): Mirror 3: 0xc356448b (stale) Mirror 4: 0x3e33af58 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x349cf): Mirror 3: 0xc356448b (stale) Mirror 4: 0x3e33af58 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x349cf: skipped 000392 1790774635.918645 write 0x02c22d thru 0x03ae4b (0xec1f bytes) 000393 1790774635.943588 mapwrite 0x03acdc thru 0x03cd55 (0x207a bytes) 000394 1790774636.135568 punch from 0x023f49 to 0x029c4e (0x5d06 bytes) 000395 1790774636.148065 mapwrite 0x005960 thru 0x00f473 (0x9b14 bytes) 000396 1790774636.251783 read 0x02f669 thru 0x03cd55 (0xd6ed bytes) 000397 1790774636.254794 mapwrite 0x0064e5 thru 0x0070a0 (0xbbc bytes) 000398 1790774636.339623 mapwrite 0x03a637 thru 0x03ffff (0x59c9 bytes) 000399 1790774636.454496 read 0x023ca4 thru 0x023df5 (0x152 bytes) 000400 1790774636.471493 write 0x00b09c thru 0x00f52a (0x448f bytes) truncating to largest ever: 0x3f7e8 000402 1790774636.474453 trunc from 0x040000 to 0x03f7e7 (0x818 bytes) 000403 1790774636.508759 mapread 0x027298 thru 0x02c079 (0x4de2 bytes) 000404 1790774636.509939 mapread 0x03d55f thru 0x03f7e7 (0x2289 bytes) 000405 1790774636.515812 trunc from 0x03f7e8 to 0x012f0b (0x2c8dc bytes) 000406 1790774636.541761 trunc from 0x012f0c to 0x008280 (0xac8b bytes) 000408 1790774636.576458 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x8281): Mirror 4: 0x5d9db33e (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x8281): Mirror 4: 0x5d9db33e (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x8281: skipped 000409 1790774636.971380 write 0x01ac2a thru 0x01d467 (0x283e bytes) 000410 1790774636.974490 mapwrite 0x00c9f3 thru 0x0119a3 (0x4fb1 bytes) 000411 1790774637.087371 write 0x03104e thru 0x03a10f (0x90c2 bytes) 000412 1790774637.090960 write 0x03d7f6 thru 0x03ffff (0x280a bytes) 000413 1790774637.092610 trunc from 0x040000 to 0x039106 (0x6ef9 bytes) 000414 1790774637.132130 trunc from 0x039107 to 0x03a4d6 (0x13d0 bytes) 000415 1790774637.150016 punch from 0x029074 to 0x035659 (0xc5e6 bytes) 000416 1790774637.159357 mapread 0x00074f thru 0x005c87 (0x5539 bytes) 000418 1790774637.178384 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) 000419 1790774637.378676 write 0x01710d thru 0x01c790 (0x5684 bytes) 000420 1790774637.405364 mapread 0x00d384 thru 0x017391 (0xa00e bytes) 000421 1790774637.435240 punch from 0x039889 to 0x03a4d6 (0xc4e bytes) 000424 1790774637.468440 read 0x018e0a thru 0x01cae0 (0x3cd7 bytes) 000426 1790774637.473188 write 0x033dd8 thru 0x03594d (0x1b76 bytes) 000429 1790774637.498963 trunc from 0x03a4d7 to 0x016418 (0x240be bytes) 000430 1790774637.546214 write 0x00f8c5 thru 0x019c2f (0xa36b bytes) 000431 1790774637.551739 mapread 0x00e6a9 thru 0x019c2f (0xb587 bytes) 000435 1790774637.562730 trunc from 0x019c30 to 0x00b7de (0xe451 bytes) 000436 1790774637.626191 trunc from 0x00b7df to 0x033ac6 (0x282e8 bytes) 000437 1790774637.660419 mapread 0x022af0 thru 0x02a941 (0x7e52 bytes) 000438 1790774637.681607 write 0x017685 thru 0x019980 (0x22fc bytes) 000439 1790774637.720499 mapwrite 0x024155 thru 0x031ea4 (0xdd50 bytes) 000442 1790774637.855863 trunc from 0x033ac7 to 0x00563e (0x2e488 bytes) 000443 1790774637.884117 mapread 0x003f1e thru 0x00563e (0x1721 bytes) 000444 1790774637.904131 write 0x02609e thru 0x0306ee (0xa651 bytes) 000446 1790774637.909534 mapwrite 0x038919 thru 0x03cfb6 (0x469e bytes) 000447 1790774638.017169 read 0x002d26 thru 0x006496 (0x3771 bytes) 000448 1790774638.030544 read 0x026d7e thru 0x02b98b (0x4c0e bytes) 000449 1790774638.031950 read 0x021e48 thru 0x02acaa (0x8e63 bytes) 000450 1790774638.033219 mapread 0x0190f8 thru 0x019925 (0x82e bytes) 000453 1790774638.037173 mapread 0x01f746 thru 0x01fb7a (0x435 bytes) 000454 1790774638.038724 trunc from 0x03cfb7 to 0x013158 (0x29e5e bytes) 000455 1790774638.066322 read 0x002176 thru 0x006486 (0x4311 bytes) 000456 1790774638.067677 mapread 0x007f5c thru 0x00eec4 (0x6f69 bytes) 000457 1790774638.070353 trunc from 0x013159 to 0x002547 (0x10c11 bytes) 000458 1790774638.092578 trunc from 0x002548 to 0x018fce (0x16a87 bytes) truncating to largest ever: 0x3fb87 000461 1790774638.120344 trunc from 0x018fcf to 0x03fb86 (0x26bb8 bytes) 000462 1790774638.144928 trunc from 0x03fb87 to 0x01b423 (0x24763 bytes) 000463 1790774638.175242 read 0x004686 thru 0x00d56b (0x8ee6 bytes) 000465 1790774638.198222 punch from 0x000c69 to 0x00c9fb (0xbd93 bytes) 000466 1790774638.204639 punch from 0x018f83 to 0x01b423 (0x24a1 bytes) 000469 1790774638.217488 mapread 0x019b69 thru 0x01b423 (0x18bb bytes) 000471 1790774638.262530 trunc from 0x01b424 to 0x003065 (0x183be bytes) 000472 1790774638.305528 mapwrite 0x015fc6 thru 0x01f798 (0x97d3 bytes) 000473 1790774638.428835 punch from 0x00b3d5 to 0x01b3be (0xffea bytes) 000474 1790774638.437025 mapread 0x01bd8b thru 0x01dbde (0x1e54 bytes) 000476 1790774638.446015 trunc from 0x01f799 to 0x01295e (0xce3a bytes) 000477 1790774638.465329 mapread 0x001d82 thru 0x00c81a (0xaa99 bytes) 000481 1790774638.535181 punch from 0x00484d to 0x006cbc (0x2470 bytes) 000483 1790774638.543531 trunc from 0x01295f to 0x0177d3 (0x4e75 bytes) 000484 1790774638.568816 write 0x00edc1 thru 0x00f995 (0xbd5 bytes) 000486 1790774638.587830 read 0x00d82c thru 0x0166f1 (0x8ec6 bytes) 000488 1790774638.607923 read 0x013d6e thru 0x0177d3 (0x3a66 bytes) 000490 1790774638.613968 mapread 0x000a56 thru 0x000bb7 (0x162 bytes) 000491 1790774638.617776 write 0x018477 thru 0x023bbc (0xb746 bytes) 000492 1790774638.626588 write 0x01f3e2 thru 0x0221b8 (0x2dd7 bytes) 000493 1790774638.629369 read 0x0113d2 thru 0x01c1d0 (0xadff bytes) 000494 1790774638.633159 mapwrite 0x005886 thru 0x014a99 (0xf214 bytes) 000496 1790774638.778951 mapread 0x01741b thru 0x018957 (0x153d bytes) 000498 1790774638.781837 read 0x00234c thru 0x0085a3 (0x6258 bytes) 000501 1790774638.786561 trunc from 0x023bbd to 0x032327 (0xe76b bytes) 000502 1790774638.850805 write 0x013697 thru 0x01f135 (0xba9f bytes) 000503 1790774638.856266 read 0x00a6ff thru 0x00f4ac (0x4dae bytes) 000504 1790774638.857187 mapwrite 0x03da3e thru 0x03ffff (0x25c2 bytes) 000506 1790774638.961248 trunc from 0x040000 to 0x03e242 (0x1dbd bytes) 000508 1790774638.994232 read 0x03ca86 thru 0x03e242 (0x17bd bytes) 000510 1790774638.998020 mapwrite 0x028b87 thru 0x031efd (0x9377 bytes) 000511 1790774639.075824 read 0x038b2c thru 0x03e242 (0x5717 bytes) 000514 1790774639.081591 punch from 0x001de1 to 0x00c120 (0xa340 bytes) 000517 1790774639.090534 trunc from 0x03e243 to 0x01bcf9 (0x22549 bytes) 000518 1790774639.147356 write 0x009666 thru 0x010b81 (0x751c bytes) 000519 1790774639.170374 read 0x003a6f thru 0x0112c8 (0xd85a bytes) 000520 1790774639.173158 punch from 0x012ce7 to 0x019fe4 (0x72fe bytes) 000523 1790774639.214252 mapread 0x01bc09 thru 0x01bcf9 (0xf1 bytes) 000525 1790774639.217536 mapread 0x01a432 thru 0x01bcf9 (0x18c8 bytes) 000528 1790774639.224019 mapwrite 0x0281c3 thru 0x02f463 (0x72a1 bytes) 000529 1790774639.324589 punch from 0x007429 to 0x009d9d (0x2975 bytes) 000532 1790774639.330595 punch from 0x028fca to 0x02d029 (0x4060 bytes) 000534 1790774639.342382 mapread 0x00acbc thru 0x01807f (0xd3c4 bytes) 000535 1790774639.346807 write 0x02d74c thru 0x0353a4 (0x7c59 bytes) 000536 1790774639.362377 MIRROR_EXTEND to 2 mirrors 000537 1790774640.481389 punch from 0x023151 to 0x02a7ce (0x767e bytes) 000538 1790774640.572600 punch from 0x00fd29 to 0x017e09 (0x80e1 bytes) 000541 1790774640.581379 mapwrite 0x02b8c7 thru 0x03b2c7 (0xfa01 bytes) 000543 1790774640.721855 write 0x03ebc1 thru 0x03ffff (0x143f bytes) 000544 1790774640.722969 mapwrite 0x0087dd thru 0x0183a1 (0xfbc5 bytes) 000546 1790774640.847754 punch from 0x01475f to 0x01b2e9 (0x6b8b bytes) 000547 1790774640.864918 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) 000549 1790774641.046906 mapwrite 0x0216e9 thru 0x022cc3 (0x15db bytes) 000550 1790774641.097888 mapwrite 0x031351 thru 0x03cfa3 (0xbc53 bytes) 000552 1790774641.179094 trunc from 0x040000 to 0x00385e (0x3c7a1 bytes) 000553 1790774641.201029 write 0x0274a9 thru 0x029f52 (0x2aaa bytes) 000554 1790774641.202333 write 0x0117cd thru 0x01f013 (0xd847 bytes) 000556 1790774641.224143 mapwrite 0x0288f1 thru 0x0309e0 (0x80f0 bytes) 000558 1790774641.398319 write 0x011221 thru 0x0157ec (0x45cc bytes) 000560 1790774641.401108 mapread 0x028591 thru 0x02ae73 (0x28e3 bytes) 000562 1790774641.404398 write 0x00059e thru 0x001a12 (0x1475 bytes) 000563 1790774641.418880 trunc from 0x0309e1 to 0x000538 (0x304a8 bytes) 000567 1790774641.474866 punch from 0x00052e to 0x000538 (0xb bytes) 000569 1790774641.493614 mapread 0x0000ee thru 0x000538 (0x44b bytes) 000570 1790774641.505576 mapwrite 0x037874 thru 0x03bacf (0x425c bytes) 000573 1790774641.606943 trunc from 0x03bad0 to 0x01ba03 (0x200cc bytes) 000574 1790774641.634678 mapwrite 0x023c46 thru 0x025df1 (0x21ac bytes) 000576 1790774641.730958 mapread 0x00e0ae thru 0x01a216 (0xc169 bytes) 000577 1790774641.749423 mapread 0x003fa8 thru 0x00950a (0x5563 bytes) 000578 1790774641.758208 mapwrite 0x03e446 thru 0x03ffff (0x1bba bytes) 000583 1790774641.849440 write 0x011045 thru 0x020756 (0xf712 bytes) 000584 1790774641.851208 trunc from 0x040000 to 0x01116f (0x2ee90 bytes) 000585 1790774641.877586 punch from 0x00afb9 to 0x01116f (0x61b7 bytes) 000586 1790774641.888560 punch from 0x010e3b to 0x01116f (0x335 bytes) 000588 1790774641.896562 write 0x0348c3 thru 0x037a96 (0x31d4 bytes) 000590 1790774641.901315 mapwrite 0x002104 thru 0x004e5e (0x2d5b bytes) 000592 1790774641.966434 punch from 0x02b51c to 0x03289e (0x7383 bytes) 000594 1790774641.973367 punch from 0x0051fa to 0x00ce37 (0x7c3e bytes) 000599 1790774641.982739 write 0x00fc8a thru 0x0145bc (0x4933 bytes) 000600 1790774641.997105 mapread 0x02cad7 thru 0x0333dd (0x6907 bytes) 000601 1790774642.009995 trunc from 0x037a97 to 0x02e283 (0x9813 bytes) 000603 1790774642.029619 punch from 0x002367 to 0x00d87f (0xb519 bytes) 000604 1790774642.045813 punch from 0x000c72 to 0x010716 (0xfaa5 bytes) 000605 1790774642.053437 write 0x014461 thru 0x019edc (0x5a7c bytes) 000606 1790774642.062985 trunc from 0x02e284 to 0x01b9b7 (0x128cc bytes) 000609 1790774642.088365 mapwrite 0x00cefe thru 0x0186b4 (0xb7b7 bytes) 000610 1790774642.181008 write 0x0014d9 thru 0x00a87f (0x93a7 bytes) 000612 1790774642.201098 mapread 0x01b871 thru 0x01b9b7 (0x147 bytes) 000615 1790774642.230463 write 0x01e2c3 thru 0x026b80 (0x88be bytes) 000616 1790774642.250602 write 0x012664 thru 0x01a73c (0x80d9 bytes) 000617 1790774642.264650 MIRROR_EXTEND to 2 mirrors 000618 1790774643.580094 read 0x020444 thru 0x026b80 (0x673d bytes) 000620 1790774643.591311 read 0x00c6e9 thru 0x011d7f (0x5697 bytes) 000622 1790774643.602484 read 0x0115c8 thru 0x01a761 (0x919a bytes) 000623 1790774643.603014 mapwrite 0x036bba thru 0x03e454 (0x789b bytes) 000624 1790774643.742008 write 0x03ec51 thru 0x03ffff (0x13af bytes) 000625 1790774643.743116 trunc from 0x040000 to 0x0035e5 (0x3ca1a bytes) 000627 1790774643.775482 trunc from 0x0035e6 to 0x00a362 (0x6d7d bytes) 000629 1790774643.798394 write 0x03286c thru 0x03ffff (0xd794 bytes) 000631 1790774643.802311 mapread 0x03d8a8 thru 0x03ffff (0x2758 bytes) 000632 1790774643.803494 write 0x016e12 thru 0x022913 (0xbb02 bytes) 000633 1790774643.814277 mapread 0x009987 thru 0x010b0e (0x7188 bytes) 000634 1790774643.856532 mapread 0x021d03 thru 0x0275f7 (0x58f5 bytes) 000635 1790774643.889376 read 0x03cc7d thru 0x03ffff (0x3383 bytes) 000637 1790774643.890474 mapread 0x031113 thru 0x0399d0 (0x88be bytes) 000638 1790774643.906525 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x29f1f976 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xa3526340 Mirror 3: 0xa3526340 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000639 1790774645.521087 punch from 0x02ce6e to 0x0331e2 (0x6375 bytes) 000640 1790774645.717051 mapwrite 0x003023 thru 0x00a48c (0x746a bytes) 000641 1790774645.999393 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x29f1f976 (stale) Mirror 3: 0xa3526340 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x8972295 Mirror 4: 0x8972295 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000643 1790774648.138608 punch from 0x03c7be to 0x03e6f0 (0x1f33 bytes) 000644 1790774648.236579 write 0x0275c4 thru 0x02be5d (0x489a bytes) 000645 1790774648.251148 write 0x01c4ea thru 0x02ae24 (0xe93b bytes) 000646 1790774648.263850 mapread 0x014036 thru 0x018de7 (0x4db2 bytes) 000648 1790774648.290554 read 0x03b08d thru 0x03ffff (0x4f73 bytes) 000649 1790774648.300193 write 0x02f22a thru 0x03d61a (0xe3f1 bytes) 000650 1790774648.308323 mapwrite 0x02cac9 thru 0x03190b (0x4e43 bytes) 000651 1790774648.447932 read 0x03866e thru 0x03ffff (0x7992 bytes) 000653 1790774648.449941 write 0x03beb8 thru 0x03e93e (0x2a87 bytes) 000654 1790774648.458107 mapread 0x00273b thru 0x01117b (0xea41 bytes) 000656 1790774648.491783 MIRROR_RESYNC 4 mirrors 000658 1790774650.204556 mapwrite 0x0105a6 thru 0x01f8bc (0xf317 bytes) 000660 1790774650.452582 read 0x002573 thru 0x006b30 (0x45be bytes) 000661 1790774650.482083 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x22d8f003 (stale) Mirror 4: 0x22d8f003 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x22d8f003 (stale) Mirror 4: 0x22d8f003 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000662 1790774650.832126 write 0x02d0b5 thru 0x0344fb (0x7447 bytes) 000663 1790774650.841363 read 0x01a2ba thru 0x01b88c (0x15d3 bytes) 000664 1790774650.847135 punch from 0x00fbeb to 0x014de6 (0x51fc bytes) 000666 1790774650.864932 trunc from 0x040000 to 0x0131f9 (0x2ce06 bytes) 000667 1790774650.888213 write 0x02b173 thru 0x02ea8e (0x391c bytes) 000668 1790774650.889899 mapwrite 0x03d9b0 thru 0x03eaac (0x10fd bytes) 000669 1790774650.969450 mapwrite 0x017f2e thru 0x01f3b1 (0x7484 bytes) 000670 1790774651.096433 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x3eaad): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3eaad): Mirror 3: 0xefd1593c (stale) Mirror 4: 0xefd1593c (stale) CRC-32 checksum value for chunk [0, 0x3eaad): Mirror 1: 0x82123ffb Mirror 5: 0x82123ffb lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3eaad: skipped 000672 1790774652.884962 trunc from 0x03eaad to 0x012ae6 (0x2bfc6 bytes) 000673 1790774653.080309 read 0x009f1c thru 0x010b1b (0x6c00 bytes) 000674 1790774653.112150 mapwrite 0x0384b4 thru 0x03ffff (0x7b4c bytes) 000676 1790774653.346726 read 0x00016e thru 0x00bf7f (0xbe12 bytes) 000677 1790774653.376074 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x22d8f003 (stale) Mirror 5: 0xc1952f9 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x22d8f003 (stale) Mirror 5: 0xc1952f9 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000678 1790774653.916463 mapwrite 0x028c76 thru 0x032de4 (0xa16f bytes) 000680 1790774654.007769 write 0x0389f0 thru 0x03d101 (0x4712 bytes) 000681 1790774654.020203 read 0x03136e thru 0x037c3f (0x68d2 bytes) 000682 1790774654.029390 mapread 0x005f1a thru 0x012bf8 (0xccdf bytes) 000683 1790774654.048896 mapread 0x00a642 thru 0x00dadd (0x349c bytes) 000686 1790774654.050660 mapread 0x01c448 thru 0x01de25 (0x19de bytes) 000688 1790774654.067505 write 0x001a34 thru 0x003334 (0x1901 bytes) 000689 1790774654.068880 punch from 0x00b02d to 0x017434 (0xc408 bytes) 000691 1790774654.091756 read 0x00d28a thru 0x010619 (0x3390 bytes) 000692 1790774654.107453 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0xc1952f9 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0xc1952f9 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000693 1790774654.577871 punch from 0x03477d to 0x03f9de (0xb262 bytes) 000694 1790774654.600070 trunc from 0x040000 to 0x002868 (0x3d797 bytes) 000695 1790774654.628943 punch from 0x00001f to 0x002868 (0x284a bytes) 000696 1790774654.639486 mapread 0x000b93 thru 0x0018d0 (0xd3e bytes) 000697 1790774654.669163 trunc from 0x002869 to 0x02723b (0x249d3 bytes) 000698 1790774654.720758 punch from 0x01163f to 0x01d548 (0xbf0a bytes) 000701 1790774654.733236 punch from 0x00f723 to 0x01f3c8 (0xfca6 bytes) 000702 1790774654.760296 trunc from 0x02723c to 0x013bb6 (0x13685 bytes) 000703 1790774654.812756 punch from 0x01042e to 0x013bb6 (0x3789 bytes) 000704 1790774654.825212 trunc from 0x013bb7 to 0x01cec2 (0x930c bytes) 000706 1790774654.889268 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x1cec3): Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1cec3): Mirror 5: 0xcaf4780a (stale) CRC-32 checksum value for chunk [0, 0x1cec3): Mirror 1: 0x51658d7b Mirror 6: 0x51658d7b lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1cec3: skipped 000708 1790774656.882057 read 0x0188fd thru 0x01cec2 (0x45c6 bytes) 000709 1790774656.904551 trunc from 0x01cec3 to 0x034f07 (0x18045 bytes) 000710 1790774657.009672 write 0x004736 thru 0x00e9dd (0xa2a8 bytes) 000711 1790774657.027841 MIRROR_RESYNC 3 mirrors 000712 1790774658.222292 read 0x033c2b thru 0x034f07 (0x12dd bytes) 000713 1790774658.226326 read 0x003f06 thru 0x00dad8 (0x9bd3 bytes) 000714 1790774658.238016 mapwrite 0x02984c thru 0x035e11 (0xc5c6 bytes) 000715 1790774658.451440 mapwrite 0x01ca23 thru 0x029d8a (0xd368 bytes) 000716 1790774658.585623 read 0x022e18 thru 0x02fc62 (0xce4b bytes) 000717 1790774658.592402 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x35e12): Mirror 6: 0x10c11d86 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x35e12): Mirror 6: 0x10c11d86 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x35e12: skipped 000718 1790774658.913863 mapwrite 0x0196b0 thru 0x027849 (0xe19a bytes) 000720 1790774659.014235 punch from 0x00cc5b to 0x0124ec (0x5892 bytes) 000721 1790774659.020744 mapread 0x031174 thru 0x035e11 (0x4c9e bytes) 000722 1790774659.032370 read 0x033f44 thru 0x035e11 (0x1ece bytes) 000725 1790774659.034613 read 0x011e87 thru 0x01a3b4 (0x852e bytes) 000726 1790774659.048366 punch from 0x026944 to 0x029d6f (0x342c bytes) 000728 1790774659.055541 mapwrite 0x02b2ca thru 0x02f327 (0x405e bytes) 000729 1790774659.117225 read 0x022150 thru 0x02bfbc (0x9e6d bytes) 000731 1790774659.137425 MIRROR_RESYNC 2 mirrors 000733 1790774660.103573 punch from 0x033cd4 to 0x035e11 (0x213e bytes) 000734 1790774660.179358 mapwrite 0x00ca6b thru 0x01976e (0xcd04 bytes) 000735 1790774660.288137 write 0x025e54 thru 0x0286fd (0x28aa bytes) 000737 1790774660.303468 punch from 0x0192c9 to 0x019972 (0x6aa bytes) 000738 1790774660.315781 mapwrite 0x03f978 thru 0x03ffff (0x688 bytes) 000740 1790774660.407907 mapwrite 0x0129a6 thru 0x020b38 (0xe193 bytes) 000744 1790774660.450964 read 0x018320 thru 0x022d21 (0xaa02 bytes) 000745 1790774660.459674 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: 0x975be3f6 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x5514d6c2 Mirror 7: 0x5514d6c2 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000746 1790774661.594467 MIRROR_RESYNC 3 mirrors 000747 1790774662.438309 mapwrite 0x0272d9 thru 0x02e5bf (0x72e7 bytes) 000748 1790774662.633590 trunc from 0x040000 to 0x01d5a6 (0x22a59 bytes) 000749 1790774662.701762 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x1d5a7): Mirror 6: (stale - read error) Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1d5a7): Mirror 6: 0x50b79a5f (stale) Mirror 7: 0x50b79a5f (stale) CRC-32 checksum value for chunk [0, 0x1d5a7): Mirror 1: 0x50b79a5f Mirror 8: 0x50b79a5f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1d5a7: skipped 000751 1790774664.443290 trunc from 0x01d5a7 to 0x0294da (0xbf34 bytes) 000753 1790774664.578358 mapwrite 0x033eca thru 0x03ffff (0xc136 bytes) 000754 1790774664.713373 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: 0x5514d6c2 (stale) Mirror 7: 0x5514d6c2 (stale) Mirror 8: 0x2bcf9710 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xb1d4d1e2 Mirror 9: 0xb1d4d1e2 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000756 1790774665.946593 mapread 0x035864 thru 0x03b032 (0x57cf bytes) 000761 1790774665.976948 write 0x03d81a thru 0x03ffff (0x27e6 bytes) 000762 1790774666.070932 read 0x029378 thru 0x03540e (0xc097 bytes) 000763 1790774666.081351 read 0x02c3d0 thru 0x02e6fe (0x232f bytes) 000765 1790774666.089269 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: 0x5514d6c2 (stale) Mirror 7: 0x5514d6c2 (stale) Mirror 8: 0x2bcf9710 (stale) Mirror 9: 0xb1d4d1e2 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x2c4f4821 Mirror 10: 0x2c4f4821 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000766 1790774667.346602 trunc from 0x040000 to 0x01a724 (0x258db bytes) 000768 1790774667.472322 MIRROR_RESYNC 6 mirrors 000769 1790774669.061303 write 0x027ad7 thru 0x02d5d1 (0x5afb bytes) 000771 1790774669.141031 trunc from 0x02d5d2 to 0x03bea3 (0xe8d2 bytes) 000773 1790774669.173887 trunc from 0x03bea4 to 0x026ef9 (0x14faa bytes) 000775 1790774669.202271 read 0x01eb1a thru 0x026ef9 (0x83e0 bytes) 000776 1790774669.214272 read 0x0046d3 thru 0x00e707 (0xa035 bytes) 000778 1790774669.235007 MIRROR_RESYNC 6 mirrors 000779 1790774671.333664 write 0x02404a thru 0x029434 (0x53eb bytes) 000781 1790774671.486063 punch from 0x00ef0c to 0x014f0b (0x6000 bytes) 000782 1790774671.522289 write 0x00b6fd thru 0x01b23d (0xfb41 bytes) 000783 1790774671.570143 punch from 0x01001c to 0x0194d1 (0x94b6 bytes) 000784 1790774671.592659 read 0x015ab7 thru 0x01d535 (0x7a7f bytes) 000789 1790774671.612582 mapwrite 0x01e0de thru 0x023274 (0x5197 bytes) 000792 1790774671.703416 punch from 0x00bcc0 to 0x0188d1 (0xcc12 bytes) 000793 1790774671.714586 read 0x010ccc thru 0x01b700 (0xaa35 bytes) 000794 1790774671.731175 MIRROR_SPLIT mirror 3 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x29435): Mirror 6: 0x1b9f950a (stale) Mirror 7: 0x1b9f950a (stale) Mirror 9: 0x1b9f950a (stale) Mirror 10: 0x1b9f950a (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x29435): Mirror 6: 0x1b9f950a (stale) Mirror 7: 0x1b9f950a (stale) Mirror 9: 0x1b9f950a (stale) Mirror 10: 0x1b9f950a (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x29435: skipped 000795 1790774672.323814 write 0x02a0d6 thru 0x02ffc3 (0x5eee bytes) 000797 1790774672.332648 mapwrite 0x007f42 thru 0x017146 (0xf205 bytes) 000799 1790774672.490137 write 0x01c574 thru 0x026c1e (0xa6ab bytes) 000800 1790774672.504552 trunc from 0x02ffc4 to 0x007bfc (0x283c7 bytes) 000801 1790774672.531912 mapwrite 0x02e39d thru 0x030c22 (0x2886 bytes) 000802 1790774672.648930 trunc from 0x030c23 to 0x01c5f0 (0x14632 bytes) 000803 1790774672.669237 trunc from 0x01c5f1 to 0x037dfc (0x1b80c bytes) 000805 1790774672.689370 write 0x03dbe0 thru 0x03ffff (0x2420 bytes) 000806 1790774672.690479 mapwrite 0x015770 thru 0x021a04 (0xc295 bytes) 000807 1790774672.764852 punch from 0x01b001 to 0x0259ef (0xa9ef bytes) 000811 1790774672.770262 mapwrite 0x0323dd thru 0x035840 (0x3464 bytes) 000815 1790774672.827543 write 0x019055 thru 0x0276cb (0xe677 bytes) 000816 1790774672.834505 mapwrite 0x013e81 thru 0x016264 (0x23e4 bytes) 000817 1790774672.894274 punch from 0x01004a to 0x01ea13 (0xe9ca bytes) 000818 1790774672.899921 trunc from 0x040000 to 0x02c033 (0x13fcc bytes) 000819 1790774672.919735 mapread 0x0143fe thru 0x0231e0 (0xede3 bytes) 000820 1790774672.936335 read 0x0101b5 thru 0x01d8e5 (0xd731 bytes) 000821 1790774672.937664 mapwrite 0x019698 thru 0x0287f1 (0xf15a bytes) 000822 1790774672.992525 trunc from 0x02c034 to 0x024081 (0x7fb2 bytes) 000823 1790774673.010081 punch from 0x022637 to 0x024081 (0x1a4b bytes) 000824 1790774673.016414 punch from 0x00645f to 0x00b313 (0x4eb5 bytes) 000825 1790774673.022896 read 0x0227d0 thru 0x022a6c (0x29d bytes) 000827 1790774673.034916 write 0x023af1 thru 0x0306b8 (0xcbc8 bytes) 000828 1790774673.037339 read 0x00719a thru 0x00f3bb (0x8222 bytes) 000829 1790774673.043431 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x306b9): Mirror 7: 0xaea8be90 (stale) Mirror 9: 0xaea8be90 (stale) Mirror 10: 0xaea8be90 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x306b9): Mirror 7: 0xaea8be90 (stale) Mirror 9: 0xaea8be90 (stale) Mirror 10: 0xaea8be90 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x306b9: skipped 000830 1790774673.394515 MIRROR_RESYNC 4 mirrors 000831 1790774674.881642 mapread 0x021fea thru 0x02c7c1 (0xa7d8 bytes) 000833 1790774674.898851 mapread 0x01107a thru 0x01cf97 (0xbf1e bytes) 000835 1790774674.914529 read 0x012e0d thru 0x021edc (0xf0d0 bytes) 000837 1790774674.915177 mapwrite 0x00a191 thru 0x015597 (0xb407 bytes) 000840 1790774675.035724 write 0x02a83a thru 0x0383da (0xdba1 bytes) 000841 1790774675.043398 trunc from 0x0383db to 0x03cab4 (0x46da bytes) 000843 1790774675.070460 read 0x030c0f thru 0x03cab4 (0xbea6 bytes) 000844 1790774675.081481 mapread 0x013296 thru 0x016fa2 (0x3d0d bytes) 000845 1790774675.082374 punch from 0x014e88 to 0x0194ea (0x4663 bytes) 000846 1790774675.114615 mapread 0x00eae4 thru 0x01be8a (0xd3a7 bytes) 000847 1790774675.167051 mapread 0x001e67 thru 0x00e211 (0xc3ab bytes) 000851 1790774675.175520 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3cab5): Mirror 9: 0x5e037807 (stale) Mirror 10: 0x5e037807 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3cab5): Mirror 9: 0x5e037807 (stale) Mirror 10: 0x5e037807 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3cab5: skipped 000852 1790774675.723205 trunc from 0x03cab5 to 0x02aea3 (0x11c11 bytes) 000853 1790774675.788269 mapwrite 0x000504 thru 0x00b6a8 (0xb1a5 bytes) 000854 1790774675.983683 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2aea4): Mirror 10: 0x86e64660 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2aea4): Mirror 10: 0x86e64660 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2aea4: skipped 000855 1790774676.576094 trunc from 0x02aea4 to 0x03aa4d (0xfbaa bytes) 000856 1790774676.611072 mapwrite 0x0215e3 thru 0x021e54 (0x872 bytes) 000857 1790774676.732758 mapread 0x032477 thru 0x03aa4d (0x85d7 bytes) 000858 1790774676.777533 read 0x00e2a2 thru 0x018344 (0xa0a3 bytes) 000859 1790774676.803949 punch from 0x02229d to 0x029ea5 (0x7c09 bytes) 000860 1790774676.829476 mapwrite 0x01d2b4 thru 0x0259df (0x872c bytes) 000861 1790774676.985200 mapwrite 0x006878 thru 0x00bdc4 (0x554d bytes) 000862 1790774677.093010 trunc from 0x03aa4e to 0x00a418 (0x30635 bytes) 000865 1790774677.138303 MIRROR_RESYNC 2 mirrors 000866 1790774678.574898 MIRROR_RESYNC 2 mirrors 000867 1790774679.340237 mapwrite 0x010117 thru 0x01a1de (0xa0c8 bytes) 000868 1790774679.758657 read 0x0054d4 thru 0x012364 (0xce91 bytes) 000869 1790774679.790627 write 0x02f6d8 thru 0x039942 (0xa26b bytes) 000871 1790774679.792536 mapread 0x02ebd9 thru 0x034d01 (0x6129 bytes) 000872 1790774679.841205 punch from 0x00adcf to 0x00dea6 (0x30d8 bytes) 000873 1790774679.897271 read 0x027db3 thru 0x02dd56 (0x5fa4 bytes) 000874 1790774679.903208 punch from 0x0355eb to 0x039942 (0x4358 bytes) 000875 1790774679.913349 mapread 0x008e35 thru 0x00d724 (0x48f0 bytes) 000876 1790774679.951355 read 0x039014 thru 0x039942 (0x92f bytes) 000877 1790774679.953298 write 0x011960 thru 0x01acc7 (0x9368 bytes) 000878 1790774679.959562 write 0x019703 thru 0x01cca2 (0x35a0 bytes) 000879 1790774679.979479 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x39943): Mirror 10: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x39943): Mirror 10: 0x782b874e (stale) CRC-32 checksum value for chunk [0, 0x39943): Mirror 1: 0xdc7a632f Mirror 11: 0xdc7a632f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x39943: skipped 000880 1790774681.604066 trunc from 0x039943 to 0x039b62 (0x220 bytes) 000882 1790774681.699086 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x39b63): Mirror 10: (stale - read error) Mirror 11: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x39b63): Mirror 10: 0x8b40c920 (stale) Mirror 11: 0xd449bd0f (stale) CRC-32 checksum value for chunk [0, 0x39b63): Mirror 1: 0xd449bd0f Mirror 12: 0xd449bd0f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x39b63: skipped 000883 1790774683.386547 write 0x025e3c thru 0x02f6ea (0x98af bytes) 000884 1790774683.508243 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x39b63): Mirror 10: 0x8b40c920 (stale) Mirror 11: 0xd449bd0f (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x39b63): Mirror 10: 0x8b40c920 (stale) Mirror 11: 0xd449bd0f (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x39b63: skipped 000885 1790774683.999510 MIRROR_RESYNC 3 mirrors 000886 1790774685.761874 mapwrite 0x000778 thru 0x00ef6b (0xe7f4 bytes) 000888 1790774686.016988 mapread 0x0078a6 thru 0x00e0a8 (0x6803 bytes) 000889 1790774686.020445 punch from 0x00317c to 0x00715f (0x3fe4 bytes) 000891 1790774686.034272 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x39b63): Mirror 10: (stale - read error) Mirror 11: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x39b63): Mirror 10: 0x3725fbf8 (stale) Mirror 11: 0x3725fbf8 (stale) CRC-32 checksum value for chunk [0, 0x39b63): Mirror 1: 0x4287ba1c Mirror 12: 0x4287ba1c lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x39b63: skipped 000892 1790774687.498525 read 0x01d0cc thru 0x020ed6 (0x3e0b bytes) 000895 1790774687.533071 punch from 0x006218 to 0x007b01 (0x18ea bytes) 000896 1790774687.654333 mapread 0x0353b4 thru 0x035d36 (0x983 bytes) 000900 1790774687.668010 trunc from 0x039b63 to 0x0274bd (0x126a5 bytes) 000901 1790774687.733571 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x274be): Mirror 10: 0xf11d2582 (stale) Mirror 12: 0xb07d67cb (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x274be): Mirror 10: 0xf11d2582 (stale) Mirror 12: 0xb07d67cb (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x274be: skipped 000903 1790774688.205194 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x274be): Mirror 12: 0xb07d67cb (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x274be): Mirror 12: 0xb07d67cb (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x274be: skipped 000904 1790774688.582496 read 0x00d07f thru 0x0117f0 (0x4772 bytes) 000906 1790774688.596244 read 0x01980f thru 0x0274bd (0xdcaf bytes) 000907 1790774688.610325 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x274be): Mirror 12: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x274be): Mirror 12: 0xb07d67cb (stale) CRC-32 checksum value for chunk [0, 0x274be): Mirror 1: 0xa4902178 Mirror 13: 0xa4902178 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x274be: skipped 000909 1790774690.023143 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x274be): Mirror 12: 0xb07d67cb (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x274be): Mirror 12: 0xb07d67cb (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x274be: skipped 000910 1790774690.562383 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) 000912 1790774690.891202 write 0x024450 thru 0x025136 (0xce7 bytes) 000914 1790774690.934840 write 0x0215b9 thru 0x030c99 (0xf6e1 bytes) 000915 1790774690.954348 trunc from 0x030c9a to 0x034454 (0x37bb bytes) 000916 1790774690.986317 punch from 0x00bb5b to 0x015dca (0xa270 bytes) 000917 1790774691.002075 write 0x03c8ab thru 0x03ffff (0x3755 bytes) 000918 1790774691.003064 write 0x0207bf thru 0x02e532 (0xdd74 bytes) 000919 1790774691.009517 read 0x007e02 thru 0x0147a3 (0xc9a2 bytes) 000920 1790774691.030108 punch from 0x039e69 to 0x03b3d1 (0x1569 bytes) 000923 1790774691.046084 punch from 0x03d427 to 0x03ffff (0x2bd9 bytes) 000924 1790774691.056209 punch from 0x03dfb5 to 0x03ffff (0x204b bytes) 000925 1790774691.062418 mapread 0x031135 thru 0x03153c (0x408 bytes) 000926 1790774691.071025 punch from 0x02234c to 0x02e2a2 (0xbf57 bytes) 000927 1790774691.085927 read 0x015c3f thru 0x018f9b (0x335d bytes) 000928 1790774691.101216 punch from 0x037864 to 0x03a068 (0x2805 bytes) 000929 1790774691.106973 mapread 0x00c7c0 thru 0x0193c5 (0xcc06 bytes) 000930 1790774691.108206 trunc from 0x040000 to 0x03690f (0x96f0 bytes) 000931 1790774691.150974 mapwrite 0x031168 thru 0x03da11 (0xc8aa bytes) 000932 1790774691.439115 mapread 0x00c722 thru 0x01550e (0x8ded bytes) 000933 1790774691.440580 trunc from 0x03da12 to 0x01fcac (0x1dd65 bytes) 000934 1790774691.465445 read 0x008707 thru 0x00e558 (0x5e52 bytes) 000937 1790774691.468813 punch from 0x004072 to 0x0086c6 (0x4655 bytes) 000940 1790774691.490369 write 0x01bbf9 thru 0x020175 (0x457d bytes) 000943 1790774691.500406 write 0x01849b thru 0x01d1fb (0x4d61 bytes) 000944 1790774691.504997 punch from 0x017ae6 to 0x020175 (0x8690 bytes) 000945 1790774691.527530 trunc from 0x020176 to 0x027b45 (0x79d0 bytes) 000947 1790774691.573217 write 0x025964 thru 0x0318f6 (0xbf93 bytes) 000948 1790774691.600850 trunc from 0x0318f7 to 0x0149bb (0x1cf3b bytes) 000950 1790774691.656079 write 0x0155aa thru 0x01e2d8 (0x8d2f bytes) 000951 1790774691.665439 trunc from 0x01e2d9 to 0x037711 (0x19439 bytes) 000952 1790774691.699866 read 0x02fd78 thru 0x0367fc (0x6a85 bytes) 000954 1790774691.732961 trunc from 0x037712 to 0x00a769 (0x2cfa8 bytes) 000955 1790774691.780248 punch from 0x002cfd to 0x00a769 (0x7a6d bytes) 000957 1790774691.795371 punch from 0x004188 to 0x009914 (0x578d bytes) 000958 1790774691.805252 read 0x005393 thru 0x00a769 (0x53d7 bytes) 000959 1790774691.822011 MIRROR_EXTEND to 2 mirrors 000960 1790774693.341724 mapread 0x003f63 thru 0x00a769 (0x6807 bytes) 000961 1790774693.368144 read 0x0070de thru 0x00a769 (0x368c bytes) 000963 1790774693.369194 mapread 0x002d82 thru 0x007e60 (0x50df bytes) 000964 1790774693.370831 mapread 0x00490a thru 0x00731e (0x2a15 bytes) 000967 1790774693.372993 mapwrite 0x01262b thru 0x01bca8 (0x967e bytes) 000968 1790774693.575822 mapwrite 0x00a3ea thru 0x00f54a (0x5161 bytes) 000970 1790774693.667955 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d0:0x0]: Invalid argument (22) 000971 1790774693.847328 write 0x03da7d thru 0x03ffff (0x2583 bytes) 000972 1790774693.851098 write 0x032fda thru 0x0374bc (0x44e3 bytes) 000973 1790774693.865129 mapwrite 0x02f8ea thru 0x039120 (0x9837 bytes) 000974 1790774694.062297 mapwrite 0x00613f thru 0x011263 (0xb125 bytes) 000977 1790774694.167982 read 0x0012bc thru 0x00610d (0x4e52 bytes) 000978 1790774694.170767 mapwrite 0x0390ab thru 0x03b0c8 (0x201e bytes) 000980 1790774694.257223 punch from 0x02422c to 0x02f52c (0xb301 bytes) 000982 1790774694.266184 punch from 0x004366 to 0x00999e (0x5639 bytes) 000983 1790774694.272079 read 0x028abf thru 0x036cda (0xe21c bytes) 000985 1790774694.293338 trunc from 0x040000 to 0x00c381 (0x33c7e bytes) 000987 1790774694.323944 read 0x0002fb thru 0x005cf4 (0x59fa bytes) 000988 1790774694.325201 mapread 0x00ad1f thru 0x00c381 (0x1663 bytes) 000989 1790774694.326491 mapwrite 0x01f4ca thru 0x02c5cf (0xd106 bytes) 000990 1790774694.449883 mapread 0x01565c thru 0x018253 (0x2bf8 bytes) 000991 1790774694.457262 MIRROR_EXTEND to 2 mirrors 000992 1790774695.548614 mapread 0x00c92f thru 0x01881f (0xbef1 bytes) 000994 1790774695.614160 write 0x0178cc thru 0x01cbbe (0x52f3 bytes) 000998 1790774695.821477 MIRROR_RESYNC 2 mirrors 000999 1790774697.411608 MIRROR_RESYNC 2 mirrors 001000 1790774698.012163 write 0x02b71e thru 0x030468 (0x4d4b bytes) All operations completed A-OK!