== sanity-flr test 70a: flr mode fsx test ================ 02:00:22 (1787119222) Seed set to 9224 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read 000007 1787119224.541316 mapwrite 0x0343ae thru 0x038706 (0x4359 bytes) truncating to largest ever: 0x25e33 000014 1787119224.629146 trunc from 0x038707 to 0x025e32 (0x128d4 bytes) 000016 1787119224.646615 write 0x03489e thru 0x03ffff (0xb762 bytes) 000017 1787119224.652148 read 0x028ff6 thru 0x02fd0c (0x6d17 bytes) 000018 1787119224.659988 mapread 0x029dd3 thru 0x031353 (0x7581 bytes) 000019 1787119224.665833 read 0x01281d thru 0x015cbf (0x34a3 bytes) 000020 1787119224.718510 write 0x0097dd thru 0x00eb20 (0x5344 bytes) 000022 1787119224.721566 read 0x0367fd thru 0x03ffff (0x9803 bytes) 000023 1787119224.722661 read 0x028efb thru 0x036749 (0xd84f bytes) 000028 1787119224.727066 mapread 0x031304 thru 0x03f3e8 (0xe0e5 bytes) 000030 1787119224.731827 mapwrite 0x02dfe6 thru 0x03337c (0x5397 bytes) 000031 1787119224.786196 trunc from 0x040000 to 0x0037ca (0x3c835 bytes) 000034 1787119224.813440 mapwrite 0x016041 thru 0x020f99 (0xaf59 bytes) 000035 1787119224.926277 trunc from 0x020f9a to 0x021c5d (0xcc4 bytes) 000036 1787119224.962623 read 0x01d9fd thru 0x021c5d (0x4261 bytes) 000038 1787119224.985222 mapwrite 0x0042a7 thru 0x013149 (0xeea3 bytes) 000039 1787119225.092446 mapwrite 0x0185cf thru 0x01c9e9 (0x441b bytes) 000042 1787119225.204435 write 0x036371 thru 0x0369c6 (0x656 bytes) 000045 1787119225.210505 read 0x0161cb thru 0x0243b2 (0xe1e8 bytes) 000047 1787119225.236297 read 0x00d4e2 thru 0x018def (0xb90e bytes) 000048 1787119225.237306 mapwrite 0x01dc95 thru 0x024e6c (0x71d8 bytes) 000052 1787119225.287636 read 0x0195a1 thru 0x01baaf (0x250f bytes) 000054 1787119225.289745 write 0x0105e4 thru 0x01d60d (0xd02a bytes) 000056 1787119225.293010 write 0x02d8b3 thru 0x03bb82 (0xe2d0 bytes) truncating to largest ever: 0x28fa9 000057 1787119225.295559 trunc from 0x03bb83 to 0x028fa8 (0x12bda bytes) truncating to largest ever: 0x3b60e 000059 1787119225.318061 trunc from 0x028fa9 to 0x03b60d (0x12665 bytes) 000061 1787119225.372217 mapwrite 0x00c712 thru 0x00e4fc (0x1deb bytes) 000062 1787119225.457032 trunc from 0x03b60e to 0x021a88 (0x19b85 bytes) 000065 1787119225.517095 MIRROR_EXTEND to 2 mirrors truncating to largest ever: 0x3f188 000066 1787119226.159227 trunc from 0x021a89 to 0x03f187 (0x1d6ff bytes) 000067 1787119226.252435 write 0x0097ae thru 0x00f10b (0x595e bytes) 000069 1787119226.263942 trunc from 0x03f188 to 0x03adc1 (0x43c6 bytes) 000070 1787119226.283813 write 0x03bcaa thru 0x03d592 (0x18e9 bytes) 000071 1787119226.286228 mapwrite 0x033e22 thru 0x0380a2 (0x4281 bytes) 000075 1787119226.395633 trunc from 0x03d593 to 0x00a6e6 (0x32eac bytes) 000076 1787119226.432731 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0xa6e7): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xa6e7): Mirror 2: 0x9a255a88 (stale) CRC-32 checksum value for chunk [0, 0xa6e7): Mirror 1: 0xa95b04f8 Mirror 3: 0xa95b04f8 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xa6e7: skipped 000077 1787119227.485677 trunc from 0x00a6e7 to 0x00b86c (0x1186 bytes) 000078 1787119227.650081 mapwrite 0x02bd42 thru 0x02da6b (0x1d2a bytes) 000080 1787119227.818314 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2da6c): Mirror 3: 0x36d94a2e (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2da6c): Mirror 3: 0x36d94a2e (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2da6c: skipped 000085 1787119228.283197 mapwrite 0x02f27e thru 0x03a6cd (0xb450 bytes) 000086 1787119228.465616 write 0x0236cd thru 0x028291 (0x4bc5 bytes) 000089 1787119228.494856 trunc from 0x03a6ce to 0x023526 (0x171a7 bytes) 000090 1787119228.555450 trunc from 0x023527 to 0x00488f (0x1ec97 bytes) 000091 1787119228.595161 write 0x01fd0a thru 0x023d60 (0x4057 bytes) 000093 1787119228.608096 trunc from 0x023d61 to 0x036faa (0x1324a bytes) 000095 1787119228.657928 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x36fab): Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x36fab): Mirror 3: 0xad5a02e1 (stale) CRC-32 checksum value for chunk [0, 0x36fab): Mirror 1: 0xca4b9216 Mirror 4: 0xca4b9216 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x36fab: skipped 000096 1787119230.280375 mapwrite 0x036602 thru 0x03ffff (0x99fe bytes) 000097 1787119230.417077 write 0x01e2e5 thru 0x01fac1 (0x17dd bytes) 000099 1787119230.431126 read 0x00902e thru 0x00a430 (0x1403 bytes) 000101 1787119230.438914 mapread 0x00966d thru 0x00d394 (0x3d28 bytes) 000103 1787119230.453228 MIRROR_RESYNC 3 mirrors 000104 1787119232.463626 read 0x02dc27 thru 0x03a446 (0xc820 bytes) 000105 1787119232.476168 mapwrite 0x0251ca thru 0x02cb81 (0x79b8 bytes) 000106 1787119232.686053 trunc from 0x040000 to 0x03071f (0xf8e0 bytes) 000107 1787119232.738213 mapread 0x02c798 thru 0x03071f (0x3f88 bytes) 000108 1787119232.780411 read 0x023d76 thru 0x02d74c (0x99d7 bytes) 000111 1787119232.793030 mapwrite 0x023776 thru 0x02d749 (0x9fd4 bytes) 000112 1787119232.950200 mapread 0x009c79 thru 0x00b671 (0x19f9 bytes) 000114 1787119232.969599 mapread 0x02b034 thru 0x03071f (0x56ec bytes) 000115 1787119232.973903 read 0x011a7c thru 0x017d9b (0x6320 bytes) 000119 1787119233.007328 write 0x0192b7 thru 0x01ace9 (0x1a33 bytes) 000120 1787119233.015911 mapread 0x02e186 thru 0x02f644 (0x14bf bytes) 000122 1787119233.017849 read 0x00e68f thru 0x01393b (0x52ad bytes) 000126 1787119233.046988 mapread 0x019e3c thru 0x020349 (0x650e bytes) 000127 1787119233.048521 write 0x00d9e4 thru 0x0193a8 (0xb9c5 bytes) 000131 1787119233.060927 mapread 0x01e9c8 thru 0x027136 (0x876f bytes) 000132 1787119233.066142 mapread 0x020ad6 thru 0x02727f (0x67aa bytes) 000133 1787119233.068042 mapwrite 0x036d5a thru 0x03ffff (0x92a6 bytes) 000134 1787119233.270155 write 0x0185c8 thru 0x027513 (0xef4c bytes) 000135 1787119233.279041 read 0x022d17 thru 0x028bcf (0x5eb9 bytes) 000136 1787119233.284654 write 0x028597 thru 0x03664c (0xe0b6 bytes) 000137 1787119233.292650 read 0x01c000 thru 0x01f5a6 (0x35a7 bytes) 000138 1787119233.310468 MIRROR_RESYNC 3 mirrors 000140 1787119235.171965 read 0x00b3db thru 0x00f4bb (0x40e1 bytes) 000144 1787119235.200487 mapwrite 0x033bc0 thru 0x035384 (0x17c5 bytes) 000146 1787119235.389728 mapwrite 0x023465 thru 0x026ddd (0x3979 bytes) 000148 1787119235.533082 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 4: 0x638c80b2 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 4: 0x638c80b2 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000150 1787119236.071037 write 0x017c9b thru 0x01f82a (0x7b90 bytes) 000152 1787119236.092313 mapwrite 0x02c464 thru 0x03157e (0x511b bytes) 000153 1787119236.260710 mapwrite 0x00011b thru 0x00bd11 (0xbbf7 bytes) 000154 1787119236.404010 mapread 0x02866a thru 0x032e34 (0xa7cb bytes) 000155 1787119236.405660 trunc from 0x040000 to 0x03ca1c (0x35e3 bytes) 000156 1787119236.447216 trunc from 0x03ca1d to 0x00a762 (0x322ba bytes) 000157 1787119236.478935 mapwrite 0x00d6b8 thru 0x00ea15 (0x135e bytes) 000159 1787119236.598287 write 0x011484 thru 0x018d42 (0x78bf bytes) 000161 1787119236.606948 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x18d43): Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x18d43): Mirror 4: 0x649e8e12 (stale) CRC-32 checksum value for chunk [0, 0x18d43): Mirror 1: 0x6bbe0be7 Mirror 5: 0x6bbe0be7 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x18d43: skipped 000165 1787119237.452337 read 0x0124ca thru 0x018d42 (0x6879 bytes) 000167 1787119237.500149 read 0x000291 thru 0x009626 (0x9396 bytes) 000171 1787119237.563682 write 0x00ce55 thru 0x00e020 (0x11cc bytes) 000172 1787119237.684134 write 0x003fb3 thru 0x00ada5 (0x6df3 bytes) 000177 1787119237.699040 trunc from 0x018d43 to 0x0043f6 (0x1494c bytes) 000178 1787119237.754788 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x43f7): Mirror 5: 0xd9a6628c (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x43f7): Mirror 5: 0xd9a6628c (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x43f7: skipped 000179 1787119238.144143 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x43f7): Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x43f7): Mirror 5: 0xd9a6628c (stale) CRC-32 checksum value for chunk [0, 0x43f7): Mirror 1: 0x2d29dc9f Mirror 6: 0x2d29dc9f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x43f7: skipped 000180 1787119239.115139 write 0x00121c thru 0x0057f7 (0x45dc bytes) 000183 1787119239.239539 read 0x001507 thru 0x0057f7 (0x42f1 bytes) 000184 1787119239.247421 MIRROR_RESYNC 3 mirrors 000185 1787119240.218933 mapread 0x004e67 thru 0x0057f7 (0x991 bytes) 000186 1787119240.235105 mapread 0x0041de thru 0x0057f7 (0x161a bytes) 000187 1787119240.236554 mapwrite 0x017380 thru 0x01bd3b (0x49bc bytes) 000189 1787119240.515990 mapread 0x0067a4 thru 0x014fd9 (0xe836 bytes) 000190 1787119240.538853 trunc from 0x01bd3c to 0x03b673 (0x1f938 bytes) 000191 1787119240.569803 write 0x017177 thru 0x02514b (0xdfd5 bytes) 000192 1787119240.576393 trunc from 0x03b674 to 0x008fe1 (0x32692 bytes) 000193 1787119240.595147 mapread 0x008ec7 thru 0x008fe1 (0x11b bytes) 000194 1787119240.618307 MIRROR_RESYNC 3 mirrors 000195 1787119241.834472 MIRROR_RESYNC 3 mirrors 000197 1787119242.394655 mapwrite 0x025bbb thru 0x03410d (0xe553 bytes) 000200 1787119242.778813 mapread 0x01afb8 thru 0x01d41e (0x2467 bytes) 000203 1787119242.792460 trunc from 0x03410e to 0x014b32 (0x1f5db bytes) 000205 1787119242.823815 write 0x005467 thru 0x013c65 (0xe7ff bytes) 000206 1787119242.839404 mapread 0x0048c3 thru 0x00b6ab (0x6de9 bytes) 000208 1787119242.858396 trunc from 0x014b33 to 0x008774 (0xc3be bytes) 000211 1787119242.883851 mapread 0x0034f9 thru 0x0058dd (0x23e5 bytes) 000213 1787119242.886454 mapwrite 0x02c258 thru 0x038c15 (0xc9be bytes) 000217 1787119243.004570 write 0x01247f thru 0x015581 (0x3103 bytes) 000223 1787119243.042431 write 0x031cc3 thru 0x03d4e2 (0xb820 bytes) 000224 1787119243.047049 write 0x019670 thru 0x0209a3 (0x7334 bytes) 000225 1787119243.067027 trunc from 0x03d4e3 to 0x018f42 (0x245a0 bytes) 000229 1787119243.093882 write 0x036b49 thru 0x03ae64 (0x431c bytes) 000230 1787119243.095468 mapwrite 0x017e97 thru 0x027d49 (0xfeb3 bytes) 000233 1787119243.186770 mapread 0x00473d thru 0x013695 (0xef59 bytes) 000235 1787119243.200087 write 0x00a05e thru 0x01043a (0x63dd bytes) 000236 1787119243.201360 write 0x017407 thru 0x025605 (0xe1ff bytes) 000237 1787119243.204985 trunc from 0x03ae65 to 0x00b9eb (0x2f479 bytes) 000238 1787119243.227015 mapwrite 0x0314ca thru 0x035d66 (0x489d bytes) 000239 1787119243.315077 read 0x00870a thru 0x0167db (0xe0d2 bytes) 000241 1787119243.328575 mapwrite 0x0021d4 thru 0x009ac4 (0x78f1 bytes) 000242 1787119243.378874 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x35d67): Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x35d67): Mirror 5: 0x36d523ef (stale) Mirror 6: 0x36d523ef (stale) CRC-32 checksum value for chunk [0, 0x35d67): Mirror 1: 0x311b83c2 Mirror 7: 0x311b83c2 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x35d67: skipped 000243 1787119244.420522 read 0x01dd29 thru 0x01e7e0 (0xab8 bytes) 000244 1787119244.440591 MIRROR_RESYNC 4 mirrors 000245 1787119245.586453 write 0x00332b thru 0x00dea0 (0xab76 bytes) 000247 1787119245.744626 write 0x01a5a3 thru 0x020d0a (0x6768 bytes) 000249 1787119245.760209 mapwrite 0x031e36 thru 0x037344 (0x550f bytes) 000250 1787119245.873645 write 0x03b81f thru 0x03ffff (0x47e1 bytes) 000251 1787119245.875784 write 0x01e1fe thru 0x029349 (0xb14c bytes) 000252 1787119245.882217 mapread 0x015c6c thru 0x019928 (0x3cbd bytes) 000253 1787119245.892035 read 0x02ade2 thru 0x038851 (0xda70 bytes) 000254 1787119245.899365 mapwrite 0x0137ce thru 0x0167a3 (0x2fd6 bytes) 000255 1787119245.948983 read 0x00737a thru 0x012264 (0xaeeb bytes) 000256 1787119245.950424 write 0x010d2a thru 0x02073e (0xfa15 bytes) 000257 1787119245.952400 trunc from 0x040000 to 0x01f995 (0x2066a bytes) 000259 1787119245.973110 write 0x02e374 thru 0x038ee7 (0xab74 bytes) 000260 1787119245.975076 write 0x000a83 thru 0x00ff6d (0xf4eb bytes) 000262 1787119245.985799 mapwrite 0x0389db thru 0x03ec18 (0x623e bytes) 000264 1787119246.083623 read 0x02907c thru 0x033e9b (0xae20 bytes) 000265 1787119246.091005 trunc from 0x03ec19 to 0x00f82b (0x2f3ed bytes) 000267 1787119246.121003 write 0x010a6e thru 0x014ff0 (0x4583 bytes) 000268 1787119246.122299 mapwrite 0x00fb69 thru 0x01a365 (0xa7fd bytes) 000272 1787119246.235624 trunc from 0x01a366 to 0x019962 (0xa03 bytes) 000273 1787119246.263748 mapwrite 0x01a3ff thru 0x022e6c (0x8a6e bytes) 000274 1787119246.406635 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x22e6d): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x22e6d): Mirror 5: 0xbdc14ad4 (stale) Mirror 6: 0xbdc14ad4 (stale) Mirror 7: 0xbdc14ad4 (stale) CRC-32 checksum value for chunk [0, 0x22e6d): Mirror 1: 0x5c038134 Mirror 8: 0x5c038134 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x22e6d: skipped 000276 1787119247.239719 MIRROR_RESYNC 5 mirrors 000277 1787119248.613461 trunc from 0x022e6d to 0x02a2bf (0x7453 bytes) 000278 1787119248.876198 write 0x02b96e thru 0x031a5a (0x60ed bytes) 000279 1787119248.888059 trunc from 0x031a5b to 0x014355 (0x1d705 bytes) 000280 1787119248.935790 read 0x013d48 thru 0x014355 (0x60e bytes) 000281 1787119248.956407 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x14356): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x14356): Mirror 5: 0x44bde717 (stale) Mirror 6: 0x44bde717 (stale) Mirror 7: 0x44bde717 (stale) Mirror 8: 0x44bde717 (stale) CRC-32 checksum value for chunk [0, 0x14356): Mirror 1: 0x44bde717 Mirror 9: 0x44bde717 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x14356: skipped 000282 1787119250.290154 mapwrite 0x00fca7 thru 0x013617 (0x3971 bytes) 000283 1787119250.533302 read 0x0104da thru 0x014355 (0x3e7c bytes) 000284 1787119250.540144 mapread 0x0013dd thru 0x00ac8e (0x98b2 bytes) 000285 1787119250.562165 MIRROR_SPLIT mirror 3 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x14356): Mirror 5: 0x44bde717 (stale) Mirror 6: 0x44bde717 (stale) Mirror 8: 0x44bde717 (stale) Mirror 9: 0x44bde717 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x14356): Mirror 5: 0x44bde717 (stale) Mirror 6: 0x44bde717 (stale) Mirror 8: 0x44bde717 (stale) Mirror 9: 0x44bde717 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x14356: skipped 000286 1787119251.186213 mapwrite 0x0141f8 thru 0x021187 (0xcf90 bytes) 000287 1787119251.295346 read 0x013fc1 thru 0x01e5af (0xa5ef bytes) 000290 1787119251.297391 mapwrite 0x018a5b thru 0x01ac89 (0x222f bytes) 000292 1787119251.339185 mapread 0x001744 thru 0x00585d (0x411a bytes) 000293 1787119251.356323 mapwrite 0x01d2f5 thru 0x029122 (0xbe2e bytes) 000296 1787119251.447552 trunc from 0x029123 to 0x0031bf (0x25f63 bytes) 000298 1787119251.472917 mapread 0x002f1e thru 0x0031bf (0x2a2 bytes) 000299 1787119251.475225 write 0x008a97 thru 0x00e51e (0x5a88 bytes) 000301 1787119251.478105 mapread 0x00267a thru 0x00d011 (0xa998 bytes) 000302 1787119251.485008 mapread 0x00b2f8 thru 0x00e51e (0x3227 bytes) 000303 1787119251.486322 mapread 0x0051ed thru 0x00e51e (0x9332 bytes) 000305 1787119251.490148 mapread 0x008086 thru 0x00e51e (0x6499 bytes) 000310 1787119251.500856 mapread 0x0058c8 thru 0x0070c5 (0x17fe bytes) 000311 1787119251.503472 read 0x00b196 thru 0x00cdfb (0x1c66 bytes) 000315 1787119251.507902 trunc from 0x00e51f to 0x022551 (0x14033 bytes) 000316 1787119251.525181 mapread 0x004776 thru 0x00e0cf (0x995a bytes) 000319 1787119251.534035 trunc from 0x022552 to 0x003ef4 (0x1e65d bytes) 000320 1787119251.558119 write 0x0118c2 thru 0x01cb99 (0xb2d8 bytes) 000321 1787119251.561844 read 0x005795 thru 0x01059c (0xae08 bytes) 000322 1787119251.570982 trunc from 0x01cb9a to 0x00513c (0x17a5d bytes) 000323 1787119251.595216 trunc from 0x00513d to 0x0260df (0x20fa3 bytes) 000324 1787119251.622921 read 0x0111e4 thru 0x012395 (0x11b2 bytes) 000330 1787119251.648553 mapwrite 0x007b1a thru 0x009f6e (0x2455 bytes) 000331 1787119251.736678 trunc from 0x0260e0 to 0x01e0f6 (0x7fe9 bytes) 000332 1787119251.759870 trunc from 0x01e0f7 to 0x032b96 (0x14aa0 bytes) 000333 1787119251.788961 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x32b97): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x32b97): Mirror 5: 0x9e176215 (stale) Mirror 6: 0x9e176215 (stale) Mirror 8: 0x9e176215 (stale) Mirror 9: 0x6bed9e77 (stale) CRC-32 checksum value for chunk [0, 0x32b97): Mirror 1: 0x7de3c745 Mirror 10: 0x7de3c745 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x32b97: skipped 000335 1787119253.695457 mapwrite 0x018967 thru 0x019b89 (0x1223 bytes) 000338 1787119254.042121 MIRROR_SPLIT mirror 1 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x32b97): Mirror 6: 0x9e176215 (stale) Mirror 8: 0x9e176215 (stale) Mirror 9: 0x6bed9e77 (stale) Mirror 10: 0x7de3c745 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x32b97): Mirror 6: 0x9e176215 (stale) Mirror 8: 0x9e176215 (stale) Mirror 9: 0x6bed9e77 (stale) Mirror 10: 0x7de3c745 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x32b97: skipped 000345 1787119254.896719 trunc from 0x032b97 to 0x039578 (0x69e2 bytes) 000347 1787119254.938907 trunc from 0x039579 to 0x0275a3 (0x11fd5 bytes) 000348 1787119254.991058 write 0x0020b7 thru 0x0037cd (0x1717 bytes) 000349 1787119255.035146 write 0x031c61 thru 0x0342a6 (0x2646 bytes) 000351 1787119255.043213 read 0x01faac thru 0x0279bb (0x7f10 bytes) 000353 1787119255.076127 trunc from 0x0342a7 to 0x015eaf (0x1e3f7 bytes) 000354 1787119255.119200 mapwrite 0x01346e thru 0x0172f0 (0x3e83 bytes) 000355 1787119255.283937 mapwrite 0x017c88 thru 0x01f58a (0x7903 bytes) 000357 1787119255.399657 trunc from 0x01f58b to 0x02a6e1 (0xb157 bytes) 000359 1787119255.424523 read 0x012a67 thru 0x019ace (0x7068 bytes) 000360 1787119255.425206 mapwrite 0x022dc7 thru 0x02b54f (0x8789 bytes) 000362 1787119255.511067 trunc from 0x02b550 to 0x039413 (0xdec4 bytes) 000363 1787119255.537200 MIRROR_RESYNC 5 mirrors 000368 1787119257.529922 read 0x0152c7 thru 0x01e71e (0x9458 bytes) 000370 1787119257.547633 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) 000371 1787119258.237741 trunc from 0x039414 to 0x0255c4 (0x13e4f bytes) 000372 1787119258.380785 mapwrite 0x00ae43 thru 0x01a33d (0xf4fb bytes) 000373 1787119258.483448 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x255c5): Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x255c5): Mirror 8: 0x5299a3b5 (stale) Mirror 9: 0x5299a3b5 (stale) Mirror 10: 0x5299a3b5 (stale) CRC-32 checksum value for chunk [0, 0x255c5): Mirror 1: 0x524e9ce6 Mirror 11: 0x524e9ce6 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x255c5: skipped 000374 1787119259.810638 write 0x035afd thru 0x03ffff (0xa503 bytes) 000377 1787119259.909288 write 0x018cf9 thru 0x0260a8 (0xd3b0 bytes) 000378 1787119259.933405 write 0x03c44b thru 0x03ffff (0x3bb5 bytes) 000382 1787119259.955356 mapread 0x03faf1 thru 0x03ffff (0x50f bytes) 000383 1787119259.968088 mapwrite 0x003488 thru 0x005360 (0x1ed9 bytes) 000384 1787119260.090701 write 0x007a3b thru 0x009c5b (0x2221 bytes) 000386 1787119260.094077 read 0x00d64a thru 0x00f729 (0x20e0 bytes) 000387 1787119260.095060 read 0x01f856 thru 0x023360 (0x3b0b bytes) 000389 1787119260.098176 trunc from 0x040000 to 0x018199 (0x27e66 bytes) 000391 1787119260.163088 MIRROR_RESYNC 5 mirrors 000392 1787119262.012659 read 0x0055a9 thru 0x01324d (0xdca5 bytes) 000395 1787119262.024091 read 0x0171e1 thru 0x018199 (0xfb9 bytes) 000396 1787119262.033001 read 0x0138cd thru 0x018199 (0x48cd bytes) 000398 1787119262.034189 trunc from 0x01819a to 0x032f3e (0x1ada5 bytes) 000400 1787119262.126821 read 0x023070 thru 0x02c144 (0x90d5 bytes) 000402 1787119262.153981 mapread 0x002298 thru 0x00aa8f (0x87f8 bytes) 000403 1787119262.156725 mapread 0x00d8e4 thru 0x0178fd (0xa01a bytes) 000404 1787119262.162754 mapwrite 0x018e2c thru 0x021000 (0x81d5 bytes) 000406 1787119262.257737 trunc from 0x032f3f to 0x01c549 (0x169f5 bytes) 000408 1787119262.288457 write 0x033db2 thru 0x034be2 (0xe31 bytes) 000409 1787119262.291667 trunc from 0x034be3 to 0x00673d (0x2e4a5 bytes) 000410 1787119262.314906 mapwrite 0x027189 thru 0x034511 (0xd389 bytes) 000412 1787119262.593610 mapwrite 0x03e986 thru 0x03ffff (0x167a bytes) 000414 1787119262.714668 write 0x003131 thru 0x0073c0 (0x4290 bytes) 000416 1787119262.726979 mapwrite 0x010406 thru 0x016b7f (0x677a bytes) 000417 1787119262.857906 write 0x03e4d1 thru 0x03ffff (0x1b2f bytes) 000418 1787119262.862729 trunc from 0x040000 to 0x02226a (0x1dd95 bytes) 000420 1787119262.888139 mapread 0x009bfb thru 0x00aab6 (0xebc bytes) 000421 1787119262.906730 write 0x03d63e thru 0x03ffff (0x29c2 bytes) 000422 1787119262.912803 write 0x01f90c thru 0x02ee0a (0xf4ff bytes) 000423 1787119262.941853 MIRROR_SPLIT mirror 1 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 9: 0xae781ec9 (stale) Mirror 10: 0xae781ec9 (stale) Mirror 11: 0xae781ec9 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 9: 0xae781ec9 (stale) Mirror 10: 0xae781ec9 (stale) Mirror 11: 0xae781ec9 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000424 1787119263.468303 mapwrite 0x0186cf thru 0x01f0af (0x69e1 bytes) 000425 1787119263.580395 read 0x036b8e thru 0x037fac (0x141f bytes) 000426 1787119263.596383 mapwrite 0x011f5d thru 0x01ca11 (0xaab5 bytes) 000427 1787119263.703447 mapwrite 0x01ae4d thru 0x02a582 (0xf736 bytes) 000429 1787119263.804854 trunc from 0x040000 to 0x03b0b2 (0x4f4d bytes) 000431 1787119263.834645 trunc from 0x03b0b3 to 0x00d5f4 (0x2dabe bytes) 000432 1787119263.875498 write 0x00ab5a thru 0x012194 (0x763b bytes) 000438 1787119263.886046 read 0x003262 thru 0x00b07a (0x7e19 bytes) 000439 1787119263.923160 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x12195): Mirror 10: 0xf2dec5e4 (stale) Mirror 11: 0xf2dec5e4 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x12195): Mirror 10: 0xf2dec5e4 (stale) Mirror 11: 0xf2dec5e4 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x12195: skipped 000440 1787119264.421621 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x12195): Mirror 11: 0xf2dec5e4 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x12195): Mirror 11: 0xf2dec5e4 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x12195: skipped 000441 1787119264.879865 read 0x00a911 thru 0x012194 (0x7884 bytes) 000442 1787119264.905747 read 0x00bd38 thru 0x00de14 (0x20dd bytes) 000443 1787119264.907105 mapwrite 0x024e26 thru 0x031cdf (0xceba bytes) 000444 1787119265.157723 trunc from 0x031ce0 to 0x017854 (0x1a48b bytes) 000445 1787119265.199093 read 0x0056e1 thru 0x00765b (0x1f7b bytes) 000446 1787119265.199938 read 0x0005ba thru 0x00a739 (0xa180 bytes) 000451 1787119265.237929 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) 000452 1787119265.579632 trunc from 0x017855 to 0x02c4bf (0x14c6b bytes) 000453 1787119265.608115 mapread 0x008367 thru 0x00da52 (0x56ec bytes) 000455 1787119265.639229 mapwrite 0x024d95 thru 0x0276de (0x294a bytes) 000456 1787119265.728687 mapwrite 0x028835 thru 0x02c272 (0x3a3e bytes) 000458 1787119265.842029 mapwrite 0x0238e6 thru 0x02b1c1 (0x78dc bytes) 000459 1787119265.948229 read 0x0079d3 thru 0x00ced3 (0x5501 bytes) 000463 1787119265.957771 trunc from 0x02c4c0 to 0x00772f (0x24d90 bytes) 000464 1787119265.998917 read 0x001977 thru 0x0073e5 (0x5a6f bytes) 000466 1787119266.009854 read 0x0075ae thru 0x00772f (0x182 bytes) 000467 1787119266.012993 read 0x0000e9 thru 0x00270a (0x2622 bytes) 000470 1787119266.021901 MIRROR_EXTEND to 2 mirrors 000471 1787119267.158816 mapread 0x006998 thru 0x00772f (0xd98 bytes) 000472 1787119267.178110 write 0x003a7f thru 0x009f07 (0x6489 bytes) 000473 1787119267.283769 mapwrite 0x007ca6 thru 0x013e5a (0xc1b5 bytes) 000476 1787119267.428876 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x13e5b): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x13e5b): Mirror 2: 0x829a6df3 (stale) CRC-32 checksum value for chunk [0, 0x13e5b): Mirror 1: 0x30a27032 Mirror 3: 0x30a27032 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x13e5b: skipped 000481 1787119268.716352 read 0x005c5f thru 0x00717e (0x1520 bytes) 000484 1787119268.736089 read 0x002afc thru 0x00d897 (0xad9c bytes) 000486 1787119268.759851 trunc from 0x013e5b to 0x02bfb6 (0x1815c bytes) 000489 1787119268.910064 trunc from 0x02bfb7 to 0x0026d7 (0x298df bytes) 000490 1787119268.944531 write 0x035bdb thru 0x036191 (0x5b7 bytes) 000491 1787119268.947069 mapwrite 0x02749b thru 0x0372ae (0xfe14 bytes) 000492 1787119269.119476 MIRROR_RESYNC 3 mirrors 000493 1787119270.597095 mapread 0x00beaa thru 0x01204b (0x61a2 bytes) 000494 1787119270.625735 trunc from 0x0372af to 0x03035e (0x6f50 bytes) 000495 1787119270.780620 mapwrite 0x0096a1 thru 0x00fcf3 (0x6653 bytes) 000496 1787119270.867328 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x3035f): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3035f): Mirror 2: 0x97dbbd49 (stale) Mirror 3: 0x97dbbd49 (stale) CRC-32 checksum value for chunk [0, 0x3035f): Mirror 1: 0xb82589f3 Mirror 4: 0xb82589f3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3035f: skipped 000497 1787119272.322092 read 0x01cd66 thru 0x01ceb5 (0x150 bytes) 000499 1787119272.378154 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3035f): Mirror 2: 0x97dbbd49 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3035f): Mirror 2: 0x97dbbd49 (stale) CRC-32 checksum value for chunk [0, 0x3035f): Mirror 1: 0xb82589f3 Mirror 4: 0xb82589f3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3035f: skipped 000500 1787119273.445499 write 0x031e5a thru 0x03ceb7 (0xb05e bytes) 000501 1787119273.615057 trunc from 0x03ceb8 to 0x0076fb (0x357bc bytes) 000502 1787119273.665835 mapread 0x0029a8 thru 0x0076fb (0x4d54 bytes) 000503 1787119273.698494 mapread 0x002b98 thru 0x0076fb (0x4b64 bytes) 000504 1787119273.699728 write 0x01ce69 thru 0x02caf2 (0xfc8a bytes) 000505 1787119273.708066 mapwrite 0x0132af thru 0x01c22b (0x8f7d bytes) 000506 1787119273.801036 mapwrite 0x03d949 thru 0x03ffff (0x26b7 bytes) 000507 1787119273.892177 mapwrite 0x010867 thru 0x017efa (0x7694 bytes) 000508 1787119273.971222 mapread 0x00555f thru 0x0061a6 (0xc48 bytes) 000509 1787119273.993065 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x9fa88f36 (stale) Mirror 4: 0x861c7510 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xb29740a Mirror 5: 0xb29740a lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000510 1787119275.378942 mapwrite 0x013189 thru 0x02234b (0xf1c3 bytes) 000511 1787119275.659344 mapread 0x034057 thru 0x03ffff (0xbfa9 bytes) 000512 1787119275.710215 trunc from 0x040000 to 0x03c28b (0x3d74 bytes) 000515 1787119275.773532 mapwrite 0x03fe58 thru 0x03ffff (0x1a8 bytes) 000516 1787119275.899405 trunc from 0x040000 to 0x03d66a (0x2995 bytes) 000517 1787119275.933951 mapread 0x0326a3 thru 0x03d66a (0xafc8 bytes) 000518 1787119275.942151 read 0x01fa33 thru 0x02cd36 (0xd304 bytes) 000521 1787119275.959069 mapwrite 0x03b972 thru 0x03ffff (0x468e bytes) 000522 1787119276.113055 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x9fa88f36 (stale) Mirror 5: 0xb29740a (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x9fa88f36 (stale) Mirror 5: 0xb29740a (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000523 1787119276.610748 mapread 0x038a1f thru 0x039465 (0xa47 bytes) 000525 1787119276.630600 mapread 0x008b9f thru 0x00dfb2 (0x5414 bytes) 000526 1787119276.653580 write 0x0008e8 thru 0x00c896 (0xbfaf bytes) 000529 1787119276.679147 mapread 0x03284e thru 0x03ffff (0xd7b2 bytes) 000530 1787119276.735844 write 0x028911 thru 0x0317f3 (0x8ee3 bytes) 000531 1787119276.758170 write 0x0386f1 thru 0x038b03 (0x413 bytes) 000532 1787119276.759848 trunc from 0x040000 to 0x03bd1b (0x42e4 bytes) 000533 1787119276.829343 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3bd1c): Mirror 5: 0x1b1ad0ce (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3bd1c): Mirror 5: 0x1b1ad0ce (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3bd1c: skipped 000534 1787119277.328097 mapread 0x0201a5 thru 0x0298c9 (0x9725 bytes) 000537 1787119277.381605 read 0x018ac5 thru 0x021c53 (0x918f bytes) 000539 1787119277.416887 mapread 0x0172f0 thru 0x024115 (0xce26 bytes) 000541 1787119277.439098 write 0x010112 thru 0x010318 (0x207 bytes) 000543 1787119277.446227 MIRROR_RESYNC 2 mirrors 000547 1787119278.500736 write 0x02749a thru 0x03269d (0xb204 bytes) 000548 1787119278.641697 mapread 0x034a11 thru 0x03bd1b (0x730b bytes) 000550 1787119278.679905 mapwrite 0x0241db thru 0x02a40e (0x6234 bytes) 000555 1787119278.769330 mapread 0x0272a7 thru 0x031284 (0x9fde bytes) 000556 1787119278.771153 write 0x015d12 thru 0x01d2b9 (0x75a8 bytes) 000557 1787119278.806583 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) 000558 1787119279.011871 mapwrite 0x030214 thru 0x033c4b (0x3a38 bytes) 000559 1787119279.132435 mapread 0x00555e thru 0x0136d4 (0xe177 bytes) 000561 1787119279.163682 MIRROR_EXTEND to 2 mirrors 000562 1787119280.062007 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) 000563 1787119280.325186 trunc from 0x03bd1c to 0x0134c5 (0x28856 bytes) 000564 1787119280.363073 trunc from 0x0134c6 to 0x000f63 (0x12562 bytes) 000565 1787119280.380737 mapread 0x000496 thru 0x000f63 (0xace bytes) 000568 1787119280.388226 write 0x037e15 thru 0x03bd9e (0x3f8a bytes) 000570 1787119280.390386 mapread 0x01d830 thru 0x020d2c (0x34fd bytes) 000572 1787119280.410882 trunc from 0x03bd9f to 0x0116b5 (0x2a6e9 bytes) 000573 1787119280.470493 mapwrite 0x027a55 thru 0x029cb2 (0x225e bytes) 000575 1787119280.553104 mapwrite 0x01a08b thru 0x020c48 (0x6bbe bytes) 000577 1787119280.644132 read 0x01dd63 thru 0x01de27 (0xc5 bytes) 000581 1787119280.648925 read 0x002b51 thru 0x0033a2 (0x852 bytes) 000583 1787119280.664990 write 0x03d3c5 thru 0x03ffff (0x2c3b bytes) 000584 1787119280.666672 mapread 0x03fdb6 thru 0x03ffff (0x24a bytes) 000586 1787119280.669719 read 0x0054cd thru 0x013aea (0xe61e bytes) 000587 1787119280.671585 mapwrite 0x00bb5c thru 0x01976e (0xdc13 bytes) 000588 1787119280.743230 read 0x0125fb thru 0x01b26f (0x8c75 bytes) 000589 1787119280.746174 mapread 0x024ea6 thru 0x02b404 (0x655f bytes) 000590 1787119280.784002 mapread 0x01521a thru 0x01e2b8 (0x909f bytes) 000592 1787119280.803424 MIRROR_EXTEND to 2 mirrors 000596 1787119281.957154 mapread 0x0280f8 thru 0x02d8a6 (0x57af bytes) 000601 1787119282.001767 read 0x00cd0e thru 0x0134b5 (0x67a8 bytes) 000604 1787119282.012593 write 0x02a9bd thru 0x03358e (0x8bd2 bytes) 000606 1787119282.140696 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) 000609 1787119282.329203 read 0x03648c thru 0x03ffff (0x9b74 bytes) 000610 1787119282.342755 mapread 0x00f4ab thru 0x019bc3 (0xa719 bytes) 000612 1787119282.365928 trunc from 0x040000 to 0x0126c3 (0x2d93c bytes) 000613 1787119282.391573 read 0x00a781 thru 0x00d46c (0x2cec bytes) 000614 1787119282.392386 read 0x00c77c thru 0x0126c3 (0x5f48 bytes) 000617 1787119282.402000 write 0x01152b thru 0x02137e (0xfe54 bytes) 000618 1787119282.408012 mapwrite 0x021fc7 thru 0x02eb56 (0xcb90 bytes) 000619 1787119282.503848 read 0x02b7f6 thru 0x02eb56 (0x3361 bytes) 000621 1787119282.506163 trunc from 0x02eb57 to 0x03c8ff (0xdda9 bytes) 000622 1787119282.527085 write 0x03b893 thru 0x03ffff (0x476d bytes) 000623 1787119282.536441 mapread 0x039fba thru 0x03dc78 (0x3cbf bytes) 000624 1787119282.537834 mapwrite 0x020a59 thru 0x028421 (0x79c9 bytes) 000627 1787119282.630654 mapread 0x01025c thru 0x013973 (0x3718 bytes) 000633 1787119282.641871 read 0x01c2c2 thru 0x01f645 (0x3384 bytes) 000636 1787119282.649050 read 0x007fa8 thru 0x00b5c3 (0x361c bytes) 000637 1787119282.653034 trunc from 0x040000 to 0x016959 (0x296a6 bytes) 000641 1787119282.687886 write 0x01faaa thru 0x02609e (0x65f5 bytes) 000646 1787119282.693673 trunc from 0x02609f to 0x000a14 (0x2568a bytes) 000648 1787119282.719664 mapwrite 0x0337e8 thru 0x03cb7e (0x9397 bytes) 000649 1787119282.806161 mapwrite 0x020924 thru 0x02d1cd (0xc8aa bytes) 000650 1787119282.906897 mapwrite 0x0298c2 thru 0x03578c (0xbecb bytes) 000651 1787119283.019202 trunc from 0x03cb7f to 0x029919 (0x13265 bytes) 000652 1787119283.075381 trunc from 0x02991a to 0x01f921 (0x9ff8 bytes) 000653 1787119283.131289 read 0x015f58 thru 0x01dc8c (0x7d35 bytes) 000655 1787119283.141118 read 0x012f73 thru 0x01308f (0x11d bytes) 000657 1787119283.145862 trunc from 0x01f922 to 0x02d4d3 (0xdbb2 bytes) 000661 1787119283.186536 write 0x01fa00 thru 0x023b28 (0x4129 bytes) 000663 1787119283.200188 write 0x0138a6 thru 0x015f04 (0x265f bytes) 000665 1787119283.204922 trunc from 0x02d4d4 to 0x004503 (0x28fd0 bytes) 000666 1787119283.235938 mapwrite 0x02a263 thru 0x03203a (0x7dd8 bytes) 000668 1787119283.392713 write 0x017746 thru 0x01c2bc (0x4b77 bytes) 000669 1787119283.421938 MIRROR_EXTEND to 2 mirrors 000673 1787119284.884163 trunc from 0x03203b to 0x01b302 (0x16d38 bytes) 000676 1787119284.989155 write 0x02c96f thru 0x034a82 (0x8114 bytes) 000680 1787119284.995158 mapwrite 0x021224 thru 0x024ab6 (0x3893 bytes) 000681 1787119285.110199 mapwrite 0x00f43a thru 0x01e5e8 (0xf1af bytes) 000683 1787119285.294554 mapwrite 0x000e2b thru 0x00ef6d (0xe143 bytes) 000685 1787119285.457764 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x34a83): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x34a83): Mirror 2: 0xa2fc53b5 (stale) CRC-32 checksum value for chunk [0, 0x34a83): Mirror 1: 0xd097ca97 Mirror 3: 0xd097ca97 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x34a83: skipped 000686 1787119286.719083 mapread 0x032228 thru 0x034a82 (0x285b bytes) 000687 1787119286.769709 MIRROR_RESYNC 3 mirrors 000688 1787119287.430619 write 0x00a279 thru 0x01702e (0xcdb6 bytes) 000689 1787119287.547538 read 0x01c92e thru 0x0279d5 (0xb0a8 bytes) 000692 1787119287.563430 mapread 0x011cb2 thru 0x01c64f (0xa99e bytes) 000693 1787119287.602133 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x34a83): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x34a83): Mirror 2: 0xd097ca97 (stale) Mirror 3: 0xd097ca97 (stale) CRC-32 checksum value for chunk [0, 0x34a83): Mirror 1: 0x55fa2ddc Mirror 4: 0x55fa2ddc lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x34a83: skipped 000694 1787119288.589532 trunc from 0x034a83 to 0x01f445 (0x1563d bytes) 000697 1787119288.799811 MIRROR_RESYNC 4 mirrors 000698 1787119290.318869 mapread 0x00523b thru 0x01151f (0xc2e5 bytes) 000701 1787119290.366806 mapread 0x00989e thru 0x017422 (0xdb85 bytes) 000702 1787119290.372504 MIRROR_RESYNC 4 mirrors 000704 1787119290.726096 trunc from 0x01f446 to 0x013486 (0xbfbf bytes) 000706 1787119290.805527 mapwrite 0x0034c9 thru 0x00b721 (0x8259 bytes) 000708 1787119290.907231 mapwrite 0x0117b5 thru 0x01bf99 (0xa7e5 bytes) 000710 1787119291.008701 write 0x03d3a9 thru 0x03ffff (0x2c57 bytes) 000711 1787119291.009944 trunc from 0x040000 to 0x013392 (0x2cc6d bytes) 000712 1787119291.028870 read 0x01141c thru 0x013392 (0x1f77 bytes) 000713 1787119291.030532 read 0x01259f thru 0x013392 (0xdf4 bytes) 000714 1787119291.032195 mapwrite 0x01fc54 thru 0x02a12a (0xa4d7 bytes) 000715 1787119291.151100 read 0x017edd thru 0x01ee2f (0x6f53 bytes) 000717 1787119291.152999 trunc from 0x02a12b to 0x017505 (0x12c25 bytes) 000718 1787119291.183522 trunc from 0x017506 to 0x000ddb (0x1672a bytes) 000720 1787119291.206657 write 0x01eb99 thru 0x02e2db (0xf743 bytes) 000721 1787119291.209438 mapwrite 0x008b20 thru 0x016fef (0xe4d0 bytes) 000722 1787119291.412743 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2e2dc): Mirror 2: 0xc6ef18b4 (stale) Mirror 4: 0xc6ef18b4 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2e2dc): Mirror 2: 0xc6ef18b4 (stale) Mirror 4: 0xc6ef18b4 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2e2dc: skipped 000724 1787119291.852090 read 0x02358e thru 0x028297 (0x4d0a bytes) 000726 1787119291.866751 write 0x01348e thru 0x0155df (0x2152 bytes) 000728 1787119291.886897 read 0x018246 thru 0x01c122 (0x3edd bytes) 000730 1787119291.904874 mapread 0x008e1e thru 0x015b7c (0xcd5f bytes) 000731 1787119291.906539 mapread 0x02b84d thru 0x02bd64 (0x518 bytes) 000734 1787119291.910543 write 0x01cbc3 thru 0x02a74a (0xdb88 bytes) 000736 1787119291.913900 mapwrite 0x035795 thru 0x039885 (0x40f1 bytes) 000739 1787119292.031085 read 0x01fbaa thru 0x024b40 (0x4f97 bytes) 000740 1787119292.031922 read 0x021c75 thru 0x02329c (0x1628 bytes) 000742 1787119292.034605 write 0x001184 thru 0x0109c1 (0xf83e bytes) 000744 1787119292.037186 trunc from 0x039886 to 0x004cfb (0x34b8a bytes) 000747 1787119292.065043 read 0x0001a4 thru 0x001add (0x193a bytes) 000751 1787119292.070631 mapread 0x002085 thru 0x0033e6 (0x1362 bytes) 000752 1787119292.078610 trunc from 0x004cfc to 0x034baa (0x2feaf bytes) 000753 1787119292.095867 trunc from 0x034bab to 0x014e02 (0x1fda8 bytes) 000755 1787119292.120068 trunc from 0x014e03 to 0x0339f8 (0x1ebf6 bytes) 000757 1787119292.172902 trunc from 0x0339f9 to 0x005b17 (0x2dee1 bytes) 000758 1787119292.221543 mapread 0x00444d thru 0x005b17 (0x16cb bytes) 000759 1787119292.232779 mapread 0x0030d8 thru 0x005b17 (0x2a40 bytes) 000762 1787119292.234894 write 0x007160 thru 0x00c6a6 (0x5547 bytes) 000763 1787119292.258468 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0xc6a7): Mirror 4: 0xadd7c493 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xc6a7): Mirror 4: 0xadd7c493 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xc6a7: skipped 000764 1787119292.572991 mapwrite 0x028ace thru 0x0340d1 (0xb604 bytes) 000765 1787119292.689661 mapread 0x029139 thru 0x0317ac (0x8674 bytes) 000766 1787119292.690871 trunc from 0x0340d2 to 0x02e774 (0x595d bytes) 000770 1787119292.717898 mapwrite 0x010b5e thru 0x01de02 (0xd2a5 bytes) 000771 1787119292.838956 write 0x01f329 thru 0x0217af (0x2487 bytes) 000773 1787119292.841713 mapread 0x020fd5 thru 0x021bfb (0xc27 bytes) 000780 1787119292.855634 mapwrite 0x00c0cb thru 0x0152d8 (0x920e bytes) 000782 1787119292.933868 MIRROR_RESYNC 2 mirrors 000783 1787119293.886705 MIRROR_EXTEND to 3 mirrors 000784 1787119295.076130 trunc from 0x02e775 to 0x0100c4 (0x1e6b0 bytes) 000785 1787119295.183783 read 0x002c06 thru 0x0100c4 (0xd4bf bytes) 000786 1787119295.195351 trunc from 0x0100c5 to 0x000db6 (0xf30e bytes) 000787 1787119295.217038 read 0x0000ab thru 0x000db6 (0xd0c bytes) 000788 1787119295.224488 write 0x03526f thru 0x036750 (0x14e2 bytes) 000789 1787119295.226201 mapread 0x036723 thru 0x036750 (0x2e bytes) 000790 1787119295.236028 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x36751): Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x36751): Mirror 4: 0xb627bef8 (stale) Mirror 5: 0xb627bef8 (stale) CRC-32 checksum value for chunk [0, 0x36751): Mirror 1: 0xcd4fb79e Mirror 6: 0xcd4fb79e lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x36751: skipped 000793 1787119297.122899 mapread 0x031309 thru 0x036750 (0x5448 bytes) 000794 1787119297.193085 trunc from 0x036751 to 0x00c9d1 (0x29d7f bytes) 000795 1787119297.292869 trunc from 0x00c9d2 to 0x01d7ef (0x10e1e bytes) 000796 1787119297.319052 read 0x00cea8 thru 0x0160dc (0x9235 bytes) 000797 1787119297.331519 trunc from 0x01d7f0 to 0x021950 (0x4161 bytes) 000799 1787119297.359743 MIRROR_RESYNC 4 mirrors 000800 1787119299.120718 trunc from 0x021951 to 0x036092 (0x14742 bytes) 000801 1787119299.186203 mapread 0x0166de thru 0x01bd51 (0x5674 bytes) 000804 1787119299.217471 mapread 0x01d69e thru 0x02d4cc (0xfe2f bytes) 000806 1787119299.241490 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x36093): Mirror 5: 0x929ab53e (stale) Mirror 6: 0x929ab53e (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x36093): Mirror 5: 0x929ab53e (stale) Mirror 6: 0x929ab53e (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x36093: skipped 000808 1787119299.694348 read 0x02c08d thru 0x036092 (0xa006 bytes) 000809 1787119299.716988 trunc from 0x036093 to 0x0147ae (0x218e4 bytes) 000810 1787119299.741076 trunc from 0x0147af to 0x036c5e (0x224b0 bytes) 000812 1787119299.775726 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x36c5f): Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x36c5f): Mirror 5: 0xaf2e5942 (stale) Mirror 6: 0xaf2e5942 (stale) CRC-32 checksum value for chunk [0, 0x36c5f): Mirror 1: 0xaf2e5942 Mirror 7: 0xaf2e5942 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x36c5f: skipped 000813 1787119301.217430 trunc from 0x036c5f to 0x004da9 (0x31eb5 bytes) 000814 1787119301.413170 mapread 0x0016c0 thru 0x004da9 (0x36ea bytes) 000818 1787119301.463113 read 0x004400 thru 0x004da9 (0x9aa bytes) 000820 1787119301.466725 read 0x004b8f thru 0x004da9 (0x21b bytes) 000821 1787119301.481192 MIRROR_RESYNC 4 mirrors 000822 1787119302.769736 mapwrite 0x00324e thru 0x00a0e0 (0x6e93 bytes) 000824 1787119302.993078 read 0x0040f0 thru 0x0054fa (0x140b bytes) 000825 1787119302.996867 trunc from 0x00a0e1 to 0x03dea6 (0x33dc6 bytes) 000827 1787119303.047099 write 0x037a48 thru 0x03fbf3 (0x81ac bytes) 000828 1787119303.064078 MIRROR_SPLIT mirror 3 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3fbf4): Mirror 5: 0x374b1214 (stale) Mirror 6: 0x374b1214 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3fbf4): Mirror 5: 0x374b1214 (stale) Mirror 6: 0x374b1214 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3fbf4: skipped 000829 1787119303.501512 trunc from 0x03fbf4 to 0x0160ee (0x29b05 bytes) 000830 1787119303.543359 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x160ef): Mirror 5: (stale - read error) Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x160ef): Mirror 5: 0x97e1d825 (stale) Mirror 6: 0x97e1d825 (stale) CRC-32 checksum value for chunk [0, 0x160ef): Mirror 1: 0xb0bd14a1 Mirror 7: 0xb0bd14a1 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x160ef: skipped 000831 1787119304.781750 mapread 0x0057b1 thru 0x006949 (0x1199 bytes) 000832 1787119304.809000 mapwrite 0x01972c thru 0x0245d4 (0xaea9 bytes) 000833 1787119305.122886 write 0x014f53 thru 0x01e3f3 (0x94a1 bytes) 000835 1787119305.129340 read 0x01027a thru 0x01bfa0 (0xbd27 bytes) 000836 1787119305.149443 MIRROR_RESYNC 4 mirrors 000837 1787119306.488925 MIRROR_EXTEND to 5 mirrors 000839 1787119307.584840 mapwrite 0x003d74 thru 0x006b17 (0x2da4 bytes) 000843 1787119307.779662 mapread 0x0057f2 thru 0x009d09 (0x4518 bytes) 000849 1787119307.797600 write 0x02940d thru 0x0313fe (0x7ff2 bytes) 000852 1787119307.804563 read 0x0129fc thru 0x01581e (0x2e23 bytes) 000855 1787119307.825767 mapread 0x00913b thru 0x011ff4 (0x8eba bytes) 000857 1787119307.848715 mapread 0x019230 thru 0x022cb0 (0x9a81 bytes) 000858 1787119307.875010 write 0x035eab thru 0x03ffff (0xa155 bytes) 000859 1787119307.883139 trunc from 0x040000 to 0x01e42b (0x21bd4 bytes) 000861 1787119307.986469 mapread 0x00248a thru 0x003197 (0xd0e bytes) 000866 1787119308.007968 MIRROR_SPLIT mirror 4 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1e42c): Mirror 5: 0xd6c29bb8 (stale) Mirror 6: 0xd6c29bb8 (stale) Mirror 7: 0xd6c29bb8 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1e42c): Mirror 5: 0xd6c29bb8 (stale) Mirror 6: 0xd6c29bb8 (stale) Mirror 7: 0xd6c29bb8 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1e42c: skipped 000868 1787119308.437258 mapwrite 0x02e368 thru 0x03c1e3 (0xde7c bytes) 000869 1787119308.537887 mapread 0x012930 thru 0x022523 (0xfbf4 bytes) 000870 1787119308.563558 write 0x000de6 thru 0x00a9e0 (0x9bfb bytes) 000871 1787119308.576663 read 0x02b1ef thru 0x02de03 (0x2c15 bytes) 000872 1787119308.581568 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x3c1e4): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3c1e4): Mirror 5: 0xc10813b9 (stale) Mirror 6: 0xc10813b9 (stale) Mirror 7: 0xc10813b9 (stale) CRC-32 checksum value for chunk [0, 0x3c1e4): Mirror 1: 0x1f83b48f Mirror 8: 0x1f83b48f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3c1e4: skipped 000874 1787119309.863967 write 0x025b17 thru 0x0334ec (0xd9d6 bytes) 000876 1787119309.989796 trunc from 0x03c1e4 to 0x01dd85 (0x1e45e bytes) 000877 1787119310.020699 mapwrite 0x030a22 thru 0x035cb5 (0x5294 bytes) 000878 1787119310.100564 mapwrite 0x009347 thru 0x0157c5 (0xc47f bytes) 000879 1787119310.196602 write 0x00a11d thru 0x015fb8 (0xbe9c bytes) 000882 1787119310.207028 mapread 0x029163 thru 0x02aca0 (0x1b3e bytes) 000883 1787119310.224657 mapwrite 0x003333 thru 0x00efb4 (0xbc82 bytes) 000884 1787119310.296990 trunc from 0x035cb6 to 0x03839b (0x26e6 bytes) 000886 1787119310.335062 trunc from 0x03839c to 0x02897f (0xfa1c bytes) 000887 1787119310.367434 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x28980): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x28980): Mirror 5: 0xc58d0bf5 (stale) Mirror 6: 0xc58d0bf5 (stale) Mirror 7: 0xc58d0bf5 (stale) Mirror 8: 0x5b6ba860 (stale) CRC-32 checksum value for chunk [0, 0x28980): Mirror 1: 0xca3bf4c6 Mirror 9: 0xca3bf4c6 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x28980: skipped 000891 1787119311.782808 mapread 0x01d7b0 thru 0x026cd8 (0x9529 bytes) 000893 1787119311.838928 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x28980): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x28980): Mirror 5: 0xc58d0bf5 (stale) Mirror 6: 0xc58d0bf5 (stale) Mirror 7: 0xc58d0bf5 (stale) Mirror 8: 0x5b6ba860 (stale) CRC-32 checksum value for chunk [0, 0x28980): Mirror 1: 0xca3bf4c6 Mirror 9: 0xca3bf4c6 Mirror 10: 0xca3bf4c6 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x28980: skipped 000894 1787119313.042967 write 0x03a377 thru 0x03bdd5 (0x1a5f bytes) 000895 1787119313.103015 MIRROR_EXTEND to 8 mirrors CRC-32 checksum value for chunk [0, 0x3bdd6): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3bdd6): Mirror 5: 0x9130fbe6 (stale) Mirror 6: 0x9130fbe6 (stale) Mirror 7: 0x9130fbe6 (stale) Mirror 8: 0xa5712b3c (stale) Mirror 9: 0xfa4936fc (stale) Mirror 10: 0xfa4936fc (stale) CRC-32 checksum value for chunk [0, 0x3bdd6): Mirror 1: 0x469887f5 Mirror 11: 0x469887f5 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3bdd6: skipped 000896 1787119314.427448 MIRROR_RESYNC 8 mirrors 000898 1787119316.180661 mapread 0x0093d6 thru 0x0191c7 (0xfdf2 bytes) 000902 1787119316.210321 trunc from 0x03bdd6 to 0x0255f6 (0x167df bytes) 000904 1787119316.309940 mapread 0x011284 thru 0x01bcf4 (0xaa71 bytes) 000908 1787119316.339463 MIRROR_RESYNC 8 mirrors 000909 1787119318.268688 read 0x021084 thru 0x0255f6 (0x4573 bytes) 000911 1787119318.281022 mapwrite 0x03319e thru 0x03983e (0x66a1 bytes) 000916 1787119318.463662 write 0x025338 thru 0x033855 (0xe51e bytes) 000918 1787119318.473559 read 0x028ebb thru 0x029925 (0xa6b bytes) 000919 1787119318.474978 trunc from 0x03983f to 0x03e8ba (0x507c bytes) truncating to largest ever: 0x3fcf7 000923 1787119318.506473 trunc from 0x03e8bb to 0x03fcf6 (0x143c bytes) 000925 1787119318.522052 mapwrite 0x03afe9 thru 0x03bc64 (0xc7c bytes) 000926 1787119318.596056 read 0x02e133 thru 0x02fe51 (0x1d1f bytes) 000927 1787119318.596839 mapread 0x038cfb thru 0x03fcf6 (0x6ffc bytes) 000929 1787119318.601056 write 0x01836b thru 0x01a1ce (0x1e64 bytes) 000932 1787119318.604974 mapread 0x00d03a thru 0x017fb9 (0xaf80 bytes) 000936 1787119318.611857 trunc from 0x03fcf7 to 0x016f3f (0x28db7 bytes) 000938 1787119318.659446 MIRROR_SPLIT mirror 4 to 7 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x180:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x16f40): Mirror 5: 0xe32adfc (stale) Mirror 6: 0xe32adfc (stale) Mirror 7: 0xe32adfc (stale) Mirror 9: 0xe32adfc (stale) Mirror 10: 0xe32adfc (stale) Mirror 11: 0xe32adfc (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x16f40): Mirror 5: 0xe32adfc (stale) Mirror 6: 0xe32adfc (stale) Mirror 7: 0xe32adfc (stale) Mirror 9: 0xe32adfc (stale) Mirror 10: 0xe32adfc (stale) Mirror 11: 0xe32adfc (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x16f40: skipped 000940 1787119319.310495 mapread 0x015232 thru 0x016f3f (0x1d0e bytes) 000941 1787119319.323587 trunc from 0x016f40 to 0x000360 (0x16bdf bytes) 000942 1787119319.345851 mapread 0x00014b thru 0x000360 (0x216 bytes) 000943 1787119319.356609 trunc from 0x000361 to 0x00baf4 (0xb794 bytes) 000946 1787119319.373523 write 0x02bd85 thru 0x02d6d3 (0x194f bytes) 000947 1787119319.374835 mapwrite 0x018c6f thru 0x02809c (0xf42e bytes) 000949 1787119319.444462 read 0x013788 thru 0x0148d8 (0x1151 bytes) 000952 1787119319.451444 MIRROR_RESYNC 7 mirrors 000955 1787119321.568403 mapwrite 0x005af7 thru 0x00bd53 (0x625d bytes) 000956 1787119321.831948 mapread 0x0082a0 thru 0x017fd4 (0xfd35 bytes) 000958 1787119321.850220 mapread 0x01bc07 thru 0x020f94 (0x538e bytes) 000959 1787119321.895871 mapwrite 0x033e3a thru 0x0340ad (0x274 bytes) 000960 1787119322.000428 read 0x01f2c8 thru 0x02062e (0x1367 bytes) 000961 1787119322.001146 mapwrite 0x03c62f thru 0x03f9b1 (0x3383 bytes) 000966 1787119322.083952 mapread 0x02c8ef thru 0x039873 (0xcf85 bytes) 000967 1787119322.108462 MIRROR_RESYNC 7 mirrors 000968 1787119324.116732 read 0x01f550 thru 0x02a360 (0xae11 bytes) 000969 1787119324.142006 mapwrite 0x0137bd thru 0x015e45 (0x2689 bytes) 000971 1787119324.468509 mapwrite 0x03344d thru 0x03d49f (0xa053 bytes) 000972 1787119324.589507 write 0x03cf6b thru 0x03ffff (0x3095 bytes) 000975 1787119324.601743 mapwrite 0x01caa9 thru 0x025cab (0x9203 bytes) 000977 1787119324.770463 write 0x01395d thru 0x01a576 (0x6c1a bytes) 000980 1787119324.789399 MIRROR_EXTEND to 8 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 5: 0x6eb4e6a1 (stale) Mirror 6: 0x6eb4e6a1 (stale) Mirror 7: 0x6eb4e6a1 (stale) Mirror 9: 0x6eb4e6a1 (stale) Mirror 10: 0x6eb4e6a1 (stale) Mirror 11: 0x6eb4e6a1 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xa97a0d48 Mirror 12: 0xa97a0d48 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000981 1787119326.182898 mapread 0x011eac thru 0x019566 (0x76bb bytes) 000986 1787119326.321783 write 0x01d0d1 thru 0x024d0c (0x7c3c bytes) 000987 1787119326.507405 read 0x00fe9b thru 0x01f7cf (0xf935 bytes) 000988 1787119326.532438 mapwrite 0x03dc09 thru 0x03f105 (0x14fd bytes) 000989 1787119326.622535 trunc from 0x040000 to 0x0348a9 (0xb756 bytes) 000990 1787119326.654267 trunc from 0x0348aa to 0x01d193 (0x17716 bytes) 000991 1787119326.703924 MIRROR_EXTEND to 9 mirrors CRC-32 checksum value for chunk [0, 0x1d194): Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1d194): Mirror 5: 0x9bcce6af (stale) Mirror 6: 0x9bcce6af (stale) Mirror 7: 0x9bcce6af (stale) Mirror 9: 0x9bcce6af (stale) Mirror 10: 0x9bcce6af (stale) Mirror 11: 0x9bcce6af (stale) Mirror 12: 0x18bb07b3 (stale) CRC-32 checksum value for chunk [0, 0x1d194): Mirror 1: 0x94dc29ee Mirror 13: 0x94dc29ee lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1d194: skipped 000992 1787119328.009687 write 0x01d536 thru 0x0205eb (0x30b6 bytes) 000994 1787119328.184846 read 0x01995b thru 0x01d8fd (0x3fa3 bytes) 000996 1787119328.194632 mapwrite 0x007ab6 thru 0x01096b (0x8eb6 bytes) 000998 1787119328.417703 MIRROR_RESYNC 9 mirrors 000999 1787119330.691491 mapwrite 0x02122c thru 0x02fc8f (0xea64 bytes) All operations completed A-OK!