== sanity-flr test 70a: flr mode fsx test ================ 21:44:17 (1787708657) Seed set to 8659 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x23c73 000003 1787708659.663852 trunc from 00000000 to 0x023c72 (0x23c73 bytes) 000004 1787708659.695445 write 0x032106 thru 0x03ffff (0xdefa bytes) 000005 1787708659.703710 mapread 0x035c76 thru 0x03f058 (0x93e3 bytes) 000006 1787708659.708100 write 0x015a00 thru 0x0235a0 (0xdba1 bytes) truncating to largest ever: 0x3a922 000007 1787708659.733050 trunc from 0x040000 to 0x03a921 (0x56de bytes) 000008 1787708659.747897 mapread 0x0195e8 thru 0x021318 (0x7d31 bytes) 000009 1787708659.750411 write 0x0039e6 thru 0x007753 (0x3d6e bytes) 000010 1787708659.805131 trunc from 0x03a922 to 0x01fdea (0x1ab37 bytes) 000011 1787708659.838010 mapwrite 0x033876 thru 0x039a3c (0x61c7 bytes) 000012 1787708660.009477 MIRROR_EXTEND to 2 mirrors 000016 1787708660.909604 mapwrite 0x03c44f thru 0x03ffff (0x3bb1 bytes) 000017 1787708661.116716 read 0x034677 thru 0x036a81 (0x240b bytes) 000018 1787708661.124823 mapread 0x02bbfd thru 0x02c5f4 (0x9f8 bytes) 000022 1787708661.141595 write 0x02921f thru 0x0370a5 (0xde87 bytes) 000023 1787708661.143833 mapread 0x03f915 thru 0x03ff12 (0x5fe bytes) truncating to largest ever: 0x3fcf0 000025 1787708661.146636 trunc from 0x040000 to 0x03fcef (0x310 bytes) 000026 1787708661.179730 read 0x0020d0 thru 0x011a29 (0xf95a bytes) 000027 1787708661.191802 mapread 0x00800e thru 0x00cd2b (0x4d1e bytes) 000028 1787708661.193470 read 0x027f48 thru 0x028848 (0x901 bytes) 000029 1787708661.194381 mapwrite 0x037707 thru 0x03ffff (0x88f9 bytes) 000030 1787708661.297893 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xe7a18183 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x8e045f03 Mirror 3: 0x8e045f03 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000031 1787708662.529510 read 0x0395ac thru 0x03e0f3 (0x4b48 bytes) 000032 1787708662.542500 write 0x010212 thru 0x014e6c (0x4c5b bytes) 000034 1787708662.649399 write 0x0334a0 thru 0x03c836 (0x9397 bytes) 000035 1787708662.672187 write 0x031fe7 thru 0x03cd10 (0xad2a bytes) 000036 1787708662.682148 mapwrite 0x0199e5 thru 0x0289a8 (0xefc4 bytes) 000037 1787708662.794778 mapwrite 0x02e91f thru 0x03d697 (0xed79 bytes) 000038 1787708662.874068 trunc from 0x040000 to 0x00e82b (0x317d4 bytes) 000039 1787708662.937698 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0xe82c): Mirror 3: 0x3a034ad0 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xe82c): Mirror 3: 0x3a034ad0 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xe82c: skipped 000044 1787708663.310602 trunc from 0x00e82c to 0x012d4c (0x4521 bytes) 000047 1787708663.339582 mapread 0x0075e1 thru 0x012d4c (0xb76c bytes) 000048 1787708663.383968 mapwrite 0x0271ec thru 0x032ab6 (0xb8cb bytes) 000050 1787708663.503626 mapwrite 0x03200a thru 0x037502 (0x54f9 bytes) 000052 1787708663.575070 trunc from 0x037503 to 0x0255bb (0x11f47 bytes) 000053 1787708663.594948 read 0x02330f thru 0x0250e0 (0x1dd2 bytes) 000055 1787708663.597254 mapwrite 0x00c7e6 thru 0x00ee4e (0x2669 bytes) 000056 1787708663.716917 mapwrite 0x0323ae thru 0x03d429 (0xb07c bytes) 000057 1787708663.886629 write 0x01b2e4 thru 0x020673 (0x5390 bytes) 000058 1787708663.908960 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) 000060 1787708664.134688 write 0x014bf3 thru 0x0249fd (0xfe0b bytes) 000061 1787708664.151121 write 0x009fa2 thru 0x00e325 (0x4384 bytes) 000062 1787708664.168145 mapread 0x01cc02 thru 0x02512e (0x852d bytes) 000068 1787708664.187241 mapread 0x029fce thru 0x0383e1 (0xe414 bytes) 000069 1787708664.241725 MIRROR_EXTEND to 2 mirrors 000070 1787708665.237716 read 0x016467 thru 0x0195b2 (0x314c bytes) 000072 1787708665.264056 mapread 0x02eb0c thru 0x03c6ef (0xdbe4 bytes) 000076 1787708665.349468 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) 000077 1787708665.520611 read 0x01a323 thru 0x01ef3f (0x4c1d bytes) 000078 1787708665.534555 mapread 0x0114d1 thru 0x01d369 (0xbe99 bytes) 000079 1787708665.559705 write 0x011c36 thru 0x01a632 (0x89fd bytes) 000082 1787708665.591899 trunc from 0x03d42a to 0x006098 (0x37391 bytes) 000084 1787708665.634533 mapread 0x003521 thru 0x006098 (0x2b78 bytes) 000088 1787708665.649727 write 0x03e3a2 thru 0x03ffff (0x1c5e bytes) 000090 1787708665.652504 mapread 0x03ba76 thru 0x03d645 (0x1bd0 bytes) 000091 1787708665.661998 trunc from 0x040000 to 0x01746a (0x28b95 bytes) 000092 1787708665.702597 write 0x0113ce thru 0x01a495 (0x90c8 bytes) 000096 1787708665.722713 MIRROR_EXTEND to 2 mirrors 000100 1787708666.928153 mapwrite 0x014954 thru 0x023ca0 (0xf34d bytes) 000102 1787708667.200455 write 0x0091e1 thru 0x011a8b (0x88ab bytes) 000103 1787708667.207100 mapwrite 0x039667 thru 0x03ffff (0x6999 bytes) 000104 1787708667.340953 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) 000109 1787708667.505574 mapwrite 0x0310c4 thru 0x034b77 (0x3ab4 bytes) 000110 1787708667.611540 MIRROR_EXTEND to 2 mirrors 000111 1787708668.316545 read 0x0041b2 thru 0x00d15a (0x8fa9 bytes) 000112 1787708668.340417 write 0x00a23d thru 0x00e64e (0x4412 bytes) 000114 1787708668.418370 MIRROR_RESYNC 2 mirrors 000115 1787708669.460953 trunc from 0x040000 to 0x00ab1d (0x354e2 bytes) 000121 1787708669.729305 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) 000122 1787708669.927884 mapwrite 0x03c7f2 thru 0x03ffff (0x380e bytes) 000124 1787708670.040185 write 0x01d2a9 thru 0x02450b (0x7263 bytes) 000128 1787708670.056482 read 0x0365d3 thru 0x0372cc (0xcfa bytes) 000131 1787708670.067576 trunc from 0x040000 to 0x024c03 (0x1b3fc bytes) 000134 1787708670.093179 write 0x019889 thru 0x01f12b (0x58a3 bytes) 000137 1787708670.100486 mapread 0x015d1e thru 0x019de9 (0x40cc bytes) 000138 1787708670.113336 mapwrite 0x01d918 thru 0x02d07c (0xf765 bytes) 000140 1787708670.339296 MIRROR_EXTEND to 2 mirrors 000141 1787708671.524113 write 0x025b44 thru 0x03340b (0xd8c8 bytes) 000146 1787708671.615949 mapwrite 0x029a81 thru 0x02c61e (0x2b9e bytes) 000147 1787708671.672794 read 0x006376 thru 0x00ead0 (0x875b bytes) 000148 1787708671.690480 mapwrite 0x01a04c thru 0x027184 (0xd139 bytes) 000150 1787708671.787949 mapwrite 0x03da1c thru 0x03ffff (0x25e4 bytes) 000151 1787708671.928119 MIRROR_RESYNC 2 mirrors 000152 1787708673.132042 trunc from 0x040000 to 0x035777 (0xa888 bytes) 000153 1787708673.242694 write 0x0021b6 thru 0x00d3fa (0xb245 bytes) 000154 1787708673.262457 write 0x02ef20 thru 0x0332a8 (0x4389 bytes) 000156 1787708673.284446 mapread 0x015146 thru 0x023b5e (0xea19 bytes) 000158 1787708673.343235 mapread 0x0146bd thru 0x02093b (0xc27f bytes) 000165 1787708673.358603 write 0x000474 thru 0x002126 (0x1cb3 bytes) 000166 1787708673.367530 trunc from 0x035778 to 0x03775b (0x1fe4 bytes) 000170 1787708673.416101 mapread 0x00b154 thru 0x01852a (0xd3d7 bytes) 000172 1787708673.429864 read 0x029eb8 thru 0x03775b (0xd8a4 bytes) 000176 1787708673.460735 write 0x0215a8 thru 0x0282ad (0x6d06 bytes) 000180 1787708673.468492 mapwrite 0x012a1d thru 0x01a070 (0x7654 bytes) 000181 1787708673.564371 write 0x010865 thru 0x01b394 (0xab30 bytes) 000182 1787708673.567563 trunc from 0x03775c to 0x01af74 (0x1c7e7 bytes) 000183 1787708673.594770 trunc from 0x01af75 to 0x01c1c3 (0x124f bytes) 000185 1787708673.621861 mapwrite 0x003aa7 thru 0x00b57d (0x7ad7 bytes) 000186 1787708673.707426 mapread 0x0194e6 thru 0x01c1c3 (0x2cde bytes) 000187 1787708673.713320 trunc from 0x01c1c4 to 0x0239c9 (0x7806 bytes) 000188 1787708673.728416 trunc from 0x0239ca to 0x02c273 (0x88aa bytes) 000189 1787708673.743885 mapread 0x011fef thru 0x015202 (0x3214 bytes) 000190 1787708673.753375 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) 000192 1787708673.978587 write 0x0023fc thru 0x007f82 (0x5b87 bytes) 000193 1787708673.996287 read 0x020d79 thru 0x02b452 (0xa6da bytes) 000197 1787708674.012265 read 0x01887a thru 0x01a807 (0x1f8e bytes) 000198 1787708674.039023 read 0x0067c1 thru 0x010d1a (0xa55a bytes) 000199 1787708674.039898 mapread 0x002e61 thru 0x00a57e (0x771e bytes) 000202 1787708674.044224 read 0x0072b6 thru 0x011de6 (0xab31 bytes) 000203 1787708674.045069 trunc from 0x02c274 to 0x036221 (0x9fae bytes) 000205 1787708674.070547 read 0x022ad0 thru 0x023f1c (0x144d bytes) 000208 1787708674.078260 MIRROR_EXTEND to 2 mirrors 000209 1787708675.439293 read 0x00d433 thru 0x014206 (0x6dd4 bytes) 000210 1787708675.465229 trunc from 0x036222 to 0x00cf31 (0x292f0 bytes) 000211 1787708675.583576 write 0x03630d thru 0x03ffff (0x9cf3 bytes) 000213 1787708675.588354 mapread 0x03cbe7 thru 0x03e644 (0x1a5e bytes) 000215 1787708675.591540 read 0x002235 thru 0x00e1a2 (0xbf6e bytes) 000218 1787708675.608952 read 0x0305bd thru 0x03f50c (0xef50 bytes) 000219 1787708675.615125 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xcbf27e91 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xbf0bebca Mirror 3: 0xbf0bebca lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000221 1787708677.033999 MIRROR_RESYNC 3 mirrors 000223 1787708678.099678 read 0x03309e thru 0x0351f4 (0x2157 bytes) 000224 1787708678.109389 mapread 0x01e33a thru 0x0299eb (0xb6b2 bytes) 000226 1787708678.162928 mapwrite 0x014686 thru 0x01d094 (0x8a0f bytes) 000228 1787708678.478040 write 0x01aa34 thru 0x0216cf (0x6c9c bytes) 000229 1787708678.479260 mapwrite 0x01949c thru 0x01a89d (0x1402 bytes) 000230 1787708678.571008 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xbf0bebca (stale) Mirror 3: 0xbf0bebca (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x3725cc5a Mirror 4: 0x3725cc5a lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000231 1787708679.781274 MIRROR_RESYNC 4 mirrors 000232 1787708680.677496 mapread 0x03ece2 thru 0x03ffff (0x131e bytes) 000233 1787708680.691122 mapwrite 0x019a49 thru 0x01b897 (0x1e4f bytes) 000234 1787708680.862990 mapread 0x011223 thru 0x015782 (0x4560 bytes) 000235 1787708680.872512 mapread 0x035b69 thru 0x03e1e3 (0x867b bytes) 000236 1787708680.889577 trunc from 0x040000 to 0x02f64d (0x109b2 bytes) 000237 1787708680.923610 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2f64e): Mirror 2: 0xcafab0a7 (stale) Mirror 4: 0xcafab0a7 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2f64e): Mirror 2: 0xcafab0a7 (stale) Mirror 4: 0xcafab0a7 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2f64e: skipped 000238 1787708681.317384 write 0x02a712 thru 0x037b0b (0xd3fa bytes) 000240 1787708681.329180 mapread 0x005e11 thru 0x009b07 (0x3cf7 bytes) 000241 1787708681.357892 read 0x00470a thru 0x00d88b (0x9182 bytes) 000242 1787708681.359690 trunc from 0x037b0c to 0x00be7b (0x2bc90 bytes) 000243 1787708681.396097 trunc from 0x00be7c to 0x00cb71 (0xcf6 bytes) 000245 1787708681.425619 read 0x000dd6 thru 0x005f1d (0x5148 bytes) 000246 1787708681.437476 mapwrite 0x020780 thru 0x02b856 (0xb0d7 bytes) 000247 1787708681.602162 read 0x01e5fb thru 0x029f46 (0xb94c bytes) 000249 1787708681.621987 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x2b857): Mirror 2: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2b857): Mirror 2: 0xd055c366 (stale) Mirror 4: 0xd055c366 (stale) CRC-32 checksum value for chunk [0, 0x2b857): Mirror 1: 0x4950dc3 Mirror 5: 0x4950dc3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2b857: skipped 000252 1787708683.179816 read 0x0219a0 thru 0x024b7e (0x31df bytes) 000254 1787708683.207502 MIRROR_RESYNC 4 mirrors 000255 1787708684.469051 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) 000257 1787708684.883724 write 0x017c09 thru 0x019ed1 (0x22c9 bytes) 000259 1787708684.981426 trunc from 0x02b857 to 0x02831d (0x3539 bytes) 000260 1787708685.011609 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2831e): Mirror 4: 0x82f2f462 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2831e): Mirror 4: 0x82f2f462 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2831e: skipped 000261 1787708685.318023 mapread 0x018760 thru 0x021a48 (0x92e9 bytes) 000266 1787708685.354418 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x2831e): Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2831e): Mirror 4: 0x82f2f462 (stale) CRC-32 checksum value for chunk [0, 0x2831e): Mirror 1: 0x46b33d1a Mirror 5: 0x46b33d1a lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2831e: skipped 000269 1787708686.208774 mapread 0x00ca53 thru 0x01b0f9 (0xe6a7 bytes) 000270 1787708686.246674 write 0x0280a0 thru 0x02adc9 (0x2d2a bytes) 000274 1787708686.339062 mapread 0x000764 thru 0x00876e (0x800b bytes) 000275 1787708686.350011 trunc from 0x02adca to 0x011889 (0x19540 bytes) 000277 1787708686.379164 trunc from 0x01188a to 0x023946 (0x120bd bytes) 000278 1787708686.406059 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x23947): Mirror 4: 0x15c74c9 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x23947): Mirror 4: 0x15c74c9 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x23947: skipped 000280 1787708686.690527 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) 000281 1787708686.842527 trunc from 0x023947 to 0x00a853 (0x190f3 bytes) 000282 1787708686.866150 read 0x003f4e thru 0x006599 (0x264c bytes) 000284 1787708686.876225 trunc from 0x00a854 to 0x00fce4 (0x5491 bytes) 000288 1787708686.896903 mapread 0x001674 thru 0x006f99 (0x5926 bytes) 000290 1787708686.911628 trunc from 0x00fce5 to 0x0147dc (0x4af8 bytes) 000293 1787708686.933646 trunc from 0x0147dd to 0x02ef4f (0x1a773 bytes) 000294 1787708686.950266 mapwrite 0x01cbb4 thru 0x027c25 (0xb072 bytes) 000296 1787708687.051711 mapread 0x01965f thru 0x028d5a (0xf6fc bytes) 000299 1787708687.055133 read 0x022da5 thru 0x026cd0 (0x3f2c bytes) 000300 1787708687.055732 trunc from 0x02ef50 to 0x03af1b (0xbfcc bytes) 000301 1787708687.084543 mapwrite 0x0239a4 thru 0x0295ec (0x5c49 bytes) 000303 1787708687.133315 mapwrite 0x01f3ca thru 0x02b5b6 (0xc1ed bytes) 000304 1787708687.197833 trunc from 0x03af1c to 0x00668f (0x3488c bytes) 000306 1787708687.219297 write 0x0073d3 thru 0x00acaa (0x38d8 bytes) 000307 1787708687.220393 mapwrite 0x02d6cc thru 0x03335a (0x5c8f bytes) 000308 1787708687.318366 mapwrite 0x03e2b4 thru 0x03ffff (0x1d4c bytes) 000312 1787708687.391226 read 0x02d34d thru 0x036b70 (0x9824 bytes) 000313 1787708687.393151 write 0x01a08a thru 0x01dca4 (0x3c1b bytes) 000314 1787708687.394305 read 0x027dba thru 0x02c21b (0x4462 bytes) 000319 1787708687.398375 mapread 0x038b76 thru 0x039dd4 (0x125f bytes) 000320 1787708687.399363 trunc from 0x040000 to 0x03dc64 (0x239b bytes) 000321 1787708687.424981 mapread 0x0381e4 thru 0x03dc64 (0x5a81 bytes) 000322 1787708687.426145 mapread 0x01a138 thru 0x023ead (0x9d76 bytes) 000323 1787708687.427908 mapread 0x017352 thru 0x025989 (0xe638 bytes) 000324 1787708687.434251 mapwrite 0x03cea7 thru 0x03ffff (0x3159 bytes) 000327 1787708687.531810 read 0x026fad thru 0x030a15 (0x9a69 bytes) 000328 1787708687.532499 read 0x002dd0 thru 0x009e9d (0x70ce bytes) 000331 1787708687.535966 read 0x03401d thru 0x03d3b5 (0x9399 bytes) 000333 1787708687.537163 write 0x005f54 thru 0x013c21 (0xdcce bytes) 000334 1787708687.539541 mapread 0x0341c0 thru 0x03c63d (0x847e bytes) 000337 1787708687.546285 write 0x011dad thru 0x01580e (0x3a62 bytes) 000341 1787708687.549876 read 0x009708 thru 0x00a1d9 (0xad2 bytes) 000343 1787708687.551665 mapwrite 0x018547 thru 0x01ee7f (0x6939 bytes) 000344 1787708687.593979 mapread 0x003935 thru 0x00d37d (0x9a49 bytes) 000345 1787708687.595166 write 0x0127bf thru 0x020ad1 (0xe313 bytes) 000346 1787708687.597370 write 0x0079c3 thru 0x015f85 (0xe5c3 bytes) 000348 1787708687.602224 mapwrite 0x019ac1 thru 0x022908 (0x8e48 bytes) 000349 1787708687.666811 mapread 0x014a5c thru 0x016a74 (0x2019 bytes) 000350 1787708687.667679 write 0x0154a4 thru 0x023b33 (0xe690 bytes) 000351 1787708687.670272 write 0x03609d thru 0x03bc6e (0x5bd2 bytes) 000354 1787708687.674050 trunc from 0x040000 to 0x03c1b0 (0x3e4f bytes) 000356 1787708687.692190 mapwrite 0x01854c thru 0x01de8b (0x5940 bytes) 000357 1787708687.757727 mapwrite 0x027f10 thru 0x028e4b (0xf3c bytes) 000358 1787708687.814571 read 0x031543 thru 0x03c1b0 (0xac6e bytes) 000360 1787708687.815663 read 0x000c28 thru 0x0098cc (0x8ca5 bytes) 000361 1787708687.816205 mapwrite 0x020ff3 thru 0x02de6e (0xce7c bytes) 000364 1787708687.871926 mapread 0x02007e thru 0x02f1a0 (0xf123 bytes) 000368 1787708687.876026 trunc from 0x03c1b1 to 0x0183d7 (0x23dd9 bytes) 000369 1787708687.900856 MIRROR_EXTEND to 2 mirrors 000371 1787708688.517732 mapread 0x0012f3 thru 0x006a4c (0x575a bytes) 000372 1787708688.553436 mapwrite 0x0139cc thru 0x01e962 (0xaf97 bytes) 000374 1787708688.730211 trunc from 0x01e963 to 0x03bfe7 (0x1d685 bytes) 000379 1787708688.763650 mapread 0x020e29 thru 0x029641 (0x8819 bytes) 000381 1787708688.785307 MIRROR_RESYNC 2 mirrors 000382 1787708689.751208 mapwrite 0x013a32 thru 0x022a8a (0xf059 bytes) 000383 1787708690.113844 MIRROR_RESYNC 2 mirrors 000384 1787708691.053299 read 0x032fa8 thru 0x03b277 (0x82d0 bytes) 000385 1787708691.067700 write 0x03b470 thru 0x03ffff (0x4b90 bytes) 000386 1787708691.167985 write 0x01ad8f thru 0x01ed62 (0x3fd4 bytes) 000387 1787708691.184947 mapread 0x00d300 thru 0x01642e (0x912f bytes) 000388 1787708691.218146 mapread 0x0032a5 thru 0x003465 (0x1c1 bytes) 000393 1787708691.237989 MIRROR_RESYNC 2 mirrors 000394 1787708692.183196 mapread 0x0357bd thru 0x03ffff (0xa843 bytes) 000396 1787708692.205866 mapread 0x0363df thru 0x03f755 (0x9377 bytes) 000397 1787708692.206388 read 0x037883 thru 0x039a3c (0x21ba bytes) 000400 1787708692.207050 trunc from 0x040000 to 0x01fe96 (0x20169 bytes) 000401 1787708692.470527 read 0x01ab65 thru 0x01c621 (0x1abd bytes) 000402 1787708692.486150 mapwrite 0x01cdc3 thru 0x0293b0 (0xc5ee bytes) 000404 1787708692.658602 MIRROR_RESYNC 2 mirrors 000405 1787708693.808539 trunc from 0x0293b1 to 0x03e141 (0x14d91 bytes) 000406 1787708693.924156 read 0x01e379 thru 0x02e1fc (0xfe84 bytes) 000409 1787708693.947690 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) 000412 1787708694.128009 write 0x0165f4 thru 0x01d65f (0x706c bytes) 000414 1787708694.142184 read 0x02d320 thru 0x03079d (0x347e bytes) 000415 1787708694.148001 trunc from 0x03e142 to 0x017769 (0x269d8 bytes) 000416 1787708694.176727 write 0x024a38 thru 0x02a241 (0x580a bytes) 000418 1787708694.179200 mapread 0x009ff5 thru 0x01697f (0xc98b bytes) 000419 1787708694.222628 trunc from 0x02a242 to 0x02401c (0x6225 bytes) 000421 1787708694.243050 mapwrite 0x006b7a thru 0x0097cc (0x2c53 bytes) 000425 1787708694.297603 mapwrite 0x00975b thru 0x0166b0 (0xcf56 bytes) 000426 1787708694.386630 trunc from 0x02401d to 0x00bc6a (0x183b2 bytes) 000429 1787708694.434276 read 0x002aed thru 0x005637 (0x2b4b bytes) 000430 1787708694.442328 mapread 0x00078e thru 0x000eba (0x72d bytes) 000434 1787708694.447971 trunc from 0x00bc6b to 0x017b34 (0xbeca bytes) 000435 1787708694.481915 trunc from 0x017b35 to 0x019550 (0x1a1c bytes) 000437 1787708694.502211 trunc from 0x019551 to 0x02e87b (0x1532b bytes) 000438 1787708694.533480 mapread 0x01f763 thru 0x0265d6 (0x6e74 bytes) 000439 1787708694.546288 trunc from 0x02e87c to 0x03bdf5 (0xd57a bytes) 000440 1787708694.578659 write 0x021059 thru 0x0285bf (0x7567 bytes) 000441 1787708694.580185 write 0x00dde1 thru 0x014394 (0x65b4 bytes) 000442 1787708694.582960 mapread 0x021f31 thru 0x022372 (0x442 bytes) 000443 1787708694.584541 mapwrite 0x0078a6 thru 0x009e20 (0x257b bytes) 000445 1787708694.664103 read 0x00d5bf thru 0x015eee (0x8930 bytes) 000448 1787708694.675907 trunc from 0x03bdf6 to 0x0179bf (0x24436 bytes) 000449 1787708694.713029 mapread 0x006889 thru 0x013814 (0xcf8c bytes) 000450 1787708694.715537 write 0x03f73d thru 0x03ffff (0x8c3 bytes) 000456 1787708694.723993 mapread 0x011b71 thru 0x01efc4 (0xd454 bytes) 000461 1787708694.762269 write 0x016fec thru 0x026358 (0xf36d bytes) 000462 1787708694.770710 mapwrite 0x01c069 thru 0x028d71 (0xcd09 bytes) 000466 1787708694.932996 MIRROR_EXTEND to 2 mirrors 000467 1787708695.783059 read 0x0393a4 thru 0x03ffff (0x6c5c bytes) 000468 1787708695.801336 mapread 0x0211a3 thru 0x024e53 (0x3cb1 bytes) 000469 1787708695.843132 MIRROR_EXTEND to 3 mirrors 000470 1787708696.628699 MIRROR_EXTEND to 4 mirrors 000471 1787708697.524372 mapread 0x029003 thru 0x02c8d1 (0x38cf bytes) 000472 1787708697.553708 write 0x0084aa thru 0x010716 (0x826d bytes) 000475 1787708697.662871 trunc from 0x040000 to 0x01db53 (0x224ac bytes) 000477 1787708697.693157 read 0x014ac1 thru 0x01d195 (0x86d5 bytes) 000479 1787708697.703328 write 0x010f7a thru 0x01b44e (0xa4d5 bytes) 000480 1787708697.704889 trunc from 0x01db54 to 0x01dbe1 (0x8e bytes) 000481 1787708697.728981 read 0x01ceb3 thru 0x01dbe1 (0xd2f bytes) 000482 1787708697.729905 trunc from 0x01dbe2 to 0x005578 (0x18669 bytes) 000484 1787708697.749502 trunc from 0x005579 to 0x018c96 (0x1371e bytes) 000485 1787708697.766725 read 0x011be5 thru 0x018c96 (0x70b2 bytes) 000487 1787708697.779955 mapread 0x004ad2 thru 0x00e1db (0x970a bytes) 000488 1787708697.782298 mapwrite 0x033130 thru 0x03bf58 (0x8e29 bytes) 000489 1787708697.868267 trunc from 0x03bf59 to 0x00448f (0x37ac9 bytes) 000491 1787708697.897403 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x4490): Mirror 2: (stale - read error) Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x4490): Mirror 2: 0x72c67227 (stale) Mirror 3: 0x72c67227 (stale) Mirror 4: 0x72c67227 (stale) CRC-32 checksum value for chunk [0, 0x4490): Mirror 1: 0x72c67227 Mirror 5: 0x72c67227 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x4490: skipped 000492 1787708699.047078 mapwrite 0x0268ad thru 0x028dd8 (0x252c bytes) 000493 1787708699.231036 write 0x01bab9 thru 0x01bd01 (0x249 bytes) 000494 1787708699.240874 read 0x022dff thru 0x028dd8 (0x5fda bytes) 000495 1787708699.241691 write 0x021a2d thru 0x02f336 (0xd90a bytes) 000498 1787708699.246461 trunc from 0x02f337 to 0x02953a (0x5dfc bytes) 000499 1787708699.273561 read 0x00cd1a thru 0x016387 (0x966e bytes) 000500 1787708699.285153 read 0x000f5f thru 0x00bee7 (0xaf89 bytes) 000501 1787708699.286212 write 0x001f19 thru 0x0025d2 (0x6ba bytes) 000502 1787708699.287905 read 0x026945 thru 0x02953a (0x2bf6 bytes) 000503 1787708699.289588 mapread 0x027c39 thru 0x02953a (0x1902 bytes) 000504 1787708699.290733 trunc from 0x02953b to 0x01ab36 (0xea04 bytes) 000506 1787708699.315989 mapwrite 0x01c545 thru 0x02a830 (0xe2ec bytes) 000507 1787708699.411936 mapwrite 0x03db9d thru 0x03ffff (0x2463 bytes) 000508 1787708699.502836 MIRROR_RESYNC 5 mirrors 000515 1787708700.874369 MIRROR_SPLIT mirror 4 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) 000516 1787708701.425036 mapwrite 0x01b788 thru 0x01d7d7 (0x2050 bytes) 000519 1787708701.611970 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x79b3b7ca (stale) Mirror 4: 0x79b3b7ca (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0x79b3b7ca (stale) Mirror 4: 0x79b3b7ca (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000521 1787708701.988517 trunc from 0x040000 to 0x02451e (0x1bae1 bytes) 000523 1787708702.009653 MIRROR_RESYNC 3 mirrors 000524 1787708703.030504 MIRROR_RESYNC 3 mirrors 000525 1787708703.554193 read 0x00fb61 thru 0x012401 (0x28a1 bytes) 000526 1787708703.563550 mapwrite 0x00157b thru 0x00ed9a (0xd820 bytes) 000527 1787708703.797977 write 0x0326fc thru 0x0374d0 (0x4dd5 bytes) 000528 1787708703.799392 mapread 0x00bf86 thru 0x00d68d (0x1708 bytes) 000531 1787708703.802676 write 0x030d75 thru 0x039914 (0x8ba0 bytes) 000533 1787708703.825689 mapwrite 0x02a2a6 thru 0x02b0b5 (0xe10 bytes) 000535 1787708703.929608 trunc from 0x039915 to 0x020aac (0x18e68 bytes) 000536 1787708703.962660 mapwrite 0x00b71c thru 0x00d9ed (0x22d2 bytes) 000537 1787708704.091554 mapread 0x01a9c1 thru 0x01f51c (0x4b5c bytes) 000538 1787708704.113394 trunc from 0x020aad to 0x017f42 (0x8b6a bytes) 000539 1787708704.156693 write 0x027b62 thru 0x02b317 (0x37b6 bytes) 000542 1787708704.166206 mapwrite 0x010719 thru 0x01d9f8 (0xd2e0 bytes) 000543 1787708704.335632 write 0x014dd2 thru 0x01e34f (0x957e bytes) 000546 1787708704.342184 trunc from 0x02b318 to 0x028402 (0x2f15 bytes) 000550 1787708704.396123 read 0x0064da thru 0x00dcbe (0x77e5 bytes) 000551 1787708704.419672 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x28403): Mirror 4: 0xe5ebec7d (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x28403): Mirror 4: 0xe5ebec7d (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x28403: skipped 000552 1787708704.887634 mapread 0x01620a thru 0x017b8b (0x1982 bytes) 000554 1787708704.905066 mapwrite 0x027a5d thru 0x02f127 (0x76cb bytes) 000555 1787708704.974478 mapread 0x025569 thru 0x02f127 (0x9bbf bytes) 000556 1787708704.975508 write 0x01f778 thru 0x024ea8 (0x5731 bytes) 000558 1787708704.980940 write 0x00d842 thru 0x01b605 (0xddc4 bytes) 000559 1787708704.992422 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) 000560 1787708705.095507 trunc from 0x02f128 to 0x0120ec (0x1d03b bytes) 000561 1787708705.116203 mapwrite 0x00cda4 thru 0x019fee (0xd24b bytes) 000565 1787708705.248715 mapread 0x018ea3 thru 0x019208 (0x366 bytes) 000566 1787708705.257206 MIRROR_EXTEND to 2 mirrors 000567 1787708705.896132 read 0x00c31d thru 0x016030 (0x9d14 bytes) 000569 1787708705.913293 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) 000571 1787708706.032438 mapread 0x019526 thru 0x019fee (0xac9 bytes) 000572 1787708706.043098 trunc from 0x019fef to 0x018258 (0x1d96 bytes) 000574 1787708706.098927 write 0x01de44 thru 0x02363a (0x57f7 bytes) 000576 1787708706.101544 read 0x0203eb thru 0x02363a (0x3250 bytes) 000577 1787708706.103851 mapwrite 0x031153 thru 0x032381 (0x122f bytes) 000581 1787708706.192582 read 0x005d5a thru 0x01042a (0xa6d1 bytes) 000585 1787708706.205692 mapwrite 0x00ae45 thru 0x00d064 (0x2220 bytes) 000588 1787708706.244371 write 0x002d5e thru 0x004de6 (0x2089 bytes) 000590 1787708706.246664 write 0x009cf3 thru 0x00e3cf (0x46dd bytes) 000595 1787708706.251046 mapwrite 0x002243 thru 0x00e847 (0xc605 bytes) 000596 1787708706.298145 read 0x01baee thru 0x01ed6c (0x327f bytes) 000598 1787708706.299800 mapread 0x02e473 thru 0x032381 (0x3f0f bytes) 000600 1787708706.308422 mapwrite 0x02b857 thru 0x036a45 (0xb1ef bytes) 000601 1787708706.374960 mapread 0x02ad04 thru 0x02b490 (0x78d bytes) 000603 1787708706.377677 write 0x03789f thru 0x03ffff (0x8761 bytes) 000606 1787708706.385911 mapread 0x036cee thru 0x039e68 (0x317b bytes) 000607 1787708706.388304 mapwrite 0x0214e2 thru 0x02ca92 (0xb5b1 bytes) 000608 1787708706.438644 read 0x01321c thru 0x015e72 (0x2c57 bytes) 000612 1787708706.441175 read 0x00cafc thru 0x011162 (0x4667 bytes) 000613 1787708706.441933 mapread 0x01c5a3 thru 0x028761 (0xc1bf bytes) 000617 1787708706.445025 mapwrite 0x00d40d thru 0x01d1e4 (0xfdd8 bytes) 000618 1787708706.510599 trunc from 0x040000 to 0x027812 (0x187ed bytes) 000624 1787708706.533588 mapwrite 0x02f559 thru 0x037310 (0x7db8 bytes) 000626 1787708706.617707 read 0x01b01a thru 0x01d921 (0x2908 bytes) 000629 1787708706.620953 mapread 0x023ed9 thru 0x027434 (0x355c bytes) 000631 1787708706.623656 mapread 0x02bf48 thru 0x037310 (0xb3c9 bytes) 000632 1787708706.625624 trunc from 0x037311 to 0x023f0a (0x13406 bytes) 000634 1787708706.654129 mapwrite 0x02bc10 thru 0x02e41a (0x280b bytes) 000636 1787708706.797211 MIRROR_EXTEND to 2 mirrors 000637 1787708707.581055 mapwrite 0x0088fc thru 0x015195 (0xc89a bytes) 000638 1787708707.746390 read 0x019ff4 thru 0x01d9db (0x39e8 bytes) 000641 1787708707.754541 mapread 0x00726a thru 0x008b6a (0x1901 bytes) 000642 1787708707.755719 mapwrite 0x031256 thru 0x03ae85 (0x9c30 bytes) 000645 1787708707.847775 read 0x03837b thru 0x03ae85 (0x2b0b bytes) 000648 1787708707.850782 mapwrite 0x007279 thru 0x014452 (0xd1da bytes) 000651 1787708707.903525 mapwrite 0x03c4db thru 0x03ffff (0x3b25 bytes) 000652 1787708708.008266 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xd2369b86 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x504a3f17 Mirror 3: 0x504a3f17 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000654 1787708709.149882 mapwrite 0x024717 thru 0x02b1be (0x6aa8 bytes) 000659 1787708709.350663 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xd2369b86 (stale) Mirror 3: 0x504a3f17 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x79e05d95 Mirror 4: 0x79e05d95 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000660 1787708710.439737 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xd2369b86 (stale) Mirror 3: 0x504a3f17 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x79e05d95 Mirror 4: 0x79e05d95 Mirror 5: 0x79e05d95 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000661 1787708711.534722 read 0x03995f thru 0x03ffff (0x66a1 bytes) 000665 1787708711.558717 write 0x01f2b1 thru 0x02952d (0xa27d bytes) 000668 1787708711.676222 read 0x01c85b thru 0x02734a (0xaaf0 bytes) 000670 1787708711.690583 mapread 0x010a9c thru 0x017ab5 (0x701a bytes) 000673 1787708711.693293 trunc from 0x040000 to 0x036ede (0x9121 bytes) 000674 1787708711.721585 mapread 0x01f87c thru 0x02dd53 (0xe4d8 bytes) 000675 1787708711.725393 mapread 0x016a0c thru 0x023c02 (0xd1f7 bytes) 000680 1787708711.732487 read 0x036a21 thru 0x036ede (0x4be bytes) 000681 1787708711.739507 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x36edf): Mirror 2: (stale - read error) Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x36edf): Mirror 2: 0xe046b693 (stale) Mirror 3: 0xd6b2765e (stale) Mirror 4: 0xcf819079 (stale) Mirror 5: 0xcf819079 (stale) CRC-32 checksum value for chunk [0, 0x36edf): Mirror 1: 0xa3c018ca Mirror 6: 0xa3c018ca lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x36edf: skipped 000685 1787708712.768975 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x36edf): Mirror 2: (stale - read error) Mirror 3: (stale - read error) Mirror 4: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x36edf): Mirror 2: 0xe046b693 (stale) Mirror 3: 0xd6b2765e (stale) Mirror 4: 0xcf819079 (stale) Mirror 5: 0xcf819079 (stale) CRC-32 checksum value for chunk [0, 0x36edf): Mirror 1: 0xa3c018ca Mirror 6: 0xa3c018ca Mirror 7: 0xa3c018ca lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x36edf: skipped 000690 1787708713.916120 read 0x02a9eb thru 0x036ede (0xc4f4 bytes) 000696 1787708713.926714 write 0x01f359 thru 0x026837 (0x74df bytes) 000697 1787708714.048852 read 0x025abd thru 0x02a1fc (0x4740 bytes) 000698 1787708714.061305 mapwrite 0x028b83 thru 0x0381c4 (0xf642 bytes) 000700 1787708714.174725 read 0x03197e thru 0x0381c4 (0x6847 bytes) 000701 1787708714.175675 mapread 0x00b45c thru 0x010f06 (0x5aab bytes) 000702 1787708714.182399 MIRROR_SPLIT mirror 3 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x381c5): Mirror 2: 0x8388e089 (stale) Mirror 3: 0x4b2c9876 (stale) Mirror 5: 0x763d5b91 (stale) Mirror 6: 0xe40df6b7 (stale) Mirror 7: 0xe40df6b7 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x381c5): Mirror 2: 0x8388e089 (stale) Mirror 3: 0x4b2c9876 (stale) Mirror 5: 0x763d5b91 (stale) Mirror 6: 0xe40df6b7 (stale) Mirror 7: 0xe40df6b7 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x381c5: skipped 000703 1787708714.698519 mapread 0x02fe2d thru 0x035d41 (0x5f15 bytes) 000705 1787708714.723430 trunc from 0x0381c5 to 0x00871e (0x2faa6 bytes) 000706 1787708714.747709 mapwrite 0x026b23 thru 0x026cbb (0x199 bytes) 000707 1787708714.825292 trunc from 0x026cbc to 0x02f453 (0x8798 bytes) 000708 1787708714.844292 read 0x022c4a thru 0x024d26 (0x20dd bytes) 000711 1787708714.849574 mapread 0x017125 thru 0x01bf36 (0x4e12 bytes) 000712 1787708714.865635 mapwrite 0x024549 thru 0x02acde (0x6796 bytes) 000713 1787708714.943496 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x2f454): Mirror 2: (stale - read error) Mirror 3: (stale - read error) Mirror 5: (stale - read error) Mirror 6: (stale - read error) Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2f454): Mirror 2: 0x552e7067 (stale) Mirror 3: 0x70999a0d (stale) Mirror 5: 0xe57033b7 (stale) Mirror 6: 0x2453daa3 (stale) Mirror 7: 0x2453daa3 (stale) CRC-32 checksum value for chunk [0, 0x2f454): Mirror 1: 0x1d5afc3f Mirror 8: 0x1d5afc3f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2f454: skipped 000715 1787708716.092661 mapread 0x004700 thru 0x00fb1f (0xb420 bytes) 000716 1787708716.136382 MIRROR_SPLIT mirror 1 to 6 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2f454): Mirror 3: 0x70999a0d (stale) Mirror 5: 0xe57033b7 (stale) Mirror 6: 0x2453daa3 (stale) Mirror 7: 0x2453daa3 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2f454): Mirror 3: 0x70999a0d (stale) Mirror 5: 0xe57033b7 (stale) Mirror 6: 0x2453daa3 (stale) Mirror 7: 0x2453daa3 (stale) CRC-32 checksum value for chunk [0, 0x2f454): Mirror 1: 0x1d5afc3f Mirror 8: 0x1d5afc3f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2f454: skipped 000718 1787708717.307940 trunc from 0x02f454 to 0x010d59 (0x1e6fa bytes) 000721 1787708717.505328 mapwrite 0x02c3de thru 0x037045 (0xac68 bytes) 000722 1787708717.645034 mapread 0x00e473 thru 0x011d2d (0x38bb bytes) 000723 1787708717.658496 trunc from 0x037046 to 0x02286f (0x147d6 bytes) 000724 1787708717.694453 MIRROR_SPLIT mirror 3 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x22870): Mirror 3: 0x8db27a48 (stale) Mirror 5: 0x8db27a48 (stale) Mirror 7: 0xee4611b0 (stale) Mirror 8: 0x5a4f03f3 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x22870): Mirror 3: 0x8db27a48 (stale) Mirror 5: 0x8db27a48 (stale) Mirror 7: 0xee4611b0 (stale) Mirror 8: 0x5a4f03f3 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x22870: skipped 000726 1787708718.100415 MIRROR_RESYNC 5 mirrors 000727 1787708719.531225 read 0x00097f thru 0x00331e (0x29a0 bytes) 000728 1787708719.541966 mapwrite 0x00a6a7 thru 0x00e433 (0x3d8d bytes) 000731 1787708719.752804 MIRROR_RESYNC 5 mirrors 000735 1787708721.500611 mapread 0x0040ce thru 0x00e52c (0xa45f bytes) 000736 1787708721.539032 mapwrite 0x00eeb1 thru 0x015eb8 (0x7008 bytes) 000737 1787708721.821432 MIRROR_RESYNC 5 mirrors 000738 1787708722.758336 trunc from 0x022870 to 0x02727c (0x4a0d bytes) 000742 1787708722.821913 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x2727d): Mirror 3: (stale - read error) Mirror 5: (stale - read error) Mirror 7: (stale - read error) Mirror 8: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2727d): Mirror 3: 0x1fc6d8d0 (stale) Mirror 5: 0x1fc6d8d0 (stale) Mirror 7: 0x1fc6d8d0 (stale) Mirror 8: 0x1fc6d8d0 (stale) CRC-32 checksum value for chunk [0, 0x2727d): Mirror 1: 0x1fc6d8d0 Mirror 9: 0x1fc6d8d0 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2727d: skipped 000743 1787708723.955180 MIRROR_SPLIT mirror 2 to 5 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2727d): Mirror 3: 0x1fc6d8d0 (stale) Mirror 7: 0x1fc6d8d0 (stale) Mirror 8: 0x1fc6d8d0 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2727d): Mirror 3: 0x1fc6d8d0 (stale) Mirror 7: 0x1fc6d8d0 (stale) Mirror 8: 0x1fc6d8d0 (stale) CRC-32 checksum value for chunk [0, 0x2727d): Mirror 1: 0x1fc6d8d0 Mirror 9: 0x1fc6d8d0 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2727d: skipped 000744 1787708725.320993 read 0x01878c thru 0x01f6c1 (0x6f36 bytes) 000747 1787708725.430708 MIRROR_SPLIT mirror 2 to 4 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2727d): Mirror 3: 0x1fc6d8d0 (stale) Mirror 8: 0x1fc6d8d0 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2727d): Mirror 3: 0x1fc6d8d0 (stale) Mirror 8: 0x1fc6d8d0 (stale) CRC-32 checksum value for chunk [0, 0x2727d): Mirror 1: 0x1fc6d8d0 Mirror 9: 0x1fc6d8d0 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2727d: skipped 000748 1787708726.668231 mapread 0x01ce8e thru 0x026b27 (0x9c9a bytes) 000749 1787708726.821653 trunc from 0x02727d to 0x01cc14 (0xa668 bytes) 000750 1787708727.070474 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x1cc15): Mirror 3: 0xf85497a0 (stale) Mirror 9: 0xf85497a0 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1cc15): Mirror 3: 0xf85497a0 (stale) Mirror 9: 0xf85497a0 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1cc15: skipped 000751 1787708727.639677 mapread 0x00bf9a thru 0x01a540 (0xe5a7 bytes) 000755 1787708727.707234 mapwrite 0x015ef7 thru 0x01a530 (0x463a bytes) 000757 1787708727.870676 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x1cc15): Mirror 3: (stale - read error) Mirror 9: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1cc15): Mirror 3: 0xf85497a0 (stale) Mirror 9: 0xf85497a0 (stale) CRC-32 checksum value for chunk [0, 0x1cc15): Mirror 1: 0x8e97d6de Mirror 10: 0x8e97d6de lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1cc15: skipped 000761 1787708729.359243 mapwrite 0x0042d0 thru 0x00ff36 (0xbc67 bytes) 000763 1787708729.532219 trunc from 0x01cc15 to 0x0364d4 (0x198c0 bytes) 000764 1787708729.550066 mapread 0x035e30 thru 0x0364d4 (0x6a5 bytes) 000765 1787708729.559075 read 0x01c8f0 thru 0x02720a (0xa91b bytes) 000766 1787708729.568315 MIRROR_RESYNC 4 mirrors 000767 1787708730.504296 mapread 0x003924 thru 0x00e7c8 (0xaea5 bytes) 000768 1787708730.522169 write 0x0262c9 thru 0x026620 (0x358 bytes) 000771 1787708730.579927 trunc from 0x0364d5 to 0x01fd8a (0x1674a bytes) 000773 1787708730.596878 read 0x01e17f thru 0x01fd8a (0x1c0c bytes) 000774 1787708730.601833 read 0x018392 thru 0x01ca11 (0x4680 bytes) 000775 1787708730.612790 MIRROR_EXTEND to 5 mirrors CRC-32 checksum value for chunk [0, 0x1fd8b): Mirror 3: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1fd8b): Mirror 3: 0x5947416e (stale) Mirror 9: 0x5947416e (stale) Mirror 10: 0x5947416e (stale) CRC-32 checksum value for chunk [0, 0x1fd8b): Mirror 1: 0x5947416e Mirror 11: 0x5947416e lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1fd8b: skipped 000777 1787708731.888474 mapread 0x01e3f9 thru 0x01fd8a (0x1992 bytes) 000778 1787708731.957953 MIRROR_RESYNC 5 mirrors 000779 1787708733.331541 mapwrite 0x022092 thru 0x0270fb (0x506a bytes) 000786 1787708733.505232 mapread 0x007395 thru 0x01088d (0x94f9 bytes) 000787 1787708733.529624 write 0x020cdb thru 0x029d91 (0x90b7 bytes) 000789 1787708733.531656 trunc from 0x029d92 to 0x02c9d6 (0x2c45 bytes) 000790 1787708733.560053 MIRROR_EXTEND to 6 mirrors CRC-32 checksum value for chunk [0, 0x2c9d7): Mirror 3: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2c9d7): Mirror 3: 0x5de8d61c (stale) Mirror 9: 0x5de8d61c (stale) Mirror 10: 0x5de8d61c (stale) Mirror 11: 0x5de8d61c (stale) CRC-32 checksum value for chunk [0, 0x2c9d7): Mirror 1: 0x12d55787 Mirror 12: 0x12d55787 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2c9d7: skipped 000791 1787708734.597438 write 0x03da40 thru 0x03ffff (0x25c0 bytes) 000792 1787708734.693291 mapwrite 0x01fdcc thru 0x02bce4 (0xbf19 bytes) 000793 1787708734.814378 mapread 0x030c87 thru 0x03e67e (0xd9f8 bytes) 000794 1787708734.853066 MIRROR_EXTEND to 7 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x397453b2 (stale) Mirror 9: 0x397453b2 (stale) Mirror 10: 0x397453b2 (stale) Mirror 11: 0x397453b2 (stale) Mirror 12: 0xc0e0248 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x2689c060 Mirror 13: 0x2689c060 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000795 1787708736.328440 write 0x00858d thru 0x00daa1 (0x5515 bytes) 000799 1787708736.416301 trunc from 0x040000 to 0x02aa9e (0x15561 bytes) truncating to largest ever: 0x3ff5e 000800 1787708736.444707 trunc from 0x02aa9f to 0x03ff5d (0x154bf bytes) 000802 1787708736.469739 trunc from 0x03ff5e to 0x022e6a (0x1d0f3 bytes) 000804 1787708736.491463 write 0x02832b thru 0x02fc35 (0x790b bytes) 000805 1787708736.495169 trunc from 0x02fc36 to 0x02c66b (0x35ca bytes) 000806 1787708736.524839 write 0x00cf51 thru 0x01627f (0x932f bytes) 000807 1787708736.531057 trunc from 0x02c66c to 0x001521 (0x2b14a bytes) 000810 1787708736.552586 trunc from 0x001522 to 0x02ff72 (0x2ea51 bytes) 000814 1787708736.571028 write 0x032c82 thru 0x03c216 (0x9595 bytes) 000815 1787708736.573472 trunc from 0x03c217 to 0x021968 (0x1a8ae bytes) 000816 1787708736.594359 trunc from 0x021969 to 0x011f07 (0xfa61 bytes) 000817 1787708736.613937 read 0x00e136 thru 0x011f07 (0x3dd2 bytes) 000818 1787708736.625581 MIRROR_RESYNC 7 mirrors 000819 1787708737.755629 write 0x00f5d9 thru 0x016f4e (0x7976 bytes) 000820 1787708737.823547 trunc from 0x016f4f to 0x005293 (0x11cbb bytes) 000821 1787708737.844206 trunc from 0x005294 to 0x00fa38 (0xa7a5 bytes) 000822 1787708737.863394 read 0x00cf9f thru 0x00fa38 (0x2a9a bytes) 000823 1787708737.868080 mapread 0x0043c9 thru 0x0066bb (0x22f3 bytes) 000825 1787708737.900077 read 0x0028f8 thru 0x00b448 (0x8b51 bytes) 000829 1787708737.903700 write 0x03bc65 thru 0x03c8c7 (0xc63 bytes) 000831 1787708737.906366 write 0x0227f9 thru 0x026f88 (0x4790 bytes) 000832 1787708737.918242 trunc from 0x03c8c8 to 0x0337c9 (0x90fe bytes) 000833 1787708737.948264 MIRROR_RESYNC 7 mirrors 000834 1787708739.802528 trunc from 0x0337ca to 0x036ab0 (0x32e7 bytes) 000835 1787708739.860704 write 0x03eab9 thru 0x03ffff (0x1547 bytes) 000836 1787708739.866159 MIRROR_EXTEND to 8 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) Mirror 13: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0xc641d4e8 (stale) Mirror 9: 0xc641d4e8 (stale) Mirror 10: 0xc641d4e8 (stale) Mirror 11: 0xc641d4e8 (stale) Mirror 12: 0xc641d4e8 (stale) Mirror 13: 0xc641d4e8 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0x7e4ac5e3 Mirror 14: 0x7e4ac5e3 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000837 1787708741.342050 mapread 0x03b711 thru 0x03d8f1 (0x21e1 bytes) 000838 1787708741.450642 MIRROR_RESYNC 8 mirrors 000841 1787708743.669502 MIRROR_EXTEND to 9 mirrors 000844 1787708744.845001 MIRROR_EXTEND to 10 mirrors 000845 1787708745.750894 MIRROR_EXTEND to 11 mirrors 000848 1787708747.078851 mapread 0x0005e0 thru 0x00e8ce (0xe2ef bytes) 000850 1787708747.142414 trunc from 0x040000 to 0x017523 (0x28adc bytes) 000851 1787708747.315063 mapread 0x01200a thru 0x017523 (0x551a bytes) 000852 1787708747.353893 write 0x00b64c thru 0x0140ac (0x8a61 bytes) 000853 1787708747.355254 mapread 0x0101b1 thru 0x017523 (0x7373 bytes) 000854 1787708747.356315 mapwrite 0x03198e thru 0x03ffff (0xe672 bytes) 000856 1787708747.565677 read 0x033983 thru 0x03ffff (0xc67d bytes) 000857 1787708747.571834 write 0x025c28 thru 0x02c721 (0x6afa bytes) 000858 1787708747.582542 mapread 0x00f43c thru 0x013933 (0x44f8 bytes) 000861 1787708747.586970 write 0x00be8d thru 0x0196d4 (0xd848 bytes) 000863 1787708747.598436 write 0x01afcb thru 0x0259ab (0xa9e1 bytes) 000864 1787708747.607350 mapread 0x001b7a thru 0x006960 (0x4de7 bytes) 000866 1787708747.620624 mapwrite 0x031daa thru 0x03930d (0x7564 bytes) 000868 1787708747.711467 trunc from 0x040000 to 0x02a275 (0x15d8a bytes) 000869 1787708747.768831 mapwrite 0x018a05 thru 0x025329 (0xc925 bytes) 000871 1787708747.872436 MIRROR_EXTEND to 12 mirrors CRC-32 checksum value for chunk [0, 0x2a276): Mirror 3: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) Mirror 17: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2a276): Mirror 3: 0x1cf6cee5 (stale) Mirror 9: 0x1cf6cee5 (stale) Mirror 10: 0x1cf6cee5 (stale) Mirror 11: 0x1cf6cee5 (stale) Mirror 12: 0x1cf6cee5 (stale) Mirror 13: 0x1cf6cee5 (stale) Mirror 14: 0x1cf6cee5 (stale) Mirror 15: 0x1cf6cee5 (stale) Mirror 16: 0x1cf6cee5 (stale) Mirror 17: 0x1cf6cee5 (stale) CRC-32 checksum value for chunk [0, 0x2a276): Mirror 1: 0xec65bef0 Mirror 18: 0xec65bef0 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2a276: skipped 000872 1787708749.236364 MIRROR_EXTEND to 13 mirrors CRC-32 checksum value for chunk [0, 0x2a276): Mirror 3: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) Mirror 17: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2a276): Mirror 3: 0x1cf6cee5 (stale) Mirror 9: 0x1cf6cee5 (stale) Mirror 10: 0x1cf6cee5 (stale) Mirror 11: 0x1cf6cee5 (stale) Mirror 12: 0x1cf6cee5 (stale) Mirror 13: 0x1cf6cee5 (stale) Mirror 14: 0x1cf6cee5 (stale) Mirror 15: 0x1cf6cee5 (stale) Mirror 16: 0x1cf6cee5 (stale) Mirror 17: 0x1cf6cee5 (stale) CRC-32 checksum value for chunk [0, 0x2a276): Mirror 1: 0xec65bef0 Mirror 18: 0xec65bef0 Mirror 19: 0xec65bef0 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2a276: skipped 000873 1787708750.925792 mapread 0x01005f thru 0x017b68 (0x7b0a bytes) 000874 1787708750.959205 mapwrite 0x011db0 thru 0x0155e2 (0x3833 bytes) 000875 1787708751.187350 mapwrite 0x01ba8b thru 0x0226c5 (0x6c3b bytes) 000876 1787708751.322028 trunc from 0x02a276 to 0x026251 (0x4024 bytes) 000878 1787708751.387298 mapwrite 0x03ba66 thru 0x03ffff (0x459a bytes) 000880 1787708751.553390 MIRROR_EXTEND to 14 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 11: (stale - read error) Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) Mirror 17: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 3: 0x7e4ac5e3 (stale) Mirror 9: 0x7e4ac5e3 (stale) Mirror 10: 0x7e4ac5e3 (stale) Mirror 11: 0x7e4ac5e3 (stale) Mirror 12: 0x7e4ac5e3 (stale) Mirror 13: 0x7e4ac5e3 (stale) Mirror 14: 0x7e4ac5e3 (stale) Mirror 15: 0x7e4ac5e3 (stale) Mirror 16: 0x7e4ac5e3 (stale) Mirror 17: 0x7e4ac5e3 (stale) Mirror 18: 0x64c83786 (stale) Mirror 19: 0x64c83786 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xb6a35f2d Mirror 20: 0xb6a35f2d lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000882 1787708753.635189 mapwrite 0x01c378 thru 0x0205ac (0x4235 bytes) 000883 1787708753.987476 trunc from 0x040000 to 0x024b76 (0x1b489 bytes) 000884 1787708754.041220 MIRROR_SPLIT mirror 4 to 13 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x24b77): Mirror 3: 0x7a2a7762 (stale) Mirror 9: 0x7a2a7762 (stale) Mirror 10: 0x7a2a7762 (stale) Mirror 12: 0x7a2a7762 (stale) Mirror 13: 0x7a2a7762 (stale) Mirror 14: 0x7a2a7762 (stale) Mirror 15: 0x7a2a7762 (stale) Mirror 16: 0x7a2a7762 (stale) Mirror 17: 0x7a2a7762 (stale) Mirror 18: 0x840d4470 (stale) Mirror 19: 0x840d4470 (stale) Mirror 20: 0x7a1e5580 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x24b77): Mirror 3: 0x7a2a7762 (stale) Mirror 9: 0x7a2a7762 (stale) Mirror 10: 0x7a2a7762 (stale) Mirror 12: 0x7a2a7762 (stale) Mirror 13: 0x7a2a7762 (stale) Mirror 14: 0x7a2a7762 (stale) Mirror 15: 0x7a2a7762 (stale) Mirror 16: 0x7a2a7762 (stale) Mirror 17: 0x7a2a7762 (stale) Mirror 18: 0x840d4470 (stale) Mirror 19: 0x840d4470 (stale) Mirror 20: 0x7a1e5580 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x24b77: skipped 000886 1787708754.723888 MIRROR_EXTEND to 14 mirrors CRC-32 checksum value for chunk [0, 0x24b77): Mirror 3: (stale - read error) Mirror 9: (stale - read error) Mirror 10: (stale - read error) Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) Mirror 17: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) Mirror 20: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x24b77): Mirror 3: 0x7a2a7762 (stale) Mirror 9: 0x7a2a7762 (stale) Mirror 10: 0x7a2a7762 (stale) Mirror 12: 0x7a2a7762 (stale) Mirror 13: 0x7a2a7762 (stale) Mirror 14: 0x7a2a7762 (stale) Mirror 15: 0x7a2a7762 (stale) Mirror 16: 0x7a2a7762 (stale) Mirror 17: 0x7a2a7762 (stale) Mirror 18: 0x840d4470 (stale) Mirror 19: 0x840d4470 (stale) Mirror 20: 0x7a1e5580 (stale) CRC-32 checksum value for chunk [0, 0x24b77): Mirror 1: 0xa6c1b958 Mirror 21: 0xa6c1b958 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x24b77: skipped 000887 1787708755.827690 trunc from 0x024b77 to 0x00f33b (0x1583b bytes) 000888 1787708755.927944 trunc from 0x00f33c to 0x00f75c (0x421 bytes) 000889 1787708755.948245 trunc from 0x00f75d to 0x0277d7 (0x1807b bytes) 000890 1787708755.962503 write 0x034b9d thru 0x03fb45 (0xafa9 bytes) 000893 1787708755.972501 MIRROR_SPLIT mirror 2 to 13 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3fb46): Mirror 3: 0x583dd579 (stale) Mirror 10: 0x583dd579 (stale) Mirror 12: 0x583dd579 (stale) Mirror 13: 0x583dd579 (stale) Mirror 14: 0x583dd579 (stale) Mirror 15: 0x583dd579 (stale) Mirror 16: 0x583dd579 (stale) Mirror 17: 0x583dd579 (stale) Mirror 18: 0xb3587b0b (stale) Mirror 19: 0xb3587b0b (stale) Mirror 20: 0x966b1684 (stale) Mirror 21: 0xfb105537 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3fb46): Mirror 3: 0x583dd579 (stale) Mirror 10: 0x583dd579 (stale) Mirror 12: 0x583dd579 (stale) Mirror 13: 0x583dd579 (stale) Mirror 14: 0x583dd579 (stale) Mirror 15: 0x583dd579 (stale) Mirror 16: 0x583dd579 (stale) Mirror 17: 0x583dd579 (stale) Mirror 18: 0xb3587b0b (stale) Mirror 19: 0xb3587b0b (stale) Mirror 20: 0x966b1684 (stale) Mirror 21: 0xfb105537 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3fb46: skipped 000894 1787708756.887854 trunc from 0x03fb46 to 0x03878f (0x73b6 bytes) 000898 1787708756.933912 trunc from 0x038790 to 0x00cea9 (0x2b8e6 bytes) 000899 1787708756.964620 trunc from 0x00ceaa to 0x00715e (0x5d4b bytes) 000900 1787708756.987091 trunc from 0x00715f to 0x00f3f0 (0x8292 bytes) 000901 1787708757.015107 read 0x00e4ac thru 0x00f3f0 (0xf45 bytes) 000902 1787708757.026108 read 0x009d5b thru 0x00f3f0 (0x5696 bytes) 000907 1787708757.042151 trunc from 0x00f3f1 to 0x038599 (0x291a9 bytes) 000909 1787708757.066607 mapread 0x01005c thru 0x010254 (0x1f9 bytes) 000910 1787708757.090011 MIRROR_RESYNC 13 mirrors 000912 1787708760.371130 mapread 0x030b55 thru 0x038599 (0x7a45 bytes) 000914 1787708760.389716 mapread 0x01ce75 thru 0x01d772 (0x8fe bytes) 000915 1787708760.398493 write 0x03d362 thru 0x03ffff (0x2c9e bytes) 000916 1787708760.442874 mapwrite 0x00ebc0 thru 0x017d34 (0x9175 bytes) 000917 1787708760.550330 mapread 0x0257f9 thru 0x03458d (0xed95 bytes) 000918 1787708760.593027 trunc from 0x040000 to 0x01ae36 (0x251c9 bytes) 000919 1787708760.635636 MIRROR_RESYNC 13 mirrors 000920 1787708763.159267 mapread 0x007ccc thru 0x012f8c (0xb2c1 bytes) 000922 1787708763.210374 MIRROR_EXTEND to 14 mirrors 000924 1787708764.795206 read 0x00d1a5 thru 0x01a343 (0xd19f bytes) 000926 1787708764.807017 read 0x007cca thru 0x013206 (0xb53d bytes) 000927 1787708764.826635 read 0x011db3 thru 0x01508c (0x32da bytes) 000928 1787708764.827112 read 0x005eaa thru 0x006184 (0x2db bytes) 000929 1787708764.828445 mapread 0x011655 thru 0x017b2f (0x64db bytes) 000930 1787708764.829126 mapwrite 0x0017c7 thru 0x00237b (0xbb5 bytes) 000931 1787708765.000283 write 0x00966d thru 0x0105a6 (0x6f3a bytes) 000932 1787708765.001514 trunc from 0x01ae37 to 0x01a8cc (0x56a bytes) 000933 1787708765.030733 read 0x0165b4 thru 0x01a8cc (0x4319 bytes) 000934 1787708765.040217 MIRROR_EXTEND to 15 mirrors CRC-32 checksum value for chunk [0, 0x1a8cd): Mirror 3: (stale - read error) Mirror 10: (stale - read error) Mirror 12: (stale - read error) Mirror 13: (stale - read error) Mirror 14: (stale - read error) Mirror 15: (stale - read error) Mirror 16: (stale - read error) Mirror 17: (stale - read error) Mirror 18: (stale - read error) Mirror 19: (stale - read error) Mirror 20: (stale - read error) Mirror 21: (stale - read error) Mirror 22: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1a8cd): Mirror 3: 0x193e4335 (stale) Mirror 10: 0x193e4335 (stale) Mirror 12: 0x193e4335 (stale) Mirror 13: 0x193e4335 (stale) Mirror 14: 0x193e4335 (stale) Mirror 15: 0x193e4335 (stale) Mirror 16: 0x193e4335 (stale) Mirror 17: 0x193e4335 (stale) Mirror 18: 0x193e4335 (stale) Mirror 19: 0x193e4335 (stale) Mirror 20: 0x193e4335 (stale) Mirror 21: 0x193e4335 (stale) Mirror 22: 0x193e4335 (stale) CRC-32 checksum value for chunk [0, 0x1a8cd): Mirror 1: 0x600c635e Mirror 23: 0x600c635e lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1a8cd: skipped 000936 1787708766.485373 trunc from 0x01a8cd to 0x028cf9 (0xe42d bytes) 000938 1787708766.722496 mapwrite 0x020658 thru 0x02b62d (0xafd6 bytes) 000941 1787708766.854353 mapwrite 0x02e345 thru 0x031160 (0x2e1c bytes) 000942 1787708766.946439 mapwrite 0x03b8b6 thru 0x03d1bd (0x1908 bytes) 000943 1787708767.045442 trunc from 0x03d1be to 0x01600c (0x271b1 bytes) 000945 1787708767.068290 mapwrite 0x0193ad thru 0x02432e (0xaf82 bytes) 000946 1787708767.183205 write 0x018930 thru 0x0245c9 (0xbc9a bytes) 000948 1787708767.187173 mapwrite 0x016c8f thru 0x01a350 (0x36c2 bytes) 000950 1787708767.266084 trunc from 0x0245ca to 0x03f31d (0x1ad54 bytes) 000951 1787708767.295423 mapread 0x034c4a thru 0x03f31d (0xa6d4 bytes) 000952 1787708767.333896 MIRROR_SPLIT mirror 14 to 14 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x192:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3f31e): Mirror 3: 0x296da1ba (stale) Mirror 10: 0x296da1ba (stale) Mirror 12: 0x296da1ba (stale) Mirror 13: 0x296da1ba (stale) Mirror 14: 0x296da1ba (stale) Mirror 15: 0x296da1ba (stale) Mirror 16: 0x296da1ba (stale) Mirror 17: 0x296da1ba (stale) Mirror 18: 0x296da1ba (stale) Mirror 19: 0x296da1ba (stale) Mirror 20: 0x296da1ba (stale) Mirror 21: 0x296da1ba (stale) Mirror 22: 0x296da1ba (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3f31e): Mirror 3: 0x296da1ba (stale) Mirror 10: 0x296da1ba (stale) Mirror 12: 0x296da1ba (stale) Mirror 13: 0x296da1ba (stale) Mirror 14: 0x296da1ba (stale) Mirror 15: 0x296da1ba (stale) Mirror 16: 0x296da1ba (stale) Mirror 17: 0x296da1ba (stale) Mirror 18: 0x296da1ba (stale) Mirror 19: 0x296da1ba (stale) Mirror 20: 0x296da1ba (stale) Mirror 21: 0x296da1ba (stale) Mirror 22: 0x296da1ba (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3f31e: skipped 000953 1787708768.270063 read 0x036e72 thru 0x037403 (0x592 bytes) 000954 1787708768.280170 mapread 0x00a393 thru 0x019e6c (0xfada bytes) 000963 1787708768.335185 write 0x018812 thru 0x01fb6e (0x735d bytes) 000964 1787708768.342502 write 0x01ab24 thru 0x0269dd (0xbeba bytes) 000967 1787708768.350960 write 0x03b384 thru 0x03ffff (0x4c7c bytes) 000969 1787708768.356637 trunc from 0x040000 to 0x0181c0 (0x27e3f bytes) 000970 1787708768.376927 write 0x01f2a6 thru 0x0272f1 (0x804c bytes) 000974 1787708768.381917 write 0x006b12 thru 0x00ff70 (0x945f bytes) 000978 1787708768.385880 write 0x029fef thru 0x02c206 (0x2218 bytes) 000980 1787708768.389990 mapwrite 0x02c67f thru 0x038275 (0xbbf7 bytes) 000981 1787708768.493426 mapread 0x01ba9e thru 0x021963 (0x5ec6 bytes) 000982 1787708768.512259 write 0x024cd0 thru 0x033a50 (0xed81 bytes) 000983 1787708768.514254 read 0x02a4a7 thru 0x02e9ec (0x4546 bytes) 000984 1787708768.515293 mapread 0x02b3f3 thru 0x02cc51 (0x185f bytes) 000985 1787708768.516854 mapread 0x00e8cd thru 0x01b421 (0xcb55 bytes) 000986 1787708768.518267 read 0x02abe8 thru 0x02ae45 (0x25e bytes) 000987 1787708768.519179 mapread 0x013f3d thru 0x01ec20 (0xace4 bytes) 000990 1787708768.521595 trunc from 0x038276 to 0x029b11 (0xe764 bytes) 000991 1787708768.546411 write 0x02737e thru 0x032f04 (0xbb87 bytes) 000992 1787708768.559145 MIRROR_RESYNC 14 mirrors 000994 1787708771.285134 mapread 0x023777 thru 0x026f0a (0x3794 bytes) 000995 1787708771.313341 mapwrite 0x0295ee thru 0x03955c (0xff6f bytes) 000996 1787708771.604601 read 0x02f416 thru 0x035353 (0x5f3e bytes) 000998 1787708771.605730 write 0x019e64 thru 0x02873c (0xe8d9 bytes) All operations completed A-OK!