== sanity-flr test 70a: flr mode fsx test ================ 21:02:28 (1788570148) Seed set to 149 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read skipping zero size read skipping zero size read 000008 1788570149.919010 MIRROR_EXTEND to 2 mirrors 000009 1788570150.304373 mapwrite 0x03d372 thru 0x03ffff (0x2c8e bytes) 000012 1788570150.454473 read 0x0101a0 thru 0x0170b1 (0x6f12 bytes) 000015 1788570150.465550 read 0x03891f thru 0x03ffff (0x76e1 bytes) 000019 1788570150.468946 read 0x00a429 thru 0x00f170 (0x4d48 bytes) truncating to largest ever: 0x276bc 000020 1788570150.485023 trunc from 0x040000 to 0x0276bb (0x18944 bytes) 000021 1788570150.504016 mapread 0x0204ed thru 0x0274e1 (0x6ff5 bytes) 000023 1788570150.507750 mapwrite 0x012b21 thru 0x015f55 (0x3435 bytes) 000024 1788570150.550184 read 0x00a4b6 thru 0x011de1 (0x792c bytes) 000025 1788570150.555533 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) 000027 1788570150.710267 MIRROR_EXTEND to 2 mirrors 000028 1788570151.375459 trunc from 0x0276bc to 0x01490a (0x12db1 bytes) 000029 1788570151.440455 read 0x00b11e thru 0x010e21 (0x5d04 bytes) truncating to largest ever: 0x3221b 000030 1788570151.447876 trunc from 0x01490b to 0x03221a (0x1d910 bytes) 000036 1788570151.472829 trunc from 0x03221b to 0x02439f (0xde7b bytes) 000037 1788570151.499212 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) 000039 1788570151.618036 mapwrite 0x02ae30 thru 0x03739f (0xc570 bytes) 000043 1788570151.723847 read 0x01e69b thru 0x01e6ec (0x52 bytes) 000044 1788570151.728792 mapwrite 0x014a3f thru 0x016d5d (0x231f bytes) 000046 1788570151.779523 write 0x01a239 thru 0x028b96 (0xe95e bytes) 000047 1788570151.781121 mapwrite 0x0123b4 thru 0x01bdf1 (0x9a3e bytes) 000048 1788570151.840764 mapwrite 0x03c47c thru 0x03ffff (0x3b84 bytes) 000050 1788570151.911496 read 0x02f688 thru 0x030b98 (0x1511 bytes) 000054 1788570151.916010 mapwrite 0x01e387 thru 0x023d5d (0x59d7 bytes) 000055 1788570151.962015 mapread 0x00a40c thru 0x01602d (0xbc22 bytes) 000057 1788570151.970776 write 0x03aaaa thru 0x03ddcd (0x3324 bytes) 000063 1788570151.976118 write 0x03997b thru 0x03ffff (0x6685 bytes) 000064 1788570151.978448 mapwrite 0x014dea thru 0x0225d3 (0xd7ea bytes) 000065 1788570152.039807 write 0x01e32c thru 0x02dbc1 (0xf896 bytes) 000067 1788570152.041547 mapread 0x02e9e4 thru 0x03af20 (0xc53d bytes) 000070 1788570152.043088 mapwrite 0x02c474 thru 0x030956 (0x44e3 bytes) 000071 1788570152.095702 read 0x007c96 thru 0x00c65c (0x49c7 bytes) 000072 1788570152.096782 read 0x0167d0 thru 0x0232ba (0xcaeb bytes) 000073 1788570152.098063 mapwrite 0x01baf2 thru 0x0262eb (0xa7fa bytes) 000074 1788570152.141018 mapwrite 0x01ab4a thru 0x020ca5 (0x615c bytes) 000076 1788570152.180401 read 0x000022 thru 0x00c7ad (0xc78c bytes) 000077 1788570152.181094 mapread 0x02d6c0 thru 0x03603c (0x897d bytes) 000078 1788570152.183051 mapread 0x00321a thru 0x006fcf (0x3db6 bytes) 000081 1788570152.186410 read 0x01a410 thru 0x0220e2 (0x7cd3 bytes) 000084 1788570152.189068 write 0x03f8c1 thru 0x03ffff (0x73f bytes) 000088 1788570152.193314 mapwrite 0x00ca65 thru 0x014c6d (0x8209 bytes) 000089 1788570152.239968 mapread 0x01a28b thru 0x01abdb (0x951 bytes) 000090 1788570152.248739 MIRROR_EXTEND to 2 mirrors 000091 1788570153.114137 mapwrite 0x014168 thru 0x01a70a (0x65a3 bytes) 000092 1788570153.295128 read 0x02dae5 thru 0x03103b (0x3557 bytes) 000100 1788570153.315613 mapread 0x033170 thru 0x03ffff (0xce90 bytes) 000102 1788570153.335817 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xc5812743 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xfceb83cd Mirror 3: 0xfceb83cd lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000104 1788570154.389922 mapread 0x00daa9 thru 0x017a02 (0x9f5a bytes) 000105 1788570154.435694 mapwrite 0x02d884 thru 0x02ed59 (0x14d6 bytes) 000106 1788570154.732339 write 0x029af3 thru 0x02d89a (0x3da8 bytes) 000108 1788570154.742130 MIRROR_RESYNC 3 mirrors 000110 1788570155.864458 mapwrite 0x00d808 thru 0x00f8bb (0x20b4 bytes) 000111 1788570156.017353 write 0x0366d5 thru 0x03a475 (0x3da1 bytes) 000112 1788570156.033391 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xbedb31f6 (stale) Mirror 3: 0xbedb31f6 (stale) CRC-32 checksum value for chunk [0, 0x40000): Mirror 1: 0xce8aa4f7 Mirror 4: 0xce8aa4f7 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000115 1788570157.202506 mapwrite 0x0057d1 thru 0x00da25 (0x8255 bytes) 000118 1788570157.410955 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xbedb31f6 (stale) Mirror 4: 0xce8aa4f7 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 2: 0xbedb31f6 (stale) Mirror 4: 0xce8aa4f7 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000120 1788570157.701311 write 0x0166d6 thru 0x01ce1d (0x6748 bytes) 000121 1788570157.712809 mapread 0x008369 thru 0x015326 (0xcfbe bytes) 000122 1788570157.734144 mapwrite 0x02b4ff thru 0x02eac9 (0x35cb bytes) 000125 1788570157.788046 mapwrite 0x00fc47 thru 0x0142d8 (0x4692 bytes) 000126 1788570157.846877 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 4: 0xce8aa4f7 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 4: 0xce8aa4f7 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000127 1788570158.140747 read 0x036fc8 thru 0x03ffff (0x9038 bytes) 000128 1788570158.150215 mapread 0x0393ce thru 0x03ffff (0x6c32 bytes) 000130 1788570158.153135 trunc from 0x040000 to 0x01040f (0x2fbf0 bytes) 000133 1788570158.176051 write 0x01bd7d thru 0x022128 (0x63ac bytes) 000135 1788570158.179035 trunc from 0x022129 to 0x016a46 (0xb6e2 bytes) 000137 1788570158.206664 MIRROR_RESYNC 2 mirrors 000140 1788570158.881586 MIRROR_RESYNC 2 mirrors 000141 1788570159.215295 mapread 0x012e66 thru 0x016a46 (0x3be1 bytes) 000143 1788570159.233735 read 0x011c5a thru 0x0140af (0x2456 bytes) 000145 1788570159.234505 mapwrite 0x02cee3 thru 0x034e20 (0x7f3e bytes) 000146 1788570159.408285 write 0x01cc55 thru 0x025020 (0x83cc bytes) 000147 1788570159.417866 read 0x017585 thru 0x01df50 (0x69cc bytes) 000148 1788570159.429391 mapread 0x006a80 thru 0x00991e (0x2e9f bytes) 000149 1788570159.430846 read 0x0134c1 thru 0x01bb96 (0x86d6 bytes) 000150 1788570159.432071 write 0x033e90 thru 0x03ffff (0xc170 bytes) 000151 1788570159.434427 read 0x035577 thru 0x037f96 (0x2a20 bytes) 000152 1788570159.435559 write 0x0093b7 thru 0x017126 (0xdd70 bytes) 000153 1788570159.437765 read 0x01e23f thru 0x022a17 (0x47d9 bytes) 000154 1788570159.439280 mapwrite 0x0392d5 thru 0x03ffff (0x6d2b bytes) 000156 1788570159.512510 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) 000157 1788570159.645133 read 0x021151 thru 0x030268 (0xf118 bytes) 000159 1788570159.657977 mapread 0x012a89 thru 0x02173b (0xecb3 bytes) 000161 1788570159.684877 trunc from 0x040000 to 0x01ab32 (0x254cd bytes) 000162 1788570159.707760 write 0x03bdf3 thru 0x03fdf3 (0x4001 bytes) 000164 1788570159.711878 mapwrite 0x01befe thru 0x01f59b (0x369e bytes) truncating to largest ever: 0x394b1 000166 1788570159.885960 trunc from 0x03fdf4 to 0x0394b0 (0x6943 bytes) 000167 1788570159.964570 write 0x0335bb thru 0x033d88 (0x7ce bytes) 000168 1788570159.971011 mapread 0x007cc4 thru 0x012a5a (0xad97 bytes) 000170 1788570160.012547 MIRROR_EXTEND to 2 mirrors 000176 1788570161.455336 mapread 0x032c71 thru 0x037f11 (0x52a1 bytes) 000177 1788570161.547579 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) 000178 1788570161.723981 read 0x037b35 thru 0x0394b0 (0x197c bytes) 000179 1788570161.738748 write 0x015bf8 thru 0x01c212 (0x661b bytes) 000180 1788570161.784396 MIRROR_EXTEND to 2 mirrors 000181 1788570162.810061 trunc from 0x0394b1 to 0x00f36f (0x2a141 bytes) 000182 1788570162.881372 read 0x00c75b thru 0x00f36f (0x2c15 bytes) 000183 1788570162.890902 MIRROR_RESYNC 2 mirrors 000184 1788570163.979507 mapwrite 0x01b4d2 thru 0x02188b (0x63ba bytes) 000185 1788570164.151805 write 0x000056 thru 0x008c5d (0x8c08 bytes) 000186 1788570164.163522 read 0x016ed3 thru 0x02188b (0xa9b9 bytes) 000187 1788570164.164399 trunc from 0x02188c to 0x02c30f (0xaa84 bytes) 000188 1788570164.193722 write 0x02cef0 thru 0x034aab (0x7bbc bytes) 000189 1788570164.208354 read 0x02a69e thru 0x03477a (0xa0dd bytes) 000191 1788570164.231929 read 0x0232c1 thru 0x02660c (0x334c bytes) 000192 1788570164.232792 mapwrite 0x021914 thru 0x026332 (0x4a1f bytes) 000194 1788570164.290811 mapread 0x02e5a4 thru 0x034aab (0x6508 bytes) 000196 1788570164.293927 read 0x005094 thru 0x0110d2 (0xc03f bytes) 000199 1788570164.296613 write 0x034cf0 thru 0x03ffff (0xb310 bytes) 000202 1788570164.311850 mapread 0x0001d8 thru 0x00091a (0x743 bytes) 000204 1788570164.314613 write 0x02d3a2 thru 0x03bd06 (0xe965 bytes) 000206 1788570164.317936 write 0x00ac33 thru 0x014e09 (0xa1d7 bytes) 000207 1788570164.324536 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) 000209 1788570164.473285 mapread 0x016721 thru 0x01946b (0x2d4b bytes) 000210 1788570164.503088 trunc from 0x040000 to 0x018ecf (0x27130 bytes) 000211 1788570164.525826 mapread 0x003499 thru 0x012735 (0xf29d bytes) 000212 1788570164.539884 trunc from 0x018ed0 to 0x029cbc (0x10ded bytes) 000216 1788570164.564275 trunc from 0x029cbd to 0x038ef6 (0xf23a bytes) 000217 1788570164.585238 mapread 0x023039 thru 0x02393b (0x903 bytes) 000219 1788570164.596757 mapread 0x00577f thru 0x008392 (0x2c14 bytes) 000220 1788570164.599291 write 0x028630 thru 0x02ed13 (0x66e4 bytes) 000221 1788570164.606214 mapread 0x0086d3 thru 0x0169ef (0xe31d bytes) 000223 1788570164.608249 mapwrite 0x031b5e thru 0x03ffff (0xe4a2 bytes) 000224 1788570164.738836 mapwrite 0x02a8f0 thru 0x034181 (0x9892 bytes) 000226 1788570164.874695 read 0x030c38 thru 0x0368b0 (0x5c79 bytes) 000231 1788570164.903500 mapwrite 0x016979 thru 0x01d71b (0x6da3 bytes) 000233 1788570165.068344 mapread 0x01341b thru 0x01c67c (0x9262 bytes) 000234 1788570165.069413 write 0x00e08a thru 0x00e931 (0x8a8 bytes) 000235 1788570165.070815 read 0x011aea thru 0x01c024 (0xa53b bytes) 000240 1788570165.073627 write 0x03b43e thru 0x03e849 (0x340c bytes) 000241 1788570165.086276 read 0x026a6f thru 0x036485 (0xfa17 bytes) 000242 1788570165.087071 write 0x020255 thru 0x02b426 (0xb1d2 bytes) 000245 1788570165.098136 mapwrite 0x010e6e thru 0x0120ed (0x1280 bytes) 000248 1788570165.257692 mapread 0x00b143 thru 0x01507c (0x9f3a bytes) 000249 1788570165.270065 write 0x012956 thru 0x018d07 (0x63b2 bytes) 000250 1788570165.272454 read 0x018a69 thru 0x02270a (0x9ca2 bytes) 000251 1788570165.273358 trunc from 0x040000 to 0x008db1 (0x3724e bytes) 000252 1788570165.297957 write 0x00f891 thru 0x0109b7 (0x1127 bytes) 000253 1788570165.299062 read 0x00ea9e thru 0x0109b7 (0x1f1a bytes) 000257 1788570165.321701 mapwrite 0x00cb6d thru 0x014c0d (0x80a1 bytes) 000259 1788570165.539751 trunc from 0x014c0e to 0x038e39 (0x2422c bytes) 000262 1788570165.565462 write 0x029a8d thru 0x033da7 (0xa31b bytes) 000265 1788570165.596829 MIRROR_EXTEND to 2 mirrors 000266 1788570167.182029 trunc from 0x038e3a to 0x00c3c0 (0x2ca79 bytes) 000268 1788570167.314720 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) 000271 1788570167.507983 mapread 0x003c59 thru 0x009d61 (0x6109 bytes) 000273 1788570167.542617 mapwrite 0x0245ac thru 0x02b270 (0x6cc5 bytes) 000275 1788570167.632715 mapread 0x020ff6 thru 0x02b270 (0xa27b bytes) 000276 1788570167.635585 mapwrite 0x03d388 thru 0x03ffff (0x2c78 bytes) 000278 1788570167.720437 mapread 0x030122 thru 0x036dcb (0x6caa bytes) 000279 1788570167.731428 read 0x03e7d2 thru 0x03ffff (0x182e bytes) 000280 1788570167.732040 write 0x00f4c7 thru 0x018b66 (0x96a0 bytes) 000286 1788570167.751706 mapwrite 0x000e8a thru 0x00dbd6 (0xcd4d bytes) 000287 1788570167.839903 trunc from 0x040000 to 0x00aeaa (0x35155 bytes) 000288 1788570167.869273 write 0x014d39 thru 0x0182c8 (0x3590 bytes) 000289 1788570167.870508 write 0x0095d8 thru 0x011d06 (0x872f bytes) 000291 1788570167.886843 read 0x000ac4 thru 0x00b230 (0xa76d bytes) 000293 1788570167.889834 read 0x015ea1 thru 0x0182c8 (0x2428 bytes) 000294 1788570167.892330 mapwrite 0x01a312 thru 0x01ee58 (0x4b47 bytes) 000296 1788570168.025532 mapread 0x01a990 thru 0x01b5f1 (0xc62 bytes) 000297 1788570168.026553 mapwrite 0x01bae8 thru 0x02a916 (0xee2f bytes) 000301 1788570168.141784 write 0x023292 thru 0x02604c (0x2dbb bytes) 000302 1788570168.144313 write 0x0395ef thru 0x03ffff (0x6a11 bytes) 000303 1788570168.146634 trunc from 0x040000 to 0x023379 (0x1cc86 bytes) 000305 1788570168.181771 trunc from 0x02337a to 0x0114f7 (0x11e82 bytes) 000306 1788570168.206617 mapwrite 0x0050dc thru 0x011daa (0xcccf bytes) 000307 1788570168.304984 mapwrite 0x01c9e0 thru 0x01dbef (0x1210 bytes) 000308 1788570168.399110 MIRROR_EXTEND to 2 mirrors 000310 1788570169.580390 MIRROR_EXTEND to 3 mirrors 000311 1788570170.332477 write 0x018fa0 thru 0x0200b9 (0x711a bytes) 000314 1788570170.430212 trunc from 0x0200ba to 0x005685 (0x1aa34 bytes) 000315 1788570170.450826 mapread 0x004839 thru 0x005685 (0xe4d bytes) 000317 1788570170.469484 mapwrite 0x02b504 thru 0x031c6c (0x6769 bytes) 000321 1788570170.549577 trunc from 0x031c6d to 0x014a6e (0x1d1fe bytes) 000322 1788570170.567698 write 0x029eb4 thru 0x0384ba (0xe607 bytes) 000323 1788570170.570135 read 0x0052f1 thru 0x00bd10 (0x6a20 bytes) 000327 1788570170.585814 write 0x02f5aa thru 0x03244d (0x2ea4 bytes) 000328 1788570170.587444 mapwrite 0x033ac6 thru 0x0393a6 (0x58e1 bytes) 000330 1788570170.654501 write 0x001fbf thru 0x007904 (0x5946 bytes) 000331 1788570170.663109 write 0x00c7fe thru 0x01c6f2 (0xfef5 bytes) 000332 1788570170.670901 trunc from 0x0393a7 to 0x038517 (0xe8f bytes) 000334 1788570170.694230 mapwrite 0x002271 thru 0x00fe0e (0xdb9e bytes) 000335 1788570170.820070 mapwrite 0x030a81 thru 0x03640c (0x598c bytes) 000336 1788570170.893435 mapread 0x028492 thru 0x031a4b (0x95ba bytes) 000338 1788570170.902569 read 0x02ab80 thru 0x02f711 (0x4b92 bytes) 000339 1788570170.903274 mapwrite 0x0332af thru 0x03b066 (0x7db8 bytes) 000340 1788570170.975307 mapwrite 0x0177e2 thru 0x01bb88 (0x43a7 bytes) 000341 1788570171.013879 read 0x02123d thru 0x028482 (0x7246 bytes) 000343 1788570171.015255 read 0x005eae thru 0x00ccca (0x6e1d bytes) 000344 1788570171.015885 trunc from 0x03b067 to 0x033d7a (0x72ec bytes) 000345 1788570171.031923 trunc from 0x033d7b to 0x02135a (0x12a20 bytes) 000348 1788570171.049371 read 0x019b62 thru 0x01e9a2 (0x4e41 bytes) 000349 1788570171.050076 mapread 0x0126e6 thru 0x01ccca (0xa5e5 bytes) 000352 1788570171.054542 mapwrite 0x03eb51 thru 0x03ffff (0x14af bytes) 000353 1788570171.134125 mapread 0x01359d thru 0x021709 (0xe16d bytes) 000354 1788570171.135454 write 0x02312c thru 0x03268c (0xf561 bytes) 000356 1788570171.138960 write 0x021877 thru 0x02718d (0x5917 bytes) 000359 1788570171.141796 write 0x013616 thru 0x017ea6 (0x4891 bytes) 000361 1788570171.143873 read 0x012bd0 thru 0x01e30c (0xb73d bytes) 000362 1788570171.144740 write 0x01be79 thru 0x028538 (0xc6c0 bytes) 000363 1788570171.146824 mapread 0x029507 thru 0x02ff65 (0x6a5f bytes) 000364 1788570171.148627 write 0x01e3fd thru 0x01e890 (0x494 bytes) 000365 1788570171.149488 read 0x0198b3 thru 0x0210b3 (0x7801 bytes) 000366 1788570171.150683 read 0x0264d1 thru 0x0265e0 (0x110 bytes) 000368 1788570171.152434 write 0x025ccd thru 0x02dbe7 (0x7f1b bytes) 000370 1788570171.154310 mapwrite 0x02c615 thru 0x02f847 (0x3233 bytes) 000371 1788570171.219592 mapwrite 0x034dc5 thru 0x03d1f3 (0x842f bytes) truncating to largest ever: 0x3d4e8 000372 1788570171.265572 trunc from 0x040000 to 0x03d4e7 (0x2b18 bytes) 000373 1788570171.280296 write 0x0220e8 thru 0x02c1e8 (0xa101 bytes) 000375 1788570171.292484 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x3d4e8): Mirror 2: 0xe299cfae (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3d4e8): Mirror 2: 0xe299cfae (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3d4e8: skipped 000376 1788570171.571841 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3d4e8): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3d4e8): Mirror 2: 0xe299cfae (stale) CRC-32 checksum value for chunk [0, 0x3d4e8): Mirror 1: 0xc0713529 Mirror 3: 0xc0713529 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3d4e8: skipped 000377 1788570172.502849 mapwrite 0x006272 thru 0x01248b (0xc21a bytes) 000379 1788570172.698788 MIRROR_RESYNC 3 mirrors 000382 1788570174.167744 mapwrite 0x01d0ed thru 0x0218f4 (0x4808 bytes) 000384 1788570174.485579 trunc from 0x03d4e8 to 0x023d76 (0x19771 bytes) 000385 1788570174.540748 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x23d77): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x23d77): Mirror 2: 0xde4228e4 (stale) Mirror 3: 0xde4228e4 (stale) CRC-32 checksum value for chunk [0, 0x23d77): Mirror 1: 0x73d617ed Mirror 4: 0x73d617ed lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x23d77: skipped 000387 1788570175.722677 read 0x0231ca thru 0x023d76 (0xbad bytes) 000388 1788570175.737420 write 0x003f77 thru 0x013c12 (0xfc9c bytes) 000389 1788570175.820946 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x23d77): Mirror 3: 0xde4228e4 (stale) Mirror 4: 0x73d617ed (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x23d77): Mirror 3: 0xde4228e4 (stale) Mirror 4: 0x73d617ed (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x23d77: skipped 000390 1788570176.120589 write 0x00ec40 thru 0x0157f8 (0x6bb9 bytes) 000391 1788570176.135655 mapwrite 0x0245f4 thru 0x025a87 (0x1494 bytes) 000392 1788570176.211766 mapwrite 0x01a9db thru 0x021415 (0x6a3b bytes) 000395 1788570176.267337 mapread 0x01cac9 thru 0x02520e (0x8746 bytes) 000396 1788570176.273622 trunc from 0x025a88 to 0x029656 (0x3bcf bytes) 000397 1788570176.288320 trunc from 0x029657 to 0x0085a1 (0x210b5 bytes) 000399 1788570176.305975 write 0x03d539 thru 0x03ffff (0x2ac7 bytes) 000401 1788570176.312442 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 4: 0x31a3f6c8 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 4: 0x31a3f6c8 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000402 1788570176.561911 MIRROR_RESYNC 2 mirrors 000403 1788570177.235012 read 0x03bfbf thru 0x03ffff (0x4041 bytes) 000404 1788570177.243321 write 0x036111 thru 0x03ad52 (0x4c42 bytes) 000405 1788570177.304560 read 0x031132 thru 0x03a0a7 (0x8f76 bytes) 000406 1788570177.323442 write 0x01c4ad thru 0x024ecc (0x8a20 bytes) 000407 1788570177.324687 trunc from 0x040000 to 0x021689 (0x1e976 bytes) 000408 1788570177.343255 write 0x00d2d2 thru 0x00f1bc (0x1eeb bytes) 000409 1788570177.344320 read 0x01fa4c thru 0x021689 (0x1c3e bytes) 000411 1788570177.345533 trunc from 0x02168a to 0x0111b9 (0x104d0 bytes) 000412 1788570177.373329 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) 000415 1788570177.513354 mapwrite 0x0392a7 thru 0x03ffff (0x6d59 bytes) 000418 1788570177.586915 trunc from 0x040000 to 0x018545 (0x27aba bytes) 000422 1788570177.606959 write 0x024970 thru 0x0331c5 (0xe856 bytes) 000423 1788570177.609656 write 0x01386a thru 0x01ce00 (0x9597 bytes) 000427 1788570177.621068 write 0x03b31c thru 0x03bb2a (0x80f bytes) 000428 1788570177.622481 write 0x02161b thru 0x023c6b (0x2651 bytes) 000429 1788570177.631217 mapwrite 0x03bb66 thru 0x03ffff (0x449a bytes) 000430 1788570177.708602 mapwrite 0x002bde thru 0x005c81 (0x30a4 bytes) 000431 1788570177.806126 trunc from 0x040000 to 0x03264d (0xd9b2 bytes) 000433 1788570177.834495 mapwrite 0x0354bd thru 0x03fb08 (0xa64c bytes) 000434 1788570177.922478 read 0x002615 thru 0x010814 (0xe200 bytes) 000435 1788570177.923660 mapread 0x02e942 thru 0x030882 (0x1f41 bytes) 000438 1788570177.926607 read 0x025877 thru 0x0263f2 (0xb7c bytes) 000439 1788570177.928088 write 0x032a3d thru 0x03e277 (0xb83b bytes) 000440 1788570177.930009 mapread 0x00a248 thru 0x0170b3 (0xce6c bytes) 000441 1788570177.939643 trunc from 0x03fb09 to 0x00b9b5 (0x34153 bytes) 000442 1788570177.954735 mapwrite 0x018cd7 thru 0x01daee (0x4e18 bytes) 000444 1788570178.065894 MIRROR_EXTEND to 2 mirrors 000448 1788570179.254726 trunc from 0x01daef to 0x023699 (0x5bab bytes) 000449 1788570179.342800 read 0x0158e1 thru 0x01b9a1 (0x60c1 bytes) 000450 1788570179.349671 trunc from 0x02369a to 0x01610a (0xd58f bytes) 000452 1788570179.363623 mapwrite 0x01d926 thru 0x02a66f (0xcd4a bytes) 000453 1788570179.504397 read 0x003e51 thru 0x00af58 (0x7108 bytes) 000454 1788570179.524798 mapwrite 0x03c59e thru 0x03ce92 (0x8f5 bytes) 000460 1788570179.664973 mapread 0x03a3aa thru 0x03ce92 (0x2ae9 bytes) 000462 1788570179.666238 write 0x01faf2 thru 0x027919 (0x7e28 bytes) 000463 1788570179.668202 read 0x011e92 thru 0x018e37 (0x6fa6 bytes) 000466 1788570179.673836 read 0x029e65 thru 0x036470 (0xc60c bytes) 000467 1788570179.675131 mapread 0x0239b7 thru 0x02bb5a (0x81a4 bytes) 000468 1788570179.676100 mapwrite 0x020dd1 thru 0x02f7b6 (0xe9e6 bytes) 000469 1788570179.812843 mapwrite 0x02d80b thru 0x0333fd (0x5bf3 bytes) 000471 1788570179.962229 mapread 0x02ad6f thru 0x035683 (0xa915 bytes) 000472 1788570179.976033 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) truncating to largest ever: 0x3e75f 000474 1788570180.247238 trunc from 0x03ce93 to 0x03e75e (0x18cc bytes) 000475 1788570180.280291 trunc from 0x03e75f to 0x02a360 (0x143fe bytes) 000476 1788570180.319139 write 0x022eea thru 0x02ef16 (0xc02d bytes) 000477 1788570180.328130 mapread 0x00113b thru 0x0072d8 (0x619e bytes) 000479 1788570180.372553 write 0x02c715 thru 0x02ed24 (0x2610 bytes) 000480 1788570180.377762 read 0x023e6a thru 0x028d90 (0x4f27 bytes) 000484 1788570180.385649 read 0x0059ab thru 0x008744 (0x2d9a bytes) 000485 1788570180.387848 write 0x033d87 thru 0x03ffff (0xc279 bytes) 000487 1788570180.397440 read 0x02d04e thru 0x038437 (0xb3ea bytes) 000489 1788570180.413120 write 0x035364 thru 0x03ffff (0xac9c bytes) 000490 1788570180.414453 read 0x0016ac thru 0x009344 (0x7c99 bytes) 000492 1788570180.423417 write 0x01c799 thru 0x026f87 (0xa7ef bytes) 000494 1788570180.429798 mapread 0x0291b1 thru 0x02986e (0x6be bytes) 000495 1788570180.430800 mapwrite 0x03aeb6 thru 0x03ffff (0x514a bytes) 000499 1788570180.591129 mapread 0x03b8aa thru 0x03ffff (0x4756 bytes) 000501 1788570180.597643 write 0x007007 thru 0x010e71 (0x9e6b bytes) 000504 1788570180.609463 mapread 0x00cb58 thru 0x012ea6 (0x634f bytes) 000505 1788570180.616484 trunc from 0x040000 to 0x005e72 (0x3a18d bytes) 000506 1788570180.659342 write 0x029925 thru 0x034490 (0xab6c bytes) 000507 1788570180.660913 read 0x00f329 thru 0x013ef0 (0x4bc8 bytes) 000509 1788570180.680031 write 0x00d57b thru 0x012b87 (0x560d bytes) 000517 1788570180.688428 mapread 0x024fbc thru 0x03440b (0xf450 bytes) 000518 1788570180.690452 mapwrite 0x034a53 thru 0x03ffff (0xb5ad bytes) 000519 1788570180.836444 trunc from 0x040000 to 0x0227ee (0x1d811 bytes) 000520 1788570180.862476 mapread 0x0204ca thru 0x0227ee (0x2325 bytes) 000521 1788570180.864083 mapwrite 0x01eac6 thru 0x02c92d (0xde68 bytes) 000522 1788570181.251204 mapread 0x004c51 thru 0x00c043 (0x73f3 bytes) 000525 1788570181.261576 mapread 0x0027a9 thru 0x0092aa (0x6b02 bytes) 000527 1788570181.267064 mapread 0x00bfac thru 0x011a53 (0x5aa8 bytes) 000528 1788570181.268954 write 0x03e30b thru 0x03ffff (0x1cf5 bytes) 000529 1788570181.276557 trunc from 0x040000 to 0x02b3a5 (0x14c5a bytes) 000530 1788570181.306632 mapread 0x00dc56 thru 0x011add (0x3e88 bytes) 000533 1788570181.311393 write 0x03fc80 thru 0x03ffff (0x380 bytes) 000535 1788570181.313600 trunc from 0x040000 to 0x00e0c5 (0x31f3a bytes) 000536 1788570181.333294 read 0x006250 thru 0x00e0c5 (0x7e76 bytes) 000538 1788570181.337282 trunc from 0x00e0c6 to 0x0111b5 (0x30f0 bytes) 000539 1788570181.354556 mapread 0x003cb6 thru 0x004481 (0x7cc bytes) 000540 1788570181.355845 trunc from 0x0111b6 to 0x030751 (0x1f59c bytes) 000543 1788570181.385965 mapwrite 0x02c7e9 thru 0x035310 (0x8b28 bytes) 000544 1788570181.473025 read 0x015afd thru 0x01fcef (0xa1f3 bytes) 000545 1788570181.474304 mapread 0x01912b thru 0x0221bd (0x9093 bytes) 000546 1788570181.476598 write 0x03466c thru 0x03ffff (0xb994 bytes) 000547 1788570181.478981 read 0x0017e6 thru 0x011240 (0xfa5b bytes) 000549 1788570181.481486 write 0x03cf2e thru 0x03ffff (0x30d2 bytes) truncating to largest ever: 0x3f4f2 000552 1788570181.485818 trunc from 0x040000 to 0x03f4f1 (0xb0e bytes) 000553 1788570181.503601 read 0x03c11b thru 0x03f4f1 (0x33d7 bytes) 000557 1788570181.517724 MIRROR_EXTEND to 2 mirrors 000558 1788570182.833443 read 0x01a058 thru 0x0293d2 (0xf37b bytes) 000559 1788570182.845622 mapwrite 0x00fe0a thru 0x0164f7 (0x66ee bytes) 000561 1788570183.084424 trunc from 0x03f4f2 to 0x018625 (0x26ecc bytes) 000564 1788570183.108945 read 0x00fe7b thru 0x018625 (0x87ab bytes) 000567 1788570183.111561 read 0x016a34 thru 0x018625 (0x1bf2 bytes) 000568 1788570183.112629 write 0x01753a thru 0x01c557 (0x501e bytes) 000569 1788570183.114826 write 0x01b840 thru 0x01f669 (0x3e2a bytes) 000572 1788570183.119079 mapread 0x00b216 thru 0x00d1ea (0x1fd5 bytes) 000573 1788570183.120966 write 0x029373 thru 0x02fcaf (0x693d bytes) 000574 1788570183.122981 read 0x024310 thru 0x02eadb (0xa7cc bytes) 000575 1788570183.149283 MIRROR_RESYNC 2 mirrors 000576 1788570184.474788 mapread 0x0232c4 thru 0x02fcaf (0xc9ec bytes) 000577 1788570184.497190 mapread 0x014a34 thru 0x01c871 (0x7e3e bytes) 000579 1788570184.507033 mapwrite 0x02ea5a thru 0x039eba (0xb461 bytes) 000580 1788570184.648651 mapwrite 0x0297d0 thru 0x034fe0 (0xb811 bytes) 000585 1788570184.762478 read 0x00a6aa thru 0x011d33 (0x768a bytes) 000586 1788570184.803485 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) 000591 1788570184.975839 read 0x00f4a1 thru 0x0133aa (0x3f0a bytes) 000592 1788570184.987451 mapread 0x02bffb thru 0x037695 (0xb69b bytes) 000598 1788570185.015295 trunc from 0x039ebb to 0x0194b9 (0x20a01 bytes) 000604 1788570185.045605 trunc from 0x0194ba to 0x02164d (0x8194 bytes) 000605 1788570185.072207 mapread 0x011efd thru 0x01bf32 (0xa036 bytes) 000606 1788570185.101048 mapwrite 0x01b1b7 thru 0x029ce0 (0xeb2a bytes) 000607 1788570185.231376 mapwrite 0x0281a5 thru 0x030ab5 (0x8911 bytes) 000609 1788570185.393507 read 0x02e7aa thru 0x030ab5 (0x230c bytes) 000610 1788570185.398195 MIRROR_EXTEND to 2 mirrors 000611 1788570186.555042 trunc from 0x030ab6 to 0x00cfe5 (0x23ad0 bytes) 000612 1788570186.774519 mapwrite 0x02abb0 thru 0x031f03 (0x7354 bytes) 000615 1788570186.951337 read 0x014737 thru 0x01c729 (0x7ff3 bytes) 000616 1788570186.967854 mapread 0x02f73e thru 0x02fd34 (0x5f7 bytes) 000618 1788570186.973331 read 0x02be6d thru 0x02dc21 (0x1db5 bytes) 000619 1788570186.976301 trunc from 0x031f04 to 0x00ac12 (0x272f1 bytes) 000620 1788570187.005130 write 0x006b1c thru 0x0138bb (0xcda0 bytes) 000621 1788570187.022099 trunc from 0x0138bc to 0x03005c (0x1c7a1 bytes) 000622 1788570187.039583 mapread 0x003a3a thru 0x013103 (0xf6ca bytes) 000625 1788570187.043763 trunc from 0x03005d to 0x00f98c (0x206d0 bytes) 000626 1788570187.060334 read 0x009d2e thru 0x00f98c (0x5c5f bytes) 000630 1788570187.068336 trunc from 0x00f98d to 0x00421a (0xb772 bytes) 000634 1788570187.097116 MIRROR_RESYNC 2 mirrors 000635 1788570187.902399 write 0x004e87 thru 0x00d050 (0x81ca bytes) 000636 1788570188.013963 read 0x005266 thru 0x00d050 (0x7deb bytes) 000637 1788570188.014651 mapwrite 0x0047fb thru 0x00dedb (0x96e1 bytes) 000643 1788570188.111552 write 0x0145e5 thru 0x0209e9 (0xc405 bytes) 000645 1788570188.120471 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x209ea): Mirror 2: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x209ea): Mirror 2: 0x10ecb93a (stale) CRC-32 checksum value for chunk [0, 0x209ea): Mirror 1: 0x1b01182 Mirror 3: 0x1b01182 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x209ea: skipped 000647 1788570189.202255 mapwrite 0x038986 thru 0x03ffff (0x767a bytes) 000648 1788570189.351705 write 0x00e495 thru 0x019358 (0xaec4 bytes) 000650 1788570189.361527 mapread 0x036345 thru 0x03a54f (0x420b bytes) 000651 1788570189.363490 write 0x028f68 thru 0x02acae (0x1d47 bytes) 000652 1788570189.373041 mapwrite 0x0107dd thru 0x015a04 (0x5228 bytes) 000653 1788570189.460422 MIRROR_RESYNC 3 mirrors 000654 1788570191.049848 mapread 0x02590a thru 0x0300bd (0xa7b4 bytes) 000655 1788570191.099616 mapread 0x026cf9 thru 0x02aa82 (0x3d8a bytes) 000656 1788570191.100524 write 0x00af78 thru 0x01ad47 (0xfdd0 bytes) 000657 1788570191.248040 mapwrite 0x039fdc thru 0x03ffff (0x6024 bytes) 000659 1788570191.379032 trunc from 0x040000 to 0x02dd8b (0x12274 bytes) 000660 1788570191.428111 mapread 0x00e2fe thru 0x00f6fe (0x1401 bytes) 000661 1788570191.437334 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x2dd8c): Mirror 2: (stale - read error) Mirror 3: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2dd8c): Mirror 2: 0xee760bdd (stale) Mirror 3: 0xee760bdd (stale) CRC-32 checksum value for chunk [0, 0x2dd8c): Mirror 1: 0x934e4d5f Mirror 4: 0x934e4d5f lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2dd8c: skipped 000662 1788570192.866698 MIRROR_RESYNC 4 mirrors 000663 1788570193.915697 mapread 0x006a4e thru 0x00f757 (0x8d0a bytes) 000666 1788570193.959871 write 0x01e77e thru 0x01ea07 (0x28a bytes) 000667 1788570194.112330 write 0x02bc6c thru 0x02e47e (0x2813 bytes) 000668 1788570194.119954 mapread 0x01dd02 thru 0x028343 (0xa642 bytes) 000670 1788570194.154828 read 0x004b15 thru 0x010e78 (0xc364 bytes) 000671 1788570194.163669 read 0x025433 thru 0x02e47e (0x904c bytes) 000674 1788570194.167088 mapwrite 0x036547 thru 0x03ffff (0x9ab9 bytes) 000676 1788570194.292455 mapwrite 0x0227d1 thru 0x025c1b (0x344b bytes) 000679 1788570194.351682 mapread 0x022e5b thru 0x02586a (0x2a10 bytes) 000685 1788570194.363070 trunc from 0x040000 to 0x00b473 (0x34b8c bytes) 000687 1788570194.400633 mapread 0x00a5c6 thru 0x00b473 (0xeae bytes) 000690 1788570194.408810 read 0x0027ac thru 0x00b473 (0x8cc8 bytes) 000691 1788570194.420647 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0xb474): Mirror 3: 0x923762ce (stale) Mirror 4: 0x923762ce (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xb474): Mirror 3: 0x923762ce (stale) Mirror 4: 0x923762ce (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xb474: skipped 000694 1788570194.719002 MIRROR_RESYNC 3 mirrors 000699 1788570195.762257 read 0x00176e thru 0x004c83 (0x3516 bytes) 000701 1788570195.771195 mapread 0x006e44 thru 0x00b473 (0x4630 bytes) 000705 1788570195.823360 write 0x032298 thru 0x0389fd (0x6766 bytes) 000706 1788570195.976655 read 0x01ff0d thru 0x0217d1 (0x18c5 bytes) 000707 1788570196.012623 write 0x02421d thru 0x02895f (0x4743 bytes) 000708 1788570196.013831 read 0x035d79 thru 0x0389fd (0x2c85 bytes) 000709 1788570196.014475 mapwrite 0x038340 thru 0x03d11f (0x4de0 bytes) 000710 1788570196.261330 trunc from 0x03d120 to 0x039707 (0x3a18 bytes) 000711 1788570196.297062 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x39708): Mirror 3: (stale - read error) Mirror 4: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x39708): Mirror 3: 0xb6023996 (stale) Mirror 4: 0xb6023996 (stale) CRC-32 checksum value for chunk [0, 0x39708): Mirror 1: 0x771679ec Mirror 5: 0x771679ec lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x39708: skipped 000714 1788570197.335432 read 0x00cf93 thru 0x015e29 (0x8e97 bytes) 000717 1788570197.361981 write 0x0011d3 thru 0x00fe9b (0xecc9 bytes) 000721 1788570197.444555 MIRROR_RESYNC 4 mirrors 000723 1788570199.816307 read 0x029b28 thru 0x036b6e (0xd047 bytes) 000724 1788570199.829485 write 0x01c3bf thru 0x028fa9 (0xcbeb bytes) 000726 1788570199.997145 MIRROR_SPLIT mirror 2 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x39708): Mirror 3: 0xd0fc9531 (stale) Mirror 5: 0xd0fc9531 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x39708): Mirror 3: 0xd0fc9531 (stale) Mirror 5: 0xd0fc9531 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x39708: skipped 000727 1788570200.521134 write 0x003ad3 thru 0x0126bd (0xebeb bytes) 000729 1788570200.542690 mapread 0x0184de thru 0x019dda (0x18fd bytes) 000730 1788570200.575371 read 0x00fbae thru 0x01a6ef (0xab42 bytes) 000731 1788570200.599769 write 0x01d145 thru 0x02b3ca (0xe286 bytes) 000732 1788570200.627080 trunc from 0x039708 to 0x00ca9d (0x2cc6a bytes) 000735 1788570200.675544 write 0x01be2d thru 0x02964e (0xd822 bytes) 000736 1788570200.681312 trunc from 0x02964f to 0x0022e5 (0x27369 bytes) 000737 1788570200.710282 read 0x000879 thru 0x0022e5 (0x1a6d bytes) 000741 1788570200.721898 read 0x0010d3 thru 0x001d30 (0xc5e bytes) 000743 1788570200.724413 write 0x000c61 thru 0x00821d (0x75bd bytes) 000744 1788570200.727147 read 0x006bfe thru 0x00821d (0x1620 bytes) 000745 1788570200.728671 mapread 0x001861 thru 0x0024a2 (0xc42 bytes) 000746 1788570200.734038 read 0x0044a1 thru 0x00821d (0x3d7d bytes) 000747 1788570200.740129 MIRROR_RESYNC 3 mirrors 000749 1788570201.892900 trunc from 0x00821e to 0x03b30d (0x330f0 bytes) 000754 1788570202.022214 mapread 0x03805d thru 0x03b30d (0x32b1 bytes) 000756 1788570202.055762 write 0x01c60c thru 0x026da0 (0xa795 bytes) 000757 1788570202.072028 read 0x010f82 thru 0x01c0d5 (0xb154 bytes) 000758 1788570202.082280 trunc from 0x03b30e to 0x03053d (0xadd0 bytes) 000759 1788570202.106963 trunc from 0x03053e to 0x01ca54 (0x13ae9 bytes) 000762 1788570202.151623 read 0x00a97e thru 0x011904 (0x6f87 bytes) 000763 1788570202.184320 read 0x004fa3 thru 0x009b28 (0x4b86 bytes) 000766 1788570202.186209 mapread 0x0032d5 thru 0x010b18 (0xd844 bytes) 000767 1788570202.192767 mapwrite 0x029c15 thru 0x0373a5 (0xd791 bytes) 000770 1788570202.463079 mapread 0x01b88e thru 0x01f541 (0x3cb4 bytes) 000772 1788570202.469451 mapread 0x011eff thru 0x021b3d (0xfc3f bytes) 000774 1788570202.473260 trunc from 0x0373a6 to 0x02760c (0xfd99 bytes) 000775 1788570202.528575 read 0x012aa7 thru 0x01c36b (0x98c5 bytes) 000776 1788570202.529705 trunc from 0x02760d to 0x01cff2 (0xa61a bytes) 000777 1788570202.555518 mapread 0x017e12 thru 0x01cff2 (0x51e1 bytes) 000780 1788570202.562215 trunc from 0x01cff3 to 0x018c4c (0x43a6 bytes) 000781 1788570202.585738 read 0x00816c thru 0x008c9c (0xb31 bytes) 000782 1788570202.586620 mapwrite 0x0113fd thru 0x018f3b (0x7b3f bytes) 000786 1788570202.739967 write 0x0260ce thru 0x02ac38 (0x4b6b bytes) 000787 1788570202.746501 write 0x016d16 thru 0x017f77 (0x1262 bytes) 000789 1788570202.749549 read 0x00134e thru 0x00d80e (0xc4c1 bytes) 000791 1788570202.752091 read 0x00093b thru 0x00629f (0x5965 bytes) 000793 1788570202.753836 mapwrite 0x00c4b4 thru 0x01058e (0x40db bytes) 000794 1788570202.800423 trunc from 0x02ac39 to 0x01b8b8 (0xf380 bytes) 000795 1788570202.815242 mapwrite 0x005e62 thru 0x007f98 (0x2137 bytes) 000796 1788570202.850057 mapread 0x00264a thru 0x0122cb (0xfc82 bytes) 000798 1788570202.852019 mapread 0x00291f thru 0x006f8d (0x466f bytes) 000799 1788570202.858415 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x1b8b9): Mirror 3: (stale - read error) Mirror 5: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x1b8b9): Mirror 3: 0x70f7288b (stale) Mirror 5: 0x70f7288b (stale) CRC-32 checksum value for chunk [0, 0x1b8b9): Mirror 1: 0xafb47b27 Mirror 6: 0xafb47b27 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1b8b9: skipped 000801 1788570203.791523 mapread 0x007fb0 thru 0x00b072 (0x30c3 bytes) 000807 1788570203.862136 read 0x015e1f thru 0x01ab9b (0x4d7d bytes) 000808 1788570203.885129 mapwrite 0x039463 thru 0x03ffff (0x6b9d bytes) 000813 1788570204.013787 trunc from 0x040000 to 0x02e3db (0x11c24 bytes) 000814 1788570204.032193 mapwrite 0x01283e thru 0x013003 (0x7c6 bytes) 000816 1788570204.092042 mapwrite 0x02ab6b thru 0x02c534 (0x19ca bytes) 000817 1788570204.135346 mapread 0x020cf4 thru 0x022c59 (0x1f66 bytes) 000819 1788570204.149535 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x2e3dc): Mirror 5: 0xa94c9e35 (stale) Mirror 6: 0xd6864af7 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x2e3dc): Mirror 5: 0xa94c9e35 (stale) Mirror 6: 0xd6864af7 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2e3dc: skipped 000821 1788570204.418570 mapwrite 0x03ec3c thru 0x03ffff (0x13c4 bytes) 000822 1788570204.483841 mapread 0x009897 thru 0x01126d (0x79d7 bytes) 000823 1788570204.495147 mapwrite 0x03f0b6 thru 0x03ffff (0xf4a bytes) 000824 1788570204.527958 write 0x00c23c thru 0x01708a (0xae4f bytes) 000827 1788570204.535314 mapread 0x02ccb1 thru 0x031d6b (0x50bb bytes) 000828 1788570204.545310 read 0x03391c thru 0x03ffff (0xc6e4 bytes) 000830 1788570204.553810 mapread 0x036f53 thru 0x03ffff (0x90ad bytes) 000831 1788570204.555033 read 0x0045bc thru 0x007b2d (0x3572 bytes) 000833 1788570204.556954 mapwrite 0x030ed3 thru 0x032809 (0x1937 bytes) 000834 1788570204.601264 mapread 0x01e1a7 thru 0x02bbbd (0xda17 bytes) 000835 1788570204.617619 MIRROR_RESYNC 3 mirrors 000838 1788570205.815095 mapread 0x0018a3 thru 0x00a4c6 (0x8c24 bytes) 000839 1788570205.834498 mapwrite 0x02b798 thru 0x035161 (0x99ca bytes) 000840 1788570205.987722 write 0x03c28e thru 0x03ffff (0x3d72 bytes) 000841 1788570205.990390 mapread 0x02842d thru 0x035d78 (0xd94c bytes) 000843 1788570205.993502 read 0x0077b8 thru 0x009d80 (0x25c9 bytes) 000845 1788570206.000798 mapread 0x00096a thru 0x00b3f0 (0xaa87 bytes) 000846 1788570206.013839 MIRROR_RESYNC 3 mirrors 000848 1788570207.223108 write 0x006c65 thru 0x0107d6 (0x9b72 bytes) 000849 1788570207.320788 MIRROR_RESYNC 3 mirrors 000850 1788570208.268329 write 0x02ab6e thru 0x033510 (0x89a3 bytes) 000852 1788570208.344919 trunc from 0x040000 to 0x0397a7 (0x6858 bytes) 000854 1788570208.374804 mapread 0x008683 thru 0x018534 (0xfeb2 bytes) 000856 1788570208.390543 mapwrite 0x022f40 thru 0x0234cf (0x590 bytes) 000859 1788570208.443817 MIRROR_SPLIT mirror 1 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x397a8): Mirror 6: 0x3482eae0 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x397a8): Mirror 6: 0x3482eae0 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x397a8: skipped 000864 1788570208.734544 mapwrite 0x002299 thru 0x011eac (0xfc14 bytes) 000865 1788570208.809939 read 0x01484e thru 0x024373 (0xfb26 bytes) 000868 1788570208.821989 trunc from 0x0397a8 to 0x03519b (0x460c bytes) 000871 1788570208.842583 mapwrite 0x00081a thru 0x00fe95 (0xf67c bytes) 000873 1788570208.901397 write 0x00f2be thru 0x01c9b8 (0xd6fb bytes) 000874 1788570208.907399 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0x3519c): Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x3519c): Mirror 6: 0x510fad83 (stale) CRC-32 checksum value for chunk [0, 0x3519c): Mirror 1: 0x67a46e4e Mirror 7: 0x67a46e4e lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x3519c: skipped 000875 1788570210.196609 write 0x001b16 thru 0x0108f6 (0xede1 bytes) 000877 1788570210.379651 mapwrite 0x03db24 thru 0x03ffff (0x24dc bytes) 000878 1788570210.486335 write 0x0189d3 thru 0x0249df (0xc00d bytes) 000879 1788570210.499794 mapread 0x005cc0 thru 0x009c10 (0x3f51 bytes) 000880 1788570210.500880 mapread 0x004c8d thru 0x00558c (0x900 bytes) 000881 1788570210.501804 mapread 0x03e3aa thru 0x03ffff (0x1c56 bytes) 000882 1788570210.509789 mapread 0x02fa91 thru 0x03e5d9 (0xeb49 bytes) 000884 1788570210.518414 mapwrite 0x02c7fc thru 0x03778a (0xaf8f bytes) 000885 1788570210.563507 mapread 0x0052ab thru 0x010db4 (0xbb0a bytes) 000887 1788570210.565207 read 0x001ba4 thru 0x005874 (0x3cd1 bytes) 000888 1788570210.565856 write 0x00a340 thru 0x019a4f (0xf710 bytes) 000889 1788570210.572407 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: 0x551252f0 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 6: 0x551252f0 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000890 1788570210.798744 write 0x020ff2 thru 0x0211dd (0x1ec bytes) 000891 1788570210.808416 read 0x02c00c thru 0x02e5d0 (0x25c5 bytes) 000893 1788570210.814129 trunc from 0x040000 to 0x012617 (0x2d9e8 bytes) 000895 1788570210.832476 read 0x001f70 thru 0x009487 (0x7518 bytes) 000896 1788570210.840415 mapwrite 0x028a92 thru 0x037ee8 (0xf457 bytes) 000899 1788570210.910673 mapwrite 0x00266d thru 0x00af2a (0x88be bytes) 000902 1788570210.958790 trunc from 0x037ee9 to 0x0018bb (0x3662d bytes) 000906 1788570210.976234 mapwrite 0x0296e6 thru 0x02cb36 (0x3451 bytes) 000907 1788570211.062752 mapwrite 0x0140ef thru 0x016482 (0x2394 bytes) 000909 1788570211.127046 mapread 0x011000 thru 0x0209be (0xf9bf bytes) 000912 1788570211.183992 trunc from 0x02cb37 to 0x0117c6 (0x1b370 bytes) 000914 1788570211.199392 trunc from 0x0117c7 to 0x000632 (0x11194 bytes) 000916 1788570211.223527 mapread 0x0004c9 thru 0x000632 (0x16a bytes) 000918 1788570211.233105 mapread 0x00047b thru 0x000632 (0x1b8 bytes) 000919 1788570211.235148 trunc from 0x000633 to 0x00c23a (0xbc08 bytes) 000920 1788570211.267995 MIRROR_EXTEND to 3 mirrors CRC-32 checksum value for chunk [0, 0xc23b): Mirror 6: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0xc23b): Mirror 6: 0xd5e40aa1 (stale) CRC-32 checksum value for chunk [0, 0xc23b): Mirror 1: 0xc17cf431 Mirror 7: 0xc17cf431 lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0xc23b: skipped 000922 1788570212.157720 mapwrite 0x02b40d thru 0x038476 (0xd06a bytes) 000924 1788570212.324661 MIRROR_EXTEND to 4 mirrors CRC-32 checksum value for chunk [0, 0x38477): Mirror 6: (stale - read error) Mirror 7: (stale - read error) mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x38477): Mirror 6: 0xf902102 (stale) Mirror 7: 0x4881557 (stale) CRC-32 checksum value for chunk [0, 0x38477): Mirror 1: 0x58bc811a Mirror 8: 0x58bc811a lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x38477: skipped 000926 1788570213.147558 MIRROR_RESYNC 4 mirrors 000927 1788570214.114061 mapread 0x006a49 thru 0x00b505 (0x4abd bytes) 000930 1788570214.149289 MIRROR_SPLIT mirror 1 to 3 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) 000931 1788570214.488754 write 0x01c6b0 thru 0x01c7bb (0x10c bytes) 000932 1788570214.618025 mapread 0x003c41 thru 0x00c0b0 (0x8470 bytes) 000934 1788570214.648068 mapread 0x032257 thru 0x038476 (0x6220 bytes) 000935 1788570214.659220 write 0x0141b5 thru 0x01e8c3 (0xa70f bytes) 000937 1788570214.675931 mapread 0x03666f thru 0x038476 (0x1e08 bytes) 000940 1788570214.677680 trunc from 0x038477 to 0x00edd0 (0x296a6 bytes) 000941 1788570214.726388 mapwrite 0x03f003 thru 0x03ffff (0xffd bytes) 000942 1788570214.812865 MIRROR_SPLIT mirror 2 to 2 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) CRC-32 checksum value for chunk [0, 0x40000): Mirror 7: 0xb150b4b3 (stale) mirror op 1: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 116 CRC-32 checksum value for chunk [0, 0x40000): Mirror 7: 0xb150b4b3 (stale) lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0, 0x100000) is only valid in mirror 1: skipped lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x40000: skipped 000943 1788570215.076420 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) 000944 1788570215.200326 read 0x02820a thru 0x02aec6 (0x2cbd bytes) 000946 1788570215.206190 mapread 0x01c5b4 thru 0x022690 (0x60dd bytes) 000948 1788570215.232762 mapread 0x007ec5 thru 0x00c0c1 (0x41fd bytes) 000949 1788570215.241593 write 0x01522e thru 0x025070 (0xfe43 bytes) 000950 1788570215.247877 mapread 0x039d30 thru 0x03ffff (0x62d0 bytes) 000951 1788570215.259523 MIRROR_EXTEND to 2 mirrors 000952 1788570216.092330 mapread 0x03864d thru 0x03ffff (0x79b3 bytes) 000953 1788570216.150225 mapread 0x01c010 thru 0x021a19 (0x5a0a bytes) 000956 1788570216.215737 read 0x03cf86 thru 0x03ffff (0x307a bytes) 000957 1788570216.233215 write 0x0012e5 thru 0x0060cd (0x4de9 bytes) 000958 1788570216.395069 mapwrite 0x013beb thru 0x016c67 (0x307d bytes) 000959 1788570216.457055 write 0x0256b9 thru 0x0355a4 (0xfeec bytes) 000961 1788570216.468511 mapread 0x03ad42 thru 0x03ffff (0x52be bytes) 000962 1788570216.477039 trunc from 0x040000 to 0x007630 (0x389cf bytes) 000963 1788570216.530456 mapread 0x003d7e thru 0x007630 (0x38b3 bytes) 000969 1788570216.546944 MIRROR_RESYNC 2 mirrors 000971 1788570217.431402 write 0x02ca42 thru 0x032766 (0x5d25 bytes) 000972 1788570217.509691 MIRROR_SPLIT mirror 1 to 1 mirrors lfs mirror mirror: cannot get UNLOCK lease, ext 8 for [0x200000404:0x1d5:0x0]: Invalid argument (22) 000973 1788570217.694738 mapread 0x01224b thru 0x019aeb (0x78a1 bytes) 000974 1788570217.745872 mapread 0x02b5ac thru 0x032766 (0x71bb bytes) 000979 1788570217.772337 trunc from 0x032767 to 0x01708c (0x1b6da bytes) 000981 1788570217.814213 write 0x03a710 thru 0x03ffff (0x58f0 bytes) 000985 1788570217.826478 mapread 0x035abe thru 0x03a7f7 (0x4d3a bytes) 000986 1788570217.840531 trunc from 0x040000 to 0x013a91 (0x2c56e bytes) 000989 1788570217.860485 trunc from 0x013a92 to 0x02710a (0x13679 bytes) 000990 1788570217.896404 mapwrite 0x02d875 thru 0x034044 (0x67d0 bytes) 000991 1788570218.062007 trunc from 0x034045 to 0x03f1d3 (0xb18f bytes) 000994 1788570218.081615 write 0x01956c thru 0x01bf49 (0x29de bytes) 000995 1788570218.094481 mapwrite 0x005c14 thru 0x00b5e3 (0x59d0 bytes) 000996 1788570218.149227 write 0x03dcd9 thru 0x03ffff (0x2327 bytes) 000998 1788570218.155761 mapwrite 0x02ea23 thru 0x039046 (0xa624 bytes) 000999 1788570218.213737 write 0x000825 thru 0x000a65 (0x241 bytes) All operations completed A-OK!