== sanity-flr test 70a: flr mode fsx test ================ 19:19:10 (1746141550) striped dir -i0 -c2 -H crush2 /mnt/lustre/d70a.sanity-flr Seed set to 1551 truncating to largest ever: 0x1fe78 000001 1746141551.494608 trunc from 00000000 to 0x01fe77 (0x1fe78 bytes) 000002 1746141551.509792 write 0x004e8e thru 0x00cc06 (0x7d79 bytes) 000003 1746141551.518525 punch from 0x00eeb0 to 0x01022b (0x137c bytes) 000004 1746141551.522544 fallocate 0x00d663 thru 0x01abff (0xd59d bytes) 000005 1746141551.524710 write 0x01da45 thru 0x01db12 (0xce bytes) 000007 1746141551.530860 punch from 0x01f8a7 to 0x01fe77 (0x5d1 bytes) 000008 1746141551.536327 punch from 0x010633 to 0x01e191 (0xdb5f bytes) 000009 1746141551.547601 write 0x032c85 thru 0x033fce (0x134a bytes) 000011 1746141551.548833 mapread 0x0015bd thru 0x011291 (0xfcd5 bytes) 000012 1746141551.560425 mapwrite 0x031dcf thru 0x0338b2 (0x1ae4 bytes) 000013 1746141551.576662 punch from 0x01cf40 to 0x02c324 (0xf3e5 bytes) 000015 1746141551.579986 read 0x00be1c thru 0x00ec04 (0x2de9 bytes) truncating to largest ever: 0x22a07 000016 1746141551.580480 trunc from 0x033fcf to 0x022a06 (0x115c8 bytes) 000017 1746141551.593548 mapread 0x0163ba thru 0x020e4a (0xaa91 bytes) 000018 1746141551.601883 write 0x00674e thru 0x0161e3 (0xfa96 bytes) 000019 1746141551.603868 mapread 0x01ba8f thru 0x022a06 (0x6f78 bytes) 000020 1746141551.605821 punch from 0x00d586 to 0x014732 (0x71ad bytes) 000021 1746141551.624395 trunc from 0x01d059 to 0x0226a6 (0x564e bytes) 000023 1746141551.628784 mapread 0x016b34 thru 0x0196c4 (0x2b91 bytes) 000024 1746141551.633555 fallocate 0x007685 thru 0x016e1a (0xf796 bytes) 000025 1746141551.636663 fallocate 0x00a161 thru 0x00dadd (0x397d bytes) 000026 1746141551.643140 mapread 0x019c7f thru 0x022a06 (0x8d88 bytes) 000029 1746141551.653461 read 0x003b0b thru 0x012a2f (0xef25 bytes) 000030 1746141551.660036 punch from 0x01cb33 to 0x01d3fc (0x8ca bytes) 000031 1746141551.665581 trunc from 0x00504c to 0x013fcc (0xef81 bytes) 000032 1746141551.675372 punch from 0x0097de to 0x01211d (0x8940 bytes) 000033 1746141551.682024 trunc from 0x022a07 to 0x01ba69 (0x6f9d bytes) fallocating to largest ever: 0x2b26b 000036 1746141551.693618 fallocate 0x01e081 thru 0x02b26a (0xd1ea bytes) 000037 1746141551.701028 MIRROR_EXTEND to 2 mirrors lfs mirror mirror: fail to pread 0-4194304 of mirror 2: Invalid argument (22) lfs verify: failed to read data from mirror 2: Invalid argument. mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 22 lfs mirror mirror: fail to pread 0-4194304 of mirror 1: Invalid argument (22) lfs verify: failed to read data from mirror 1: Invalid argument. lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x1ba6a: skipped LOG DUMP (29 total operations): 1: 1746141551.494608 TRUNC_UP 0x00000 to 0x1fe78 2: 1746141551.509792 WRITE 0x04e8e thru 0x0cc06 (0x07d79 bytes) 3: 1746141551.518525 PUNCH 0x0eeb0 thru 0x1022b (0x0137c bytes) 4: 1746141551.522544 FALLOC 0x0d663 thru 0x1ac00 (0x0d59d bytes)INTERIOR 5: 1746141551.524710 WRITE 0x1da45 thru 0x1db12 (0x000ce bytes) 6: 1746141551.530860 PUNCH 0x1f8a7 thru 0x1fe77 (0x005d1 bytes) 7: 1746141551.536327 PUNCH 0x10633 thru 0x1e191 (0x0db5f bytes) 8: 1746141551.547601 WRITE 0x32c85 thru 0x33fce (0x0134a bytes) HOLE 9: 1746141551.548833 MAPREAD 0x015bd thru 0x11291 (0x0fcd5 bytes) 10: 1746141551.560425 MAPWRITE 0x31dcf thru 0x338b2 (0x01ae4 bytes) 11: 1746141551.576662 PUNCH 0x1cf40 thru 0x2c324 (0x0f3e5 bytes) 12: 1746141551.579986 READ 0x0be1c thru 0x0ec04 (0x02de9 bytes) 13: 1746141551.580480 TRUNC_DN 0x33fcf to 0x22a07 14: 1746141551.593548 MAPREAD 0x163ba thru 0x20e4a (0x0aa91 bytes) 15: 1746141551.601883 WRITE 0x0674e thru 0x161e3 (0x0fa96 bytes) 16: 1746141551.603868 MAPREAD 0x1ba8f thru 0x22a06 (0x06f78 bytes) 17: 1746141551.605821 PUNCH 0x0d586 thru 0x14732 (0x071ad bytes) 18: 1746141551.624395 ZERO 0x1d059 thru 0x226a6 (0x0564e bytes) 19: 1746141551.628784 MAPREAD 0x16b34 thru 0x196c4 (0x02b91 bytes) 20: 1746141551.633555 FALLOC 0x07685 thru 0x16e1b (0x0f796 bytes)INTERIOR 21: 1746141551.636663 FALLOC 0x0a161 thru 0x0dade (0x0397d bytes)INTERIOR 22: 1746141551.643140 MAPREAD 0x19c7f thru 0x22a06 (0x08d88 bytes) 23: 1746141551.653461 READ 0x03b0b thru 0x12a2f (0x0ef25 bytes) 24: 1746141551.660036 PUNCH 0x1cb33 thru 0x1d3fc (0x008ca bytes) 25: 1746141551.665581 ZERO 0x0504c thru 0x13fcc (0x0ef81 bytes) 26: 1746141551.675372 PUNCH 0x097de thru 0x1211d (0x08940 bytes) 27: 1746141551.682024 TRUNC_DN 0x22a07 to 0x1ba6a 28: 1746141551.693618 FALLOC 0x1e081 thru 0x2b26b (0x0d1ea bytes)PAST_EOF 29: 1746141551.701028 MIRROR_EXTEND to 2 mirrors Using seed 1551 Correct content saved for comparison (maybe hexdump "/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr" vs "/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr.fsxgood") sanity-flr test_70a: @@@@@@ FAIL: fsx FLR file /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr failed Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7249:error() = /home/green/git/lustre-release/lustre/tests/sanity-flr.sh:3351:test_70a() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7622:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7685:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7503:run_test() = /home/green/git/lustre-release/lustre/tests/sanity-flr.sh:3353:main() Dumping lctl log to /tmp/testlogs//sanity-flr.test_70a.*.1746141553.log rsync: chown "/tmp/testlogs/.sanity-flr.test_70a.debug_log.oleg117-server.1746141553.log.q2o6dK" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanity-flr.test_70a.dmesg.oleg117-server.1746141553.log.0GroZk" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1651) [generator=3.1.2]