== sanity-flr test 70a: flr mode fsx test ================ 20:38:29 (1746146309) striped dir -i0 -c2 -H crush2 /mnt/lustre/d70a.sanity-flr Seed set to 6310 000001 1746146310.493874 write 0x027270 thru 0x02d203 (0x5f94 bytes) 000002 1746146310.505755 write 0x025c92 thru 0x03365a (0xd9c9 bytes) 000003 1746146310.513292 write 0x0227a1 thru 0x028dea (0x664a bytes) 000004 1746146310.517168 mapwrite 0x034b71 thru 0x03c98f (0x7e1f bytes) truncating to largest ever: 0xb67a 000005 1746146310.603617 trunc from 0x03c990 to 0x00b679 (0x31316 bytes) 000006 1746146310.640762 trunc from 0x004248 to 0x00b679 (0x7432 bytes) 000007 1746146310.643909 read 0x008dc3 thru 0x00944a (0x688 bytes) 000008 1746146310.655581 write 0x021818 thru 0x02344a (0x1c33 bytes) 000009 1746146310.656297 mapread 0x011e4f thru 0x01d80e (0xb9c0 bytes) zero_range to largest ever: 0x21ce0 000010 1746146310.680584 trunc from 0x013b93 to 0x021cdf (0xe14d bytes) 000011 1746146310.695828 trunc from 0x0024df to 0x003d20 (0x1842 bytes) 000012 1746146310.706356 read 0x01f639 thru 0x0232a6 (0x3c6e bytes) 000013 1746146310.716601 mapwrite 0x01391a thru 0x018bc7 (0x52ae bytes) 000014 1746146310.730894 punch from 0x018fdb to 0x02344a (0xa470 bytes) 000015 1746146310.735163 mapread 0x008531 thru 0x013671 (0xb141 bytes) 000016 1746146310.736907 write 0x0338b7 thru 0x03dd66 (0xa4b0 bytes) 000018 1746146310.748431 trunc from 0x03dd67 to 0x001957 (0x3c40f bytes) 000019 1746146310.772233 read 0x0018de thru 0x001957 (0x7a bytes) fallocating to largest ever: 0x40000 000020 1746146310.779495 fallocate 0x039c98 thru 0x03ffff (0x6368 bytes) 000021 1746146310.784135 punch from 0x000f39 to 0x0018af (0x977 bytes) 000022 1746146310.795664 trunc from 0x001958 to 0x005f52 (0x45fb bytes) 000023 1746146310.827258 trunc from 0x005f53 to 0x039bb2 (0x33c60 bytes) 000025 1746146310.842936 read 0x02e1eb thru 0x039bb2 (0xb9c8 bytes) 000026 1746146310.855081 mapread 0x0087f5 thru 0x00a53f (0x1d4b bytes) 000027 1746146310.857582 trunc from 0x023516 to 0x02d5e9 (0xa0d4 bytes) 000029 1746146310.862590 fallocate 0x0120b0 thru 0x01d35a (0xb2ab bytes) 000030 1746146310.871192 write 0x001230 thru 0x0111a3 (0xff74 bytes) 000031 1746146310.874217 read 0x0141ec thru 0x0236ca (0xf4df bytes) 000034 1746146310.891271 MIRROR_EXTEND to 2 mirrors lfs mirror mirror: fail to pread 0-4194304 of mirror 2: Invalid argument (22) lfs verify: failed to read data from mirror 2: Invalid argument. mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 22 lfs mirror mirror: fail to pread 0-4194304 of mirror 1: Invalid argument (22) lfs verify: failed to read data from mirror 1: Invalid argument. lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x39bb3: skipped LOG DUMP (29 total operations): 1: 1746146310.493874 WRITE 0x27270 thru 0x2d203 (0x05f94 bytes) HOLE 2: 1746146310.505755 WRITE 0x25c92 thru 0x3365a (0x0d9c9 bytes) EXTEND 3: 1746146310.513292 WRITE 0x227a1 thru 0x28dea (0x0664a bytes) 4: 1746146310.517168 MAPWRITE 0x34b71 thru 0x3c98f (0x07e1f bytes) 5: 1746146310.603617 TRUNC_DN 0x3c990 to 0x0b67a 6: 1746146310.640762 ZERO 0x04248 thru 0x0b679 (0x07432 bytes) 7: 1746146310.643909 READ 0x08dc3 thru 0x0944a (0x00688 bytes) 8: 1746146310.655581 WRITE 0x21818 thru 0x2344a (0x01c33 bytes) HOLE 9: 1746146310.656297 MAPREAD 0x11e4f thru 0x1d80e (0x0b9c0 bytes) 10: 1746146310.680584 ZERO 0x13b93 thru 0x21cdf (0x0e14d bytes) 11: 1746146310.695828 ZERO 0x024df thru 0x03d20 (0x01842 bytes) 12: 1746146310.706356 READ 0x1f639 thru 0x232a6 (0x03c6e bytes) 13: 1746146310.716601 MAPWRITE 0x1391a thru 0x18bc7 (0x052ae bytes) 14: 1746146310.730894 PUNCH 0x18fdb thru 0x2344a (0x0a470 bytes) 15: 1746146310.735163 MAPREAD 0x08531 thru 0x13671 (0x0b141 bytes) 16: 1746146310.736907 WRITE 0x338b7 thru 0x3dd66 (0x0a4b0 bytes) HOLE 17: 1746146310.748431 TRUNC_DN 0x3dd67 to 0x01958 18: 1746146310.772233 READ 0x018de thru 0x01957 (0x0007a bytes) 19: 1746146310.779495 FALLOC 0x39c98 thru 0x40000 (0x06368 bytes)PAST_EOF 20: 1746146310.784135 PUNCH 0x00f39 thru 0x018af (0x00977 bytes) 21: 1746146310.795664 TRUNC_UP 0x01958 to 0x05f53 22: 1746146310.827258 TRUNC_UP 0x05f53 to 0x39bb3 23: 1746146310.842936 READ 0x2e1eb thru 0x39bb2 (0x0b9c8 bytes) 24: 1746146310.855081 MAPREAD 0x087f5 thru 0x0a53f (0x01d4b bytes) 25: 1746146310.857582 ZERO 0x23516 thru 0x2d5e9 (0x0a0d4 bytes) 26: 1746146310.862590 FALLOC 0x120b0 thru 0x1d35b (0x0b2ab bytes)INTERIOR 27: 1746146310.871192 WRITE 0x01230 thru 0x111a3 (0x0ff74 bytes) 28: 1746146310.874217 READ 0x141ec thru 0x236ca (0x0f4df bytes) 29: 1746146310.891271 MIRROR_EXTEND to 2 mirrors Using seed 6310 Correct content saved for comparison (maybe hexdump "/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr" vs "/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr.fsxgood") sanity-flr test_70a: @@@@@@ FAIL: fsx FLR file /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr failed Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7249:error() = /home/green/git/lustre-release/lustre/tests/sanity-flr.sh:3351:test_70a() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7622:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7685:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7503:run_test() = /home/green/git/lustre-release/lustre/tests/sanity-flr.sh:3353:main() Dumping lctl log to /tmp/testlogs//sanity-flr.test_70a.*.1746146312.log rsync: chown "/tmp/testlogs/.sanity-flr.test_70a.debug_log.oleg121-server.1746146312.log.0cJiN7" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanity-flr.test_70a.dmesg.oleg121-server.1746146312.log.8rFEY4" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1651) [generator=3.1.2]