== sanity-flr test 70a: flr mode fsx test ================ 20:10:00 (1746144600) Seed set to 4601 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002 1746144601.738048 mapwrite 0x01ec67 thru 0x0280f8 (0x9492 bytes) 000003 1746144601.854776 read 0x0126a3 thru 0x017bab (0x5509 bytes) 000005 1746144601.864771 read 0x018810 thru 0x01f5c2 (0x6db3 bytes) 000008 1746144601.871185 write 0x01de6f thru 0x028835 (0xa9c7 bytes) 000013 1746144601.880169 read 0x026744 thru 0x028835 (0x20f2 bytes) truncating to largest ever: 0x1ba78 000014 1746144601.880770 trunc from 0x028836 to 0x01ba77 (0xcdbe bytes) 000015 1746144601.892764 write 0x009403 thru 0x014338 (0xaf36 bytes) 000019 1746144601.896598 write 0x02b493 thru 0x02c924 (0x1492 bytes) 000021 1746144601.900908 mapwrite 0x03603f thru 0x0369c9 (0x98b bytes) 000022 1746144601.981180 trunc from 0x0369ca to 0x009305 (0x2d6c4 bytes) 000023 1746144601.990298 write 0x038a4b thru 0x039123 (0x6d9 bytes) 000024 1746144601.991130 mapwrite 0x031862 thru 0x035b5a (0x42f9 bytes) 000025 1746144602.046591 mapread 0x01ebd4 thru 0x0234d0 (0x48fd bytes) 000026 1746144602.065748 mapwrite 0x000012 thru 0x007c3c (0x7c2b bytes) 000028 1746144602.112195 write 0x03bc1a thru 0x03f1ba (0x35a1 bytes) 000029 1746144602.112822 mapwrite 0x016e61 thru 0x01e03e (0x71de bytes) 000033 1746144602.170023 mapwrite 0x018dc8 thru 0x026c26 (0xde5f bytes) truncating to largest ever: 0x26a67 000034 1746144602.216518 trunc from 0x03f1bb to 0x026a66 (0x18754 bytes) 000035 1746144602.225874 trunc from 0x026a67 to 0x016890 (0x101d6 bytes) 000037 1746144602.235850 mapread 0x00ef61 thru 0x00f104 (0x1a4 bytes) 000038 1746144602.248441 mapwrite 0x03bb91 thru 0x03ffff (0x446f bytes) 000040 1746144602.328826 trunc from 0x040000 to 0x0112c0 (0x2ed3f bytes) 000042 1746144602.340014 write 0x001f40 thru 0x0092d4 (0x7395 bytes) 000043 1746144602.340716 mapread 0x00a913 thru 0x00c293 (0x1981 bytes) 000044 1746144602.351565 write 0x00773f thru 0x012cc4 (0xb586 bytes) 000046 1746144602.353839 mapread 0x0025e8 thru 0x006971 (0x438a bytes) truncating to largest ever: 0x2c2f0 000047 1746144602.356343 trunc from 0x012cc5 to 0x02c2ef (0x1962b bytes) 000049 1746144602.368249 read 0x00b43b thru 0x0133b8 (0x7f7e bytes) 000051 1746144602.379904 mapwrite 0x011909 thru 0x016626 (0x4d1e bytes) 000052 1746144602.447148 MIRROR_EXTEND to 2 mirrors lfs mirror mirror: fail to pread 0-4194304 of mirror 2: Invalid argument (22) lfs verify: failed to read data from mirror 2: Invalid argument. mirror op 0: lfs mirror verify /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr: 22 lfs mirror mirror: fail to pread 0-4194304 of mirror 1: Invalid argument (22) lfs verify: failed to read data from mirror 1: Invalid argument. lfs verify: '/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr' chunk [0x100000, 0xffffffffffffffff) exceeds file size 0x2c2f0: skipped LOG DUMP (47 total operations): 1: 1746144601.735316 SKIPPED (no operation) 2: 1746144601.738048 MAPWRITE 0x1ec67 thru 0x280f8 (0x09492 bytes) 3: 1746144601.854776 READ 0x126a3 thru 0x17bab (0x05509 bytes) 4: 1746144601.864352 SKIPPED (no operation) 5: 1746144601.864771 READ 0x18810 thru 0x1f5c2 (0x06db3 bytes) 6: 1746144601.870543 SKIPPED (no operation) 7: 1746144601.870878 SKIPPED (no operation) 8: 1746144601.871185 WRITE 0x1de6f thru 0x28835 (0x0a9c7 bytes) EXTEND 9: 1746144601.879555 SKIPPED (no operation) 10: 1746144601.879880 SKIPPED (no operation) 11: 1746144601.880169 READ 0x26744 thru 0x28835 (0x020f2 bytes) 12: 1746144601.880770 TRUNC_DN 0x28836 to 0x1ba78 13: 1746144601.892764 WRITE 0x09403 thru 0x14338 (0x0af36 bytes) 14: 1746144601.893653 SKIPPED (no operation) 15: 1746144601.895900 SKIPPED (no operation) 16: 1746144601.896266 SKIPPED (no operation) 17: 1746144601.896598 WRITE 0x2b493 thru 0x2c924 (0x01492 bytes) HOLE 18: 1746144601.900565 SKIPPED (no operation) 19: 1746144601.900908 MAPWRITE 0x3603f thru 0x369c9 (0x0098b bytes) 20: 1746144601.981180 TRUNC_DN 0x369ca to 0x09306 21: 1746144601.990298 WRITE 0x38a4b thru 0x39123 (0x006d9 bytes) HOLE 22: 1746144601.991130 MAPWRITE 0x31862 thru 0x35b5a (0x042f9 bytes) 23: 1746144602.046591 MAPREAD 0x1ebd4 thru 0x234d0 (0x048fd bytes) 24: 1746144602.065748 MAPWRITE 0x00012 thru 0x07c3c (0x07c2b bytes) 25: 1746144602.112027 SKIPPED (no operation) 26: 1746144602.112195 WRITE 0x3bc1a thru 0x3f1ba (0x035a1 bytes) HOLE 27: 1746144602.112822 MAPWRITE 0x16e61 thru 0x1e03e (0x071de bytes) 28: 1746144602.169479 SKIPPED (no operation) 29: 1746144602.169737 SKIPPED (no operation) 30: 1746144602.170023 MAPWRITE 0x18dc8 thru 0x26c26 (0x0de5f bytes) 31: 1746144602.216518 TRUNC_DN 0x3f1bb to 0x26a67 32: 1746144602.225874 TRUNC_DN 0x26a67 to 0x16891 33: 1746144602.235494 SKIPPED (no operation) 34: 1746144602.235850 MAPREAD 0x0ef61 thru 0x0f104 (0x001a4 bytes) 35: 1746144602.248441 MAPWRITE 0x3bb91 thru 0x3ffff (0x0446f bytes) 36: 1746144602.327646 SKIPPED (no operation) 37: 1746144602.328826 TRUNC_DN 0x40000 to 0x112c1 38: 1746144602.340014 WRITE 0x01f40 thru 0x092d4 (0x07395 bytes) 39: 1746144602.340716 MAPREAD 0x0a913 thru 0x0c293 (0x01981 bytes) 40: 1746144602.351565 WRITE 0x0773f thru 0x12cc4 (0x0b586 bytes) EXTEND 41: 1746144602.353482 SKIPPED (no operation) 42: 1746144602.353839 MAPREAD 0x025e8 thru 0x06971 (0x0438a bytes) 43: 1746144602.356343 TRUNC_UP 0x12cc5 to 0x2c2f0 44: 1746144602.367940 SKIPPED (no operation) 45: 1746144602.368249 READ 0x0b43b thru 0x133b8 (0x07f7e bytes) 46: 1746144602.379904 MAPWRITE 0x11909 thru 0x16626 (0x04d1e bytes) 47: 1746144602.447148 MIRROR_EXTEND to 2 mirrors Using seed 4601 Correct content saved for comparison (maybe hexdump "/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr" vs "/mnt/lustre/d70a.sanity-flr/f70a.sanity-flr.fsxgood") sanity-flr test_70a: @@@@@@ FAIL: fsx FLR file /mnt/lustre/d70a.sanity-flr/f70a.sanity-flr failed Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7249:error() = /home/green/git/lustre-release/lustre/tests/sanity-flr.sh:3351:test_70a() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7622:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7685:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7503:run_test() = /home/green/git/lustre-release/lustre/tests/sanity-flr.sh:3353:main() Dumping lctl log to /tmp/testlogs//sanity-flr.test_70a.*.1746144603.log rsync: chown "/tmp/testlogs/.sanity-flr.test_70a.debug_log.oleg423-server.1746144603.log.ax61LS" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanity-flr.test_70a.dmesg.oleg423-server.1746144603.log.4CQddB" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1651) [generator=3.1.2]