== sanity-lfsck test 18g: Find out orphan OST-object and repair it (7) ========================================================== 20:59:17 (1767319157) ##### The target MDT-object is lost, but related OI mapping is there The LFSCK should recreate the lost MDT-object without affected by the stale OI mapping. ##### 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0836074 s, 25.1 MB/s [0x200004a51:0xc:0x0] /mnt/lustre/d18g.sanity-lfsck/a1/f1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 3301 0xce5 0x240000400 1 2851 0xb23 0x280000400 Inject failure to simulate lost MDT-object but keep OI mapping fail_loc=0x162e fail_loc=0 Trigger layout LFSCK on all devices to find out orphan OST-object Started LFSCK on the device lustre-MDT0000: scrub layout oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory Waiting 120s for 'completed' oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory Waiting 60s for 'completed' oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory Waiting 40s for 'completed' oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory Waiting 30s for 'completed' oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory Waiting 20s for 'completed' oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory Waiting 10s for 'completed' oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory oleg341-server: error: get_param: param_path 'mdd//lfsck_layout': No such file or directory Waiting 0s for 'completed' Update not seen after 120s: want 'completed' got '' sanity-lfsck test_18g: @@@@@@ FAIL: (2) MDS1 is not the expected 'completed' Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7487:error() = /home/green/git/lustre-release/lustre/tests/sanity-lfsck.sh:2987:test_18g() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7863:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7926:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7741:run_test() = /home/green/git/lustre-release/lustre/tests/sanity-lfsck.sh:3011:main() Dumping lctl log to /tmp/testlogs//sanity-lfsck.test_18g.*.1767319287.log rsync: chown "/tmp/testlogs/.sanity-lfsck.test_18g.debug_log.oleg341-server.1767319287.log.dnXEKd" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanity-lfsck.test_18g.dmesg.oleg341-server.1767319287.log.P5xuNA" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3]