== sanity-lfsck test 18f: Skip the failed OST(s) when handle orphan OST-objects ========================================================== 19:23:45 (1763339025) ##### The target MDT-object is lost. The LFSCK should re-create the MDT-object under .lustre/lost+found/MDTxxxx. If some OST fail to verify some OST-object(s) during the first stage-scanning, the LFSCK should skip orphan OST-objects for such OST. Others should not be affected. ##### 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0695735 s, 30.1 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0500713 s, 41.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0538044 s, 39.0 MB/s /mnt/lustre/d18f.sanity-lfsck/a1/f1 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 195 0xc3 0x280000401 /mnt/lustre/d18f.sanity-lfsck/a2/f2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 196 0xc4 0x280000401 1 67 0x43 0x2c0000401 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0684259 s, 30.6 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0694309 s, 30.2 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0729518 s, 28.7 MB/s /mnt/lustre/d18f.sanity-lfsck/a3/f3 lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 67 0x43 0x280000400 /mnt/lustre/d18f.sanity-lfsck/a4/f4 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 68 0x44 0x280000400 1 66 0x42 0x2c0000400 Inject failure, to simulate the case of missing the MDT-object fail_loc=0x1616 fail_loc=0x1616 fail_loc=0 fail_loc=0 Inject failure, to simulate the OST0 fail to handle MDT0 LFSCK request during the first-stage scanning. fail_loc=0x161c fail_val=0 Trigger layout LFSCK on all devices to find out orphan OST-object Started LFSCK on the device lustre-MDT0000: scrub layout Waiting 120s for 'partial' Waiting 100s for 'partial' Waiting 90s for 'partial' Waiting 60s for 'partial' Waiting 40s for 'partial' Waiting 30s for 'partial' Waiting 20s for 'partial' Waiting 10s for 'partial' Waiting 0s for 'partial' Update not seen after 120s: want 'partial' got 'completed' sanity-lfsck test_18f: @@@@@@ FAIL: (2) MDS2 is not the expected 'partial' Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7314:error() = /home/green/git/lustre-release/lustre/tests/sanity-lfsck.sh:2881:test_18f() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7690:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7753:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7568:run_test() = /home/green/git/lustre-release/lustre/tests/sanity-lfsck.sh:2948:main() Dumping lctl log to /tmp/testlogs//sanity-lfsck.test_18f.*.1763339161.log rsync: chown "/tmp/testlogs/.sanity-lfsck.test_18f.debug_log.oleg438-server.1763339161.log.VjJi3P" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanity-lfsck.test_18f.dmesg.oleg438-server.1763339161.log.nbptqG" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3]