== sanity-lfsck test 18e: Find out orphan OST-object and repair it (5) ========================================================== 18:50:53 (1789599053)
#####
The target MDT-object layout EA slot is occpuied by some new
created OST-object when repair dangling reference case. Such
conflict OST-object has been modified by others. To keep the
new data, the LFSCK will create a new file to refernece this
old orphan OST-object.
#####
[0x200004281:0x4:0x0]
/mnt/lustre/d18e.sanity-lfsck/a1/f1
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 0
	obdidx		 objid		 objid		 group
	     0	          3266	        0xcc2	   0x240000400

[0x200004281:0x5:0x0]
/mnt/lustre/d18e.sanity-lfsck/a1/f2
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 0
	obdidx		 objid		 objid		 group
	     0	          3267	        0xcc3	   0x240000400

[0x200004281:0x6:0x0]
/mnt/lustre/d18e.sanity-lfsck/a1/f3
lmm_stripe_count:  1
lmm_stripe_size:   4194304
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 0
	obdidx		 objid		 objid		 group
	     0	          3268	        0xcc4	   0x240000400

[0x200004281:0x7:0x0]
/mnt/lustre/d18e.sanity-lfsck/a1/f4
  lcm_layout_gen:    2
  lcm_mirror_count:  1
  lcm_entry_count:   2
    lcme_id:             1
    lcme_mirror_id:      0
    lcme_flags:          init
    lcme_extent.e_start: 0
    lcme_extent.e_end:   1048576
      lmm_stripe_count:  1
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: 0
      lmm_objects:
      -   0: { l_ost_idx:   0, l_fid: [0x240000400:0xcc5:0x0] }

    lcme_id:             2
    lcme_mirror_id:      0
    lcme_flags:          0
    lcme_extent.e_start: 1048576
    lcme_extent.e_end:   EOF
      lmm_stripe_count:  1
      lmm_stripe_size:   1048576
      lmm_pattern:       raid0
      lmm_layout_gen:    0
      lmm_stripe_offset: -1

Inject failure to make /mnt/lustre/d18e.sanity-lfsck/a1/f1 and /mnt/lustre/d18e.sanity-lfsck/a1/f2
to reference the same OST-object (which is f1's OST-object).
Then drop /mnt/lustre/d18e.sanity-lfsck/a1/f1 and its OST-object, so f2 becomes
dangling reference case, but f2's old OST-object is there.
Also the failure makes /mnt/lustre/d18e.sanity-lfsck/a1/f3 and /mnt/lustre/d18e.sanity-lfsck/a1/f4
to reference the same OST-object (which is f3's OST-object).
Then drop /mnt/lustre/d18e.sanity-lfsck/a1/f3 and its OST-object, so f4 becomes
dangling reference case, but f4's old OST-object is there.

fail_loc=0x1618
fail_loc=0
stopall to cleanup object cache
setupall
oleg347-server: oleg347-server.virtnet: executing load_modules_local
oleg347-server: oleg347-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1
oleg347-server: oleg347-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1
oleg347-server: oleg347-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1
Using TIMEOUT=20
fail_val=10
fail_loc=0x1602
debug=-1
debug_mb=150
debug=-1
debug_mb=150
Trigger layout LFSCK on all devices to find out orphan OST-object
Started LFSCK on the device lustre-MDT0000: scrub layout
Write new data to f2/f4 to modify the new created OST-object.
fail_val=0
fail_loc=0
Waiting 120s for 'completed'
Updated after 6s: want 'completed' got 'completed'
debug_mb=21
debug_mb=21
 sanity-lfsck test_18e: @@@@@@ FAIL: (6) Expect 2 orphans have been fixed, but got: 4 
  Trace dump:
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7530:error()
  = /home/green/git/lustre-release/lustre/tests/sanity-lfsck.sh:2765:test_18e()
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7907:run_one()
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7970:run_one_logged()
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7784:run_test()
  = /home/green/git/lustre-release/lustre/tests/sanity-lfsck.sh:2804:main()
Dumping lctl log to /tmp/testlogs//sanity-lfsck.test_18e.*.1789599201.log
rsync: chown "/tmp/testlogs/.sanity-lfsck.test_18e.debug_log.oleg347-server.1789599201.log.ulQX13" failed: Operation not permitted (1)
rsync: chown "/tmp/testlogs/.sanity-lfsck.test_18e.dmesg.oleg347-server.1789599201.log.D6sHqL" failed: Operation not permitted (1)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3]