== sanity-lfsck test 8: LFSCK state machine ============== 12:34:54 (1736357694) formatall oleg352-server: oleg352-server.virtnet: executing set_hostid oleg352-server: oleg352-server.virtnet: executing load_modules_local setupall oleg352-server: oleg352-server.virtnet: executing load_modules_local oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 Using TIMEOUT=20 preparing... 20 * 20 files will be created Wed Jan 8 12:35:36 EST 2025. total: 20 mkdir in 0.02 seconds: 822.65 ops/second total: 20 create in 0.02 seconds: 891.47 ops/second total: 20 mkdir in 0.02 seconds: 803.12 ops/second prepared Wed Jan 8 12:35:37 EST 2025. fail_loc=0x1603 fail_loc=0x1604 192.168.203.152@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg352-client.virtnet /mnt/lustre (opts:) fail_val=2 fail_loc=0x1601 Started LFSCK on the device lustre-MDT0000: scrub namespace Stopped LFSCK on the device lustre-MDT0000. Started LFSCK on the device lustre-MDT0000: scrub namespace fail_loc=0x80001609 Waiting 32s for 'failed' Updated after 2s: want 'failed' got 'failed' fail_loc=0x1600 Started LFSCK on the device lustre-MDT0000: scrub namespace fail_loc=0x160a stop mds1 fail_loc=0x160b start mds1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 fail_loc=0x1601 Started LFSCK on the device lustre-MDT0000: scrub namespace stop mds1 fail_loc=0x160b start mds1 oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 stop mds1 start mds1 without resume LFSCK oleg352-server: oleg352-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 pdsh@oleg352-client: oleg352-server: ssh exited with exit code 1 fail_val=2 fail_loc=0x1602 Started LFSCK on the device lustre-MDT0000: scrub namespace name: lfsck_namespace magic: 0xa06249ff version: 2 status: scanning-phase2 flags: scanned-once param: last_completed_time: N/A time_since_last_completed: N/A latest_start_time: 1736357769 time_since_latest_start: 4 seconds last_checkpoint_time: 1736357769 time_since_last_checkpoint: 4 seconds latest_start_position: 718, [0x200000007:0x1:0x0], 0x0 last_checkpoint_position: 35184372088832, N/A, N/A first_failure_position: N/A, N/A, N/A checked_phase1: 445 checked_phase2: 1 updated_phase1: 0 updated_phase2: 0 failed_phase1: 0 failed_phase2: 0 directories: 43 dirent_repaired: 0 linkea_repaired: 0 nlinks_repaired: 0 multiple_linked_checked: 0 multiple_linked_repaired: 0 unknown_inconsistency: 0 unmatched_pairs_repaired: 0 dangling_repaired: 0 multiple_referenced_repaired: 0 bad_file_type_repaired: 0 lost_dirent_repaired: 0 local_lost_found_scanned: 0 local_lost_found_moved: 0 local_lost_found_skipped: 0 local_lost_found_failed: 0 striped_dirs_scanned: 0 striped_dirs_repaired: 0 striped_dirs_failed: 0 striped_dirs_disabled: 0 striped_dirs_skipped: 0 striped_shards_scanned: 0 striped_shards_repaired: 0 striped_shards_failed: 0 striped_shards_skipped: 0 name_hash_repaired: 0 linkea_overflow_cleared: 0 agent_entries_repaired: 0 success_count: 0 run_time_phase1: 7 seconds run_time_phase2: 3 seconds average_speed_phase1: 63 items/sec average_speed_phase2: 0 objs/sec average_speed_total: 44 items/sec real_time_speed_phase1: N/A real_time_speed_phase2: 0 objs/sec current_position: [0x200000401:0x221:0x0] sanity-lfsck test_8: @@@@@@ FAIL: (23) Expect 'scanned-once,inconsistent',but got 'scanned-once' Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6421:error() = /home/green/git/lustre-release/lustre/tests/sanity-lfsck.sh:969:namespace_error() = /home/green/git/lustre-release/lustre/tests/sanity-lfsck.sh:1153:test_8() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6756:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6803:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6644:run_test() = /home/green/git/lustre-release/lustre/tests/sanity-lfsck.sh:1167:main() Dumping lctl log to /tmp/testlogs//sanity-lfsck.test_8.*.1736357773.log rsync: chown "/tmp/testlogs/.sanity-lfsck.test_8.debug_log.oleg352-server.1736357773.log.ugm42L" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanity-lfsck.test_8.dmesg.oleg352-server.1736357773.log.baUGf9" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1651) [generator=3.1.2]