== replay-ost-single test 6: Fail OST before obd_destroy ========================================================== 09:50:17 (1758808217) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg145-server mds-ost sync done. Waiting for MDT destroys to complete 1280+0 records in 1280+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.327355 s, 16.0 MB/s /mnt/lustre/d0.replay-ost-single/f6.replay-ost-single lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 18 0x12 0x280000401 fail_loc=0x80000119 Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg145-server mds-ost sync done. before_free: 7663156 after_dd_free: 7658036 took 0 seconds Failing ost1 on oleg145-server Stopping /mnt/lustre-ost1 (opts:) on oleg145-server 09:50:49 (1758808249) shut down facet: ost1 facet_host: oleg145-server facet_failover_host: oleg145-server Failover ost1 to oleg145-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg145-server: oleg145-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg145-client: oleg145-server: ssh exited with exit code 1 Started lustre-OST0000 09:51:04 (1758808264) targets are mounted 09:51:04 (1758808264) facet_failover done oleg145-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec affected facets: ost1 oleg145-server: oleg145-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg145-server: *.lustre-OST0000.recovery_status status: COMPLETE Can't lstat /mnt/lustre/d0.replay-ost-single/f6.replay-ost-single: No such file or directory Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0000-osc-MDT0001.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0001.old_sync_processed wait 40 secs maximumly for oleg145-server mds-ost sync done. Waiting for MDT destroys to complete free_before: 7663156 free_after: 7663156