== recovery-small test 50: failover MDS under load ======= 15:11:53 (1788462713) writemany pid 42438 Failing mds1 on oleg610-server Stopping /mnt/lustre-mds1 (opts:) on oleg610-server 15:12:11 (1788462731) shut down facet: mds1 facet_host: oleg610-server facet_failover_host: oleg610-server Failover mds1 to oleg610-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg610-server: oleg610-server.virtnet: executing set_default_debug -1 all pdsh@oleg610-client: oleg610-server: ssh exited with exit code 1 Started lustre-MDT0000 15:12:38 (1788462758) targets are mounted 15:12:38 (1788462758) facet_failover done oleg610-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds1 on oleg610-server Stopping /mnt/lustre-mds1 (opts:) on oleg610-server 15:14:07 (1788462847) shut down facet: mds1 facet_host: oleg610-server facet_failover_host: oleg610-server Failover mds1 to oleg610-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg610-server: oleg610-server.virtnet: executing set_default_debug -1 all pdsh@oleg610-client: oleg610-server: ssh exited with exit code 1 Started lustre-MDT0000 15:14:40 (1788462880) targets are mounted 15:14:41 (1788462881) facet_failover done oleg610-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds1 on oleg610-server Stopping /mnt/lustre-mds1 (opts:) on oleg610-server 15:16:09 (1788462969) shut down facet: mds1 facet_host: oleg610-server facet_failover_host: oleg610-server Failover mds1 to oleg610-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg610-server: oleg610-server.virtnet: executing set_default_debug -1 all pdsh@oleg610-client: oleg610-server: ssh exited with exit code 1 Started lustre-MDT0000 15:16:44 (1788463004) targets are mounted 15:16:44 (1788463004) facet_failover done oleg610-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec writemany returned 0