== recovery-small test 50: failover MDS under load ======= 09:53:26 (1787838806) writemany pid 42736 Failing mds1 on oleg426-server Stopping /mnt/lustre-mds1 (opts:) on oleg426-server 09:53:45 (1787838825) shut down facet: mds1 facet_host: oleg426-server facet_failover_host: oleg426-server Failover mds1 to oleg426-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg426-server: oleg426-server.virtnet: executing set_default_debug -1 all pdsh@oleg426-client: oleg426-server: ssh exited with exit code 1 Started lustre-MDT0000 09:54:19 (1787838859) targets are mounted 09:54:20 (1787838860) facet_failover done oleg426-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds1 on oleg426-server Stopping /mnt/lustre-mds1 (opts:) on oleg426-server 09:55:42 (1787838942) shut down facet: mds1 facet_host: oleg426-server facet_failover_host: oleg426-server Failover mds1 to oleg426-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg426-server: oleg426-server.virtnet: executing set_default_debug -1 all pdsh@oleg426-client: oleg426-server: ssh exited with exit code 1 Started lustre-MDT0000 09:56:17 (1787838977) targets are mounted 09:56:17 (1787838977) facet_failover done oleg426-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds1 on oleg426-server Stopping /mnt/lustre-mds1 (opts:) on oleg426-server 09:57:40 (1787839060) shut down facet: mds1 facet_host: oleg426-server facet_failover_host: oleg426-server Failover mds1 to oleg426-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg426-server: oleg426-server.virtnet: executing set_default_debug -1 all pdsh@oleg426-client: oleg426-server: ssh exited with exit code 1 Started lustre-MDT0000 09:58:17 (1787839097) targets are mounted 09:58:17 (1787839097) facet_failover done oleg426-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec writemany returned 0