== recovery-small test 50: failover MDS under load ======= 19:38:25 (1790897905) writemany pid 42448 Failing mds1 on oleg357-server Stopping /mnt/lustre-mds1 (opts:) on oleg357-server 19:38:41 (1790897921) shut down facet: mds1 facet_host: oleg357-server facet_failover_host: oleg357-server Failover mds1 to oleg357-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg357-server: oleg357-server.virtnet: executing set_default_debug -1 all pdsh@oleg357-client: oleg357-server: ssh exited with exit code 1 Started lustre-MDT0000 19:39:14 (1790897954) targets are mounted 19:39:14 (1790897954) facet_failover done oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds1 on oleg357-server Stopping /mnt/lustre-mds1 (opts:) on oleg357-server 19:40:36 (1790898036) shut down facet: mds1 facet_host: oleg357-server facet_failover_host: oleg357-server Failover mds1 to oleg357-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg357-server: oleg357-server.virtnet: executing set_default_debug -1 all pdsh@oleg357-client: oleg357-server: ssh exited with exit code 1 Started lustre-MDT0000 19:41:10 (1790898070) targets are mounted 19:41:10 (1790898070) facet_failover done oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds1 on oleg357-server Stopping /mnt/lustre-mds1 (opts:) on oleg357-server 19:42:27 (1790898147) shut down facet: mds1 facet_host: oleg357-server facet_failover_host: oleg357-server Failover mds1 to oleg357-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg357-server: oleg357-server.virtnet: executing set_default_debug -1 all pdsh@oleg357-client: oleg357-server: ssh exited with exit code 1 Started lustre-MDT0000 19:43:04 (1790898184) targets are mounted 19:43:04 (1790898184) facet_failover done oleg357-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec writemany returned 0