== recovery-small test 50: failover MDS under load ======= 17:06:04 (1788815164) writemany pid 42520 Failing mds1 on oleg133-server Stopping /mnt/lustre-mds1 (opts:) on oleg133-server 17:06:19 (1788815179) shut down facet: mds1 facet_host: oleg133-server facet_failover_host: oleg133-server Failover mds1 to oleg133-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg133-server: oleg133-server.virtnet: executing set_default_debug -1 all pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1 Started lustre-MDT0000 17:06:40 (1788815200) targets are mounted 17:06:40 (1788815200) facet_failover done oleg133-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds1 on oleg133-server Stopping /mnt/lustre-mds1 (opts:) on oleg133-server 17:07:53 (1788815273) shut down facet: mds1 facet_host: oleg133-server facet_failover_host: oleg133-server Failover mds1 to oleg133-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg133-server: oleg133-server.virtnet: executing set_default_debug -1 all pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1 Started lustre-MDT0000 17:08:13 (1788815293) targets are mounted 17:08:13 (1788815293) facet_failover done oleg133-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds1 on oleg133-server Stopping /mnt/lustre-mds1 (opts:) on oleg133-server 17:09:28 (1788815368) shut down facet: mds1 facet_host: oleg133-server facet_failover_host: oleg133-server Failover mds1 to oleg133-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg133-server: oleg133-server.virtnet: executing set_default_debug -1 all pdsh@oleg133-client: oleg133-server: ssh exited with exit code 1 Started lustre-MDT0000 17:09:46 (1788815386) targets are mounted 17:09:46 (1788815386) facet_failover done oleg133-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec writemany returned 0