== recovery-small test 50: failover MDS under load ======= 11:34:28 (1785080068) writemany pid 41003 Failing mds1 on oleg114-server Stopping /mnt/lustre-mds1 (opts:) on oleg114-server 11:34:46 (1785080086) shut down facet: mds1 facet_host: oleg114-server facet_failover_host: oleg114-server Failover mds1 to oleg114-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg114-server: oleg114-server.virtnet: executing set_default_debug -1 all pdsh@oleg114-client: oleg114-server: ssh exited with exit code 1 Started lustre-MDT0000 11:35:22 (1785080122) targets are mounted 11:35:22 (1785080122) facet_failover done oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds1 on oleg114-server Stopping /mnt/lustre-mds1 (opts:) on oleg114-server 11:36:44 (1785080204) shut down facet: mds1 facet_host: oleg114-server facet_failover_host: oleg114-server Failover mds1 to oleg114-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg114-server: oleg114-server.virtnet: executing set_default_debug -1 all pdsh@oleg114-client: oleg114-server: ssh exited with exit code 1 Started lustre-MDT0000 11:37:21 (1785080241) targets are mounted 11:37:21 (1785080241) facet_failover done oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec Failing mds1 on oleg114-server Stopping /mnt/lustre-mds1 (opts:) on oleg114-server 11:38:44 (1785080324) shut down facet: mds1 facet_host: oleg114-server facet_failover_host: oleg114-server Failover mds1 to oleg114-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg114-server: oleg114-server.virtnet: executing set_default_debug -1 all pdsh@oleg114-client: oleg114-server: ssh exited with exit code 1 Started lustre-MDT0000 11:39:18 (1785080358) targets are mounted 11:39:18 (1785080358) facet_failover done oleg114-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec writemany returned 0