== recovery-small test 52: failover OST under load ======= 12:52:32 (1788799952) writemany pid 48695 Failing ost1 on oleg615-server Stopping /mnt/lustre-ost1 (opts:) on oleg615-server 12:52:50 (1788799970) shut down facet: ost1 facet_host: oleg615-server facet_failover_host: oleg615-server Failover ost1 to oleg615-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg615-server: oleg615-server.virtnet: executing set_default_debug -1 all pdsh@oleg615-client: oleg615-server: ssh exited with exit code 1 Started lustre-OST0000 12:53:16 (1788799996) targets are mounted 12:53:16 (1788799996) facet_failover done oleg615-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec writemany succeeded writemany pid 49543 Failing ost1 on oleg615-server Stopping /mnt/lustre-ost1 (opts:) on oleg615-server 12:58:24 (1788800304) shut down facet: ost1 facet_host: oleg615-server facet_failover_host: oleg615-server Failover ost1 to oleg615-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg615-server: oleg615-server.virtnet: executing set_default_debug -1 all pdsh@oleg615-client: oleg615-server: ssh exited with exit code 1 Started lustre-OST0000 12:58:47 (1788800327) targets are mounted 12:58:47 (1788800327) facet_failover done oleg615-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec writemany succeeded writemany pid 50408 Failing ost1 on oleg615-server Stopping /mnt/lustre-ost1 (opts:) on oleg615-server 13:03:54 (1788800634) shut down facet: ost1 facet_host: oleg615-server facet_failover_host: oleg615-server Failover ost1 to oleg615-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg615-server: oleg615-server.virtnet: executing set_default_debug -1 all pdsh@oleg615-client: oleg615-server: ssh exited with exit code 1 Started lustre-OST0000 13:04:20 (1788800660) targets are mounted 13:04:20 (1788800660) facet_failover done oleg615-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec writemany succeeded Connected clients: oleg615-client.virtnet oleg615-client.virtnet