== recovery-small test 52: failover OST under load ======= 03:44:07 (1782632647) writemany pid 48802 Failing ost1 on oleg359-server Stopping /mnt/lustre-ost1 (opts:) on oleg359-server 03:44:21 (1782632661) shut down facet: ost1 facet_host: oleg359-server facet_failover_host: oleg359-server Failover ost1 to oleg359-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg359-server: oleg359-server.virtnet: executing set_default_debug -1 all pdsh@oleg359-client: oleg359-server: ssh exited with exit code 1 Started lustre-OST0000 03:44:37 (1782632677) targets are mounted 03:44:37 (1782632677) facet_failover done oleg359-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec writemany succeeded writemany pid 49652 Failing ost1 on oleg359-server Stopping /mnt/lustre-ost1 (opts:) on oleg359-server 03:49:52 (1782632992) shut down facet: ost1 facet_host: oleg359-server facet_failover_host: oleg359-server Failover ost1 to oleg359-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg359-server: oleg359-server.virtnet: executing set_default_debug -1 all pdsh@oleg359-client: oleg359-server: ssh exited with exit code 1 Started lustre-OST0000 03:50:08 (1782633008) targets are mounted 03:50:08 (1782633008) facet_failover done oleg359-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec writemany succeeded writemany pid 50504 Failing ost1 on oleg359-server Stopping /mnt/lustre-ost1 (opts:) on oleg359-server 03:55:21 (1782633321) shut down facet: ost1 facet_host: oleg359-server facet_failover_host: oleg359-server Failover ost1 to oleg359-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg359-server: oleg359-server.virtnet: executing set_default_debug -1 all pdsh@oleg359-client: oleg359-server: ssh exited with exit code 1 Started lustre-OST0000 03:55:37 (1782633337) targets are mounted 03:55:37 (1782633337) facet_failover done oleg359-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec writemany succeeded Connected clients: oleg359-client.virtnet oleg359-client.virtnet