== recovery-small test 52: failover OST under load ======= 00:51:33 (1782363093) writemany pid 47306 Failing ost1 on oleg159-server Stopping /mnt/lustre-ost1 (opts:) on oleg159-server 00:51:46 (1782363106) shut down facet: ost1 facet_host: oleg159-server facet_failover_host: oleg159-server Failover ost1 to oleg159-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg159-server: oleg159-server.virtnet: executing set_default_debug -1 all pdsh@oleg159-client: oleg159-server: ssh exited with exit code 1 Started lustre-OST0000 00:52:01 (1782363121) targets are mounted 00:52:01 (1782363121) facet_failover done oleg159-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec writemany succeeded writemany pid 48148 Failing ost1 on oleg159-server Stopping /mnt/lustre-ost1 (opts:) on oleg159-server 00:57:19 (1782363439) shut down facet: ost1 facet_host: oleg159-server facet_failover_host: oleg159-server Failover ost1 to oleg159-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg159-server: oleg159-server.virtnet: executing set_default_debug -1 all pdsh@oleg159-client: oleg159-server: ssh exited with exit code 1 Started lustre-OST0000 00:57:34 (1782363454) targets are mounted 00:57:34 (1782363454) facet_failover done oleg159-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec writemany succeeded writemany pid 49004 Failing ost1 on oleg159-server Stopping /mnt/lustre-ost1 (opts:) on oleg159-server 01:02:48 (1782363768) shut down facet: ost1 facet_host: oleg159-server facet_failover_host: oleg159-server Failover ost1 to oleg159-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg159-server: oleg159-server.virtnet: executing set_default_debug -1 all pdsh@oleg159-client: oleg159-server: ssh exited with exit code 1 Started lustre-OST0000 01:03:03 (1782363783) targets are mounted 01:03:03 (1782363783) facet_failover done oleg159-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec writemany succeeded Connected clients: oleg159-client.virtnet oleg159-client.virtnet