== recovery-small test 52: failover OST under load ======= 22:33:46 (1783564426) writemany pid 47311 Failing ost1 on oleg353-server Stopping /mnt/lustre-ost1 (opts:) on oleg353-server 22:34:00 (1783564440) shut down facet: ost1 facet_host: oleg353-server facet_failover_host: oleg353-server Failover ost1 to oleg353-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg353-server: oleg353-server.virtnet: executing set_default_debug -1 all pdsh@oleg353-client: oleg353-server: ssh exited with exit code 1 Started lustre-OST0000 22:34:19 (1783564459) targets are mounted 22:34:19 (1783564459) facet_failover done oleg353-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec writemany succeeded writemany pid 48161 Failing ost1 on oleg353-server Stopping /mnt/lustre-ost1 (opts:) on oleg353-server 22:39:33 (1783564773) shut down facet: ost1 facet_host: oleg353-server facet_failover_host: oleg353-server Failover ost1 to oleg353-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg353-server: oleg353-server.virtnet: executing set_default_debug -1 all pdsh@oleg353-client: oleg353-server: ssh exited with exit code 1 Started lustre-OST0000 22:39:51 (1783564791) targets are mounted 22:39:51 (1783564791) facet_failover done oleg353-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec writemany succeeded writemany pid 49009 Failing ost1 on oleg353-server Stopping /mnt/lustre-ost1 (opts:) on oleg353-server 22:45:05 (1783565105) shut down facet: ost1 facet_host: oleg353-server facet_failover_host: oleg353-server Failover ost1 to oleg353-server mount facets: ost1 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg353-server: oleg353-server.virtnet: executing set_default_debug -1 all pdsh@oleg353-client: oleg353-server: ssh exited with exit code 1 Started lustre-OST0000 22:45:22 (1783565122) targets are mounted 22:45:22 (1783565122) facet_failover done oleg353-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec writemany succeeded Connected clients: oleg353-client.virtnet oleg353-client.virtnet