[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Redundant setup

----- Original Message -----
> Any chance you could address the DNS pieces?
> -Matt
> -------- Original Message --------
> Subject: 	Redundant setup
> Date: 	Thu, 12 Sep 2013 15:42:13 +0200
> From: 	Anselm Strauss <strauss puzzle ch>
> To: 	users lists openshift redhat com
> Thinking about how to make a redundant Openshift setup I came across some
> interesting information in the web, but it still left me with some vital
> questions:
> 1. Redundant brokers
> These are said to be completely stateless. Does that also mean that my
> fail-over load balancers do not need to synchronize the state table for
> network connections as e.g. pfsync does it for BSD? So i do not need sticky
> sessions on the load balancers and if one of them dies, the next packet of a
> session can just be sent over the other load balancer and will maybe end up
> on the other broker. The client should not notice this and does not need to
> re-create the session. Also if a broker fails the other one can take over in
> the middle of open sessions.
> 2. Redundant DDNS setup
> With dynamically created apps in Openshift I see the DNS service become more
> important. A failure of the master DNS server would not interrupt running
> apps but would block Openshift from doing dynamic DNS updates and creating
> new apps. As I know BIND there is no real multi-master setup for the same
> domain, you can have only one active server that takes the DDNS updates and
> distributes them to the slave servers. What is the best practice here? Is it
> just not that important? Do you have a custom setup with a fail-over master
> DNS server or do you switch one of the slave servers to become a master and
> configure the other ones to sync from the new master? I guess it is not
> trivial to sync the DNS data with dynamic updates between multiple servers
> by hand and guarantee a consistent state.

If you're using a commercial DNS service, that service will have to provide the redundancy.  Either they'll have to provide a protocol for selecting alternate update servers with fallback or they'll have to provide their own multiple virtual hosts with the same IP to manage failover.

If you're running your own DNS with BIND, I have found indications that you can set up the slave servers to accept update queries and forward them to the master.  This means, for the most robust configuration, you could have a hidden master/slave pair which accept update queries and a set of public NS slaves which accept normal queries.  On maintenance or failure you could promote the hidden slave to master so that it will continue to accept updates.

It is important to regularly flush the update journals on the master and backup the zone files offline.

It is not possible to eliminate the zone master as a short-term SPOF, but it is possible to minimize the duration of any degradation or downtime by preparing the hidden slave for promotion.

It is critical that the DNS servers are single-purpose, and for dynamic DNS, preferably dedicated to the dynamic zones both for robustness and security.

- Mark

> 3. Redundant gears for an app
> As I see from the DNS entry of an app it points to a node directly. Initial
> traffic and also SSH login will always go to a "master" gear of an app. If
> that gear or its node fail the app will be down. I guess this is not really
> a focus now, since also the db gears are not redundant. The redundancy and
> scaling of storage is a well known feature request, especially with MongoDB.
> But that would still leave the part of the master gear of an app a single
> point of failure. Are there any plans to address that?
> Thanks for any thoughts on this.
> Cheers,
> Anselm
> _______________________________________________
> users mailing list
> users lists openshift redhat com
> http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Mark Lamourine <markllama redhat com>
Sr. Software Developer, Cloud Engineering
Red Hat, 314 Littleton Road, Westford MA 01886
Voice: +1 978 392 1093
markllama @ irc://irc.freenod.org*lopsa

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]