[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: master and etcd installation





On Jan 6, 2016, at 12:23 PM, Jason DeTiberus <jdetiber redhat com> wrote:



On Wed, Jan 6, 2016 at 12:02 PM, Srinivas Naga Kotaru (skotaru) <skotaru cisco com> wrote:
Thanks Jason for information.

Clayton mentioned master need more memory. Not sure he mean by hosting both components and etcd is the main memory eater like you said. 

I suspect Clayton was referring to the etcd service running on the master and not the master service itself, however I've cc'd him in case he wants to correct me on that.

Specifically on memory:

The master (specifically the controller) keeps caches of frequently used cluster information, so the larger your cluster gets the more memory the Openshift controller process will require (the elected master only).  The API server part tends to generate a fair amount of memory churn through garbage collection, but does not necessarily need a large resident set.  Etcd, in most deployments, will typically use more memory than both master processes combined.  

Up until etcd + master memory is 50% of your host total memory, it's probably easier to run both on the same machine.  Past that, keeping them separate will likely result in more predictable behavior and avoid OOM events (occasionally, in rare scenarios, etcd can OOM if asked to retrieve very large portions of its key space in a single call - in practice Openshift does not do this but it is possible).  


 

What is redhat best recommended placement for production deployments?

We don't officially make a recommendation either way, our quick installation methods co-reside the services for HA installations, and we have the option in the advanced installation to separate the deployment of etcd and master hosts.
 
don’t see any reference architecture guide for OSE 3.X yet from redhat.

We released a 3.0 Reference Architecture: https://access.redhat.com/articles/1755133, however for 3.1 we recommend using the native HA support instead of using pacemaker for HA. There is an updated Reference Architecture that is being worked on, but I'm not sure of when it will be completed. 

 

-- 
Srinivas Kotaru

From: Jason DeTiberus <jdetiber redhat com>
Date: Wednesday, January 6, 2016 at 12:53 AM
To: skotaru <skotaru cisco com>
Cc: "dev lists openshift redhat com" <dev lists openshift redhat com>, "users lists openshift redhat com" <users lists openshift redhat com>
Subject: Re: master and etcd installation


On Jan 6, 2016 12:58 AM, "Srinivas Naga Kotaru (skotaru)" <skotaru cisco com> wrote:
>
> Hi 
>
> What is the best way to deploy master and etcd  on production setups? 
>
> - Combine master and etcd on same nodes
>         - separate mater and etcd to its own nodes
>
> I knew that we should require minimum of 3 nodes for etcd cluster HA.  
>
> What is pros and cons of running separately or combining together apart from saving few nodes or cleaner isolation?

That is the main tradeoff, saving hosts by co-residing the services vs. isolating the services.

>
>  What other considerations we need to keep in mind 
>
> - CPU
>    - Memory

Memory will be the main resource, etcd memory usage will grow with the cluster size and the number of resources deployed.

>         - API traffic
>        - etcd cluster health traffic
>       - OVS network latency or performance? 
>       - any other factors ?? 

Masters can be scaled independently from the etcd cluster and require less resources.

The only other consideration I see is that replacing a master host (outside of the master hosting the CA) does not require restoration of data from backup, where replacing an etcd host would (that, or the etcd cluster would need to be modified through etcdctl to add a new host and remove the old host).

>
>
> -- 
> Srinivas Kotaru
>
> _______________________________________________
> dev mailing list
> dev lists openshift redhat com
> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
>




--
Jason DeTiberus

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]