[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: OSE 3.X upper limits



Kubernetes supports 150 nodes per cluster at a very specific SLA (all
calls <2s).  That is not the largest cluster you can run.  In
practice, Kubernetes and OpenShift have tested clusters up to 1k+
nodes and our concerns are optimizations in particular parts of the
code, not the maximum scale of the cluster.

On Mon, Jan 4, 2016 at 4:11 PM, Srinivas Naga Kotaru (skotaru)
<skotaru cisco com> wrote:
> Clayton
>
> This is good info to keep in mind while designing. What is the practical limit of nodes per cluster? We are thinking to use multiple clusters for non-prod/prod and internal/External apps.
>
> We heard Kubernates 1.1 support ~150 nodes per cluster.
>
>
> --
> Srinivas Kotaru
>
>
>
>
>
>
>
> On 1/4/16, 12:53 PM, "Clayton Coleman" <ccoleman redhat com> wrote:
>
>>We recommend no more than 100 containers per node at this time, while
>>performance issues in Docker and Kubernetes are being sorted out.
>>There is work on going to increase that to several hundred
>>(https://github.com/kubernetes/kubernetes/issues/19094)
>>
>>You can run many containers in one pod - you should not see a
>>significant difference between 1 container per pod and 20.  I would
>>recommend not going over 20 containers per pod, if only because we
>>haven't extensively tested it.
>>
>>If you have suitably powerful masters, you can expect to run ~10-30k
>>pods on ~1k nodes.  You may need to configure your network properly to
>>ensure that openvswitch can scale to those levels, or choose a static
>>/ non-openshift-sdn configuration like static route assignment.
>>Masters use a fair amount of memory at scale - you may need 16-32GB
>>per master.
>>
>>If you are running thousands of projects per cluster, you will also
>>need more memory than that.  Exact numbers are hard to say but ~1GB
>>per 1k of projects is not unreasonable today (can vary a lot).
>>
>>
>>On Mon, Jan 4, 2016 at 3:45 PM, Srinivas Naga Kotaru (skotaru)
>><skotaru cisco com> wrote:
>>> What are the upper limits of
>>> - How many nodes in a single cluster
>>>        -  How many pods in  a single node
>>>       -   How many containers in a single POD
>>>
>>> Looking for  practical values which can be used as a baseline for cluster
>>> design.
>>>
>>> --
>>> Srinivas Kotaru
>>>
>>> _______________________________________________
>>> dev mailing list
>>> dev lists openshift redhat com
>>> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
>>>


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]