[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Design questions around Container logs, EFK & OCP


Am 07.03.2018 um 23:47 schrieb Mohamed A. Shahat:
> Hi All, 
> My first question here, so i am hoping at least for some
> acknowledgement ! 
> _Background_
>   * OCP v3.7
Do you use the enterprise version or the opensource one?
>   * Several Worker Nodes
>   * Few Workload types 
>   * One Workload, let's call it WorkloadA is planned to have dedicated
>     Worker Nodes.
> _Objective_
>   * for WorkloadA , I'd like to send/route the Container Logs to an
>     External EFK / ELK stack other than the one that does get setup
>     with OCP
> _Motivation_ 
>   * For Workload A, an ES cluster does already exist, we would like to
>     reuse it. 
>   * There is an impression that the ES cluster that comes with OCP
>     might not necessarily scale if the team operating OCP does not
>     size it well
> _Inquiries_
>  1. Has this done before ? Yes / No ? Any comments ?
As you may know is "handle logs in a proper way" not a easy task.
There are some serious questions like the following.

* How long should the logs be preserved
* How much logs are written
* How fast are the logs written
* What's the limit of the network
* What's the limit of the remote es
* and many many more questions

>  1. Is there anyway with the fluentd pods or else to route specific
>     Workload / Pods Container logs to an external ES cluster ? 
>  2. If not, i'm willing to deploy my own fluentd pods , what do i lose
>     by excluding the WorkloadA Worker Nodes to not have the OCP
>     fluentd pods ? for example i don't want to lose any Operations /
>     OCP related / Worker Nodes related logs going to the embedded ES
>     cluster, all i need is to have the Container Logs of WorkloadA to
>     another ES cluster.
Have you looked at the following doc part?

Opensource origin:


As in the doc described you can send the collected fluentd logs to a
external es cluster.

You can find the source of the openshift logging solution in this repo.

> Looking forward to hearing from you, 
> Thanks, 

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]