[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Heptio Contour

On Jan 24, 2018, at 2:05 PM, Srinivas Naga Kotaru (skotaru) <skotaru cisco com> wrote:



Good analyses. That is exactly am looking for. Thanks for great info.


Also happy that, you already did prototype and compared with current OCP routing solution.


Also can u share your thoughts on how ambassador fit this eco system? my research shows, ambassador would be a good fit for north/south ingress controller where as Istio would be great fit for east/west service traffic. Both use Envoy internally.

I do expect all of these proxies to become more interesting over time, and I agree east/west and north/south are obvious places for differentiation.  We might have multiple interlocking proxies with different focuses.  


Then ambassador would be another competitor to contour?


I knew lot of moving parts on routes, ingress and services side but none is prime time ready for high scale workloads.  



Srinivas Kotaru

From: Clayton Coleman <ccoleman redhat com>
Date: Wednesday, January 24, 2018 at 10:32 AM
To: Srinivas Naga Kotaru <skotaru cisco com>
Cc: users <users lists openshift redhat com>
Subject: Re: Heptio Contour


At this point in time, contour is still pretty new, so expect some rough edges.  I did a prototype of routes with envoy (similar to contour, but preserving the router features) a few months back, and identified a set of challenges which made it not a great fit as a replacement for the OOTB openshift router


In general, when comparing to haproxy and the general state, here's the list


PROs (envoy):


* supports http2 natively

* deeper insight into traffic passing through


CONs (envoy):


* scale is not great right now - even using dynamic programming i couldn't get much above 100 backends before hitting the wall (wouldn't scale to very large, dense clusters)

* memory use is much higher than haproxy - so another density challenge (I was using 30GB at 1k backends, vs 5GB for 15k backends that we see with haproxy)

* web sockets can't be transparent - so you have to run another port for them instead of sharing the HTTP port

* SNI passthrough not ready, maybe in 6mo

* reencrypt was really hacky, I couldn't get it to work right now (again, 6mo should be fixed)

* general fragility - was easy to break when programming config


CONs (contour, vs openshift router):


* None of the security isolation stuff (preventing one tenant from using someone else's hostname)

* None of the protection against bad certs (preventing someone from breaking the router by using someone else's tenant)

* No status reporting back


I think the biggest long term challenge with envoy will be pure scale - HAProxy is at least two orders of magnitude more efficient right now, and I think it will be a while before envoy even gets close.  So if you have 10k+ frontends, haproxy is the only game in town.  On ingress vs routes, routes are still more polished, so it's really just a "do you want the features routes have that ingress doesn't.


On the other downsides to envoy, I expect to see progress over the next year or two to fixing it.  I had originally done the prototype expecting that maybe we would use envoy as the "out of the box" plugin to the router (continuing to support routes and ingress and all the features, but with envoy underneath), but the biggest challenge is that envoy isn't really better than haproxy for the feature set that ingress and routes expose.  Where envoy really shines is in something like istio, where you have a richer model for the ingress / service definition that can use the extra bells and whistles.  Ultimately ingress + annotations and routes are both targeted at "simple, high scale" use of web frontends.  I would expect a lot of people to have their apps "grow up" and use node ports or istio ingress as the apps get bigger and more important.  I don't see them as directly competing.




On Fri, Jan 19, 2018 at 1:36 PM, Srinivas Naga Kotaru (skotaru) <skotaru cisco com> wrote:

How it is different than Openshift router and what extra benefits it brings? Anyone educate me to understand differences or possible use cases where it fit into eco system? Or replacing ingress controller or will it solve ingress controller 244 address limitations?







Srinivas Kotaru

users mailing list
users lists openshift redhat com


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]