[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [4.x]: any future plans for proxy-mode: ipvs ?



Hi Clayton, Dan,

thanks for taking the time to respond, much appreciated.

On Mon, Jun 10, 2019 at 5:46 PM Dan Williams <dcbw redhat com> wrote:
On Sat, 2019-06-08 at 14:52 -0700, Clayton Coleman wrote:
> OVN implements kube services on nodes without kube-proxy - is there a
> specific feature gap between ipvs and ovn services you see that needs
> to be filled?

I'd love to hear the answer to that question too :)

[DC]: without knowing in details the OVN's lb implementation i doubt i can call it a gap ;) Saying that let me give our use case which we used back in 1.5 and still using in 3.7.
Being in the video processing/ encoding space we have some apps pods which needs to talk to a hardware storage data plane IPs over which various different video segments (different chunk size: 2/6 seconds and different bitrate) are getting written/ pulled.
Now the pods do talk to a K8s service (2 ports) which is mapped to a big endpoint list (300-600 endpoint IPs). As such (if i remember correctly) we ended up with # of iptables rules =  # of pods (2000) x 2 (K8s service ports) x # of endpoints.

Now what we've seen in the past was that load balancing traffic distribution was not hitting all endpoints (some where getting hit harder than others).
As such we thought that maybe with the ipvs getting stable in K8s 1.12+ we should try and see:
  • if various ipvs load balancing algorithm will provide better alternatives
  • if ipvs DSR will make any improvements
  • refreshing the iptables rules will be faster

The proxy implementation is usually tightly coupled to the network
plugin implementation. Some network plugins use kube-proxy while others
have their own internal load balancing implementation, like ovn-
kubernetes.

The largest issue we've seen with the iptables-based kube-proxy (as
opposed to IPVS-based kube-proxy) is iptables contention, and since
OVN's load-balancing/proxy implementation does not use iptables this is
not a concern for OVN.
[DC]:   Dan - would you mind pointing me to the code which deals with the OVN's lb logic ? looked in [1] but i guess i'm missing something else? (maybe looking in the wrong repo)

[1] https://github.com/openshift/origin/blob/master/pkg/cmd/openshift-sdn/proxy.go

Independently of that, we are planning to have a standalone kube-proxy
daemonset that 3rd party plugins (like Calico) can use which could be
run in IPVS mode if needed:

https://github.com/openshift/release/pull/3711

[DC]: i guess this is based on the [2] and if so, you mind (for my own curiosity) helping me understand the difference between OpenShiftSDN and OVNKubernetes networkType ? what new problems does the new OVNKubernetes type solve?
 
[2] https://github.com/ovn-org/ovn-kubernetes

That's waiting on Clayton for an LGTM for the mirroring bits (hint hint
:)

Dan

> > On Jun 8, 2019, at 4:08 PM, Daniel Comnea <comnea dani gmail com>
> > wrote:
> >
> > Hi,
> >
> > Are there any future plans in 4.x lifecycle to decouple kube-proxy
> > from OVN and allow setting/ running K8s upstream kube-proxy in ipvs
> > mode ?
> >
> > Cheers,
> > Dani
> > _______________________________________________
> > dev mailing list
> > dev lists openshift redhat com
> > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
>
> _______________________________________________
> dev mailing list
> dev lists openshift redhat com
> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
>
>


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]