[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Routing SPI





> > > >
> > > >
> > > >
> > > > Hello everyone,
> > > >
> > > >
> > > >
> > > > We are currently playing with the Routing SPI, on the latest master
> > > > of
> > > > origin.
> > > >
> > > >
> > > >
> > > > It works quite well. I published a gist explaining how to use it :
> > > > https://gist.github.com/Filirom1/7334311
> > > >
> > > >
> > > >
> > > > Our goal is to build an nginx OpenShift LoadBalancer that will bypass
> > Node
> > > > Apache, HAProxy and Node Proxy.
> >
> > This is not something we enable well yet. He's not talking about HA apps.
> > He's talking about DIY routing/proxying for all apps.
> >
> > Gears bind to internal ports. You have to go through either the Node httpd
> > or Node tcp proxy (which is now iptables-based, not haproxy) to reach them
> > at all. And I'd like to point out, apps that are not scaled don't currently
> > expose any ports via the tcp proxy, so the only way to reach them is via
> > the httpd proxy.
> 
> 
> If I want the DIY router to point to every applications (not scalable, not
> ha), I have to expose ports of not scalable application.
> 
> I can remove this `if` statement and the port_interface will be present in
> mongodb.
> https://github.com/openshift/origin-server/blob/73bdccc445fed10d0ac5c9d12d3a9ae1d7604e25/controller/app/models/application.rb#L2020
> 
> But it means that it will increase significantly the number of exposed
> ports. Is it a problem for you ?
> 


Yes. It would be a problem for large installations and we do not want to make this a default. Warrants a config option at the very least.


> 
> Then, I need to publish the endpoints to the Routing SPI.
> 
> I tried to remove `self.ha` in
> https://github.com/openshift/origin-server/blob/73bdccc445fed10d0ac5c9d12d3a9ae1d7604e25/controller/app/models/application.rb#L1503

That is required, but don't forget to remove the self.ha check from remove_endpoint notification too :).


> 
> Bit it didn't work.
> 
> Now I think that I have to add PublishRoutingInfoOp in pending_ops for
> every application creation. But I don't know where to do it.
> 
> Any idea ?
> 

You dont need a separate op for that.
Ports need to be exposed for non-scalable apps... somewhere around the end of function 'calculate_add_component_ops'.
https://github.com/openshift/origin-server/blob/master/controller/app/models/application.rb#L2020

Uncheck the self.scalable around the block where ExposePortOp.new happens. I think that should do it.

/Rajat



> 
> 
> > > > Another thing I notice, is that `add_gear` and `delete_gear` messages
> > are
> > > > only published if the application is HA. Is it wanted ?
> > > >
> > > > Sounds like a bug - I'll make sure one is filed.
> > >
> > > It is an intended outcome. Not a bug. Isn't the entire routing SPI
> > > meant for applications who want to have the HA routing layer? So we
> > > skip the hassle for those who are not designated so.
> > > We can always change the behaviour, counter-arguments please?
> >
> > For this user, it's not just HA apps. It's all ports on all apps. Whether
> > that's a use case we want to support is not clear to me.
> >
> > > > The last question is how can I force every http request to pass
> > through the
> > > > nginx (nginx servers are different from node servers), and ssh/git
> > requests
> > > > to access the node directly ?
> >
> > The only way I can see doing this is if HTTP requests and git/ssh requests
> > have different hostnames. Which they do if it's an HA app (the ha-app-name
> > can then be pointed to the router); otherwise not. So the only way to do it
> > for all apps currently is to make all apps HA. Do we want to do it for all
> > apps?
> >
> > Is it maybe valuable to introduce an option to always create two different
> > app names, one for web and one for ssh? Maybe app-namespace continues to be
> > web (may be pointed to router if desired), and git-app-namespace always
> > points to node? At least something like this split is implied by the desire
> > to route web access and ssh access differently for ALL apps.
> >
> 
> I finally found a solution with aliases.
> 
> If the application `myapp.priv.company.com` has an alias to `
> myapp.company.com`, then `*.priv.company.com` could be used for ssh/git and
> http private access. And `*.company.com` will point to a DIY router that
> will only accept http requests.
> 
> DNS delegation will only be used of internal names.
> Public names could be handled by a DNS wildcard entry pointing to the DIY
> router, or by a more complex system.
> 
> 
> >
> > > The difference between web requests vs ssh/git requests should be
> > > resolved using the DNS entries.
> > > There should be one DNS meant for web requests and another for
> > > ssh/git. And that is what the event 'make-ha' does.
> > >
> > >  - A given app's default dns is not HA, because it points to the
> > >  first-gear/head-gear/deploy-gear of the app. e.g.
> > >  appname-namespace.rhcloud.com
> > >  - If one uses the appname-namespace.rhcloud.com DNS then both the
> > >  git/ssh and web/http requests land on the first gear without the
> > >  router knowing anything about it.
> > >  - Enter the HA DNS entry - ha-appname-namespace.rhcloud.com ! This
> > >  dns points to the router with the assumption that some
> > >  vhost/mod-rewrite/nginx-http-rewrite exists in the router to do
> > >  forward/reverse proxy with the appropriate url mapping.
> > >
> > > With two DNS entries, one pointing to the router and the other to the
> > > head-gear, we do not have to worry about 'move' etc. The fallout is
> > > that git-push/ssh is not HA. Only the web requests become HA.
> > > Eventually we can figure out how to resolve/re-map git/ssh DNS if
> > > the head gear/node goes down.
> >
> > _______________________________________________
> > dev mailing list
> > dev lists openshift redhat com
> > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> >
> 
> Cheers
> 
> Romain
> 


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]