[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Routing SPI

2013/11/13 Luke Meyer <lmeyer redhat com>

----- Original Message -----
> From: "Rajat Chopra" <rchopra redhat com>
> To: "Clayton Coleman" <ccoleman redhat com>
> Cc: dev lists openshift redhat com
> Sent: Monday, November 11, 2013 4:58:00 PM
> Subject: Re: Routing SPI
> ----- Original Message -----
> > From: "Clayton Coleman" <ccoleman redhat com>
> > To: "Philibert Romain" <romain philibert worldline com>
> > Cc: dev lists openshift redhat com
> > Sent: Monday, November 11, 2013 7:11:00 AM
> > Subject: Re: Routing SPI
> >
> > Hey Romain - didn't see any responses to the public list, so I'll
> > throw a
> > couple of answers in where I know.
> >
> > On Nov 6, 2013, at 9:10 AM, Philibert Romain <
> > romain philibert worldline com
> > > wrote:
> >
> >
> >
> >
> >
> >
> > Hello everyone,
> >
> >
> >
> > We are currently playing with the Routing SPI, on the latest master
> > of
> > origin.
> >
> >
> >
> > It works quite well. I published a gist explaining how to use it :
> > https://gist.github.com/Filirom1/7334311
> >
> >
> >
> > Our goal is to build an nginx OpenShift LoadBalancer that will bypass Node
> > Apache, HAProxy and Node Proxy.

This is not something we enable well yet. He's not talking about HA apps. He's talking about DIY routing/proxying for all apps.

Gears bind to internal ports. You have to go through either the Node httpd or Node tcp proxy (which is now iptables-based, not haproxy) to reach them at all. And I'd like to point out, apps that are not scaled don't currently expose any ports via the tcp proxy, so the only way to reach them is via the httpd proxy.

If I want the DIY router to point to every applications (not scalable, not ha), I have to expose ports of not scalable application.

I can remove this `if` statement and the port_interface will be present in mongodb.

But it means that it will increase significantly the number of exposed ports. Is it a problem for you ?

Then, I need to publish the endpoints to the Routing SPI.

I tried to remove `self.ha` in https://github.com/openshift/origin-server/blob/73bdccc445fed10d0ac5c9d12d3a9ae1d7604e25/controller/app/models/application.rb#L1503

Bit it didn't work.

Now I think that I have to add PublishRoutingInfoOp in pending_ops for every application creation. But I don't know where to do it.

Any idea ?

> > Another thing I notice, is that `add_gear` and `delete_gear` messages are
> > only published if the application is HA. Is it wanted ?
> >
> > Sounds like a bug - I'll make sure one is filed.
> It is an intended outcome. Not a bug. Isn't the entire routing SPI
> meant for applications who want to have the HA routing layer? So we
> skip the hassle for those who are not designated so.
> We can always change the behaviour, counter-arguments please?

For this user, it's not just HA apps. It's all ports on all apps. Whether that's a use case we want to support is not clear to me.

> > The last question is how can I force every http request to pass through the
> > nginx (nginx servers are different from node servers), and ssh/git requests
> > to access the node directly ?

The only way I can see doing this is if HTTP requests and git/ssh requests have different hostnames. Which they do if it's an HA app (the ha-app-name can then be pointed to the router); otherwise not. So the only way to do it for all apps currently is to make all apps HA. Do we want to do it for all apps?

Is it maybe valuable to introduce an option to always create two different app names, one for web and one for ssh? Maybe app-namespace continues to be web (may be pointed to router if desired), and git-app-namespace always points to node? At least something like this split is implied by the desire to route web access and ssh access differently for ALL apps.

I finally found a solution with aliases.

If the application `myapp.priv.company.com` has an alias to `myapp.company.com`, then `*.priv.company.com` could be used for ssh/git and http private access. And `*.company.com` will point to a DIY router that will only accept http requests.

DNS delegation will only be used of internal names.
Public names could be handled by a DNS wildcard entry pointing to the DIY router, or by a more complex system.

> The difference between web requests vs ssh/git requests should be
> resolved using the DNS entries.
> There should be one DNS meant for web requests and another for
> ssh/git. And that is what the event 'make-ha' does.
>  - A given app's default dns is not HA, because it points to the
>  first-gear/head-gear/deploy-gear of the app. e.g.
>  appname-namespace.rhcloud.com
>  - If one uses the appname-namespace.rhcloud.com DNS then both the
>  git/ssh and web/http requests land on the first gear without the
>  router knowing anything about it.
>  - Enter the HA DNS entry - ha-appname-namespace.rhcloud.com ! This
>  dns points to the router with the assumption that some
>  vhost/mod-rewrite/nginx-http-rewrite exists in the router to do
>  forward/reverse proxy with the appropriate url mapping.
> With two DNS entries, one pointing to the router and the other to the
> head-gear, we do not have to worry about 'move' etc. The fallout is
> that git-push/ssh is not HA. Only the web requests become HA.
> Eventually we can figure out how to resolve/re-map git/ssh DNS if
> the head gear/node goes down.

dev mailing list
dev lists openshift redhat com



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]