[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Routing SPI






----- Original Message -----
> From: "Philibert Romain" <romain philibert worldline com>
> To: "Rajat Chopra" <rchopra redhat com>
> Cc: "Clayton Coleman" <ccoleman redhat com>, dev lists openshift redhat com
> Sent: Tuesday, November 19, 2013 6:04:37 AM
> Subject: RE: Routing SPI
> 
> Hi Rajat,
> 
> In my case, I found that using mongodb is easier than mcollective.
> In mongodb, tha application object contains app_name and namespace
> information. I don't have order issues, ...
> Is using mcollective preferable than mongodb ? Is the API more stable than
> mongodb data format ? What is the point to using Mcollective ?


Mcollective call gets the information from the real source - the node. In case mongo becomes outdated, the information can be rebuilt from what is actually there on the node. If mongo is up-to-date either way is good.
You can get the app_name and other app info from the gear's uuid as well. The call to 'app, gear = Application.find_by_gear_uuid(gear_uuid.to_s)' gives the needful.



> 
> Nethertheless, I found interesting features in ApplicationContainerProxy.
> I found that find_available takes a non_ha_server_identities to guarantee
> that gears of an HA application are preferably on different node.
> What happens if I manually move (oo-admin-move) the 2 HA gears on the same
> node? Is there any warning that I will break the HA ?

No such warning produced today. It would be a useful improvement to 'move' indeed.

/Rajat


> 
> Cheers
> Romain
> 
> -----Message d'origine-----
> De : Rajat Chopra [mailto:rchopra redhat com]
> Envoyé : lundi 18 novembre 2013 22:51
> À : Philibert Romain
> Cc : Clayton Coleman; dev lists openshift redhat com
> Objet : Re: Routing SPI
> 
> 
> 
> Hi!
>   Try this, out of the box. You can get the entire map of all gears'
>   endpoints with one mcollective call. Use it to populate outdated mongo, or
>   the router.
> 
> 
> #!/usr/bin/env oo-ruby
> require '/var/www/openshift/broker/config/environment'
> Rails.configuration.msg_broker[:rpc_options][:timeout] = 600 gear_map =
> OpenShift::ApplicationContainerProxy.get_all_gears_endpoints
> 
> 
> As this runs for all gears across all applications and nodes (should be few
> minutes), there would still be stuff that gets created/deleted. The code to
> translate the gear_map above to initialize the routing table will have to
> take care of out of order events that show up while it runs.
> 
> This still suffers from the drawbacks you have mentioned. i.e. capacity
> overflow. Also, the nodes that do not respond will have its gears missing in
> the map without any warning.
> 
> Example entry in gear_map :
> 
>           { "528a815bc7042fbaae000007" =>
>                     [  { "cartridge_name" => 'haproxy-1.4',
>                           "external_port" => 38032,
>                           "internal_address" => '127.1.250.1',
>                           "internal_port" => '8080' ,
>                           "protocols" => ['tcp', 'http', 'https', 'ws',
>                           'wss'],
>                           "type" => ['load_balancer']
>                         }
>                     ]
>           }
> 
> 
> Hope this helps.
> Rajat
> 
> 
> 
> 
> 
> ----- Original Message -----
> > From: "Philibert Romain" <romain philibert worldline com>
> > To: "Clayton Coleman" <ccoleman redhat com>
> > Cc: dev lists openshift redhat com
> > Sent: Monday, November 18, 2013 9:39:01 AM
> > Subject: RE: Routing SPI
> >
> > Hi
> >
> > Now I seed the initial routing state directly from mongodb:
> > https://github.com/worldline/openshift-nginx-routing/commit/551c0b7f01
> > 5c652f508d250e98ba30674228ece6 But I think that one day the database
> > format may change, and the nginx router will be broken.
> >
> > So the solution looks to be, as you mentioned earlier, to invoke a
> > oo-routing script to retrieve the initial state.
> > I could do it in ruby and make a Pull Request, but right now, the
> > solution I have in mind is not so good:
> >  * broker-util looks to be the right place for this script, but our
> > router is  not located on the broker (to avoid conflicts when
> > listening on port 80 and  443).
> >  * if there are millions of applications, the oo-routing-script may
> > have  memory issue to print the whole routing table, if not
> > implemented as a  stream
> >  * the same thing for the application parsing the oo-routing-script
> >
> > Did you plan to implement such a script ? Are you interested by it ?
> > Do you have an idea how to do it well ?
> >
> > Cheers
> > Romain
> >
> > -----Message d'origine-----
> > De : Clayton Coleman [mailto:ccoleman redhat com] Envoyé : jeudi 14
> > novembre 2013 18:40 À : Philibert Romain Cc : Arunabha Ghosh; Luke
> > Meyer; dev lists openshift redhat com Objet : Re: Routing SPI
> >
> >
> >
> >
> > ----- Original Message -----
> > >
> > >
> > > Hi,
> > >
> > >
> > >
> > > Thank you everybody for all your answers.
> > >
> > >
> > >
> > > I confirm that my goal is to develop a DIY routing/proxying based on
> > > the routing SPI.
> > >
> > >
> > >
> > > I think I will not use the gear_group? include=endpoints API because
> > > it means a lot of requests to be sent (List users, for each user,
> > > list applications, and for each application retrieve the endpoints).
> > > I will wait for the REST endpoints API, or I will retrieve the data
> > > directly from mongodb.
> >
> > We actually hadn't planned on a more complex endpoint because of the
> > volume of data for all apps - if you need to seed an initial state
> > that's something that direct mongo access is more suitable.  Having an
> > oo-* script that would bulk invoke the routing SPI for all
> > applications in the DB wouldn't be that hard to implement - that might
> > be a good compromise for people migrating an existing installation onto the
> > SPI.
> >
> > >
> > >
> > >
> > >
> > >
> > > But our second goal is to recommend every application to be HA.
> > >
> > >
> > >
> > > Is there a way I can change the default setting for app creation
> > > from not-scalable & not-HA to scalable & HA ? I mean, where should I
> > > start in the source code, if I want to do that ?
> > >
> > >
> > >
> > > I was also wondering why the HA prefix is
> > > `ha-appname-domain.example.com` and not `appname-domain.ha.example.com` ?
> > >
> > > Having a wildcard DNS entry `*.ha.example.com` pointing to the
> > > reverse proxy would ease installing OpenShift. Public DNS delegation
> > > would become optional.
> > >
> > >
> > >
> > > DNS entry not HA (appname-domain.example.com) would be delegated to
> > > a private DNS for ssh/git.
> > >
> > >
> > >
> > > Cheers
> > >
> > > Romain
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > De : dev-bounces lists openshift redhat com
> > > [mailto:dev-bounces lists openshift redhat com] De la part de
> > > Arunabha Ghosh Envoyé : mercredi 13 novembre 2013 20:46 À : Luke Meyer Cc
> > > :
> > > dev lists openshift redhat com Objet : Re: Routing SPI
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Wed, Nov 13, 2013 at 11:15 AM, Luke Meyer < lmeyer redhat com > wrote:
> > >
> > >
> > >
> > >
> > > ----- Original Message -----
> > > > From: "Rajat Chopra" < rchopra redhat com >
> > > > To: "Clayton Coleman" < ccoleman redhat com >
> > > > Cc: dev lists openshift redhat com
> > >
> > >
> > > > Sent: Monday, November 11, 2013 4:58:00 PM
> > > > Subject: Re: Routing SPI
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > From: "Clayton Coleman" < ccoleman redhat com >
> > > > > To: "Philibert Romain" < romain philibert worldline com >
> > > > > Cc: dev lists openshift redhat com
> > > > > Sent: Monday, November 11, 2013 7:11:00 AM
> > > > > Subject: Re: Routing SPI
> > > > >
> > > > > Hey Romain - didn't see any responses to the public list, so
> > > > > I'll throw a couple of answers in where I know.
> > > > >
> > > > > On Nov 6, 2013, at 9:10 AM, Philibert Romain <
> > > > > romain philibert worldline com
> > > > > > wrote:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hello everyone,
> > > > >
> > > > >
> > > > >
> > > > > We are currently playing with the Routing SPI, on the latest
> > > > > master of origin.
> > > > >
> > > > >
> > > > >
> > > > > It works quite well. I published a gist explaining how to use it :
> > > > > https://gist.github.com/Filirom1/7334311
> > > > >
> > > > >
> > > > >
> > > > > Our goal is to build an nginx OpenShift LoadBalancer that will
> > > > > bypass Node Apache, HAProxy and Node Proxy.
> > >
> > >
> > > This is not something we enable well yet. He's not talking about HA apps.
> > > He's talking about DIY routing/proxying for all apps.
> > >
> > > Gears bind to internal ports. You have to go through either the Node
> > > httpd or Node tcp proxy (which is now iptables-based, not haproxy)
> > > to reach them at all. And I'd like to point out, apps that are not
> > > scaled don't currently expose any ports via the tcp proxy, so the
> > > only way to reach them is via the httpd proxy.
> > >
> > >
> > >
> > > > > Another thing I notice, is that `add_gear` and `delete_gear`
> > > > > messages are only published if the application is HA. Is it wanted ?
> > > > >
> > > > > Sounds like a bug - I'll make sure one is filed.
> > > >
> > > > It is an intended outcome. Not a bug. Isn't the entire routing SPI
> > > > meant for applications who want to have the HA routing layer? So
> > > > we skip the hassle for those who are not designated so.
> > > > We can always change the behaviour, counter-arguments please?
> > >
> > >
> > > For this user, it's not just HA apps. It's all ports on all apps.
> > > Whether that's a use case we want to support is not clear to me.
> > >
> > >
> > >
> > > > > The last question is how can I force every http request to pass
> > > > > through the nginx (nginx servers are different from node
> > > > > servers), and ssh/git requests to access the node directly ?
> > >
> > >
> > > The only way I can see doing this is if HTTP requests and git/ssh
> > > requests have different hostnames. Which they do if it's an HA app
> > > (the ha-app-name can then be pointed to the router); otherwise not.
> > > So the only way to do it for all apps currently is to make all apps HA.
> > > Do we want to do it for all apps?
> > >
> > > Is it maybe valuable to introduce an option to always create two
> > > different app names, one for web and one for ssh? Maybe
> > > app-namespace continues to be web (may be pointed to router if
> > > desired), and git-app-namespace always points to node? At least
> > > something like this split is implied by the desire to route web
> > > access and ssh access differently for ALL apps.
> > >
> > >
> > >
> > >
> > >
> > > I'd vote for this one. Reading the docs on OSO, the single namespace
> > > for both ssh and web access seemed a little bit odd as it forced
> > > both the serving of the app and dev access to the app to the same entity.
> > > Separating them out will enable quite a few useful use cases like
> > > the central load balancing.
> > > Further, one can imagine the git repo for an app to be located on a
> > > centralized service which receives the upload, packages the app and
> > > deploys it onto nodes. This would fit in very well with potential
> > > future use cases like docker based apps where the git upload is
> > > received by OSO, packaged as a docker container and deployed on the
> > > node with the app being routed to by the centralized load balancing
> > > and routing service.
> > >
> > >
> > >
> > >
> > >
> > >
> > > > The difference between web requests vs ssh/git requests should be
> > > > resolved using the DNS entries.
> > > > There should be one DNS meant for web requests and another for
> > > > ssh/git. And that is what the event 'make-ha' does.
> > > >
> > > > - A given app's default dns is not HA, because it points to the
> > > > first-gear/head-gear/deploy-gear of the app. e.g.
> > > > appname-namespace.rhcloud.com
> > > > - If one uses the appname-namespace.rhcloud.com DNS then both the
> > > > git/ssh and web/http requests land on the first gear without the
> > > > router knowing anything about it.
> > > > - Enter the HA DNS entry - ha-appname-namespace.rhcloud.com ! This
> > > > dns points to the router with the assumption that some
> > > > vhost/mod-rewrite/nginx-http-rewrite exists in the router to do
> > > > forward/reverse proxy with the appropriate url mapping.
> > > >
> > > > With two DNS entries, one pointing to the router and the other to
> > > > the head-gear, we do not have to worry about 'move' etc. The
> > > > fallout is that git-push/ssh is not HA. Only the web requests become
> > > > HA.
> > > > Eventually we can figure out how to resolve/re-map git/ssh DNS if
> > > > the head gear/node goes down.
> > >
> > >
> > > _______________________________________________
> > > dev mailing list
> > > dev lists openshift redhat com
> > > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > >
> > >
> > >
> > >
> > >
> > >
> > > Ce message et les pièces jointes sont confidentiels et réservés à
> > > l'usage exclusif de ses destinataires. Il peut également être
> > > protégé par le secret professionnel. Si vous recevez ce message par
> > > erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire.
> > > L'intégrité du message ne pouvant être assurée sur Internet, la
> > > responsabilité de Worldline ne pourra être recherchée quant au
> > > contenu de ce message. Bien que les meilleurs efforts soient faits
> > > pour maintenir cette transmission exempte de tout virus,
> > > l'expéditeur ne donne aucune garantie à cet égard et sa
> > > responsabilité ne saurait être recherchée pour tout dommage résultant
> > > d'un virus transmis.
> > >
> > > This e-mail and the documents attached are confidential and intended
> > > solely for the addressee; it may also be privileged. If you receive
> > > this e-mail in error, please notify the sender immediately and
> > > destroy it. As its integrity cannot be secured on the Internet, the
> > > Worldline liability cannot be triggered for the message content.
> > > Although the sender endeavours to maintain a computer virus-free
> > > network, the sender does not warrant that this transmission is
> > > virus-free and will not be liable for any damages resulting from any
> > > virus transmitted.
> > >
> > > _______________________________________________
> > > dev mailing list
> > > dev lists openshift redhat com
> > > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > >
> >
> >
> > Ce message et les pièces jointes sont confidentiels et réservés à
> > l'usage exclusif de ses destinataires. Il peut également être protégé
> > par le secret professionnel. Si vous recevez ce message par erreur,
> > merci d'en avertir immédiatement l'expéditeur et de le détruire.
> > L'intégrité du message ne pouvant être assurée sur Internet, la
> > responsabilité de Worldline ne pourra être recherchée quant au contenu
> > de ce message. Bien que les meilleurs efforts soient faits pour
> > maintenir cette transmission exempte de tout virus, l'expéditeur ne
> > donne aucune garantie à cet égard et sa responsabilité ne saurait être
> > recherchée pour tout dommage résultant d'un virus transmis.
> >
> > This e-mail and the documents attached are confidential and intended
> > solely for the addressee; it may also be privileged. If you receive
> > this e-mail in error, please notify the sender immediately and destroy
> > it. As its integrity cannot be secured on the Internet, the Worldline
> > liability cannot be triggered for the message content. Although the
> > sender endeavours to maintain a computer virus-free network, the
> > sender does not warrant that this transmission is virus-free and will
> > not be liable for any damages resulting from any virus transmitted.
> >
> > _______________________________________________
> > dev mailing list
> > dev lists openshift redhat com
> > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> >
> 
> 
> Ce message et les pièces jointes sont confidentiels et réservés à l'usage
> exclusif de ses destinataires. Il peut également être protégé par le secret
> professionnel. Si vous recevez ce message par erreur, merci d'en avertir
> immédiatement l'expéditeur et de le détruire. L'intégrité du message ne
> pouvant être assurée sur Internet, la responsabilité de Worldline ne pourra
> être recherchée quant au contenu de ce message. Bien que les meilleurs
> efforts soient faits pour maintenir cette transmission exempte de tout
> virus, l'expéditeur ne donne aucune garantie à cet égard et sa
> responsabilité ne saurait être recherchée pour tout dommage résultant d'un
> virus transmis.
> 
> This e-mail and the documents attached are confidential and intended solely
> for the addressee; it may also be privileged. If you receive this e-mail in
> error, please notify the sender immediately and destroy it. As its integrity
> cannot be secured on the Internet, the Worldline liability cannot be
> triggered for the message content. Although the sender endeavours to
> maintain a computer virus-free network, the sender does not warrant that
> this transmission is virus-free and will not be liable for any damages
> resulting from any virus transmitted.
> 


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]