[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

RE: Routing SPI



Right ;) I understand now, why an mcollective call is important.
Thank you !

-----Message d'origine-----
De : Rajat Chopra [mailto:rchopra redhat com]
Envoyé : mardi 19 novembre 2013 19:17
À : Philibert Romain
Cc : Clayton Coleman; dev lists openshift redhat com
Objet : Re: Routing SPI






----- Original Message -----
> From: "Philibert Romain" <romain philibert worldline com>
> To: "Rajat Chopra" <rchopra redhat com>
> Cc: "Clayton Coleman" <ccoleman redhat com>,
> dev lists openshift redhat com
> Sent: Tuesday, November 19, 2013 7:06:18 AM
> Subject: RE: Routing SPI
>
> Thank you Rajat.
>
> Is there any scripts that synchronize an outdated mongo with
> mcollective informations coming from node ?
> Is it something usual (outdated mongodb) whith big openshift
> production instances ?
>


No generic script is available to populate mongodb from the nodes. And by 'outdated' I dont mean a broken/corrupted mongo. We rely on it as source of truth for most things.

An example of 'outdated' mongo is with routing endpoints itself. We introduced routing endpoints about two months back, but what happens to the applications that were created before it? Mongo will be outdated for an application that was created, lets say, six months ago. The mcoll call becomes useful in such cases.

No doubts on Mongo's integrity here :).

/Rajat



> I was thinking that mongodb was reliable, and in case of
> inconsistency, the mongodb data was pushed to gears. As described in oo-admin-repair.
> If mongodb is not consistent with nodes, the whole broker has to be
> fixed, not only routers...
>
> Romain
>
> -----Message d'origine-----
> De : Rajat Chopra [mailto:rchopra redhat com] Envoyé : mardi 19
> novembre 2013 15:46 À : Philibert Romain Cc : Clayton Coleman;
> dev lists openshift redhat com Objet : Re: Routing SPI
>
>
>
>
>
>
> ----- Original Message -----
> > From: "Philibert Romain" <romain philibert worldline com>
> > To: "Rajat Chopra" <rchopra redhat com>
> > Cc: "Clayton Coleman" <ccoleman redhat com>,
> > dev lists openshift redhat com
> > Sent: Tuesday, November 19, 2013 6:04:37 AM
> > Subject: RE: Routing SPI
> >
> > Hi Rajat,
> >
> > In my case, I found that using mongodb is easier than mcollective.
> > In mongodb, tha application object contains app_name and namespace
> > information. I don't have order issues, ...
> > Is using mcollective preferable than mongodb ? Is the API more
> > stable than mongodb data format ? What is the point to using Mcollective ?
>
>
> Mcollective call gets the information from the real source - the node.
> In case mongo becomes outdated, the information can be rebuilt from
> what is actually there on the node. If mongo is up-to-date either way is good.
> You can get the app_name and other app info from the gear's uuid as
> well. The call to 'app, gear =
> Application.find_by_gear_uuid(gear_uuid.to_s)' gives the needful.
>
>
>
> >
> > Nethertheless, I found interesting features in ApplicationContainerProxy.
> > I found that find_available takes a non_ha_server_identities to
> > guarantee that gears of an HA application are preferably on different node.
> > What happens if I manually move (oo-admin-move) the 2 HA gears on
> > the same node? Is there any warning that I will break the HA ?
>
> No such warning produced today. It would be a useful improvement to 'move'
> indeed.
>
> /Rajat
>
>
> >
> > Cheers
> > Romain
> >
> > -----Message d'origine-----
> > De : Rajat Chopra [mailto:rchopra redhat com] Envoyé : lundi 18
> > novembre 2013 22:51 À : Philibert Romain Cc : Clayton Coleman;
> > dev lists openshift redhat com Objet : Re: Routing SPI
> >
> >
> >
> > Hi!
> >   Try this, out of the box. You can get the entire map of all gears'
> >   endpoints with one mcollective call. Use it to populate outdated mongo,
> >   or
> >   the router.
> >
> >
> > #!/usr/bin/env oo-ruby
> > require '/var/www/openshift/broker/config/environment'
> > Rails.configuration.msg_broker[:rpc_options][:timeout] = 600
> > gear_map =
> > OpenShift::ApplicationContainerProxy.get_all_gears_endpoints
> >
> >
> > As this runs for all gears across all applications and nodes (should
> > be few minutes), there would still be stuff that gets
> > created/deleted. The code to translate the gear_map above to
> > initialize the routing table will have to take care of out of order events that show up while it runs.
> >
> > This still suffers from the drawbacks you have mentioned. i.e.
> > capacity overflow. Also, the nodes that do not respond will have its
> > gears missing in the map without any warning.
> >
> > Example entry in gear_map :
> >
> >           { "528a815bc7042fbaae000007" =>
> >                     [  { "cartridge_name" => 'haproxy-1.4',
> >                           "external_port" => 38032,
> >                           "internal_address" => '127.1.250.1',
> >                           "internal_port" => '8080' ,
> >                           "protocols" => ['tcp', 'http', 'https', 'ws',
> >                           'wss'],
> >                           "type" => ['load_balancer']
> >                         }
> >                     ]
> >           }
> >
> >
> > Hope this helps.
> > Rajat
> >
> >
> >
> >
> >
> > ----- Original Message -----
> > > From: "Philibert Romain" <romain philibert worldline com>
> > > To: "Clayton Coleman" <ccoleman redhat com>
> > > Cc: dev lists openshift redhat com
> > > Sent: Monday, November 18, 2013 9:39:01 AM
> > > Subject: RE: Routing SPI
> > >
> > > Hi
> > >
> > > Now I seed the initial routing state directly from mongodb:
> > > https://github.com/worldline/openshift-nginx-routing/commit/551c0b
> > > 7f01
> > > 5c652f508d250e98ba30674228ece6 But I think that one day the
> > > database format may change, and the nginx router will be broken.
> > >
> > > So the solution looks to be, as you mentioned earlier, to invoke a
> > > oo-routing script to retrieve the initial state.
> > > I could do it in ruby and make a Pull Request, but right now, the
> > > solution I have in mind is not so good:
> > >  * broker-util looks to be the right place for this script, but
> > > our router is  not located on the broker (to avoid conflicts when
> > > listening on port 80 and  443).
> > >  * if there are millions of applications, the oo-routing-script
> > > may have  memory issue to print the whole routing table, if not
> > > implemented as a  stream
> > >  * the same thing for the application parsing the
> > > oo-routing-script
> > >
> > > Did you plan to implement such a script ? Are you interested by it ?
> > > Do you have an idea how to do it well ?
> > >
> > > Cheers
> > > Romain
> > >
> > > -----Message d'origine-----
> > > De : Clayton Coleman [mailto:ccoleman redhat com] Envoyé : jeudi
> > > 14 novembre 2013 18:40 À : Philibert Romain Cc : Arunabha Ghosh;
> > > Luke Meyer; dev lists openshift redhat com Objet : Re: Routing SPI
> > >
> > >
> > >
> > >
> > > ----- Original Message -----
> > > >
> > > >
> > > > Hi,
> > > >
> > > >
> > > >
> > > > Thank you everybody for all your answers.
> > > >
> > > >
> > > >
> > > > I confirm that my goal is to develop a DIY routing/proxying
> > > > based on the routing SPI.
> > > >
> > > >
> > > >
> > > > I think I will not use the gear_group? include=endpoints API
> > > > because it means a lot of requests to be sent (List users, for
> > > > each user, list applications, and for each application retrieve the endpoints).
> > > > I will wait for the REST endpoints API, or I will retrieve the
> > > > data directly from mongodb.
> > >
> > > We actually hadn't planned on a more complex endpoint because of
> > > the volume of data for all apps - if you need to seed an initial
> > > state that's something that direct mongo access is more suitable.
> > > Having an
> > > oo-* script that would bulk invoke the routing SPI for all
> > > applications in the DB wouldn't be that hard to implement - that
> > > might be a good compromise for people migrating an existing
> > > installation onto the SPI.
> > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > But our second goal is to recommend every application to be HA.
> > > >
> > > >
> > > >
> > > > Is there a way I can change the default setting for app creation
> > > > from not-scalable & not-HA to scalable & HA ? I mean, where
> > > > should I start in the source code, if I want to do that ?
> > > >
> > > >
> > > >
> > > > I was also wondering why the HA prefix is
> > > > `ha-appname-domain.example.com` and not
> > > > `appname-domain.ha.example.com` ?
> > > >
> > > > Having a wildcard DNS entry `*.ha.example.com` pointing to the
> > > > reverse proxy would ease installing OpenShift. Public DNS
> > > > delegation would become optional.
> > > >
> > > >
> > > >
> > > > DNS entry not HA (appname-domain.example.com) would be delegated
> > > > to a private DNS for ssh/git.
> > > >
> > > >
> > > >
> > > > Cheers
> > > >
> > > > Romain
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > De : dev-bounces lists openshift redhat com
> > > > [mailto:dev-bounces lists openshift redhat com] De la part de
> > > > Arunabha Ghosh Envoyé : mercredi 13 novembre 2013 20:46 À : Luke
> > > > Meyer Cc
> > > > :
> > > > dev lists openshift redhat com Objet : Re: Routing SPI
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Nov 13, 2013 at 11:15 AM, Luke Meyer < lmeyer redhat com
> > > > >
> > > > wrote:
> > > >
> > > >
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > From: "Rajat Chopra" < rchopra redhat com >
> > > > > To: "Clayton Coleman" < ccoleman redhat com >
> > > > > Cc: dev lists openshift redhat com
> > > >
> > > >
> > > > > Sent: Monday, November 11, 2013 4:58:00 PM
> > > > > Subject: Re: Routing SPI
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ----- Original Message -----
> > > > > > From: "Clayton Coleman" < ccoleman redhat com >
> > > > > > To: "Philibert Romain" < romain philibert worldline com >
> > > > > > Cc: dev lists openshift redhat com
> > > > > > Sent: Monday, November 11, 2013 7:11:00 AM
> > > > > > Subject: Re: Routing SPI
> > > > > >
> > > > > > Hey Romain - didn't see any responses to the public list, so
> > > > > > I'll throw a couple of answers in where I know.
> > > > > >
> > > > > > On Nov 6, 2013, at 9:10 AM, Philibert Romain <
> > > > > > romain philibert worldline com
> > > > > > > wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Hello everyone,
> > > > > >
> > > > > >
> > > > > >
> > > > > > We are currently playing with the Routing SPI, on the latest
> > > > > > master of origin.
> > > > > >
> > > > > >
> > > > > >
> > > > > > It works quite well. I published a gist explaining how to use it :
> > > > > > https://gist.github.com/Filirom1/7334311
> > > > > >
> > > > > >
> > > > > >
> > > > > > Our goal is to build an nginx OpenShift LoadBalancer that
> > > > > > will bypass Node Apache, HAProxy and Node Proxy.
> > > >
> > > >
> > > > This is not something we enable well yet. He's not talking about
> > > > HA apps.
> > > > He's talking about DIY routing/proxying for all apps.
> > > >
> > > > Gears bind to internal ports. You have to go through either the
> > > > Node httpd or Node tcp proxy (which is now iptables-based, not
> > > > haproxy) to reach them at all. And I'd like to point out, apps
> > > > that are not scaled don't currently expose any ports via the tcp
> > > > proxy, so the only way to reach them is via the httpd proxy.
> > > >
> > > >
> > > >
> > > > > > Another thing I notice, is that `add_gear` and `delete_gear`
> > > > > > messages are only published if the application is HA. Is it
> > > > > > wanted ?
> > > > > >
> > > > > > Sounds like a bug - I'll make sure one is filed.
> > > > >
> > > > > It is an intended outcome. Not a bug. Isn't the entire routing
> > > > > SPI meant for applications who want to have the HA routing
> > > > > layer? So we skip the hassle for those who are not designated so.
> > > > > We can always change the behaviour, counter-arguments please?
> > > >
> > > >
> > > > For this user, it's not just HA apps. It's all ports on all apps.
> > > > Whether that's a use case we want to support is not clear to me.
> > > >
> > > >
> > > >
> > > > > > The last question is how can I force every http request to
> > > > > > pass through the nginx (nginx servers are different from
> > > > > > node servers), and ssh/git requests to access the node directly ?
> > > >
> > > >
> > > > The only way I can see doing this is if HTTP requests and
> > > > git/ssh requests have different hostnames. Which they do if it's
> > > > an HA app (the ha-app-name can then be pointed to the router); otherwise not.
> > > > So the only way to do it for all apps currently is to make all apps HA.
> > > > Do we want to do it for all apps?
> > > >
> > > > Is it maybe valuable to introduce an option to always create two
> > > > different app names, one for web and one for ssh? Maybe
> > > > app-namespace continues to be web (may be pointed to router if
> > > > desired), and git-app-namespace always points to node? At least
> > > > something like this split is implied by the desire to route web
> > > > access and ssh access differently for ALL apps.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > I'd vote for this one. Reading the docs on OSO, the single
> > > > namespace for both ssh and web access seemed a little bit odd as
> > > > it forced both the serving of the app and dev access to the app
> > > > to the same entity.
> > > > Separating them out will enable quite a few useful use cases
> > > > like the central load balancing.
> > > > Further, one can imagine the git repo for an app to be located
> > > > on a centralized service which receives the upload, packages the
> > > > app and deploys it onto nodes. This would fit in very well with
> > > > potential future use cases like docker based apps where the git
> > > > upload is received by OSO, packaged as a docker container and
> > > > deployed on the node with the app being routed to by the
> > > > centralized load balancing and routing service.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > The difference between web requests vs ssh/git requests should
> > > > > be resolved using the DNS entries.
> > > > > There should be one DNS meant for web requests and another for
> > > > > ssh/git. And that is what the event 'make-ha' does.
> > > > >
> > > > > - A given app's default dns is not HA, because it points to
> > > > > the first-gear/head-gear/deploy-gear of the app. e.g.
> > > > > appname-namespace.rhcloud.com
> > > > > - If one uses the appname-namespace.rhcloud.com DNS then both
> > > > > the git/ssh and web/http requests land on the first gear
> > > > > without the router knowing anything about it.
> > > > > - Enter the HA DNS entry - ha-appname-namespace.rhcloud.com !
> > > > > This dns points to the router with the assumption that some
> > > > > vhost/mod-rewrite/nginx-http-rewrite exists in the router to
> > > > > do forward/reverse proxy with the appropriate url mapping.
> > > > >
> > > > > With two DNS entries, one pointing to the router and the other
> > > > > to the head-gear, we do not have to worry about 'move' etc.
> > > > > The fallout is that git-push/ssh is not HA. Only the web
> > > > > requests become HA.
> > > > > Eventually we can figure out how to resolve/re-map git/ssh DNS
> > > > > if the head gear/node goes down.
> > > >
> > > >
> > > > _______________________________________________
> > > > dev mailing list
> > > > dev lists openshift redhat com
> > > > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Ce message et les pièces jointes sont confidentiels et réservés
> > > > à l'usage exclusif de ses destinataires. Il peut également être
> > > > protégé par le secret professionnel. Si vous recevez ce message
> > > > par erreur, merci d'en avertir immédiatement l'expéditeur et de
> > > > le détruire.
> > > > L'intégrité du message ne pouvant être assurée sur Internet, la
> > > > responsabilité de Worldline ne pourra être recherchée quant au
> > > > contenu de ce message. Bien que les meilleurs efforts soient
> > > > faits pour maintenir cette transmission exempte de tout virus,
> > > > l'expéditeur ne donne aucune garantie à cet égard et sa
> > > > responsabilité ne saurait être recherchée pour tout dommage
> > > > résultant d'un virus transmis.
> > > >
> > > > This e-mail and the documents attached are confidential and
> > > > intended solely for the addressee; it may also be privileged. If
> > > > you receive this e-mail in error, please notify the sender
> > > > immediately and destroy it. As its integrity cannot be secured
> > > > on the Internet, the Worldline liability cannot be triggered for the message content.
> > > > Although the sender endeavours to maintain a computer virus-free
> > > > network, the sender does not warrant that this transmission is
> > > > virus-free and will not be liable for any damages resulting from
> > > > any virus transmitted.
> > > >
> > > > _______________________________________________
> > > > dev mailing list
> > > > dev lists openshift redhat com
> > > > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > > >
> > >
> > >
> > > Ce message et les pièces jointes sont confidentiels et réservés à
> > > l'usage exclusif de ses destinataires. Il peut également être
> > > protégé par le secret professionnel. Si vous recevez ce message
> > > par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire.
> > > L'intégrité du message ne pouvant être assurée sur Internet, la
> > > responsabilité de Worldline ne pourra être recherchée quant au
> > > contenu de ce message. Bien que les meilleurs efforts soient faits
> > > pour maintenir cette transmission exempte de tout virus,
> > > l'expéditeur ne donne aucune garantie à cet égard et sa
> > > responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis.
> > >
> > > This e-mail and the documents attached are confidential and
> > > intended solely for the addressee; it may also be privileged. If
> > > you receive this e-mail in error, please notify the sender
> > > immediately and destroy it. As its integrity cannot be secured on
> > > the Internet, the Worldline liability cannot be triggered for the
> > > message content. Although the sender endeavours to maintain a
> > > computer virus-free network, the sender does not warrant that this
> > > transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.
> > >
> > > _______________________________________________
> > > dev mailing list
> > > dev lists openshift redhat com
> > > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > >
> >
> >
> > Ce message et les pièces jointes sont confidentiels et réservés à
> > l'usage exclusif de ses destinataires. Il peut également être
> > protégé par le secret professionnel. Si vous recevez ce message par
> > erreur, merci d'en avertir immédiatement l'expéditeur et de le
> > détruire. L'intégrité du message ne pouvant être assurée sur
> > Internet, la responsabilité de Worldline ne pourra être recherchée
> > quant au contenu de ce message. Bien que les meilleurs efforts
> > soient faits pour maintenir cette transmission exempte de tout
> > virus, l'expéditeur ne donne aucune garantie à cet égard et sa
> > responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis.
> >
> > This e-mail and the documents attached are confidential and intended
> > solely for the addressee; it may also be privileged. If you receive
> > this e-mail in error, please notify the sender immediately and
> > destroy it. As its integrity cannot be secured on the Internet, the
> > Worldline liability cannot be triggered for the message content.
> > Although the sender endeavours to maintain a computer virus-free
> > network, the sender does not warrant that this transmission is
> > virus-free and will not be liable for any damages resulting from any
> > virus transmitted.
> >
>
>
> Ce message et les pièces jointes sont confidentiels et réservés à
> l'usage exclusif de ses destinataires. Il peut également être protégé
> par le secret professionnel. Si vous recevez ce message par erreur,
> merci d'en avertir immédiatement l'expéditeur et de le détruire.
> L'intégrité du message ne pouvant être assurée sur Internet, la
> responsabilité de Worldline ne pourra être recherchée quant au contenu
> de ce message. Bien que les meilleurs efforts soient faits pour
> maintenir cette transmission exempte de tout virus, l'expéditeur ne
> donne aucune garantie à cet égard et sa responsabilité ne saurait être
> recherchée pour tout dommage résultant d'un virus transmis.
>
> This e-mail and the documents attached are confidential and intended
> solely for the addressee; it may also be privileged. If you receive
> this e-mail in error, please notify the sender immediately and destroy
> it. As its integrity cannot be secured on the Internet, the Worldline
> liability cannot be triggered for the message content. Although the
> sender endeavours to maintain a computer virus-free network, the
> sender does not warrant that this transmission is virus-free and will
> not be liable for any damages resulting from any virus transmitted.
>


Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité de Worldline ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis.

This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]