[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Support for distributed applications



It's a bit difficult to get endpoints today from the REST API.  You can listen to the publish-http-endpoint-url event sent to HAProxy in order to collect those on the gear, but it's harder to get from externally.  The gear_groups API lists the gears, but not the ports.

As part of the HA pep, we will expose an endpoints REST API call that returns a hostname, port, protocol, type, cart, and gear id for each visible endpoint.  That'll assist in getting that info.  (https://github.com/openshift/openshift-pep/blob/master/openshift-pep-005.md#routing-table-data-model-wip)

For now, there is no great option.

----- Original Message -----
> thanks for all the replies!
> 
> Distributed applications are thus no issue: simply pass the gear events
> around without the need for a component such as 'gearfinder'. Nice that
> this is supported.
> 
> I guess I do still need a gearfinder application to let other applications
> find the tcp endpoints? We are not doing HTTP between applications thus I
> need a way to discover from ApplicationA where the registered endpoints for
> ApplicationB are. Or could I do that gear-groups call for this purpose as
> well (using the broker auth key from ApplicationA)?
> 
> 
> On Wed, Jul 10, 2013 at 6:36 PM, Clayton Coleman <ccoleman redhat com>wrote:
> 
> > And the key part of the hook example is parsing a structured response of
> > key values, vs something simpler.  There are other ways to serialize the
> > gear data across the wire.
> >
> > On Jul 10, 2013, at 12:22 PM, Chris Alfonso <calfonso redhat com> wrote:
> >
> > > On 10/07/13 17:59 +0200, Romain wrote:
> > >>
> > https://github.com/smarterclayton/openshift-redis-cart/blob/master/hooks/subscribe-redis-connection-info
> > >> returns
> > >> a 404
> > >>
> > >>
> > >> 2013/7/10 Clayton Coleman <ccoleman redhat com>
> > >>
> > >>> I would recommend looking at the Redis cart for an example of doing
> > gear
> > >>> discovery.
> > >>>
> > >>>
> > >>>
> > https://github.com/smarterclayton/openshift-redis-cart/blob/master/hooks/publish-redis-connection-info
> > >>>
> > >>>
> > https://github.com/smarterclayton/openshift-redis-cart/blob/master/hooks/subscribe-redis-connection-info
> > >
> > >
> > https://github.com/smarterclayton/openshift-redis-cart/blob/master/hooks/set-redis-connection-info
> > >
> > > The hook names just need to match what they are named in the manifest
> > >
> > https://github.com/smarterclayton/openshift-redis-cart/blob/master/metadata/manifest.yml
> > >
> > >>> There is an open story to simplify the way this info is passed to the
> > >>> connection hook, and also to expose a mechanism so that the subscribe
> > hook
> > >>> can set an ENV variable that the web framework has access to.  If
> > you're
> > >>> using erlang as a web framework (vs. a connected service cartridge)
> > then
> > >>> you don't need that second part.
> > >>>
> > >>> You could also use the broker auth key stored in the gear to make a
> > call
> > >>> to the REST API for the app and get the gear topology (via the
> > gear-groups
> > >>> call) and then generate it that way - the hooks are the *more* correct
> > way
> > >>> of doing that today.
> > >>>
> > >>> ----- Original Message -----
> > >>> > Responded at the forum so those riding Google can see it too...
> > >>> >
> > >>>
> > https://www.openshift.com/forums/openshift/clustered-applications-knowing-the-topology#comment-32203
> > >>> >
> > >>> > Can discuss here further, but I think the pub-sub cartridge
> > capabilities
> > >>> can
> > >>> > get you what you need. I also think having a scalable Erlang
> > cartridge
> > >>> would
> > >>> > be pretty cool :)
> > >>> >
> > >>> > ----- Original Message -----
> > >>> > From: "Thijs Terlouw" <thijsterlouw gmail com>
> > >>> > To: dev lists openshift redhat com
> > >>> > Sent: Wednesday, July 10, 2013 9:56:29 AM
> > >>> > Subject: Support for distributed applications
> > >>> >
> > >>> >
> > >>> > This question is an extension from an OpenShift forum post [1].
> > >>> >
> > >>> > We at Spilgames are using Erlang as a core component of our
> > >>> architecture. One
> > >>> > of the core strengths of Erlang is the support for building
> > distributed
> > >>> > applications. Erlang also makes it trivial to hook various components
> > >>> > together in a network and execute RPC calls between them. These
> > features
> > >>> all
> > >>> > rely on being able to interconnect the components. A custom TCP-based
> > >>> binary
> > >>> > protocol is used between Erlang components. Erlang wants to build a
> > fully
> > >>> > connected network where all components can talk directly with
> > eachother.
> > >>> In
> > >>> > order to do this, we need to know the endpoints of the other
> > instances
> > >>> > (host/ip + listen port).
> > >>> >
> > >>> > Currently in OpenShift all Gears are fully isolated containers and
> > those
> > >>> > containers normally do not talk to eachother. This prevents building
> > any
> > >>> > kind of distributed application, including distributed Erlang. We
> > >>> believe we
> > >>> > can work around this with custom components that register the new
> > >>> instances,
> > >>> > but we also believe that this functionality is generic enough that
> > the
> > >>> > platform should offer support for it. Let's first talk about the
> > proposed
> > >>> > work-around and then about let's come back to the vision that RedHat
> > has
> > >>> for
> > >>> > OpenShift PaaS.
> > >>> >
> > >>> > For the work-around I am thinking of introducing a "gearfinder" REST
> > >>> > application. This gearfinder starts in the primary gear [2] and
> > offers a
> > >>> > REST component. It is thus available through a fixed location (dns).
> > >>> > Whenever a new gear starts, it attempts to register with the
> > gearfinder.
> > >>> The
> > >>> > new gear also asks the gearfinder for a list of known gears, which
> > >>> returns a
> > >>> > list of information such as cartridge-name, gear-id, listen ip+port
> > and
> > >>> > host. My Erlang application can then get all the unique hosts and
> > >>> connect to
> > >>> > each of those EPMDs to get a list of nodes on that host. Next it can
> > >>> connect
> > >>> > to the Erlang gears indirectly via the proxy. See attached picture;
> > the
> > >>> > numbers in the notes help to understand the proposed flow. Basically
> > this
> > >>> > component is just keeping track of the gears, which is information
> > that
> > >>> is
> > >>> > probably already in the system.
> > >>> >
> > >>> > Whether distributed applications should be supported, depends on the
> > >>> vision
> > >>> > that RedHat has for OpenShift. Does RedHat just want simple web-based
> > >>> > applications that run in a silo? Or do you also want to offer a
> > platform
> > >>> > that can be used to build complex distributed applications. I believe
> > >>> with a
> > >>> > few minor tweaks it can do both. Do note that my example is probably
> > not
> > >>> > suitable for the Online platform, but those issues can be solved. I
> > think
> > >>> > distributed applications are not disallowed by the PaaS concept. A
> > PaaS
> > >>> > should try to support as many different languages and architectures
> > as
> > >>> > possible. It would also be a nice competitive advantage that
> > OpenShift
> > >>> has
> > >>> > over other PaaS provider; I believe nobody offers this yet.
> > >>> >
> > >>> >
> > >>> > [1]
> > >>> >
> > >>>
> > https://www.openshift.com/forums/openshift/clustered-applications-knowing-the-topology
> > >>> > [2] thinking about solutions to avoid the SPOF
> > >>> >
> > >>> > _______________________________________________
> > >>> > dev mailing list
> > >>> > dev lists openshift redhat com
> > >>> > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > >>> >
> > >>> > _______________________________________________
> > >>> > dev mailing list
> > >>> > dev lists openshift redhat com
> > >>> > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > >>> >
> > >>>
> > >>> _______________________________________________
> > >>> dev mailing list
> > >>> dev lists openshift redhat com
> > >>> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > >
> > >> _______________________________________________
> > >> dev mailing list
> > >> dev lists openshift redhat com
> > >> http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > >
> >
> 
> 
> 
> --
> Thijs Terlouw,
> http://www.startinchina.com
> 


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]