[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: web consoles as broker plugins






On Mon, Jul 8, 2013 at 9:57 AM, Clayton Coleman <ccoleman redhat com> wrote:


----- Original Message -----
>
>
> ----- Original Message -----
> > From: "Mark Lamourine" <markllama redhat com>
>
> To really muddle things up, in thinking about the admin-console, I've thought
> that it should include at least a little bit of this diagnostic capability.
> Why? Because I foresee that any or all of the inputs the admin console might
> consult may not be available. Log collection is a good example; also
> optional usage tracking in MongoDB. But even the DB or mcollective might not
> be available, and I think it would be a better interface to say "you can't
> see this data value because it's not available - and here's why - and here's
> what you can do about it" rather than just assume that anything will be
> available.

Knowing what broker plugins are loaded would be nice.

At least with the three left that respond to the instance() method, you can ask the factory for an instance and determine what class you get.  We could add a method to return the @provider attribute which contains either the factory itself or the plugin class.
 

>
> > I've always wondered a bit about the "use the presence of a plugin
> > config file to decide to load the plugin" model.  That's what causes
> > the broker to burn if the plugins are missing. I've thought it would
> > be cleaner to have settings in the broker.conf that indicate which
> > plugins to load.  If that were the model, detecting missing plugins
> > would just be "read the config and note if the plugins are
> > selected".  Then you could *try* loading them and if they fail or
> > have missing config values the plugin itself could reflect and
> > present a list of missing values.
> >
> > It would also be reasonable to have the abstract classes for the
> > plugins load and inform the broker that they need to be overridden.
>
> As implemented, you can't even get past bundler collecting gems if these
> files are wrong. I'm not sure what we want to consider the minimum viable
> setup that we need to assume in order to bootstrap setup and diagnostics,
> but I think the broker and accompanying config are too complex. A
> setup/diagnose app should be so simple and fail-safe that it can be useful
> without httpd config or any other components existing, and can't be broken
> by whether optional or essential components exist or not.

Very good argument.


Bundler forces everything the code knows about to load and check.  Are there rules saying you can't load additional gems after   It does break (!?!?) the bundler mechanism, but do we care?  Or is it really not possible? 
>
> I have mixed feelings about the existing plugin include model. It does allow
> nicely for the typical RPM practice of dropping a config file somewhere to
> "configure thyself in" rather than modifying config, and it's easy for
> Gemfile to understand. On the other hand, bundler's "interface" to figuring
> out when you have a problem is not just obtuse but downright misleading.
> Would be nice if the method for including plugins landed outside bundler's
> purview entirely.

Can't really, otherwise you can't include required gems in a sane way.

Well I wouldn't call bundler "sane" but what do we really lose if we subvert it?
 

>
> > > In most cases, broker failures during load can be confined to the
> > > broker, and
> > > the appropriate fallback logic can be selected. Fallback *could* be
> > > defined
> > > as "load the setup engine and mark all controller routes to return
> > > X". On
> > > the other hand, setup mode is something that should exist for a
> > > limited
> > > amount of time and then be completely disabled (to prevent security
> > > exploitation), which is an argument for more separation.
> > >
> > >
> > >
> > >
> > >
> > > I am not aware of a way to make a rails engine quit loading itself
> > > if it is
> > > unable to find the appropriate config while allowing other rails
> > > engines to
> > > continue the
> > > load. If this is possible then maybe it does not make any
> > > difference.
> > >
> > > Should be fairly easy - I can think of several ways. It may not be
> > > worth the
> > > effort now, of course. What's important is that the controller
> > > engine be in
> > > a certain state if load if a plugin fails - not the rails app as a
> > > whole.
> > > Today, failing fast from plugins is useful because it helps isolate
> > > problems. In the future clean and coherent troubleshooting on
> > > startup via
> > > helpful translation of errors may be more important.
> >
> > And the point of my original comment was "we should consider this
> > when discussing changes to the Rails broker/console/admin routing
> > and when designing the setup application."
> >
> >
> > >
> > >
> > >
> > >
> > >
> > > I am also looking at the initial setup like the wordpress like
> > > setup step
> > > where you specify the db/user/pass, select the appropriate
> > > auth/dns/node-container
> > > plugins. And once set up, don't need to invoke the setup again till
> > > a migrate
> > > or something is necessary. What I am thinking off may just be
> > > different from
> > > what
> > > you are imagining the setup rails engine will do.
> > >
> > > Both Drupal and Wordpress implement this setup flow inside
> > > themselves - its
> > > different than rails of course mostly because they have the ability
> > > to throw
> > > away their loaded environment. The parallels don't imply we have to
> > > do it
> > > that way - but the ability to use the load of the framework to test
> > > the
> > > settings is certainly an important part of their experience.
> > >
> > >
> > >
> > >
> > >
> > > --kr
> > >
> > > On Jul 7, 2013, at 8:58 AM, Clayton Coleman < ccoleman redhat com >
> > > wrote:
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Jul 7, 2013, at 11:46 AM, Krishna Raman < kraman gmail com >
> > > wrote:
> > >
> > >
> > >
> > > I wanted to keep it out of the discussion because the integrated
> > > web setup is
> > > special. It needs to be running while the other components are not.
> > >
> > > Why? There's no real reason why the broker has to crash and burn if
> > > it's
> > > improperly configured, nor is there any real requirement that all
> > > the broker
> > > setup code is run once. In fact, over time, as we move to improve
> > > how the
> > > oo-* scripts boot and load (to reduce their dependence on the
> > > entire rails
> > > infrastructure) we will already need most of the initialization
> > > code to be
> > > cleanly abstracted from rails (invoked, but not coupled with).
> > >
> > > I'm not putting constraints on how the setup should work, merely
> > > highlighting
> > > that all of our arguments need to be framed around justifying *why*
> > > something is different/special - i.e. not a properly decoupled
> > > rails engine
> > > that can be hosted in many places. Choosing to put integrated setup
> > > someplace else is a valid decision, made with the understanding
> > > that there
> > > is a reason to deviate from the norm.
> > >
> > >
> > >
> > >
> > > And once it has finished setting everything up, it will be turned
> > > off and the
> > > others will be turned on. I don't mind it being a plugin, but it
> > > most likely
> > > will not be run in the same process space as the broker or console
> > > as the
> > > config entered needed by the broker/console plugins would not be
> > > present
> > > while the setup is running.
> > >
> > > --kr
> > >
> > > On Jul 7, 2013, at 8:35 AM, Clayton Coleman < ccoleman redhat com >
> > > wrote:
> > >
> > >
> > >
> > >
> > > Mark's right - a component like "integrated web setup" (regardless
> > > of when or
> > > whether its implemented) is appropriate to mount as a plugin to the
> > > broker,
> > > and needs the flexibility to be rerouted to a base URL. It's not
> > > broker
> > > setup to an end deployer - its OpenShift setup.
> > >
> > > Eventually we will absolutely be required to share a number of
> > > common web UI
> > > components between the various parts of Openshift, especially web
> > > assets,
> > > layouts, common visual helpers, etc. Regardless of the app ->
> > > passenger ->
> > > route topology, we will start sharing the underlying code. That
> > > will allow
> > > us to make the right choices here.
> > >
> > > The implication beyond that is we need to be developing and
> > > integrating
> > > *isolated* components, that can be flexibly moved across rails apps
> > > in the
> > > future. Things like namespacing our rails plugins (which we aren't
> > > great at
> > > today), ensuring we keep core library code properly independent,
> > > and
> > > thinking through cases like these as Luke has done.
> > >
> > > On Jul 7, 2013, at 8:36 AM, Mark Lamourine < markllama gmail com >
> > > wrote:
> > >
> > >
> > >
> > >
> > > Krishna: the reason I put my foot in was because it had stopped
> > > being about
> > > the admin console and had started being about integrating the web
> > > functions
> > > in the web application of the broker. The desire for another
> > > component could
> > > affect the decision.
> > >
> > > In this context I'm just concerned with its presence, not it's
> > > requirements
> > > (other then being available and avoiding conflct.)
> > >
> > > - Mark
> > >
> > >
> > > On Sun, Jul 7, 2013 at 1:27 AM, Krishna Raman < kraman gmail com >
> > > wrote:
> > >
> > >
> > >
> > > Hey Mark,
> > >
> > > Harrison and I will be looking into the setup application for
> > > prebuilt VM
> > > images as part of Origin improvements we have been discussing.
> > > Prefer not to mix that requirement into the discussion about admin
> > > console.
> > >
> > > --kr
> > >
> > > On Jul 6, 2013, at 6:10 PM, Mark Lamourine < markllama gmail com >
> > > wrote:
> > >
> > >
> > >
> > >
> > > Just to throw gas on the fire, an additional "application" that I'd
> > > like to
> > > see is a "setup" tool which would be presented to the root (/) of
> > > the broker
> > > web (possibly by redirect to some other path) when the broker has
> > > been
> > > installed and started but not configured.
> > >
> > > The purpose of this app would be to guide the new implementer
> > > through the
> > > process of configuring the database information and
> > > selecting/configuring
> > > the broker plugins. It would not be expected to actually install or
> > > setup
> > > the back-end services ala an all in one, but might guide the
> > > installer to a
> > > tutorial for each one as needed.
> > >
> > > I'm not suggesting that this be added to existing planned work, I'm
> > > just
> > > wondering how something like this might fit into the presentation
> > > of an
> > > extended broker (REST interface, developer app console etc.)
> > >
> > > - Mark
> > >
> > >
> > > On Fri, Jul 5, 2013 at 11:30 AM, Luke Meyer < lmeyer redhat com >
> > > wrote:
> > >
> > >
> > > I'd like to solicit some feedback on this before getting too deep
> > > into it and
> > > see what pieces of the puzzle I may be unaware of.
> > >
> > > We're beginning work on adding an administrative console - a webapp
> > > dashboard
> > > of sorts for administrators to visualize and navigate some basic
> > > OpenShift
> > > entities (initially read-only). Draft PEP is at
> > > https://github.com/openshift/openshift-pep/blob/master/openshift-pep-001.md
> > > (very much a work in progress).
> > >
> > > Since this will involve pulling in basically all the same domain
> > > concepts as
> > > are already present in the broker, the thought has been to provide
> > > this as a
> > > broker plugin (a rails engine). In that case, it could be
> > > optionally added
> > > to the broker app via configuration and would use the same process
> > > space.
> > >
> > > We could consider the same for the developer console, though it
> > > performs all
> > > access via the REST API and doesn't share anything in the way of
> > > actual
> > > domain objects. Still, having a completely separate server for it,
> > > or even
> > > just a separate Passenger instance, seems unnecessary.
> > >
> > > Now for some implementation nitty-gritty. The way we have things
> > > today, the
> > > broker host httpd (where authentication is likely configured)
> > > proxies
> > > /broker to the broker application bound to localhost and running in
> > > a single
> > > Passenger instance scoped to the /broker URI ("RackBaseURI
> > > /broker"). So
> > > anything that runs in the broker space has to begin with /broker
> > > and if
> > > httpd is responsible for authentication, it occurs based on this
> > > URI as
> > > well. This is not ideal for the admin console, which isn't
> > > conceptually
> > > "part of" the broker, authenticates separately (if at all), and
> > > which may
> > > need to proxy differently (or, not at all).
> > >
> > > Is there really a thought that we will ever add another Passenger
> > > definition
> > > to the /broker one now existing?
> > >
> > > My thought was to remove "RackBaseURI /broker" from the Passenger
> > > broker conf
> > > and just adjust the broker routes scope, which now begins with
> > > "/rest" to be
> > > "/broker/rest" (any routes added by plugins would need to be
> > > similarly
> > > adjusted). The broker doesn't have any assets to worry about so
> > > that seems
> > > to be all that is necessary; I did this and ran the API tests and
> > > all
> > > passed, so this seems viable.
> > >
> > > This would enable mounting the admin console at "/admin-console" on
> > > the
> > > localhost server without having it automatically proxied externally
> > > -
> > > administrators could choose to configure access to that however
> > > they wanted
> > > (proxying, port-forwarding, auth). There are certainly other ways
> > > to handle
> > > this; it just seems the simplest conceptually. Likewise, just mount
> > > the
> > > developer console at /console in the same app.
> > >
> > > Some pros of this approach:
> > > 1. Fewer optional services to chkconfig/start and account for
> > > during
> > > upgrades/outages.
> > > 2. Simpler proxy/auth config at the host httpd.
> > > 3. Fewer processes taking up memory and initialization time.
> > >
> > > Some cons:
> > > 1. Coupling components like this potentially raises the complexity
> > > of the app
> > > / bloats each process.
> > > 2. Is there a potential security risk in external-facing broker app
> > > sharing
> > > process space with admin functions?
> > > 3. Have to have a broker running where you want an admin console
> > > (already
> > > required for developer console due to auth requirements).
> > >
> > > What else have I not considered?
> > >
> > > _______________________________________________
> > > dev mailing list
> > > dev lists openshift redhat com
> > > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > >
> > >
> > >
> > > --
> > > ----
> > > Mark Lamourine < markllama gmail com >
> > > Dad, Hubbie, Software Developer, System Administrator, Road Cyclist
> > > _______________________________________________
> > > dev mailing list
> > > dev lists openshift redhat com
> > > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > >
> > >
> > >
> > >
> > > --
> > > ----
> > > Mark Lamourine < markllama gmail com >
> > > Dad, Hubbie, Software Developer, System Administrator, Road Cyclist
> > >
> > >
> > > _______________________________________________
> > > dev mailing list
> > > dev lists openshift redhat com
> > > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > >
> > >
> > >
> > > _______________________________________________
> > > dev mailing list
> > > dev lists openshift redhat com
> > > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > >
> >
> > _______________________________________________
> > dev mailing list
> > dev lists openshift redhat com
> > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> >
>

_______________________________________________
dev mailing list
dev lists openshift redhat com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev



--
----
Mark Lamourine <markllama gmail com>
Dad, Hubbie, Software Developer, System Administrator, Road Cyclist

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]