[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Some thoughts on shared access to applications

On 07/09/2013 10:52 PM, Clayton Coleman wrote:

On Jul 9, 2013, at 6:55 PM, Chris Alfonso <calfonso redhat com> wrote:

On 09/07/13 15:04 -0400, Clayton Coleman wrote:

----- Original Message -----
On 09/07/13 14:38 -0400, Luke Meyer wrote:
----- Original Message -----

From: "Clayton Coleman" <ccoleman redhat com>
To: "Luke Meyer" <lmeyer redhat com>
Cc: dev lists openshift redhat com
Sent: Tuesday, July 9, 2013 1:36:30 PM
Subject: Re: Some thoughts on shared access to applications

----- Original Message -----
I can think of at least three reasons why people may want to group

1. Access control - I want to collaborate with group X on this set
of apps,
but not another.
2. Grouping layers of a composed app, allowing them to use each
other as
resources while possibly scaling separately - for instance, a
layer, an API layer, a UI layer that are all separate apps.
3. Access to / specification of shared resources; a Jenkins app is
a good
example, but also external DB definitions or other resources.

Not trying to list everything here, but it shouldn't be hard to see
that the
different purposes are likely to be orthogonal, and it would
probably be bad
to use the domain as the scope for all of them. What is a domain
anyway? Right now it's nothing but a device for keeping app names
unique on
the cloud domain. Does it need to be anything more than that? It
seems to me
that group membership for other purposes should be more of a
tagging system.

Elaborate? Define tagging?
Specify each grouping that an app belongs in as a tag on the app. To
illustrate the point with speculative functionality:

This sorta sounds like the AWS concept of a security context. Would having a
separate entity (security context) associated with all the other primary
entities help. It seems like we are trying to derive access control using
using entity relationships that are there for other purposes rather than
replying on one discriptor of access control, such as a security context.
So in AWS, every entity is associated with one security context?  Can you compose contexts?
I was suggesting we borrow the aws IAM rbac policies and permissions idea and
just called it a security context as an umbrella entity that could be
associated with the first account created. Additional users, domains, and
applications could them be added to the security context based upon whatever
policies are created. I'm not saying this is trivial by any means; however, I
am saying it was complicated enough to protect AWS resources that an entire
permissions system and management console was created to address the problem
rather than try to derive permissions and policies based upon entity
relationships inately.
I can't tell whether that's a good thing or a bad thing :).  I'm certainly sympathetic to ensuring we solve the right problems with access control.  A lot of the initial modeling was a combination of hierarchical access control (projects have acl, which the resources within them inherit) and is similar to the github model (just complex enough to solve 98% of use cases and no more).

Hypothetically, how would a security context model work?

A user has one security context to start

All resources they create have that security context by default.

Applications are still part of a single domain 

A user account owns applications for the purposes of billing

In order to allow teams to simplify billing and resource management, an owner needs to be able to delegate creation of apps to others under his account.  Hard to imagine this will be easy to model to users without a domain ownership relation.

Therefore, a domain has a security context as well.  Apps created in the domain inherit that security context as well.

Users have the ability to create multiple security contexts.  A security context is a list of users and roles.
I would first refer to this as our 'user' security model (as opposed to application security model - more on that below).

Amazon's IAM system is built up of security policies, groups and users (and roles - more below).  Security policies define what actions can be done on what resources.  Those policies can be attached to groups or users.  Users can also be a part of groups and inherit any policies attached to the group.

An example of a security policies ( http://aws.amazon.com/iam/#detailed_description) would be something like:

  "Statement": [
      "Sid": "Stmt123456789012345",
      "Action": [
      "Effect": "Allow",
      "Resource": "*"

This rule allows the action ListAllMyBuckets to be execute on the s3 system across all resources.  Resources and actions support a range of specifiers from very granular to wildcard.

Assigning a security context to a resource changes the materialized membership list on any resource.

There is no need to overlay two distinct security contexts on a single resource.
I believe the AWS approach in this case is to take the most restrictive interpretation in the case of overlaps.

Security contexts need to be clearly labelled by their users

When adding users to a security context, the resources that are owned by that context must be clearly communicated to users (to prevent unintended access escalation).  This is a similar problem as to Teams as described originally.

A domain owner must be able to clearly identify the security contexts in play in each application - they cannot assume that the permissions on the domain apply to the apps (if apps are allowed to change permissions).

To add one person to a specific app, you will have to either duplicate the security context (and it stops being in sync with the original) or implement a security context inheritance model.  

To deny access, the same problem applies (although this is not a very common case anyway)
In the AWS model, the 'effect' is used to allow or deny.

A security context needs to have a role which allows a member to alter the security context.
If we try and align with the AWS terminology, 'role' is actually used to define the application model.  I'm not as familiar with the specifics but conceptually, a role is used to assign security permissions that will be applied to instances on creation.  That keeps you from having to copy user permissions to the instances themselves.  I typically think of this in line with Kerberos service principal type access.

Only the owner of a resource can change what security context is associated with a resource, because otherwise they could be removed from access.
This has been one of the weaknesses I've seen in Amazon's model.  To my knowledge, there is no concept of 'owner'.  For example, I've always wanted to create a rule that allows users to only terminate instances that they have created.  However, that mapping and ownership is not propagated to the rules.

More complicated permission models within a context beyond a simple role seem unlikely - even the several roles sketched out below are pushing the number of gradations you can create.

Other characteristics of a model like this I'm missing?


If you (a developer) want to allow developers on team A to access a set of
apps, you tag all those apps with "access_group: team_A" and somewhere you
define who is in team A and what role(s) they get (perhaps this is in
LDAP), and that is applied to whatever apps you give that tag.

If you want to compose a set of apps into one meta-app and have them find
each other according to rules you define, then you tag them as
"composed_app: app_A" and the broker connects them with the necessary env
vars (something like domain variables today, with Jenkins as the "build

If you want to specify a set of env vars that will be used by multiple apps
(say, to access a DB) then you tag all those apps as "resource_group: db_A"
and something like "rhc set-env db_password=foo --tagged db_A" sets the var
for all those apps.

And just in case a domain (as we know it today) actually does correspond to
something they want to group by (e.g. dev/qa/prod domains with different
versions of the same apps may correspond with a resource they all use), it
could be an implicit tag ("rhc set-env foo=bar --tagged domain_A"), or
there could just be a way to associate all current and future members of a
domain explicitly with a tag.

The same app could be in multiple groupings, which is what makes me think of
tags. So, apps in multiple domains or groupings could use with the same
Jenkins builder just by tagging it multiple times, but it would still be
easy to have a separate Jenkins for say dev apps and prod apps.

----- Original Message -----
From: "Clayton Coleman" <ccoleman redhat com>
To: "meghdoot bhattacharya" <meghdoot_b yahoo com>
Cc: dev lists openshift redhat com
Sent: Monday, July 8, 2013 5:15:45 PM
Subject: Re: Some thoughts on shared access to applications

One of the biggest challenges, as we move from the single-domain
per account
model, is to adapt how we think about domain-wide concepts like
today. The simple path we've started on is a domain-wide

I have a domain A
When I add Jenkins in A, all other applications have access to it
Jenkins contributes "domain" environment variables that all other
apps have
access to.
Once Jenkins exists, I can add the jenkins client cart to other
apps in A
The Jenkins client cart uses the domain wide environment variable
to check
whether it's installable
If Jenkins is removed, it removes the jenkis client cart from other
apps in

If we start to use domains to scope sets of users, how does Jenkins
react? I
personally would be frustrated having to have one Jenkins per
because Jenkins either costs me money or provisioning space (Online
Enterprise). Hypothetically, what sorts of changes are required to
concepts like a shared Jenkins across domains:

Each user may have different access to different domains
- adding the Jenkins Client and pointing it to a Jenkins server
domain B
may mean that some users in domain A can't see the server
- the info the client needs is from variables (url, user, and
password) so
the broker access is redundant

The Jenkins server needs to know which Jenkins clients are
connected to it
so it can disable the client carts if it's removed
- because that could be every domain in the server, we'd need to
track the
"connection" between the server and client differently

The Jenkins client won't have access to the domain wide environment
- instead, the Jenkins client needs to be able to get access to the
about the jenkins server on install
- the broker could pass it, but then the broker has to know about
of environment variables
- The env var would have to be removed on deletion

Thinking through the problem, it seems like what we would be trying
to do is
register a "connection" between a jenkins server and client. The
needs to expose a set of arbitrary info that the client must
consume. That
connection seems to be very similar to cartridge hooks, just
between apps.
One possible approach might be to allow client cartridges to
indicate they
have an external dependency that must be satisfied (app with
component X) on
creation, and then allow the client and server to expose an
connection hook that the broker could resolve prior to creation.

For example:

1) User attempts to add jenkins-client to an app
2) Broker sees that jenkins-client declares a "jenkins-server"
that is cross-application
3) Broker looks in the domain for an app that can satisfy that
a) if it finds multiple, or none, the user will have to specify the
appropriate app
b) when adding a cart, user would have to tell the REST API what
should satisfy the dependency, including any app they have access
c) some thought must be given to backwards compatibility with
4) The broker checks for an external subscribe connection hook on
client cart and attempts to match that to an external publish hook
in the
server cart
5) If it succeeds, it runs the server publish hook, then includes
output on the cartridge creation (or triggers the hook as part of
6) Cart configure succeeds or fails based on whether the input from
#5 is
acceptable, broker reports that back

Obviously speculative, and is not the first problem we would go
solve... but
there are some interesting parallels there to app<->app connection
hooks and
how we would track and manage that.

----- Original Message -----
Much needed and highly anticipated feature. Not only for
considerations where different groups: dev, release engineers,
24X7 Ops
would want to operate on the same application for different
aspects of
management, the simplest case during feature development where a
dev asks
the teammate to debug an app in their gear independently can be
supported without the need of sharing explicitly ssh keys and rhc
Something like multiple domains per User can be used to have
isolated mesh
interconnected apps/topology.
Personally from my view point I like the role classification

From: "dev-request lists openshift redhat com"
<dev-request lists openshift redhat com>
To: dev lists openshift redhat com
Sent: Saturday, July 6, 2013 9:00 AM
Subject: dev Digest, Vol 16, Issue 23

Send dev mailing list submissions to
dev lists openshift redhat com

To subscribe or unsubscribe via the World Wide Web, visit
or, via email, send a message with subject or body 'help' to
dev-request lists openshift redhat com

You can reach the person managing the list at
dev-owner lists openshift redhat com

When replying, please edit your Subject line so it is more
than "Re: Contents of dev digest..."

Today's Topics:

1. Some thoughts on shared access to applications (Clayton
2. Munin plugins for OpenShift (Tim Bielawa)


Message: 1
Date: Fri, 5 Jul 2013 16:27:48 -0400 (EDT)
From: Clayton Coleman < ccoleman redhat com >
To: Openshift Dev < dev lists openshift redhat com >
Subject: Some thoughts on shared access to applications
< 2128282006 24947482 1373056068712 JavaMail root redhat com >
Content-Type: text/plain; charset=utf-8

There's been discussion recently about how multiple users could
access an
individual application. There have been various comments on it
over time -
wanted to summarize the latest thinking and start a discussion
about the
various features/behavior.

Current model:

A CloudUser *owns* a Domain, which *contains* zero or more
A Domain is a logical grouping of Applications

-> domain (contains)
-> owner (owns)

Application has SSH keys from its owner
The usage and limitations of the application reflect the
capabilities of
owner domain (the ability to upload client certificates, etc)

Possible changes:

A CloudUser may own many Domains
A Domain has one or more Members
An Application has one or more Members
All Members of a Domain are Members of the contained Applications
A Member has a Role within an Application or Domain
A Member who has a Role on a Domain, has an equivalent Role on
Members have *view* access on the resource and any contained
Applications receive SSH keys from all Members with sufficient
A CloudUser has *view* rights on CloudUsers on the Domains they
are a
(i.e. you can only see users you collaborate with)
A configuration setting would allow CloudUsers to view all other
Otherwise, adding a user would require sharing an Invitation via
a unique

Possible future additions:

A Team has one or more Members
A Team has *access* to one or more Domains
A Team has a Role, that is assigned to all Members when they are
added to
A Member may be assigned a more limited Role on a specific

Here are some proposed Roles (roles inherit the permissions of
Multiple names are suggested as examples

- can view the application or domain in the REST API
- no access to viewing environment variables or SSL certificates

- can stop/start/restart individual gears or the entire
- can change scale factors
- can read and set environment variables
- can view and set aliases, including SSL certificates

- can add/remove cartridges
- can trigger builds or deployments
- can create applications in a domain (and delete apps they

- can add/remove users, change roles, delete applications, delete

- can transfer ownership of a domain

We've discussed splitting SSH access off into a separate variable
- a Gear

- the Member's SSH keys are not transferred to the gear, and so
has no SSH
Git access

- can SSH to the gear
- can push changes to Git

* In the future, it might be nice to have a role in between none
edit/write where the user has only read access to the gear. This
is a
complex implementation though and needs more thought.

* All owners and admin/managers are implicitly
edit/write/push/ssh enabled
(since they can change their own roles).

Some simplifying assumptions we've discussed:

All access to applications is implicit by virtue of Domain

- You are granted access to an application by virtue of being a
Member of
containing domain
- The point of a Domain is to group applications. Since grouped
often have security interconnections, it's natural to make the
that access to one is implicitly granting access to all. Multiple
can be used to partition applications.
- This also simplifies the access control protection on domains -
clients may need to access only a single app, but want to use the
API to navigate to the app. Backwards compatibility is pretty

Team membership would be denormalized to the Applications

- Allows fast read access control patterns for showing a list of
you have access to and checking membership
- Has the possibility to introduce "membership storms" where
adding a user
a team triggers jobs on hundreds of applications. Team membership
might be queued centrally to avoid this
- The "team" would also be a member on the app
- Denormalization has to be handled cleanly so that removing a
user from
domain doesn't remove the Member entry for a user from the team

Any sort of external group integration would map to a Team

- For instance, to map an LDAP group to a Team, you would store a
bit of
metadata with the Team and then when the user logs in you could
do an
initial LDAP groups -> team check to provision the user. A
process would manage LDAP removal. Only active OpenShift users
would be
materialized into the Team (prevents excessive group operations)

Inheritance of roles

- Rather than do complex permission bits, it's conceptually
simpler to have
set of nested roles where each level lets you do more things. If
we can get
the roles right, that makes the modelling and behavior a lot
easier to
- A lot of operations on the app are destructive (i.e. stopping
an app
the app is useless). So most of the higher roles are ordered from
least to
most destructiveness

Queries that have to be fast:

- List of applications I have access to
- List of domains that I have access to
- Do I have access to this domain/application?

The most efficient model for those queries is going to be a list
of members
on each application object, with an index on the nested member
ids. We are
explicitly assuming high hundreds of users on each application,
which means
the index selectivity will be high. Also, given that the list of
applications may span tens or hundreds of domains, the info
needed for
generating a REST API response needs to be denormalized from the
the application (probably just domain_namespace).

I'm sure I've left some stuff out - please ask if something
fundamental is


Message: 2
Date: Fri, 05 Jul 2013 17:11:40 -0400
From: Tim Bielawa < tbielawa redhat com >
To: "Openshift Dev" < dev lists openshift redhat com >
Subject: Munin plugins for OpenShift
Message-ID: < 87wqp4k8z7 fsf deepfryer rdu redhat com >
Content-Type: text/plain; charset="us-ascii"

I wrote a simple munin plugin for charting gears present/idle on
[0]. All the relevant bits are available on GitHub [1].

Does anyone else have any useful plugins written for
Munin+OpenShift? I
couldn't find any in a standard 5-second Google search for "munin

Obligatory pretty pictures (fancier ones are in references):



I'd love to collaborate if there's any interest in this beyond
what I've
done already (pull requests accepted).


[0] http://blog.lnx.cx/2013/07/05/a-munin-plugin-for-openshift/
[1] https://github.com/tbielawa/openshift-munin-plugins

Tim Bielawa, Software Engineer/Scribe
We are all cloud
GCA - Production Control Team (RDU)
919.332.6411 Cell | IRC: tbielawa (#gca)
1BA0 4FAB 4C13 FBA0 A036 4958 AD05 E75E 0333 AE37
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 162 bytes
Desc: not available
URL: <


dev mailing list
dev lists openshift redhat com

End of dev Digest, Vol 16, Issue 23

dev mailing list
dev lists openshift redhat com

dev mailing list
dev lists openshift redhat com

dev mailing list
dev lists openshift redhat com

dev mailing list
dev lists openshift redhat com

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]