[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Building out new REST resources - new things to know!

Copied from a post I made upstream - this applies to all REST resources in OpenShift, 
which over the next few sprints we'll need to refactor to pick up the benefits of the 
changes made upstream.


Pull https://github.com/GoogleCloudPlatform/kubernetes/pull/4248 made the 
first set of changes to reduce the overall complexity of building out 
RESTful resource storage.  Up until now, we had several interfaces and 
objects we would implement each time we wanted to add a resource:

   1. RESTStorage (pkg/apiserver/interfaces.go#RESTStorage) an interface 
   for defining CRUD operations on a REST endpoint
      1. A REST implementation that called into a Registry for each type 
         1. This implementation does most of the validations and basic 
         checking before invoking the registry, and has a lot of duplicated code 
         across resource types
         2. Sometimes these objects are wrapped - the wrapper must 
         implement all the methods the wrapped REST object also implements
      2. Registry (e.g. pkg/registry/service/registry.go) an interface for 
   defining CRUD operations against a backend
      1. An etcd Registry implementation (e.g. pkg/registry/etcd/etcd.go) 
      that knows how to store API objects in etcd as key values
         1. This implementation is doing extremely similar things across 
         types, although there are some transactional actions that differ from type 
         to type
      3. A client interface (pkg/client/client.go) which is either a 
   Namespacer (the resource is namespace scoped) or not (the resource is root 

The Registry and Client interfaces are very similar - they take and return 
strongly typed objects for CRUD operations.  The REST implementation is 
generically typed and sometimes returns api.Status instead of the basic 
type for restful style actions (although with limitations).  I'll refer to 
these as "REST storage" or "storage" objects from now on

Anyone familiar with implementing one of these patterns a) has to duplicate 
a lot of code, b) has to write a lot of similar tests, and c) tends to make 
slightly different distinctions about where code should go.  

We have several ongoing goals in this area:

   1. Make API objects "dumb" - have the minimal amount logic around our 
   CRUD operations to allow clients to set both spec and status (depending on 
   who they are)
   2. Ruthlessly crush code duplication and bugs that arise from it
   3. Keep the client interfaces and the storage interfaces very similar so 
   that they can be treated as remote or local as needed
   4. Keep open the possibility of implementing other storage types beyond 
   etcd in the future (e.g. a SQL store or another key/value store)

For a while we have been building out generic Etcd code to apply to 
registries.  With #4248, the first of the "big" objects (pods) was 
converted to use genericetcd, with some significant enhancements added.

The generic Etcd object (https://github.com/GoogleCloudPlatform/kubernetes/blob/72da3b44244b2d9ecb5a650334aec7528808ff6f/pkg/registry/generic/etcd/etcd.go#L32) 
is a helper that provides CRUD methods that match the RESTStorage 
interfaces.  You provide a number of specific behavior functions, and it 
follows a standard pattern for creating, updating, deleting, and retrieving 
objects.  For those implementing a new type or refactoring an existing one, 
there's a set of steps you can follow:

   1. Create "pkg/registry/<type>/etcd/etcd.go" to implement RESTStorage 
   for your type for etcd
      1. It should look a bit like 
      2. If you implement all the common methods for your type, you can 
      embed the store. If you want to only expose some methods, nest it as a 
      member and wrap the methods
      3. If your type has related REST resources (like Bindings for Pods) 
      you will want to wrap the methods - Bindings, for example, have very 
      limited semantics
      4. If you have wrapper style classes, in the short term they should 
      be converted to Decorator functions on the Store. In the long term they 
      should be replaced by controller loops that write back to the store by 
      updating the Status field directly - more work on this is being done now so 
      stay tuned.
   2. Create validations for creation (ValidateType) and update 
      1. Note that update types take the old and the new object, and you 
      need to validate that updates are allowed
      2. Be sure to call ValidateObjectMeta instead of doing it yourself
   3. Create a create and update strategy in "pkg/registry/<type>/rest.go"
      1. It should look like 
      2. The strategy abstracts the normal rules for updating the resource 
      (for the base resource path), things like clearing status on create (since 
      end users typically don't status), generating names, setting UIDs and 
      creation timestamps, etc
      3. If you have referential or custom logic that runs on create or 
      update that applies to external resources (external load balancers), we 
      need to abstract it into the strategy.  Sync with me as you hit cases like 
      this as there are other refactors in progress
   4. Put other common helpers (that operate only on your internal API 
   type) in "pkg/registry/<type>/rest.go" as well
      1. This is code that anyone implementing RESTStorage on a different 
      backend would need
   5. If you need a registry interface, create an adapter in 
   "pkg/registry/<type>/registry.go" that adapts the RESTStorage interfaces by 
   casting return types
      1. Eventually we'd have an adapter for the client interfaces that 
      does this (we're pretty close) that hides remote REST and local REST and 
      uses the same path.  For now though...
   6. Add tests to "pkg/registry/<type>/etcd/etcd.go" for your type
      1. Because we've unified the REST and Registry implementations, you 
      only need to write one set of tests
      2. To make it easier to write tests, there's the beginning of a 
      generic REST test suite in "pkg/api/rest/resttest" that ensures you meet 
      common behavior.  Help wanted to flush this out further. Most types should 
      behave the same for most operations, so focus on testing the things that 
      make your type different from standard rest resources
      3. When unifying test cases for existing resources, you will need to 
      convert from using registrytest to writing to etcd.  In most cases this 
      should be straightforward, but there's boilerplate we can take away to make 
      it easier to mock
If you are currently adding new rest types or considering refactors, please 
look to adapt your type to the new mechanism.  I've got services and RCs 
lined up, but the remainder all need conversion.  Theres still more cleanup 
and features to add to genericetcd.Etcd, but we're getting much closer to 
not having to do anything to implement new REST types.  Let me know if you 
have feature requests and I'll try to ensure everyone gets what they need.

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]