[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: PHP developers workflow discussion




----- Original Message -----
> ----- Original Message -----
> > From: "Clayton Coleman" <ccoleman redhat com>
> > To: "Vojtech Vitek" <vvitek redhat com>
> > Cc: dev lists openshift redhat com
> > Sent: Friday, March 15, 2013 7:15:56 PM
> > Subject: Re: PHP developers workflow discussion
> > 
> > 
> > 
> > ----- Original Message -----
> > > Hi,
> > > this thread is meant to start discussion about the possible
> > > improvements
> > > for the PHP (and other interpreted scripting languages)
> > > developers.
> > > 
> > > 1) Self-modifying apps, built-in installers
> > > 
> > > PHP frameworks often use the self-modifying functionality of it's
> > > own
> > > source files. Not just for storing the application settings, but
> > > also
> > > for updating/installing the functionality.
> > > 
> > > As an example, both Wordpress/Drupal have built-in installers
> > > that
> > > can
> > > be used via the administration; so the site admins are able to
> > > update/install new plugins and themes on-the-fly.
> > > 
> > > Normally, all the changes are made in the app's source code
> > > itself:
> > >  Drupal: app-root/repo/php/sites/*/{modules,themes}
> > >  Wordpress: app-root/repo/php/wp-content/{plugins,themes}
> > > 
> > > This is nothing we would like on OpenShift, right? Because
> > > whenever
> > > the Developer git pushes, these changes are all wiped out..
> > > 
> > > So what's the workaround? They could probably use the persistent
> > > storage
> > > (symlink the directories in git relatively to the
> > > $OPENSHIFT_DATA_DIR),
> > > - but is that enough? Would that be even maintainable?
> > > 
> > 
> > As someone who recently rewrote the Drupal/Wordpress quikcstarts I
> > can give some feedback I've heard from people opening bugs against
> > me.
> > 
> > There are two important sets of users:
> > 
> > 1) Those who want to use drupal/wordpress more like an admin
> > 2) Those who are more comfortable dealing with it more like a
> > developer
> > 
> > Group #1 wants the content in the data dir, and they expect
> > Wordpress
> > "to just work".  Without shared storage though, scaling those apps
> > is hard because the data dirs have to be synced, and then
> > replicated.  Down the road, we want this to just work with scaling
> > but we're not there yet
> > 
> > Group #2 wants to check the code in and push changes out, they're
> > ok
> > with doing some hacking to get there.  For these guys, if they
> > check
> > their plugins in they can scale their apps, but they lose the web
> > experience (mostly)
> > 
> > However, both of these frameworks have different requirements for
> > the
> > data dir.  What might be nice here is some sort of magic in the
> > cartridge - perhaps a ".openshift-links" marker file in a directory
> > that will autosymlink an ${OPENSHIFT_DATA_DIR} equivalent at build
> > time.  For Wordpress this could "almost" work - we might still have
> > to add some more logic.  This starts to stray dangerously into
> > proscribing a format, but the alternative is to start hacking the
> > same code into the build scripts.  The alternative might be a
> > helper
> > that lives in the build hook that autosymlinks dirs.
> > 
> > > 
> > > 2) SFTP to the app-root/repo/
> > > 
> > >  - By the way, this is the default workflow used by the Zend
> > >  Studio
> > >    while developing the Zend apps on OpenShift right now..
> > > 
> > > I've talked to many PHP developers about their feel and
> > > satisfaction
> > > with the OpenShift PaaS and most of them told me they really hate
> > > the
> > > "slow" git push deployment.
> > > 
> > > They can't develop on OpenShift as quickly as they're used to, so
> > > they're basically forced to use their own local Apache for
> > > development.
> > > 
> > > This is against the whole PaaS idea; and also far-away from the
> > > ideal
> > > test/production release management.. See
> > > https://openshift.redhat.com/community/blogs/release-management-in-the-cloud
> > > 
> > > What the PHP developers call for is usually the SFTP/FTP access
> > > to
> > > the app-root/repo/ directory using their favorite IDE to be able
> > > to make the changes on-the-fly. Why? The thing is, that PHP is
> > > kind
> > > of those "Try it and see if it works." languages.
> > 
> > As a note, we've had a bunch of discussions about this previously
> > and
> > this is definitely something we'd like to implement in both the
> > clients and the deploy model.  We've so far discussed the
> > possibility of a "no restart git deploy", and there's a client code
> > branch that does this transparently in the background:
> > 
> >   https://github.com/smarterclayton/rhc/commit/c73bd032672ec0b627271da93004e8c356c4b57c
> > 
> > In this model the idea was that you could start the client up,
> > which
> > watches a directory, then makes a commit everytime you save onto a
> > branch.  The gear would be in a "no restart on deploy" mode that
> > would take incoming commits and lay them down to disk.  Once you're
> > done, you get to decide whether to commit or not, and the client
> > would tell openshift to go back into "regular mode".
> 
> I think that the "auto-commit" solution could be potentially
> very dangerous, as the space consumed by git metadata could grow
> rapidly and cause the application to run out of the disk space
> during the production - and without any developer's notice.

I doubt it - we can easily run tidy at the end of the day.  And in scenarios where the metadata would be large, we would skip out of that process.  Also, any user checking big files into source code has other issues - this isn't going to make that problem any worse.

> 
> I'd rather let the developer commit the changes himself:
> 
> /just an ugly example of deploy and rhc tools integration/
> 
> $ git push gear master
> ... rejected by post-receive hook:
> ... There are unsaved changes in the gear repository.
> ... a) Use --force to override the changes.
> ... b) Use rhc commit "message" to save the changes
> ...    remotely and then git pull/merge to your
> ...    local repository.
> ... c) ssh into the gear... and do the git commit/push
> ....   manually.

That's certainly a very good idea - I don't see any reason not to support that independent of other discussions.

> 
> > 
> > > 
> > > =
> > > 
> > > So what comes to my mind to improve the usability for the PHP
> > > developers
> > > (and to be able to use PHP apps with self-modifying
> > > functionality)
> > > is:
> > > - let them access the app-root/repo/ for development (well, they
> > > can
> > >   do it right now, until they git push and wipe-out all the
> > >   changes..)
> > > - let them decide if the app-root/repo/ was git repository or not
> > > (means do not remove the .git/ metadata when cloning from bare
> > > git
> > > repo)
> > > - let them git commit/push from that gear's repository using SSH
> > > - let them git commit/push from Web Console (as in advanced Cloud
> > > IDEs)
> >  
> > An interesting idea - certainly this seems reasonable for a user to
> > request this sort of config (copy to a non-bare git repo on the
> > server).  Has some potential headaches - if you're using SFTP or
> > rsync though on the client, the question comes down to whether you
> > want to avoid git locally, or "hide" git from the user.  There are
> > a
> > hundred ways to do deploys and the biggest worry I'd have is that
> > we
> > give too many options.
> > 
> > So the question for you is - do PHP developers not want to see Git?
> >  Should they see Git?  Is Git inevitably going to trickle down into
> > every facet of the programming world over time?  Should we be a
> > frontend to Git?  All of those questions may change over the next
> > 3-5 years, but it'd be good to get a coherent perspective of
> > whether
> > we should hide Git from PHP developers (because it scares them) or
> > ease them into it (so they are ready for where the industry is
> > going).
> 
> Well, we force the developers to use git anyway, don't we? So hiding
> .git/ directory in app-root/repo/ doesn't make much sense to me
> (if not speaking of lowering the disk space).

We force them today.  We don't necessarily force them in the future.

> 
> The Drupal community use git as the default SCM for more than two
> years now (they switched from svn). On the other hand, the Wordpress
> guys use svn till now.
> 
> I think we can't speak of general preferences of all the PHP
> developers
> but rather what we can offer them :) Right now it's just git, right?

The underlying question that we should all be struggling with (feel free to wrestle with it at night) - do we have 2 deployment models, or N models?

Models:
  1) You push code to git, we run hooks that do things which can be builds, Jenkins, etc
     (current) 
  2) You push binaries to some location on the gear and tell us, we run hooks that do things
     (future, lots of enterprise customers, WAR files)

Are all other workflows subclasses of these workflows (i.e. Git-SVN is 1, PHP as you described in your other email where user saves and it's uploaded via SFTP is 2) or are they distinct workflows, i.e.:

  3) OpenShift gear runs a SVN server that has similar, but slightly different exit hooks from Git
  4) OpenShift gear runs a Mercurial repo because you just like mercurial better
  5) Zend runs an SFTP server and calls our hooks when we push things
  N) invent random tech, run it on OpenShift as your deploy frontend

The implicit danger in 3-N is that we lose the ability to keep all of those services in sync with an evolving deploy model.  It seems unlikely we'll want to support these extra scenarios directly (although folks should certainly feel free to investigate or hack them in) in any short term scenario.

It may be that what we really want to do for everything that is not Git+Binary is support a pull model in the gear, where the gear is limited to handling #1 and #2 from a build/hook/deploy process, but there is a process where by

  external code or changes     =>  binaries or Git changesets                         =>  standard deploy 
    GitHub                           git fetch upstream && git reset ...                    run git hooks and build
    SVN repo                         git-svn integration                                      "
    Mercurial repo                   git-hg integration (almost 1:1 feature set)              "
    3rd party binary server          pull binaries to the deploy on disk location           we haven't designed this yet

This takes us out of the business of having an infinitely flexible deploy model, and puts the onus on the integrator to build a process that converts a source of changes into either Git changesets or binaries.

To get back to your question then - I like the idea you raised about the git --force detecting changes (although that may be too easy to trigger by someone unintentionally writing to the app dir), allowing folks to copy exploded contents directly to the gear, and then being sensitive to the possibility that we'd have to help smooth the SVN transition for certain sets of the PHP world with something that makes it easier (their app hooked to an external SVN repo, a translation layer converting SVN changesets to Git, and then commands they can run from the CLI to deploy a specific svn commit or trigger a pull).

> 
> - Vojtech
> 
> > 
> > > - let them do the above also using the popular Cloud IDEs (eg.
> > > Cloud9)
> > > 
> > > 
> > > We would need of course to think of the possible git conflicts
> > > that
> > > would come with the app-root/repo/ git repository. But in my
> > > opinion,
> > > this all would be wort it.
> > > 
> > > 
> > > - Vojtech
> > > 
> > > ---
> > > Vojtech Vitek
> > > Red Hat, Inc.
> > > Brno, Czech Rep.
> > > GSM: +420608260892
> > > 
> > > _______________________________________________
> > > dev mailing list
> > > dev lists openshift redhat com
> > > http://lists.openshift.redhat.com/openshiftmm/listinfo/dev
> > > 
> > 
> 


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]