Some answers inline.
Can somebody reply on this if possible? Don't want to bug Brenton since he is probably jet lagged now.
Otherwise I will wait for him.
Sent from my iPhone
Further additions to my comments inline and couple of new questions. Question of unreachable gears and auto syncing of them is important. Our largest web app runs in thousands of boxes :)
My comments/questions inline.
On Jan 22, 2013, at 11:54 PM, Brenton Leanhardt <bleanhar redhat com
+++ meghdoot bhattacharya [22/01/13 17:47 -0800]:
I had couple of questions on the workflow.
1. When we git push to the master, how does the nodes get the their git repo updated? Does the git hooks in the master send a message to the broker and that in turn uses mcollective to broadcast the message to the nodes and the gears in turn do a git pull from the master? Or do the Haproxy distribute the code/artifacts directly? I am imagining the nodes have to do a pull from the master.
If you look in a scaled gear's git repo you will see a post receive
hook called 'post-receive'. The chain of events looks like this:
3 post_start_app function in the abstract cartridge 4 deploy.sh
in the haproxy cartridge
in the haproxy cartridge (rsync)
>> So am I correct in stating the git operations only matter between local dev repo and master haproxy node. Between the master node and node gears we are just syncing content without using as such git operations.
Yes, this is correct. The git push only happens between your local repo and the repo on the haproxy gear. After that the code is rsync'd to the other gears.
But the git hooks in node gears come in use or not?
Only on the haproxy gear
>> Looks like I am wrong. There is no git repo in the individual gear nodes and looks like there is just a repo/content directory. And the sync_gears script runs the pre scripts, installs content and runs the post scripts in a ssh loop. Am I right?
Yes, Sync gear calls the pre, and post scripts. But these are not git hooks.
Also sync_gears.sh in step 5 comes from the primary
cartridge, so I am guessing instead of
haproxy it will be nodejs
sync_gears.sh for the scaled app.
The sync_gears.sh comes from the abstract cartridge which is inherited by the other cartridges. Its the same script no matter which cartridge you call it from.
Hmm, in that case my 2nd question becomes all the more relevant because there may be rsync failures on some of the nodes and how do we deal with it.
And why are we not using mcollective notification and git pull from individual nodes for this?
Orchestrated git, push, and gear resynchronization are on the roadmap as part of rolling restarts and upgrades. Rsync is the simplest thing that actually works, so that's where we started. Also, git scaled pushes will continue to work even in the presence of an mcollective outage. If there isn't a resynchronization mechanism though there should be.
And the other question is does the mongo db keep track of any of these code pushes so that it may be used for the source of truth for deployment related queries.
Git push and sync are purely on the nodes. They do not interact with the broker so mongo is not notified of git pushes at the moment.
2. If few nodes get offline and come back up and had missed the new git push, how do they sync up automatically?
AFAIK, currently the node will not sync up. You will need to issue another git push for the sync to take place.
dev mailing list
dev lists openshift redhat com