Thanks for all your help. I've somewhat nearly got there, but having problems testing it because of the binary file being quite large and openshift only having a finite execution time for third party cartridges.
Is most of the creation time taken up by downloading the cart binaries ? Or by compilation / other?
Downloading the binaries.
If anyone's got any suggestions on a workaround for that limit would be much appreciated
- Is there a way to have an environment variable of some sort which could list all the gears running? I would like to create the whole cartridge similar to a web-app style so that it can scale infinitely. >From what I understand, environment variables set are global across all the gears upon first run, but does that mean they can be dynamically updated? I need this to write some logic to prevent a split brain. This is because the cluster requires the first "gear" to be brought up with the slightly different command.
Currently the way to get this information is to use a connection hook (which gives you variables from all of the gears) and then do the same configuration on each gear. If you look at the redis cart you can see an example of the parsing of the data, although we'd like to make this simpler in the future by piping JSON to the stdin of the hook. For now, the example below demonstrates breaking the info up and then using it to assign a master (the gear with the lowest alphabetical ordered uuid the first time the cluster starts up).
Thanks for the reply. I have no clue where to start with ruby, tbh none of it actually makes sense to me :( Would I just need to modify a few variables to grab just the output? With the reddis cartridge, what happens if all the nodes crash and are brought up all at the same time or the master crashes and the other nodes are running. I haven't used reddis, but with mariadb this would cause issues which would need some sort of logic or manual intervention.
- Is it possible to use the haproxy routing to loadbalance the requests to the cluster? I'm not so familiar with haproxy, but I'm assuming as it can run like a tcp loadbalancer similar to LVS (which I currently use for my external cluster) I don't see why it couldn't work. It'd obviously need a custom port?
Scalable apps are load-balanced by default, but for HTTP only. There is a second haproxy on the node routing tcp connections inter-gears. The entry point is $OPENSHIFT_GEAR_DNS.
I don't think it is possible to accept external connections other than http[s] on app's haproxy.
This may be interesting, is $OPENSHIFT_GEAR_DNS unique per scaled gear? When you say external connections does that include other apps within the openshift environment? eg. App1 mariadb and app2 php
Yes, It's unique since has gear UUID on it.
External connections are any connection external to gear itself: another app or somewhere outside openshift
Hmm... this makes it hard as I was planning on building the cluster to be shared with multiple PHP applications (within openshift).
- I think I read somewhere, that by default the gears will try to do negative affinity by default when scaling out? eg. be hosted on a separate node. Is that the default behavior, or am I thinking of something else?
AFAIK, broker chooses the least full node. Could someone please confirm this?
- Am I wrong to assume this won't work with Online as it requires some external dependencies from the MariaDB repo (not SCL)?
You could deploy mariadb binary along with your cartridge.
Where would I need to save these binary files? It also a few dependencies would I need to package those up too?
Create a versions/<version-number>/ and install mariadb there, with any mariadb necessary libs (some may be installed on host already). Config file and/or startup script may be edited so mariadb prefix dir point to this directory.
I'll give that a try.
Do you know if there's a link lurking around somewhere which explains this procedure? I'm a little confused about how to do this.