We have been running Origin since last November and i'd like to share some experiences, pains and thoughts.
Our origin cluster has about 25 servers including masters,nodes and routers. We have roughly 500 applications exposing services and a bunch of HPA firing up containers all the time.
1) Resource consumption: i noticed during the day a increase of memory consumption due multiple reloads, a lot of process keep running until the connections is finished or OOM kill. Other issue regarding restarts is that due to TCP SYN DROP iptables we are facing some high latencies. What can we do to reduce restart overhead ?
2) Metrics: Would be nice to pull some metrics from the routers, something like general network i/o and per endpoint traffic, i found a prometheus export but due to process restart the endpoint states are cleaned. HAProxy 1.6 have a fix for that (http://blog.haproxy.com/2015/10/14/whats-new-in-haproxy-1-6/). Do we have plans to upgrade to 1.6 ? What kind of metrics do we have available today?
Diego Castro / The CloudFather
GetupCloud.com - Eliminamos a Gravidade