[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Socket activation



On Sep 4, 2013, at 1:54 PM, David Strauss <david davidstrauss net> wrote:

> On Wed, Sep 4, 2013 at 12:22 PM, Krishna Raman <kraman gmail com> wrote:
>> Could make this a configuration/setup option. If the router/switch misbehaves with it gets SYN failures then use a haproxy,
>> else live with 1 SYN failure per gear un-idle.
>> 
>> What do you think?
> 
> I'm just wary of relying on failure-retry mechanisms for normal
> operations. It takes network monitoring from "no SYN-ACK failures
> should occur under normal conditions" to "there is some amount
> expected, and expectations might even be exceeded under normal
> conditions " Noise like that makes the ops job hard without getting
> either false positives (on widespread activation) or false negatives
> (on network failure). Even if you can tolerate it, it's not a fun
> model to work with in practice.
> 
> It might be possible to map gear activations to SYN-ACK failures to
> discover network failure, but network and daemon monitoring are rarely
> integrated. If you're at a colo facility, they're not even managed by
> the same parties.
> 
> Using haproxy to mask the SYN-ACK failure is interesting. But, unlike
> the socat shim *behind* socket activation, the haproxy would have to
> persistently run for every container/gear on the host. That would
> really hurt density. What would you think of haproxy *behind* socket
> activation as the shim? I don't think it supports it at the moment,
> but it would not be hard to implement.

We already run a haproxy on the node to do port-forwarding and it seems to
perform fine under load and at density. Its not as fast as iptables but at least it 
will hide the SYN-ACK problem.

Running it inside the gear is still not suitable because of the extra memory
consumption.

> 
> I'd also like to note that the dropped-SYN activation model won't work
> for most UDP protocols. Some will retry. Some will just have data go
> missing. If you're using statsd or GELF, that would be pretty
> annoying.

Perhaps the node level haproxy is best in this case. It might be able to handle 
this better as well.

> 
> -- 
> David Strauss
>   | david davidstrauss net
>   | +1 512 577 5827 [mobile]



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]