Found that in the logs:
curl: (56) Recv failure: Connection reset by peer
and some seconds later, we have the expected 200 response. I opened 3 more shells on the nodes directly, and did the same curl command, but on the endpoint directly (something like 10.1.2.9:8080), and it was working like a charm.
I have destroyed the router and ipfailover pods, with the same result. I ended up with rebooting all the VMs, and it seems to be ok now, but it's hard to reproduce a random issue :(
Also, as I said, origin-node is using 20% cpu on all node (bi-xeon CPU E5-2690 v2 @ 3.00GHz), which seems a lot to me.