Welcome! Log In Create A New Profile

Advanced

Connection timed out while connecting to upstream, syn packet volume is high

December 12, 2011 01:24PM
I've been troubleshooting this issue on and off the last couple days and have not been able to eliminate this error from nginx:

2011/12/12 13:12:32 [error] 27810#0: *36209658 connect() failed (110: Connection timed out) while connecting to upstream

Our setup is a single machine running nginx that serves as reverse proxy to two separate apache servers. The setup is handling perhaps a few dozen requests per second. The boxes themselves are almost idle, no issue with load in that respect.

The upstream apache servers are not getting hit hard and are no where near their maxclient settings.

This error was happening every second or two on the nginx side until I set "net.ipv4.tcp_syncookies = 1" on the apache machines (not sure why it wasn't turned out by default). After making that change the frequency dropped to 1 or 2 connection time outs per minute. Similarly, leaving syncookies off but increasing "net.ipv4.tcp_max_syn_backlog" alleviated the issue as well, though didn't stop it entirely.

If I use tcdump to count syn packets I see 8x the volume to these nginx-fronted apache machines that I see for other datacenters handling far more load (hundreds of concurrent connections) that are behind traditional hardware load balancers.

Is there a reason nginx would be generating so many syn packets? Or, anyone have recommended settings for alleviating it as much as possible at least?

Any help is greatly appreciated.
Subject Author Posted

Connection timed out while connecting to upstream, syn packet volume is high

gtuhl December 12, 2011 01:24PM

RE: Connection timed out while connecting to upstream, syn packet volume is high

Lukas Tribus December 12, 2011 02:16PM

Re: RE: Connection timed out while connecting to upstream, syn packet volume is high

gtuhl December 12, 2011 02:37PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 121
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready