Welcome! Log In Create A New Profile

Advanced

Re: Strange $upstream_response_time latency spikes with reverse proxy

All files from this thread

File Name File Size   Posted by Date  
107.3 KB open | download jay 03/18/2013 Read message
102 KB open | download jay 03/18/2013 Read message
Maxim Dounin
March 17, 2013 07:44AM
Hello!

On Sun, Mar 17, 2013 at 02:23:20AM -0700, Jason Oster wrote:

[...]

> > 1) A trivial one. Listen queue of your backend service is
> > exhausted, and the SYN packet is dropped due to this. This
> > can be easily fixed by using bigger listen queue, and also
> > easy enough to track as there are listen queue overflow
> > counters available in most OSes.
>
> Overflow queue is configured to 1024 on these hosts, though
> nothing changes when I increase it. I can however make the delay
> much longer by making the queue smaller.

On "these hosts"? Note that listen queue aka backlog size is
configured in _applications_ which call listen(). At a host level
you may only configure somaxconn, which is maximum allowed listen
queue size (but an application may still use anything lower, even
just 1).

Make sure to check actual listen queue sizes used on listen
sockets involved. On Linux (you are using Linux, right?) this
should be possible with "ss -nlt" (or "netstat -nlt").

> > 2) Some other queue in the network stack is exhausted. This
> > might be nontrivial to track (but usually possible too).
>
> This is interesting, and could very well be it! Do you have any
> suggestions on where to start looking?

I'm not a Linux expert, but quick search suggests it should be
possible with dropwatch, see e.g. here:

http://prefetch.net/blog/index.php/2011/07/11/using-netstat-and-dropwatch-to-observe-packet-loss-on-linux-servers/

--
Maxim Dounin
http://nginx.org/en/donation.html

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Strange $upstream_response_time latency spikes with reverse proxy

jay March 14, 2013 10:08PM

Re: Strange $upstream_response_time latency spikes with reverse proxy

Maxim Dounin March 15, 2013 04:22AM

Re: Strange $upstream_response_time latency spikes with reverse proxy

jay March 16, 2013 04:38AM

Re: Strange $upstream_response_time latency spikes with reverse proxy

nginxorg March 16, 2013 11:06AM

Re: Strange $upstream_response_time latency spikes with reverse proxy

jay March 17, 2013 05:20AM

Re: Strange $upstream_response_time latency spikes with reverse proxy Attachments

jay March 18, 2013 06:10PM

Re: Strange $upstream_response_time latency spikes with reverse proxy

Andrei Belov March 19, 2013 05:50AM

Re: Strange $upstream_response_time latency spikes with reverse proxy

jay March 19, 2013 06:44PM

Re: Strange $upstream_response_time latency spikes with reverse proxy

jay March 22, 2013 10:14PM

Re: Strange $upstream_response_time latency spikes with reverse proxy

Maxim Dounin March 16, 2013 07:40PM

Re: Strange $upstream_response_time latency spikes with reverse proxy

jay March 17, 2013 05:24AM

Re: Strange $upstream_response_time latency spikes with reverse proxy

Maxim Dounin March 17, 2013 07:44AM

Re: Strange $upstream_response_time latency spikes with reverse proxy

jay March 18, 2013 05:20PM

Re: Strange $upstream_response_time latency spikes with reverse proxy

Maxim Dounin March 19, 2013 10:20AM

Re: Strange $upstream_response_time latency spikes with reverse proxy

jay March 19, 2013 06:18PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 208
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready