All files from this thread

File Name	File Size		Posted by	Date
image003.jpg	3.2 KB	open | download	John Melom	03/26/2018	Read message
rawRespScatterplot.png	42.3 KB	open | download	John Melom	03/26/2018	Read message

Hello!

On Mon, Mar 26, 2018 at 08:21:27PM +0000, John Melom wrote:

> I am load testing our system using Jmeter as a load generator.
> We execute a script consisting of an https request executing in
> a loop. The loop does not contain a think time, since at this
> point I am not trying to emulate a “real user”. I want to get a
> quick look at our system capacity. Load on our system is
> increased by increasing the number of Jmeter threads executing
> our script. Each Jmeter thread references different data.
>
> Our system is in AWS with an ELB fronting Nginx, which serves as
> a reverse proxy for our Docker Swarm application cluster.
>
> At moderate loads, a subset of our https requests start
> experiencing to a 1 second delay in addition to their normal
> response time. The delay is not due to resource contention.
> System utilizations remain low. The response times cluster
> around 4 values: 0 millilseconds, 50 milliseconds, 1 second,
> and 1.050 seconds. Right now, I am most interested in
> understanding and eliminating the 1 second delay that gives the
> clusters at 1 second and 1.050 seconds.
>
> The attachment shows a response time scatterplot from one of our
> runs. The x-axis is the number of seconds into the run, the
> y-axis is the response time in milliseconds. The plotted data
> shows the response time of requests at the time they occurred in
> the run.
>
> If I run the test bypassing the ELB and Nginx, this delay does
> not occur.
> If I bypass the ELB, but include Nginx in the request path, the
> delay returns.
>
> This leads me to believe the 1 second delay is coming from
> Nginx.

There are no magic 1 second delays in nginx - unless you've
configured something explicitly.

Most likely, the 1 second delay is coming from TCP retransmission
timeout during connection establishment due to listen queue
overflows. Check "netstat -s" to see if there are any listen
queue overflows on your hosts.

[...]

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx