Welcome! Log In Create A New Profile

Advanced

Performance plateau when keepalives enabled

March 08, 2010 11:30AM
Hi,

I am benchmarking nginx 0.7.65 with httperf simulating a typical user visit which lasts max 10 seconds including pause time (i.e. hit page, grab inline elements such as images/css/js, pause, hit another page, etc.) I am serving only static html and images/css/javascript.

httperf is running from two separate client machines both within 10ms ping from the webserver. While httperf is running, I am also browsing the site on my workstation (200ms from the webserver) to see how it feels for a user.

When I run the benchmark with keepalive_timeout 0, nginx is very very fast and accepts new connections instantly. Concurrent connections never goes above 5 since the benchmark clients are so close to the server. However,

When I run the benchmark with keepalive_timeout 65, nginx is very slow (5-20 seconds delay) to accept new connections as soon as it reaches 1500 concurrent connections. It plateaus here: it completes all the requests but it never exceeds approximately 1500 concurrent connections.

Is this normal? Everywhere I read tells me to leave keepalive on and it shouldn't make any difference in nginx. So is this a keepalive problem, or a problem handling >1500 concurrent connections? What should I look at to fix it?

The box is 97% idle and it is not waiting on disk.

OS is Linux 2.6 with fs.file-max=65535 also before starting nginx, ulimit -n 65535 is called (from /etc/init.d/nginx)

There are no errors at all in the nginx error.log. Not for file descriptors, or anything. I watched error.log at "info" logging level and the only messages at all are the keepalive connection closing messages, which I believe are normal.

The nginx.conf has (amongst other things):

worker_processes 4; # it's quad core machine
worker_rlimit_nofile 6144;
events {
worker_connections 3072;
use epoll;
}

http {
sendfile on;
tcp_push on;
tcp_nodelay on;
}

Thanks
Steve
Subject Author Posted

Performance plateau when keepalives enabled

powderhound March 08, 2010 11:30AM

Re: Performance plateau when keepalives enabled

Maxim Dounin March 08, 2010 05:10PM

Re: Performance plateau when keepalives enabled

powderhound March 08, 2010 05:21PM

Re: Performance plateau when keepalives enabled

powderhound March 08, 2010 06:15PM

Re: Performance plateau when keepalives enabled

powderhound March 08, 2010 06:42PM

Re: Performance plateau when keepalives enabled

powderhound March 08, 2010 09:35PM

Re: Performance plateau when keepalives enabled

powderhound March 08, 2010 11:04PM

Re: Performance plateau when keepalives enabled

James Matthews March 08, 2010 06:10PM

Re: Performance plateau when keepalives enabled

James Matthews March 08, 2010 06:36PM

Re: Performance plateau when keepalives enabled

Momchil Ivanov March 11, 2010 08:28AM

Re: Performance plateau when keepalives enabled

Vinay Y S March 21, 2010 04:30AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 175
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 500 on July 15, 2024
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready