Welcome! Log In Create A New Profile

Advanced

[Patch] SO_REUSEPORT support from master process

Lu, Yingqi
August 22, 2014 12:56PM
Dear All,

The "SO_REUSEPORT support for listen sockets support" patches submitted by Sepherosa Ziehau are posted and discussed in [1] and [2]. Last update on the threads was 09/05/2013 and the patch is not included in the current Nginx code. Reading from the discussion, my understanding is that his patch makes a dedicated listen socket for each of the child process. In order to make sure at any given time there is always a listen socket available, the patch makes the first worker process different/special than the rest.

Here, I am proposing a simpler way to enable the SO_REUSEPORT support. It is just to create and configure certain number of listen sockets in the master process with SO_REUSEPORT enabled. All the children processes can inherit. In this case, we do not need to worry about ensuring 1 available listen socket at the run time. The number of the listen sockets to be created is calculated based on the number of active CPU threads. With big system that has more CPU threads (where we have the scalability issue), there are more duplicated listen sockets created to improve the throughput and scalability. With system that has only 8 or less CPU threads, there will be only 1 listen socket. This makes sure duplicated listen sockets only being created when necessary. In case that SO_REUSEPORT is not supported by the OS, it will fall back to the default/original behavior (this is tested on Linux kernel 3.8.8 where SO_REUSEPORT is not supported).

This prototype patch has been tested on an Intel modern dual socket platform with a three tier open source web server workload (PHP+Nginx/memcached/MySQL). The web server has 2 IP network interfaces configured for testing. The Linux kernel used for testing is 3.13.9. Data show:

Case 1: with single listen statement (Listen 80) specified in the configuration file, there is 46.3% throughout increase.
Case 2: with dual listen statements (for example, Listen 192.168.1.1:80 and Listen 192.168.1.2:80), there is 10% throughput increase.

Both testing cases keep everything the same except the patch itself to get above result.

The reason that Case1 has bigger performance gains is that Case1 by default only has 1 listen socket while Case2 by default already has 2.

Please review it and let me know your questions and comments.

[1] http://forum.nginx.org/read.php?29,241283,241283
[2] http://forum.nginx.org/read.php?29,241470,241470

1. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel
Subject Author Views Posted

[Patch] SO_REUSEPORT support from master process Attachments

Lu, Yingqi 1100 August 22, 2014 12:56PM

RE: [Patch] SO_REUSEPORT support from master process

Lu, Yingqi 449 August 22, 2014 01:00PM

Re: [Patch] SO_REUSEPORT support from master process

Sepherosa Ziehau 474 August 27, 2014 05:10AM

RE: [Patch] SO_REUSEPORT support from master process

luy 1156 August 27, 2014 12:26PM

Re: [Patch] SO_REUSEPORT support from master process

Sepherosa Ziehau 496 August 28, 2014 05:28AM

RE: [Patch] SO_REUSEPORT support from master process

luy 417 August 27, 2014 01:36PM

RE: [Patch] SO_REUSEPORT support from master process

luy 959 September 18, 2014 04:08PM

Re: [Patch] SO_REUSEPORT support from master process

Valentin V. Bartenev 377 September 19, 2014 04:38AM

RE: [Patch] SO_REUSEPORT support from master process

luy 455 September 19, 2014 11:54AM

Re: [Patch] SO_REUSEPORT support from master process

Valentin V. Bartenev 407 September 19, 2014 12:50PM

RE: [Patch] SO_REUSEPORT support from master process

luy 452 September 19, 2014 01:34PM

RE: [Patch] SO_REUSEPORT support from master process

luy 385 September 20, 2014 06:24PM

RE: [Patch] SO_REUSEPORT support from master process

luy 403 September 20, 2014 07:02PM

RE: [Patch] SO_REUSEPORT support from master process

luy 488 September 20, 2014 07:46PM

RE: [Patch] SO_REUSEPORT support from master process

luy 395 September 23, 2014 11:38AM

RE: [Patch] SO_REUSEPORT support from master process

luy 447 September 23, 2014 11:48AM

RE: [Patch] SO_REUSEPORT support from master process

luy 491 September 29, 2014 03:46PM

RE: [Patch] SO_REUSEPORT support from master process

luy 411 October 07, 2014 03:34PM

Re: [Patch] SO_REUSEPORT support from master process

Maxim Dounin 417 October 08, 2014 09:00AM

RE: [Patch] SO_REUSEPORT support from master process

luy 379 October 08, 2014 01:08PM

RE: [Patch] SO_REUSEPORT support from master process

luy 391 October 08, 2014 02:26PM

RE: [Patch] SO_REUSEPORT support from master process

luy 366 October 30, 2014 06:26PM

Re: [Patch] SO_REUSEPORT support from master process

Sepherosa Ziehau 421 November 16, 2014 04:08AM

Re: [Patch] SO_REUSEPORT support from master process

Maxim Dounin 599 November 16, 2014 06:52AM



Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 60
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready