Welcome! Log In Create A New Profile

Advanced

Re: SO_REUSEPORT

November 24, 2015 05:46PM
Issue found.

If worker_processes is set at the start of the config file the feature
works fine, if it is set at the end of the config file it does not.

On Wed, Nov 25, 2015 at 9:38 AM, SplitIce <mat999@gmail.com> wrote:

> Ok,
>
> I have found it to be a configuration bug. With a fresh configuration
> (default configuration from package, with reuseport added) and reuseport
> enabled the feature works (same 3.18 kernel, same nginx binary).
>
> Now I just need to identify which line of our production configuration
> creates this bug.
>
> I will update when I know more.
>
> On Wed, Nov 25, 2015 at 12:42 AM, Valentin V. Bartenev <vbart@nginx.com>
> wrote:
>
>> On Wednesday 25 November 2015 00:25:19 SplitIce wrote:
>> > Hi all,
>> >
>> > I couldn't find anything in the mailing list about this issue, surely we
>> > are not the only one?
>> >
>> > When activating reuseport I am seeing all requests be served from a
>> single
>> > nginx process. All others are just idling (SIGALARM interruption of
>> > epoll_wait / epoll_wait timeout according to strace).
>> >
>> > Process 442 attached - interrupt to quit
>> > epoll_wait(60, 8225010, 512, 4294967295) = -1 EINTR (Interrupted system
>> > call)
>> > --- SIGALRM (Alarm clock) @ 0 (0) ---
>> > rt_sigreturn(0xe) = -1 EINTR (Interrupted system
>> call)
>> > epoll_wait(60, 8225010, 512, 4294967295) = -1 EINTR (Interrupted system
>> > call)
>> > --- SIGALRM (Alarm clock) @ 0 (0) ---
>> >
>> >
>> >
>> > This only occurs with reuseport, as soon as it is disabled the load is
>> > correctly distributed again.
>> >
>> > Configuration:
>> > worker_processes 12; # 2x8 cores on server
>> > multiple server blocks on different IP's and ports with reuseaddr.
>> > Linux kernel: 3.18.20
>> >
>> > Server nic has interrupts over all cores:
>> >
>> > # sudo ethtool -S eth0 |grep rx | grep pack
>> > rx_packets: 11244443305
>> > rx_queue_0_packets: 1381842455
>> > rx_queue_1_packets: 1373383493
>> > rx_queue_2_packets: 1490287703
>> > rx_queue_3_packets: 1440591930
>> > rx_queue_4_packets: 1378550073
>> > rx_queue_5_packets: 1373473609
>> > rx_queue_6_packets: 1437806438
>> >
>> >
>> > We have also experimented with disabling iptables and anything else on
>> the
>> > server that could be interfering. I have also loaded it onto three other
>> > fresh servers with the same kernel (same OS image), but with different
>> nic
>> > cards (with and without multiple rx queues) with no changes.
>> >
>> > This has me stumped. Ideas?
>> >
>>
>> You should try another kernel.
>>
>> wbr, Valentin V. Bartenev
>>
>> _______________________________________________
>> nginx-devel mailing list
>> nginx-devel@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>>
>
>
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel
Subject Author Views Posted

SO_REUSEPORT

splitice 563 November 24, 2015 08:26AM

Re: SO_REUSEPORT

Valentin V. Bartenev 246 November 24, 2015 08:44AM

Re: SO_REUSEPORT

splitice 359 November 24, 2015 09:02AM

Re: SO_REUSEPORT

splitice 359 November 24, 2015 09:04AM

Re: SO_REUSEPORT

splitice 363 November 24, 2015 05:40PM

Re: SO_REUSEPORT

splitice 405 November 24, 2015 05:46PM

Re: SO_REUSEPORT

Maxim Dounin 277 November 25, 2015 08:18AM



Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 57
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready