Welcome! Log In Create A New Profile

Advanced

Re: limit_conn is dropping valid connections and causing memory leaks on nginx reload

Valentin V. Bartenev
October 02, 2017 08:54AM
On Saturday 30 September 2017 06:05:15 Dejan Grofelnik Pelzel wrote:
> Hello,
>
> We are running the nginx 1.13.5 with HTTP/2 in a proxy_pass proxy_cache
> configuration with clients having relatively long open connections. Our
> system does automatic reloads for any new configuration and we recently
> introduced a limit_conn to some of the config files. After that, I've
> started noticing a rapid drop in connections and outgoing network every-time
> the system would perform a configuration reload. Even stranger, on every
> reload the memory usage would go up for about 1-2GB until ultimately
> everything crashed if the reloads were too frequent. The memory usage did go
> down after old workers were released, but that could take up to 30 minutes,
> while the configuration could get reloaded up to twice per minute.
>
> We used the following configuration as recommended by pretty much any
> example:
> limit_conn_zone $binary_remote_addr zone=1234con:10m;
> limit_conn zone1234con 10;
>
> I was able to verify the connection drop by doing a simple ab test, for
> example, I would run ab -c 100 -n -k 1000 https://127.0.0.1/file.bin
> 990 of the connections went through, however, 10 would still be active.
> Immediately after the reload, those would get dropped as well. Adding -r
> option would help the problem, but that doesn't fix our problem.
>
> Finally, after I tried to create a workaround, I've configured the limit
> zone to:
> limit_conn_zone "v$binary_remote_addr" zone=1234con:10m;
>
> Suddenly everything magically started to work. The connections were not
> being dropped, the limit worked as expected and even more surprisingly the
> memory usage was not going up anymore. I've been tearing my hair out almost
> all day yesterday trying to figure this out. While I was very happy to see
> this resolved, I am now confused as to why nginx behaves in such a way.
>
> I'm thinking this might likely be a bug, so I'm just wondering if anyone
> could explain why it is happening or has a similar problem.
>
> Thank you!
>

Have you checked error log?

wbr, Valentin V. Bartenev
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

limit_conn is dropping valid connections and causing memory leaks on nginx reload

Dejan Grofelnik Pelzel September 30, 2017 06:05AM

Re: limit_conn is dropping valid connections and causing memory leaks on nginx reload

Anoop Alias September 30, 2017 06:14AM

Re: limit_conn is dropping valid connections and causing memory leaks on nginx reload

Lucas Rolff September 30, 2017 06:16AM

Re: limit_conn is dropping valid connections and causing memory leaks on nginx reload

Dejan Grofelnik Pelzel September 30, 2017 08:47AM

Re: limit_conn is dropping valid connections and causing memory leaks on nginx reload

Francis Daly October 01, 2017 07:40PM

Re: limit_conn is dropping valid connections and causing memory leaks on nginx reload

Valentin V. Bartenev October 02, 2017 08:54AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 253
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready