Welcome! Log In Create A New Profile

Advanced

Re: nginx cache growing well above max_size threshold

September 14, 2017 02:12PM
You can actually can run H/2 through HAProxy, using ALPN to determine if
the client understands H/2 I have the following (snippet of a) config that
sends to different nginx ports based on the ALPN response.

frontend https
mode tcp
bind 0.0.0.0:443 ssl crt /etc/haproxy/certs alpn h2,http/1.1 ecdhe
secp384r1
http-response set-header Strict-Transport-Security "max-age=16000000;
includeSubDomains; preload;"
timeout http-request 10s
#send all HTTP/2 traffic to a specific backend
use_backend http2-nodes if { ssl_fc_alpn -i h2 }
#send HTTP/1.1 and HTTP/1.0 to default, which don't speak HTTP/2
default_backend http1-nodes

backend http1-nodes
mode http
balance roundrobin
default-server inter 1s fall 2

server web01 10.1.1.12:80 check send-proxy
server web03 10.1.1.14:80 check send-proxy

backend http2-nodes
mode tcp
balance roundrobin
default-server inter 1s fall 2

server web01 10.1.1.12:81 check send-proxy
server web03 10.1.1.14:81 check send-proxy

Sounds like you may not want to complicate this system ATM, but just
throwing it out there. It's worked really well for me, i have had i live
for about a year.


On Thu, Sep 14, 2017 at 1:09 PM, Lucas Rolff <lucas@lucasrolff.com> wrote:

> Okay cool, I'll give it a try
>
> In our case we do not run http2 on the machines since haproxy runs in
> front as well (which doesn't support http2)
>
> I'll also try enable a bit more verbose logging on one of the machines to
> see what the logs say
>
> Thanks a lot Maxim!
>
> Best regards,
> Lucas Rolff
>
> Get Outlook for iOS https://aka.ms/o0ukef
> ------------------------------
> *From:* nginx <nginx-bounces@nginx.org> on behalf of Maxim Dounin <
> mdounin@mdounin.ru>
> *Sent:* Thursday, September 14, 2017 6:55:57 PM
> *To:* nginx@nginx.org
> *Subject:* Re: nginx cache growing well above max_size threshold
>
> Hello!
>
> On Thu, Sep 14, 2017 at 04:34:09PM +0000, Lucas Rolff wrote:
>
> > I have a minor question, so I have an nginx box using
> > proxy_cache, it has a key zone of 40 gigabyte (so it can cache
> > 320 million files), a max_size of 1500 gigabyte for the cache
> > and the inactive set to 30 days.
> >
> > However we experience that nginx goes well above the defined
> > limit - in our case the max size is 1500 gigabyte, but the cache
> > directory takes goes well above 1700 gigabyte.
> >
> > There's a total of 42.000.000 files currently on the system,
> > meaning the average filesize is about 43 kilobyte.
> >
> > Normally I know that nginx can go slightly above the limit,
> > until the cache manager purges the files, but it stays at about
> > 1700 gigabyte constantly unless we manually clear out the size.
> >
> > I see there's a change in 1.13.1 that ignores long locked cache
> > entries, is it possible that this bugfix actually fixes above
> > issue?
> >
> > Upgrading is rather time consuming and we have to ensure nginx
> > versions across the platform, so I wonder if anyone has some
> > pointers if the above bugfix would maybe solve our issue.
> > (currently the custom nginx version is based on nginx 1.10.3).
>
> https://trac.nginx.org/nginx/ticket/1163
>
> TL;DR:
>
> This behaviour indicate there is a problem somewhere, likely
> socket leaks or process crashes. Reports suggests it might be
> related to HTTP/2. The change in 1.13.1 don't fix the root cause,
> but will allow nginx to keep cache under max_size regardless of
> the problem.
>
> --
> Maxim Dounin
> http://nginx.org/
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

nginx cache growing well above max_size threshold

Lucas Rolff September 14, 2017 12:36PM

Re: nginx cache growing well above max_size threshold

Maxim Dounin September 14, 2017 12:58PM

Re: nginx cache growing well above max_size threshold

Lucas Rolff September 14, 2017 01:10PM

Re: nginx cache growing well above max_size threshold

jeffdyke September 14, 2017 02:12PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 61
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready