Welcome! Log In Create A New Profile

Advanced

Re: Nginx proxy cache purge process does not clean up items fast enough for new elements

Maxim Dounin
October 16, 2019 12:50PM
Hello!

On Wed, Oct 16, 2019 at 09:44:08AM -0400, sachin.shetty@gmail.com wrote:

> Thankyou Maxim, is there anyway I can make the cache manager a bit more
> aggressive in prune and purge? We already leave 20% of space free on the
> disks, but the concurrent request rate for large files can be huge and we
> still run in to this issue.

Most likely you are hitting space limits due to temporary files, which
aren't managed by cache manager at all. My recommendation would
be to consider proxy_cache_lock and/or "proxy_cache_use_stale
updating" to reduce number of concurrent requests trying to cache
the same files, see here:

http://nginx.org/r/proxy_cache_lock
http://nginx.org/r/proxy_cache_use_stale

Also you may want to tune things like proxy_cache_min_uses to
reduce unneeded caching on sporadic requests.

> What are your thoughts about disabling buffering on such issues? This is not
> a fatal error, so we should stop buffering and switch to streaming mode and
> let the request succeed with a error log line in error.log.

Yes, in theory. But the question is how complex and error prone
the resulting code would be. Hence the current choice is to treat
such errors as fatal.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Nginx proxy cache purge process does not clean up items fast enough for new elements

sachin.shetty@gmail.com October 16, 2019 05:24AM

Re: Nginx proxy cache purge process does not clean up items fast enough for new elements

Maxim Dounin October 16, 2019 08:58AM

Re: Nginx proxy cache purge process does not clean up items fast enough for new elements

sachin.shetty@gmail.com October 16, 2019 09:44AM

Re: Nginx proxy cache purge process does not clean up items fast enough for new elements

Maxim Dounin October 16, 2019 12:50PM

Re: Nginx proxy cache purge process does not clean up items fast enough for new elements

sachin.shetty@gmail.com October 17, 2019 08:13AM

Re: Nginx proxy cache purge process does not clean up items fast enough for new elements

Maxim Dounin October 17, 2019 08:58AM

Re: Nginx proxy cache purge process does not clean up items fast enough for new elements

Hung Nguyen October 16, 2019 09:58AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 248
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready