Welcome! Log In Create A New Profile

Advanced

Re: Memory usage in nginx proxy setup and use of min_uses

Maxim Dounin
May 17, 2021 03:08PM
Hello!

On Mon, May 17, 2021 at 02:47:33PM +0000, Lucas Rolff wrote:

> Hi Maxim,
>
> Thanks a lot for your reply!
>
> I'm indeed aware of the ~8k keys per mb of memory, I was just
> wondering if it was handled differently when min_uses are in
> use, but it does indeed make sense that nginx has to keep track
> of it somehow, and the keys zone makes the most sense!
>
> > Much like with any cache item, such keys are removed from the
> > keys_zone if no matching requests are seen during the
> > "inactive" time
>
> That's a bummer, since that still allows memory "poisoning" - it
> would be awesome to have another flag for proxy_cache_path to
> control how long keys that have not yet reached min_uses are
> kept in SHM.
> The benefit of this would be to say if min_uses have not been
> reached within let's say 5 minutes, then we purge those keys
> from SHM to clear up the memory.
>
> For controlling the cache items - ideally we wanna use query
> strings as a part of the cache key, but still ideally prevent
> memory poisoning as above - the inactive flag for min_uses would
> be pretty useful for this - while it won't prevent it fully,
> we'd still be able to somewhat control memory even if people are
> trying to do the cache/memory poisoning.

In no particular order:

- The attack you are considering is not about "poisoning". At
most, it can be used to make the cache less efficient.

- The goal "to somewhat control memory" looks confusing: the
memory used by caching is hard-limited by the keys_zone size,
and it is not possible to use more memory than configured. At
most, you can try to limit the number of keys an attacker will be
able to put into keys_zone. But using separate inactive timer for
keys not reached min_uses won't help here: an attacker who is able
to do arbitrary amount of requests will be able to flush all cache
items anyway.

- Using proxy_cache_min_uses cannot help here, regardless of how
it is handled, since nothing stops the attacker from requesting
the same resource multiple times.

In general, I see two basic options to handle things if you don't
want one to be able to reduce your cache efficiency:

1. Strictly limit which resources are to be cached, in particular,
by using appropriate proxy_cache_key, as already suggested.

2. Limit the maximum number of requests an attacker can do, so it
won't be able to cause noticeable degradation of cache efficiency.
In particular, this can be done with limit_req
(http://nginx.org/r/limit_req).

Also it is always a good idea to make sure your site works fine
without caching at all.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Memory usage in nginx proxy setup and use of min_uses

Lucas Rolff May 16, 2021 12:48PM

Re: Memory usage in nginx proxy setup and use of min_uses

Maxim Dounin May 17, 2021 10:38AM

Re: Memory usage in nginx proxy setup and use of min_uses

Lucas Rolff May 17, 2021 11:32AM

Re: Memory usage in nginx proxy setup and use of min_uses

Maxim Dounin May 17, 2021 03:08PM

Re: Memory usage in nginx proxy setup and use of min_uses

Lucas Rolff May 17, 2021 03:34PM

Re: Memory usage in nginx proxy setup and use of min_uses

Maxim Dounin May 17, 2021 09:28PM

Re: Memory usage in nginx proxy setup and use of min_uses

Lucas Rolff May 19, 2021 02:46PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 285
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready