Welcome! Log In Create A New Profile

Advanced

Proxy_Cache Locking or Waiting method?

Resicow
September 28, 2009 07:18PM
Hi Igor,

I hope you are well. I just wanted to know if you had a time frame as to
when we may start to see backend locking for proxy_cache.

I have an example where nginx will be caching large files, and sometimes
a customer will use an HTTP monitoring service to check if a file works,
and that may generate 40+ hits to the file all at the same time, causing
nginx to proxy and write the file 40 times to the disk.

Is there a way to either:

A) Cause nginx to pause connections for subsequent requests for the same
file, until the file is in cache from the first connection. Obviously
this isn't very clean, as it can cause connections to pile up.

-or-

B) Prevent nginx for writing the duplicate files to the hard disk. So if
nginx sees that the file is not in cache, but is currently being fetched
from the backend, then it can proxy the connection from the backend
still, but just not write the file to the tmp directory, etc. This would
be the ideal method I believe.

This is a huge issue when using SSDs, as there is a major amount of
wasted write cycles, which of course causes other issues like the page
cache filling up with files that will just be deleted, etc. Plus, if you
are proxying a 1GB file, then in the example above, you'll need 40GBs of
space available on the tmp volume, to hold the 40 incoming files, just
to have 39 of them deleted.

Thanks,

John
Subject Author Posted

Proxy_Cache Locking or Waiting method?

Resicow September 28, 2009 07:18PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 204
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready