Hello Jeff...

On Fri, Jan 28, 2011 at 3:08 AM, Jeff Mitchell <jeff@jefferai.org> wrote:

> Hello,
>
> We have a particular application (gitweb) that performs a particular,
> extraordinarily slow function when the home page is loaded. As the
> number of repositories has increased, this has grown to take several
> *minutes* per page view (yes, ugh).
>
> To combat this, we tried setting up cache so that this only occurs once
> an hour, but it's still causing too much of a problem...both because it
> doesn't always last an hour (sometimes we see the delay happen again
> sooner) and because when it does expire, it simply takes too long (and
> if multiple clients connect at that same time making that request, it
> gets even worse).
>

I think you look the wrong way at the problem you have. You should fix these
issues you mention which are NOT the expected behavior before trying to fix
your situation with your proposed patch of allowing a different time to a
special IP and leave a bot requesting the page.

First problem: early renewal of page.

This is not a mistery why it happens: Either your server it's returning a
header that tells nginx to refresh the cache, or you have a problem inside
your configuration. If you tell nginx to update each 60 min, it will always
respect that except backend server tells nginx to refresh the cache.

Second problem: multiple clientes renewing cache.

This can easily be fixed by using proxy_cache_stale, you can set it up to
return users the last cached page while the backend ends updating the first
request that expired. This means that after a requests pass to the backend,
while that request does not complete, nginx will return the last cached
(which already expired) version to any new request for that page.

If you fix those both problems then I think you won't need all that
workaround you want to do.

I'm not nginx guru and maybe it's me looking the wrong way at your problem
but in case it help this is how I dealt before with the same problem you
described.

Good Luck!

Guzmán Brasó
--
http://www.guruhub.com.uy
+59898674020
Montevideo, Uruguay
_______________________________________________
nginx mailing list
nginx@nginx.org
http://nginx.org/mailman/listinfo/nginx