Welcome! Log In Create A New Profile

Advanced

Re: Cache questions

July 13, 2009 04:17PM
On Mon, Jul 13, 2009 at 04:02:06PM -0400, Jim Ohlstein wrote:

>
>
> Igor Sysoev wrote:
> >On Mon, Jul 13, 2009 at 03:33:21PM +0400, Maxim Dounin wrote:
> >
> >
> >>Hello!
> >>
> >>On Fri, Jul 10, 2009 at 07:39:46PM -0400, Jim Ohlstein wrote:
> >>
> >>
> >>>I'm using the fastcgi cache for static files (images, javascript,css)
> >>>and just found multiple lines in the error log like this one:
> >>>
> >>>2009/07/10 10:22:54 [crit] 22476#0: ngx_slab_alloc() failed: no memory
> >>>in cache keys zone "one"
> >>>
> >>>So I increased the memory available for the zone and reloaded nginx. It
> >>>took over five hours to go through the cache but these are the relevant
> >>>entries:
> >>>
> >>>2009/07/10 12:11:03 [notice] 21038#0: start cache manager process 32730
> >>>2009/07/10 12:11:04 [notice] 21038#0: cache manager process 22480 exited
> >>>with code 0
> >>>
> >>>and finally
> >>>
> >>>2009/07/10 17:43:27 [notice] 32730#0: http file cache:
> >>>/usr/local/nginx/cache 11638.289M, bsize: 4096
> >>>
> >>>My questions are:
> >>>
> >>>Is that simply the total (11638.289MB or 11.4GB) of all of the file
> >>>sizes, or is that the actual disk space consumed taking into account
> >>>total number of blocks used multiplied by the block size? The number
> >>>
> >>It's size on disk (i.e. number of blocks * block size), but for
> >>files only (it doesn't take directories into account).
> >>
> >
> >Just note: nginx rounds a file size to the bsize.
> >bsize is f_bsize from statfs() or f_frsize from statvfs().
> >I'm not sure that bsize matches always a filesystem allocation unit.
> >
>
> OK, Thanks.
>
> In trying to tune this, if I set fastcgi_cache_min_uses to 2, does that
> mean that a file will only be written to the cache the second time that
> it is requested? Google translate did not give me a clear answer to this
> from the Russian documentation. I think that I could improve efficiency
> greatly if I didn't cache files on the first request.

Yes, a file will be cached only on the second request made during period
set in by "inactive" option of fastcgi_cache_path directive.

> I would know
> better if I could get some statistics. I know the last time I asked the
> answer was "not yet". Do you have any idea when this might be
> implemented even on a rudimentary basis?

You may log $upstream_cache_status.

> >>>could be quite different given what I estimate are nearly one million
> >>>mostly small files in the cache at this point.
> >>>
> >>>When I next upgrade nginx (I'm running 0.8.4), and I attempt a "graceful
> >>>upgrade" will it have to go through this entire process again?
> >>>
> >>Yes. Binary upgrade doesn't preserve shared memory zones, so
> >>cache will be rescanned again.
> >>
> >>Note that nginx uses cache even before it was completely scanned,
> >>so it shouldn't be a problem.
> >>
>
> Yes, but it uses a fair bit of system resources scanning a large cache.
> Nothing much to be done about it though.

I tried to minimize impact of cache scanning according to this comment:

/*
* if processing 100 files takes more than 200ms,
* it seems that many operations require disk i/o,
* therefore sleep 200ms
*/


--
Igor Sysoev
http://sysoev.ru/en/
Subject Author Posted

Cache questions

Jim Ohlstein July 10, 2009 07:39PM

Re: Cache questions

Maxim Dounin July 13, 2009 07:33AM

Re: Cache questions

Igor Sysoev July 13, 2009 07:48AM

Re: Cache questions

Jim Ohlstein July 13, 2009 04:02PM

Re: Cache questions

Igor Sysoev July 13, 2009 04:17PM

Re: Cache questions

Jim Ohlstein July 13, 2009 08:45PM

Re: Cache questions

Maxim Dounin July 13, 2009 10:29PM

Re: Cache questions

Jim Ohlstein July 13, 2009 11:09PM

Re: Cache questions

Igor Sysoev July 14, 2009 01:21AM

Re: Cache questions

Jim Ohlstein July 14, 2009 09:26AM

Re: Cache questions

Igor Sysoev July 14, 2009 11:11AM

Re: Cache questions

Jim Ohlstein July 14, 2009 11:28AM

Re: Cache questions

Igor Sysoev July 14, 2009 11:41AM

Re: Cache questions

Jim Ohlstein July 14, 2009 11:55AM

Re: Cache questions

Marcus Clyne July 14, 2009 03:23PM

Re: Cache questions

merlin corey July 14, 2009 05:37PM

Re: Cache questions

Maxim Dounin July 13, 2009 04:50PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 161
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 500 on July 15, 2024
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready