Welcome! Log In Create A New Profile

Advanced

Re: Using the memcache module to your advantage

Merlin
April 01, 2009 08:00PM
SSSloppy,

In short, using the memcached backend with NginX is just like publishing to
flat-files except your storage engine is in memcached and not the local
filesystem.

Going deeper, your forum software using memcached helps it cache specific
query results so it need not go to the database. NginX's memcached module
serves content directly from memcached, completely bypassing your
application and touching the database not at all. This means to utilize it
with your forum, you will need to publish, or store, the output for each URI
in memcached. In Python this can be accomplished very easily with a simple
WSGI middleware. In PHP you will need to use the output buffering functions
(ob_*). In other stuff, others.

Now the more in depth how to...

The NginX Memcached Module (documented
http://wiki.nginx.org/NginxHttpMemcachedModule and
http://sysoev.ru/nginx/docs/http/ngx_http_memcached_module.html in original
Russian) simply turns memcached into a backend for NginX to reverse proxy
to, much like another HTTP server or fastcgi, etc.

What this means is that NginX will attempt to serve resources from memcached
keys. The example from the example from the wiki is pretty succinct and
very complete:

server {
location / {
set $memcached_key $uri;
memcached_pass

name:11211;
default_type

text/html;
error_page
404 = /fallback;
}

location = /fallback {
proxy_pass backend;
}
}

[apologies if you don't have HTML mail]
Here we see that the memcached key is set to the URI requested. Next is the
memcached_pass directive which is like the rest of the *_pass directives
(proxy_pass, fastcgi_pass, etc) in that it tells which backend to go to.
The rest is just setting the default type (In my experience it seems MIME
Type is not checked from memcached) and setting a fallback location to serve
from in case the content is not in memcached yet.

What this setup then assumes is that the backend (in your case, the forum
software) will publish the page output into memcached in the key that is the
URI. As I previously mentioned, the best way to go about this is with some
kind of middleware or output buffering, in my experience. The problem with
the forum is it might need to be a little more complicated, depending on how
you want to do it.

If you want to just cache page output for a minute or two, it should be as
simple as pushing into memcached with a minute expiration time, nothing else
need be done except not serving POST requests from memcached and disallowing
your very dynamic pages with the same URI from being cached (so like
/forum/post would be not cached but /forum/main-category/this-is-a-thread
would).

However, if you want to "cache forever" it gets a lot more complicated.
You'll need to do the above, without the minute limit, and in addition to
that, you'll need to include code so that every action that changes
something on a page causes a republication. This can obviously get pretty
hairy if your application was not designed with such a thing in mind in the
first place.

At any rate, good luck!

- Merlin

On Wed, Apr 1, 2009 at 3:31 PM, SSSlippy wrote:

> Ok I am trying to figure out how to use the memcache module to my
> advantage. I run multiple forums that can put their "datastore" into
> memcached already.
>
> Would just adding the memcached server info help?
>
> Confused on how to make this work.
>
> Posted at Nginx Forum: http://forum.nginx.org/read.php?2,738,738#msg-738
>
>
>
Subject Author Posted

Using the memcache module to your advantage

SSSlippy April 01, 2009 06:31PM

Re: Using the memcache module to your advantage

Merlin April 01, 2009 08:00PM

Re: Using the memcache module to your advantage

Josh Turmel April 01, 2009 11:09PM

Re: Using the memcache module to your advantage

qctu January 15, 2010 12:33AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 301
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready