Welcome! Log In Create A New Profile

Advanced

Regex for caching static files

Posted by philipp 
Regex for caching static files
March 09, 2010 05:12AM
Hello,

we have a webapp with static content like these files:

http://www.example.com/images/banner_top.jpg?rev=102
http://www.example.com/images/banner_bottom.jpg?rev=102
http://www.example.com/login.html?rev=102
http://www.example.com/logout.html?rev=113

this is our nginx config:

server {
listen 80;
servername www.example.com;

# Dynamic content
location / {
include /etc/nginx/proxy.conf;
proxy_pass http://127.0.0.1:9000;
track_uploads uploadTracker 30s;
}

# Cache static content
location ~* \?rev=([0-9]+)$ {
proxy_cache_valid 200 60d;
proxy_cache_valid 404 5m;
proxy_cache cache;
expires 60d;
include /etc/nginx/proxy.conf;
proxy_pass http://127.0.0.1:9000;
}
}

Unfortunately these urls aren't catched by the regex. What's wrong with our regex?
Re: Regex for caching static files
March 09, 2010 06:43PM
I don't believe that nginx will look at the query string in evaluating a location

You might try something like this:

[code]
location / {
if ( $args ~* "\?rev=([0-9]+)$" ) {
proxy_cache_valid 200 60d;
proxy_cache_valid 404 5m;
proxy_cache cache;
expires 60d;
}
include /etc/nginx/proxy.conf;
proxy_pass http://127.0.0.1:9000;
track_uploads uploadTracker 30s;
}
[/code]


Note: you need to set a proxy_cache zone_name, proxy_cache_key, and proxy_cache_path. See http://wiki.nginx.org/NginxHttpProxyModule#proxy_cache. I'm guessing you have omitted this from the post for simplicity sake.

--
Jim Ohlstein
Re: Regex for caching static files
March 10, 2010 03:40AM
Hi Jim,

thanks for your fast response. I have tried your config example but nginx (0.7.65) does support proxy_cache_... options within if statements. :(

Reloading nginx configuration: [emerg]: "proxy_cache_valid" directive is not allowed here in /etc/nginx/sites-enabled/www.example.com:117

My conf looks like this:
[code]
cat /etc/nginx/conf.d/proxy_cache.conf
proxy_cache_path /var/lib/nginx/cache/ levels=1:2 keys_zone=cache:10m max_size=500M;

cat /etc/nginx/proxy.conf
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;

proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;

client_body_buffer_size 128k;

proxy_hide_header Etag;
[/code]
Re: Regex for caching static files
March 10, 2010 11:16PM
Well that's too bad... I had never tried it. I guess I'm not accustomed to seeing static content with a query string.

How about trying to capture the image files with regex like:

[code]
location ~* \.(jpe?g|gif|png|)$ { # add or remove file extensions as needed
proxy_cache_valid 200 60d;
proxy_cache_valid 404 5m;
proxy_cache cache;
expires 60d;
}
[/code]


and the other pages in a similar fashion.

--
Jim Ohlstein
Re: Regex for caching static files
March 11, 2010 12:33AM
Hi,
I basically want to solve the same problem (static content generated with CGI variables) in a more generic case. Basically my idea was to somehow limit caching based on the Content-type of the proxied request rather than the location.

It might look something like this:

(I know this isn't valid)

[code]
location / {

response-type =~ [ 'text/css', 'image/gif', 'image/jpeg', 'application/x-javascript', 'image/png' ] {
proxy_cache_valid 200 1d;
proxy_cache_valid 404 5m;
proxy_cache cache;
expires 1d;
}
# Everything else is not cached
}
[/code]



I'm drawing a blank in any way to do this with nginx currently. Is there a way, or am I right in thinking nginx can't handle this?
Re: Regex for caching static files
March 11, 2010 02:27AM
@Jim thanks for your hint, I am already using this kind of solution.

But I'd rather like to generalise my approach. Cause my web project is in a svn repository I can cache all files in case they have a revision param. I do not want to specify file types. Maybe this is a feature request?

@jayi sour idea looks good but it won't be possible to cache 404 pages based on the response type... Cause all 404 pages will be text/html or something like that independent of the "real" response-type.
Re: Regex for caching static files
March 11, 2010 10:40AM
philipp Wrote:
-------------------------------------------------------
> @Jim thanks for your hint, I am already using this
> kind of solution.
>
> But I'd rather like to generalise my approach.
> Cause my web project is in a svn repository I can
> cache all files in case they have a revision
> param. I do not want to specify file types. Maybe
> this is a feature request?
>

I see. I tried. I'm sure that your situation is not unique in that regard. It's worth inquiring about.

--
Jim Ohlstein
Re: Regex for caching static files
March 11, 2010 10:42PM
philipp Wrote:
> @jayi sour idea looks good but it won't be
> possible to cache 404 pages based on the response
> type... Cause all 404 pages will be text/html or
> something like that independent of the "real"
> response-type.


Sure, I could add a separate 404 handler:

[code]
location / {

response-type =~ [ 'text/css', 'image/gif', 'image/jpeg', 'application/x-javascript', 'image/png' ] {
proxy_cache_valid 200 1d;
proxy_cache_valid 404 5m;
proxy_cache cache;
expires 1d;
}

proxy_cache_valid 404 5m;
proxy_cache cache;
expires 1d;
# Everything else is not cached
}
[/code]


My question is more towards whether or not I can configure caching based on content response-type. I didn't see $request_type as a variable I can use anywhere in the doc.
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 258
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready