Welcome! Log In Create A New Profile

Advanced

Re: Checking multiple caches before forwarding request to upstream

Maxim Dounin
April 06, 2017 10:56AM
Hello!

On Thu, Apr 06, 2017 at 06:45:23AM -0400, sachin.shetty@gmail.com wrote:

> Hi,
>
> We want to define multiple caches based on certain request headers (time
> stamp) so that we can put files modified in last 10 days on SSDs, last 30
> days on HDDs and so on. I understand that we could use map feature to pick a
> cache dynamically which is good and works for us.
>
> But when serving a file, we want to check in all the caches because file
> modified in last 11 days could still be on SSDs, so we don't wan't to
> unnecessarily pull it out from backend and put it on HDDs.
>
> so net, net we want to check multiple caches before pulling from the
> backend. Is there a way to do that? We can only define one proxy_cache in a
> location block, so I figured if we some how use try files or error page
> attribute to failover multiple blocks and check one cache in each block, we
> could check multiple caches.
>
> following is my simplified config:
>
> server {
> listen 7800 ;
> server_name localhost;
> resolver 8.8.8.8 ipv6=off;
>
> location / {
>
> set $served_by "cache";
> set $cache_key $request_uri;
> proxy_cache cache_recent;
> proxy_cache_key $cache_key;
>
> error_page 418 = @go-to-cloud-storage; return 418;
> #try_files /does-not-exist @go-to-cloud-storage;
>
> }
>
> location @go-to-cloud-storage {
> set $served_by "cloud";
>
> proxy_cache cache_recent;
> proxy_cache_key $cache_key;
>
> proxy_set_header Host $host;
> proxy_pass https://$http_host$request_uri;
> }
>
> }
>
> But in the above config, @go-to-cloud-storage storage is always executed
> even when object is in cache for the / block.

That's because you unconditionally return 418 before any other
processing. Rewrite directives, including "return", are executed
when nginx searches for a configuration to use, see details here:

http://nginx.org/en/docs/http/ngx_http_rewrite_module.html

Moreover, proxy_cache only works when you proxy somewhere, and
there is no proxy_pass in your "location /".

To check multiple caches consider actual proxying instead, this
will allow you to naturally use multiple caching layers.

--
Maxim Dounin
http://nginx.org/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Checking multiple caches before forwarding request to upstream

sachin.shetty@gmail.com April 06, 2017 06:45AM

Re: Checking multiple caches before forwarding request to upstream

Maxim Dounin April 06, 2017 10:56AM

Re: Checking multiple caches before forwarding request to upstream

sachin.shetty@gmail.com April 06, 2017 11:04AM

Re: Checking multiple caches before forwarding request to upstream

sachin.shetty@gmail.com April 07, 2017 11:53AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 162
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready