Welcome! Log In Create A New Profile

Advanced

S3 Proxy/Cache: invalidating requests

Posted by allthetime 
S3 Proxy/Cache: invalidating requests
November 12, 2014 03:06PM
I am having a strange issue. I am trying to create a caching proxy server between my apps and a private s3 bucket.

Everything works fine if I disable the "proxy_cache" line, but if it is enabled (which I want to cache requests/images) everything breaks. It seems the entire structure/flow of requests is being mutated when caching is enabled and I don't understand why. Perhaps I am implementing caching wrong?

Here is my config, and I have included some information from tcpdumps to give you an idea about the character of the problem.

I am using the aws sdk to form requests and authentication is sent in the headers.

http {

proxy_cache_path /home/cache levels=1:2 keys_zone=S3_CACHE:10m inactive=24h max_size=500m;
proxy_temp_path /home/cache/tmp;

server {

server_name my-cache-server.com;
listen 80;

proxy_cache S3_CACHE;

location / {

proxy_buffering on;
proxy_pass http://MY_BUCKET.s3.amazonaws.com/;
proxy_pass_request_headers on;
}
}
}

I have attached tcpdumps viewed in wireshark to explain the request flow in both situations... with proxy_cache disabled (working) is the larger image.

With the cache diasbled everything flows properly. First HEAD is sent from client -> proxy -> s3, then GET is sent from client -> proxy -> s3 and then the images are returned.

But if I turn on the cache, The HEAD is sent from client -> proxy, then it stops, then a GET is sent from proxy -> S3.

What is going on here, and why isn't the request flow the same when I turn the proxy on, and how can I make it so?

I have also posted this issue on stackoverflow - http://stackoverflow.com/questions/26825933/nginx-proxy-cache-invalidating-request-to-s3/26830981?noredirect=1#comment42245919_26830981



Edited 1 time(s). Last edit at 11/12/2014 03:07PM by allthetime.
Attachments:
open | download - cache_cache_disabled copy.png (121.1 KB)
open | download - cache_cache_enabled copy.png (66.4 KB)
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 125
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 500 on July 15, 2024
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready