Hello. I have two nginx instances - let's call them upstream and downstream. Both are running Ubuntu 13.10 64-bit and nginx 1.4.1. I want to use proxy_store to mirror some rarely-changing files from upstream to downstream.

On the downstream server, I have created a /var/www directory owned by www-data (the user configured to run worker nginx processes). All files are served out of this directory. The directory (and its sub-dirs) have 755 permissions.

In theory, when I ask for a file from the downstream server, my understanding is that it should look under /var/www for it; upon not finding it, get it from upstream and store it locally in downstream; and then serve the file from downstream on an on-going basis. The upstream server should only show one access in its access log.

This is not happening. The downstream server keeps complaining that the file cannot be found locally, and continually fetches the file from upstream instead. So each access attempt to downstream for that file generates one "no such file or directory" error in the downstream error log, and a regular GET in the upstream access log.

If I instead touch a file at the location (as the www-data user) where nginx wants to find the file locally on the downstream server; do a GET for that file; and then delete the file, nginx will do the right thing (i.e., get the file from upstream, store it at that location, and then serve it). If I skip the GET, nginx continues to not save the file locally, and keeps getting it each time from upstream.

Any idea what's going on?

Here's my downstream server's config:

upstream download_servers {
server download.foobar.com;
}

server {
listen 80;
server_name www.foobar.com;

location / {
root /var/www;
index index.html;
proxy_redirect off;
}

location /download/ {
root /var/www/download/fetch/;
error_page 404 = /fetch$url;
}

location /fetch/ {
internal;
proxy_store /var/www/download${uri};
proxy_http_version 1.1;
proxy_pass http://download_servers;
proxy_store_access user:rw group:rw all:r;
}
}