Hi All

I have a rather strange issue that appears to have only started in the last few days (no changes have been made in the last few days) . I'll try and describe it as best I can but please let me know if I need to post particular configs etc.

I have a setup with an nginx load balancer in front of 2 backend nginx web servers running on CentOS 6.3. This site has been running fine up until now but recently when visiting the home page it has been showing me the page I created as the default site on the webserver (it is a discrete notice that says the host header has not been recognised on the webserver). When you visit a sub-page e.g. domain.com/about-us it is fine but visiting domain.com shows the default site. It appears that this problem can be solved by disabling the caching on the load balancer for this particular site. As soon as the cache is not used, the homepage shows correctly.

The cache on the load balancer is held in a RAM drive mapped to /cache with a total available size of 256Mb. The cache configuration is as follows

proxy_cache_path /cache levels=1:2 keys_zone=app-cache:30m max_size=126m inactive=10m;
proxy_temp_path /cache/tmp;
proxy_buffer_size 8192;
proxy_max_temp_file_size 1m;

I reduced the total size of the cache last night as I have a feeling it was causing another problem I saw recently where we came under extreme load, the cache filled to the max allowed size of 256Mb but that left no room on the mount point for the temp path to buffer files from the upstreams and started serving empty files to clients.

I've looked in the error log for the particular site and found a number of instances of

2013/01/03 05:47:24 [crit] 22889#0: *352983 pwrite() "/cache/tmp/0000028264" failed (28: No space left on device) while reading upstream, client: 121.58.173.7, server: www.domain.com, request: "GET /images/slider-plus.gif HTTP/1.1", upstream: "http://10.0.100.193:80/images/slider-plus.gif", host: "www.domain.com", referrer: "http://www.domain.com/example-page"

A df -h shows the /cache mount to only be using 1% of the available space at the moment, so I'm not sure why I should be still getting this error.

The nginx version on the load balancer is 1.0.11 (built from source)
The nginx version on the web server is 1.0.15 (installed from package from epel)

I realise that nginx is in need of updating, I only realised this yesterday. If this is the problem then I can bring forward the plans to update it, but it'd be useful to have confirmation of what this problem might be and how it can be solved.

Please feel free to request any extra info needed to help diagnose this situation.

Many thanks

Eric