I have nginx serving as both a load balancer and front end proxy for my django app served by apache2. Every day, I've been seeing the following connection time out error and some users have experienced the site being down around the same time.

2014/01/18 23:17:15 [error] 23381#0: *33868 upstream timed out (110: Connection timed out) while connecting to upstream

nginx site file:

upstream backend {
ip_hash;
server 127.0.0.1:81;
server 192.234.322.3
}
server {
listen 80;
client_max_body_size 20M;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://backend;
}
}
server {
listen 81;
client_max_body_size 20M;
server_name example.com;
access_log /var/log/nginx/example.access.log;
error_log /var/log/nginx/example.error.log;

location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_set_header REMOTE_ADDR $remote_addr;
proxy_pass http://127.0.0.1:8080;
}
}

Any ideas why this is happening? The requests that time out are not showing up in my apache2 logs either.