Hi Mex,

We shut them down one-by-one, 45 minutes apart. The issue only seems to occur when the first server listed is blocked however. We don't see the read timeouts if I leave the iptables rules enabled on the second server. I think that may be a false symptom related to ip_hash binding clients to the first server.

Here are the iptables rules:

Drop rule: iptables -I INPUT -s 192.168.1.0/24 -j DROP
Allow rule: iptables -D INPUT -s 192.168.1.0/24 -j DROP

I also thought about trying to add "down" to the servers in the upstream block, but as you said that would be rather complex to script.

The only error I see is a 499 error in the Nginx logs, followed by a 200:

ip.address - - [06/May/2013:01:50:53 -0400] "GET /home HTTP/1.1" 499 0 "-" "Mozilla 4.0"
ip.address - - [06/May/2013:01:52:04 -0400] "GET /home HTTP/1.1" 200 24781 "-" "Mozilla/5.0 (compatible; PRTG Network Monitor (www.paessler.com); Windows)"