you config is somewhat messed_up, but it think this is not an issue here.
are you sure your fastcgi_process is able to deliver more than 520 parallel connections?
http://wiki.nginx.org/EventsModule#worker_connections -> to be defined in event {}
max clients = worker_processes * worker_connections
In a reverse proxy situation, max clients becomes
max clients = worker_processes * worker_connections/4