Sorry for the late reply, was flooded with other things ;)

I tried by adding
worker_rlimit_nofile 10240;
instead and it worked with 10000 connections.

Have a few questions:
1) Is there a implicit default max number of file descriptors set for worker_rlimit_nofile?

2) How does worker_rlimit_nofile and ulimit affect the actual max number of file descriptors?

3) How does nginx handle unfinished requests that are taking a long time to complete sending? (slow upstream)

Thanks in advance for your help!