I am setting up a nginx reverse ssl proxy - I have a machine I can use with 2 E5-2650 CPU's and lots of RAM. I have nginx-1.6.0 + openssl-1.0.1h installed. I have taken into consideration most optimization suggestions out there and incorporated them. I will attach a copy of my config file.

(optimizing first connection experience is good) With my testing just for handshake + connection setup with 2K cert it is taking 3.5ms on average. I see spikes in this time every 40 or so handshakes. I would like the 90+ percentile of the handshakes to not have any jitter/variance.

testing method:
time
for i in {1..1000};
do
httperf --hog --server localhost --port 443 --ssl --uri /nginx_ping --ssl-no-reuse --num-calls 1 --num-conns 1 --rate 1 | egrep "Connection time \[ms\]\: |Reply time \[ms\]\: " | awk {'print $5'} | xargs | tr -s " " ", " >> test.log;
done;

-if you think this methodology is not right - do let me know. I have looked at the tcpdumps and made sure a full handshake is happening and then a GET request is issued

gives me: request-time, connect_time, response_time
request_time = connect_time(ssl handshake + connection setup) + response_time.


1. I want to debug why there is jitter in the handshake time - i want the 90th, 95th, 99th, 99.9th percentiles to also be around 3.5ms.
2. I want to see if i can make nginx any faster to do handshake. what is the fastest you guys think this can happen
3. how can i profile nginx and proceed to make this faster

all comments are welcome!

thanks!


not sure how to attach config:
config details:
5 workers, worker_priority -10, timer_resolution 200ms, worker_cpu_affinity to separates cores on cpu2, error_log to dev/null, use epoll, worker_conns 2000, multi_accept on, accept_mutex off, sendfile on, tcp_nopush on, tcp_nodelay on, file caches, keepalive_timeout 5000, keepalive_requests 100000, reset_timedout_connection on, client_body_timeout 10, send_timeout 2, gzip, server_tokens off, postpone_output 0. upstream: keep alive 180, proxy_buffering off, client_body_buffer_size 512K, large_client_header_buffers 4 64k, client_max_body_size 0. server: listen 443 ssl, access_log off, ssl_buffer_size 8k, ssl_session_timeout 10m, ssl_protocols SSLv3 TLSv1, ssl_ciphers RC4-MD5, ssl_prefer_server_ciphers on, ssl_session_cache shared:SSL:10m. location /nginx_ping - return 200.