Hi Guys,

I posted originally my issue on askubuntu but I think this will be a better place

http://askubuntu.com/questions/611418/intermittent-ssl-handshake-issues-on-ubuntu-12-04-and-nginx.

Original post
--------------------------------

# In simple terms

I am having issues with https handshakes. I am currently using nginx but it is most likely not an nginx issue.

# Behaviour

Web clients such as browsers will sometimes present "SSL connection error" (Chrome)

Apache benchmark will spit out several error lines and will report around 1-10% failures. Errors below will appear in random order but the first one is more common.

(1) Benchmarking mysite.net (be patient)...SSL read failed (1) - closing connection
128494120003296:error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac:s3_pkt.c:486:

(2) SSL read failed (1) - closing connection
128494120003296:error:140943FC:SSL routines:SSL3_READ_BYTES:sslv3 alert bad record mac:s3_pkt.c:1262:SSL alert number 20

# Server setup
Ubuntu:

Ubuntu 12.04 64bit with all updates and patches installed, server restarted.
Nginx:

nginx/1.6.3 - from nginx.org (deb http://nginx.org/packages/ubuntu/ precise nginx)

OpenSSL dynamically linked:

# ldd `which nginx` | grep ssl
libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f3065569000)

# strings /lib/x86_64-linux-gnu/libssl.so.1.0.0 | grep "^OpenSSL "
OpenSSL 1.0.1 14 Mar 2012

Nginx server config (with limited cyphers)
OpenSSL:

1.0.1 14 Mar 2012

#dpkg -s libssl1.0.0
Version: 1.0.1-4ubuntu5.25

#Workarounds

So far, the only workaround I found, is to narrow down available cyphers.

Instead of using Mozilla Intermediate set, I found these would work without any issues:

ssl_ciphers 'ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4';

Second option is to downgrade to stock nginx (1.1.19-1ubuntu0.7)

#Things I tried

Because I am mainly using nginx as a proxy / load balancer, I tried replacing nginx with HA-Proxy 1.5. Unfortunately I got the same problem.
I tried compiling nginx 1.6.3 with openssl 1.0.1m - no change.
On-line https/ssl validity tester did not found any issues with any of the certificates.
Disabling other nginx sites did not help either.

#Things I noticed

Interestingly this problem does not occur when using apache benchmark from the server itself or it's immediate neighbours, but it does happen when connecting from outside of the data centre. Apparently DC guys (coreix) claim not to have any DDOS prevention system in front of the servers which would cause such an issue.
This issue is happening mainly on one of the https domains and is very sporadic for remaining two - hosted on the same box.