Intermittent SSL errors (upstream certificate verify error)
June 23, 2023 06:41PM
Hello,

We have setup NGINX as a network proxy and performs 2-way TLS on either side, upstream and downstream. We noticed that randomly the container is giving 502s when connecting to upstream. The error logged is - "upstream SSL certificate verify error: (20:unable to get local issuer certificate) while SSL handshaking to upstream"

Given that most of the times SSL handshake is correct I do not want to assume that its a configuration problem. Attaching it below for reference:
Also we see multiple attempts made to different upstream IPs registered against an upstream host. This can be inferred from

$upstream_status = "-, 502 : 502 : 200"
$upstream_addr = ", IP1:8443 : IP2:8443 : IP1:8443"
As we can see, at times its the same upstream IP that is tried twice within the same request and it succeeds

The NGINX server block taking in traffic is default. In other words all domains end up in this server block. I have some other logic of treating them differently, but yes this would imply a lot of connections being made into this server section.
I am not sure if NGINX has any limits to the upstream connections that can be made from a single server block.

What could be the reason for these sporadic SSL errors? "20:unable to get local issuer certificate" implies that NGINX could not find its local truststore to verify the incoming server certificate against. But its right there...

server {
listen 9080 default_server proxy_protocol;
listen 9081 default_server ssl proxy_protocol;

server_name "_";

proxy_ssl_certificate "/ssl/client.crt";
proxy_ssl_certificate_key "/ssl/client.key";
proxy_ssl_server_name on;
proxy_ssl_trusted_certificate "/pki/ca/cacerts.pem";
proxy_ssl_verify on;
proxy_ssl_verify_depth 3;
proxy_ssl_protocols "TLSv1.2";

----
}
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 115
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 500 on July 15, 2024
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready