right now we are redirecting log information from port 9200 to port 9400 using nginx without ssl. I need to get nginx to handle both ssl and non ssl traffic. I've attempted to do this using the 497 return code.

Present Configuration File:

upstream elasticsearch {
server 10.X.X.X:9400;
keepalive 10;
}

server {
listen 9200;

location / {
auth_basic "Protected Elasticsearch";
auth_basic_user_file /etc/nginx/htpasswd.users;

proxy_pass http://elasticsearch;
proxy_redirect off;
proxy_buffering off;

proxy_http_version 1.1;
proxy_set_header Connection "Keep-Alive";
proxy_set_header Proxy-Connection "Keep-Alive";
}

}

This works great for non-SSL traffic.

I've modified the config as follows to handle ssl:

upstream elasticsearch {
server 10.X.X.X:9400;
keepalive 10;
}



server {
listen 9200 ssl;
ssl_certificate /etc/pki/tls/certs/validcert.crt;
ssl_certificate_key /etc/pki/tls/private/validcert.key;

error_page 497 301 =307 http://10.X.X.X:9400$request_uri;

location /{

auth_basic "Protected Elasticsearch";
auth_basic_user_file /etc/nginx/htpasswd.users;

proxy_pass http://elasticsearch;
proxy_redirect off;
proxy_buffering off;

proxy_http_version 1.1;
proxy_set_header Connection "Keep-Alive";
proxy_set_header Proxy-Connection "Keep-Alive";
}
}

The idea is ssl data gets redirected to the upstream elasticsearch - and http traffic gets sent along anyway. At some point we'll stop accepting unencrypted log files, but until the change can be made in the applications we need to accept both.

This configuration isn't working however. Logs aren't showing up in Kibana or Elasticsearch. Can you help me see what I'm missing? I don't see errors in the nginx (or elasticsearch) logs. I do see in nginx' log that it is redirecting http traffic to 307.

Is there a clue in the nginx access log? I don't know how to interpret these very well.

working log entry from original config:

10.X.X.X - kibana_ingest_admin [21/Jun/2023:15:11:19 +0000] "POST /_bulk HTTP/1.1" 200 472 "-" "elasticsearch-net/7.8.1+aed95253ed9d86b6e22107b3b4a6e2496c206cd4 (Microsoft Windows 6.3.9600; .NET Core 3.1.23; Elasticsearch.Net)" "-"

I am confident the 200 means OK - and I can see in Kibana that logs are reaching elasticsearch.

Looking at a 'failure' log with the new config I see:

10.X.X.X - kibana_ingest_admin [21/Jun/2023:15:11:19 +0000] "POST /_bulk HTTP/1.1" 307 171 "-" "elasticsearch-net/7.8.1+aed95253ed9d86b6e22107b3b4a6e2496c206cd4 (Microsoft Windows 10.0.14393; .NET 6.0.4; Elasticsearch.Net)" "-"

In place of the 200, I see 307 which I suspect is correct - but all these failed entries have the 307 followed by 171 - not sure what that means.

Thank You

Try this:

upstream elasticsearch {
server 10.X.X.X:9400;
keepalive 10;
}

server {
listen 9200;
return 301 https://$host:9400$request_uri;
}

server {
listen 9400 ssl;
ssl_certificate /etc/pki/tls/certs/validcert.crt;
ssl_certificate_key /etc/pki/tls/private/validcert.key;

location / {
auth_basic "Protected Elasticsearch";
auth_basic_user_file /etc/nginx/htpasswd.users;

proxy_pass http://elasticsearch;
proxy_redirect off;
proxy_buffering off;

proxy_http_version 1.1;
proxy_set_header Connection "Keep-Alive";
proxy_set_header Proxy-Connection "Keep-Alive";
}
}


The SSL certificate and key paths are specified. location / block handles the proxying to the upstream Elasticsearch server, as before.

I Hope this help you.