Hello, I believe I have a tuning issue with NGINX - hoping someone can point me in the right direction!
About 1 in 60,000 requests being proxied through Kong/NGINX are timing out. These requests are getting to the upstreamed host and are successfully logged in the load balancer in front of this upstreamed host. So either there's a network issue between that load balancer and NGINX or NGINX is simply not able/willing to process the response.
Assuming this is an NGINX tuning issue, these are my settings (note these hosts have 32 cores). Traffic is not all that high, less than 10 req/sec per instance and requests are usually satisfied in less than a second:
worker_processes auto;
worker_priority -10;
daemon off;
pid pids/nginx.pid;
error_log logs/error.log debug;
worker_rlimit_nofile 20000;
events {
worker_connections 10000;
multi_accept on;
}
Most all other config settings are "default" values. There's nothing in the Kong logs that indicate these dropped responses are being processed by Kong. There's no indication there aren't enough workers. These timeouts do not happen in clusters, they are more like singletons.
Any advice on things I should look at or diagnosis possibilities? Thanks very much, Ryan