We are running nginx and uwsgi behind Load Balancer (AWS ELB). uwsgi is able to process request in lesses than 100 ms but nginx is adding some overhead and final request time is becoming way more than uwsgi time. This is not happening for all of the requests but for only 2-3 % of requests.

Log Format

'$status [$time_local] "$request" $body_bytes_sent $request_length $request_time $upstream_response_time $upstream_connect_time $upstream_header_time $upstream_status $pipe';


200 [21/Sep/2022:11:46:46 +0000] "POST api_end_point HTTP/1.1" 7238 1546 24.848 0.036 0.000 0.036 200 .

200 [21/Sep/2022:11:46:57 +0000] "POST api_end_point HTTP/1.1" 1130 1558 2.178 0.044 0.000 0.040 200 .

200 [21/Sep/2022:11:46:56 +0000] "POST api_end_point HTTP/1.1" 1130 1565 10.212 0.028 0.000 0.024 200 .
Log 1: upstream request time is 36 ms and upstream connect time is 0 but nginx request time is 24.848 seconds.

Log 2: upstream request time is 44 ms and upstream connect time is 0 but nginx request time is 2.178 seconds.

Log 3: upstream request time is 28 ms and upstream connect time is 0 but nginx request time is 10.212 seconds.

Nginx Config:

error_log /var/log/nginx/error.log info;
worker_processes auto;
worker_rlimit_nofile 30000;

events {
worker_connections 1000;
use epoll;
multi_accept on;
}

http {
include /etc/nginx/mime.types;
default_type application/json;
client_max_body_size 5m;
client_body_buffer_size 256k;
client_header_buffer_size 1k;
large_client_header_buffers 8 64k;
client_header_timeout 1m;
client_body_timeout 2m;
send_timeout 2m;
reset_timedout_connection on;
server_tokens off;

sendfile on;
tcp_nopush on;
tcp_nodelay on;

keepalive_timeout 200;
keepalive_requests 2000;

log_format mycombined '$status [$time_local] "$request" $body_bytes_sent $request_length $request_time $upstream_response_time $upstream_connect_time $upstream_header_time $upstream_status $pipe';
map $status $loggable {
~^[23] 0;
default 1;
}

include /etc/nginx/conf.d/*.conf;
}


server {
listen our_custom_port;
server_name localhost;
access_log /var/log/nginx/access.log mycombined;
location api_end_point {
include uwsgi_params;
uwsgi_read_timeout 5s;
uwsgi_pass unix:/opt/apis/server/uwsgi_socket.sock;
uwsgi_ignore_client_abort on;
}
location /healthCheck {
include uwsgi_params;
uwsgi_read_timeout 5s;
uwsgi_pass unix:/opt/apis/server/uwsgi_socket.sock;
uwsgi_ignore_client_abort on;
}
}
I tried using gunicorn also instead of uwsgi. Similar issue persists with nginx + gunicorn also.

Any help would be highly appreciated.