Hello,

Problem: Nginx returns 400 on small amount of POST requests (1-5 per minute from 10k POST requests)
Setup: Nginx in front of Tomcat
Version: 1.8.0 (upgraded from 1.4.6 that had the same problem)
Log format: log_format main $remote_addr - $remote_user [$time_local] $request $status $request_length $body_bytes_sent $request_time $http_referer $http_user_agent $http_x_forwarded_for $upstream_response_time $upstream_status;
Main settings:
http {
proxy_buffers 8 10k;
proxy_buffer_size 10k;
large_client_header_buffers 8 10k;
keepalive_timeout 65;
}
server {
listen 80;
server_name localhost;

gzip on;
gzip_comp_level 1;
gzip_min_length 50;
gzip_proxied any;
gzip_types text/css application/javascript application/json application/xml text/plain;

location /json/ {
proxy_pass http://127.0.0.1:8080/tomcat/webApp/;
}
}


From access log:
22.22.22.22 - - [19/Oct/2015:10:12:42 +0000] POST /json/some.json HTTP/1.1 400 430 0 0.019 curl/7.35.0 11.11.11.11 - -
22.22.22.22 - - [19/Oct/2015:10:13:32 +0000] POST /json/some.json HTTP/1.1 400 256 0 59.251 - - 11.11.11.11 - -
22.22.22.22 - - [19/Oct/2015:10:14:23 +0000] POST /json/some.json HTTP/1.1 400 257 0 59.711 - - 11.11.11.11 - -
22.22.22.22 - - [19/Oct/2015:10:14:35 +0000] POST /json/some.json HTTP/1.1 400 255 0 59.536 - - 11.11.11.11 - -

What I have checked:
- the log shows that reqest wasn't sent to tomcat. Tested and tomcat processes all requests under 200ms.
- nothing is written to error_log (tried info and warn)
- the general answer of increasing large_client_header_buffers size doesn't seem valid here as $request_length is not close to max(in case someone can suggest a way of replicating these requests then I can test on 2nd environment)

1. Why is Nginx replying with HTTP 400 in 0.019s for $request_length 430?
2. Why is there up to 60s $request_time? It looks like at 60s ELB kills it and returns 5xx to client.

Thanks,
Pauls