Welcome! Log In Create A New Profile

Advanced

Prevent json truncation from large POST requests

Posted by coherent 
Prevent json truncation from large POST requests
January 09, 2018 02:24PM
Hi there,
I'm having an issue with Nginx and sending large POST requests. I have a load balancer that forwards requests to an EC2 instance that has Nginx running on port 80 and my application running on port 9928. See attached image for an architecture diagram, I can tell that submitting directly to my application is fine while submitting through nginx causes the error, which makes me think this is an nginx issue.

When I submit moderate sized POST requests, everything is fine, but when I increase the size of the POST request to multiple megabytes, my application has the error "message is too long" and that the json is invalid because it has an unexpected end of input. I think my POST request is getting truncated somewhere between nginx and my application server because my $request_body when I save it as a log, looks fine.

Here is a snippet of my debug nginx log:

5 http client request body recv 8949
*5 http client request body rest 5239409
*5 recv: fd:9 -1 of 3693359
*5 recv() not ready (11: Resource temporarily unavailable)
*5 http client request body recv -2
*5 http client request body rest 5239409
*5 event timer: 9, old: 1515520710736, new: 1515520710905
*5 post event 000055A779AE3B40
*5 delete posted event 000055A779AE3B40
*5 http run request: "/mps/updateFunction?mode=async"

From Line 1796 of this Failed NGINX Log:
https://gist.github.com/CaptainChemist/b1562b40b4a2da89bf8ed452e7cac4d4

By comparison, this is a Successful NGINX Log:
https://gist.github.com/CaptainChemist/ab920a953ead13d7244a657e1521ab71

Here are my configuration files, clearly they are kind of a mess but thanks so much for your help!

**nginx.conf**

user www-data;
worker_processes auto;
pid /run/nginx.pid;

events {
worker_connections 768;
}

http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;

include /etc/nginx/mime.types;
default_type application/octet-stream;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;

access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log debug;

include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}

**sites-enabled/default**

upstream processingServer {
server 127.0.0.1:9928;
keepalive 25600;
}

log_format postdata $request_body;
log_format upstreamlog $request_body;

server {
listen 80;
client_max_body_size 5000M;
client_body_buffer_size 5000M;
client_header_timeout 3m;
client_body_timeout 3m;
send_timeout 3m;
keepalive_timeout 100000;
location / {
access_log /var/log/nginx/postdata.log postdata;
access_log /var/log/nginx/upstream_postdata.log upstreamlog;

add_header X-external-IP 54.89.000.000;
proxy_pass http://processingServer;
proxy_send_timeout 86400s;
proxy_read_timeout 300s;
proxy_http_version 1.1;
}
}



Edited 3 time(s). Last edit at 01/09/2018 02:26PM by coherent.
Attachments:
open | download - nginx.png (46.9 KB)
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 199
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready