Welcome! Log In Create A New Profile

Advanced

nginx and upstream Content-Length

Yaniv Aknin
September 30, 2011 04:12AM
Hi,

In a recent thread on the uwsgi mailing list[1], I began suspecting that
nginx will not honor an upstream's Content-Length header. i.e., if an
upstream mentions a Content-Length of 1,000 bytes, but the connection is
broken after 500 bytes, nginx will still happily serve this entity with a
200 OK status.

This may be a known bug in nginx, I wanted to be certain I indeed understand
it correctly and raise the attention to it on the nginx mailing list -
because I think this is a very serious bug with potentially disastrous
consequences, as I describe below.

I was able to confirm this both for uwsgi_pass and proxy_pass; if the
upstream sets a Content-Length and then breaks the connection before that
length was achieved, nginx will pass this onwards to the client.
Furthermore, since the upstream protocol is HTTP 1.0 but the nginx-client
protocl is HTTP 1.1 (with keepalive), the request will simply not terminate,
because the client can't tell that the server has nothing more to send and
nginx will not break the connection, despite the fact its connection with
the upstream was broken and there's no chance this request will ever be
fulfilled.

Things get far worse with gzip compression on - nginx will remove the
Content-Length header sent by the client and replace it with chunked
encoding - /incorrect chunked encoding/, that will make the client believe
it has the full entity, even though it has only a part of this.

Think about this with regard to caching, ETags and transparent proxy caching
- if something like this happens to a cachable entity, especially one with
an ETag, especially if a large ISP's transparent proxy intercepts the
request - you might end up serving an incorrect representation of the entity
for a very long time and for many thousands of requests (!).

Anyhow, I think the only sane resolution is that nginx will honor upstream
Content-Length (and chunked encoding, if and when nginx will support it),
and intentionally close the downstream connection prematurely in case the
upstream connection is closed before the end of the Content-Length or the
last chunk is received.

I suspect the recent work in nginx 1.1.4 for
ngx_http_upstream_keepalivecould be relevant here, but not sure,
didn't read the code.

I'll be happy to hear your thoughts or provide further data.

- Yaniv

1: http://comments.gmane.org/gmane.comp.python.wsgi.uwsgi.general/2061
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

nginx and upstream Content-Length

Yaniv Aknin September 30, 2011 04:12AM

Re: nginx and upstream Content-Length

Maxim Dounin September 30, 2011 06:00AM

Re: nginx and upstream Content-Length

Yaniv Aknin September 30, 2011 10:38AM

Re: nginx and upstream Content-Length

Thomas Love September 30, 2011 11:10AM

Re: nginx and upstream Content-Length

Yaniv Aknin September 30, 2011 12:08PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 118
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready