Use case: Large uploads (hundreds of megabytes to tens of gigabytes) where nginx is serving as a reverse proxy and load balancer. The upstream servers can get bogged down, and when they do, they apply backpressure by responding with 503 status code.

Problem: Naively implemented, the client sends the entire request body off to the server, then waits to find out that the server can't handle the request. Time and network bandwidth are wasted, and the client has to retry the request.

Partial solution: Using an idempotent request method, with "proxy_request_buffering on", and "proxy_next_upstream http_503", nginx will accept the upload from the client once, but try each server in succession until one works. Fortunately, nginx will set header "Expect: 100-continue" on each proxied request and will not send the request body off to an upstream server that isn't ready to receive it. However, nginx won't even begin to send a proxied request to any upstream server until the initial request body upload from the client has completed. Also, the entire request body has to be stored somewhere local to nginx and the speed of that storage has a direct impact on the performance of the whole process.

Next solution idea: Have the *client* set header "Expect: 100-continue". Then the client won't send the request body until nginx can find an upstream server to handle the request. However, this is not how things work today. Nginx will unconditionally accept the request with "100 Continue" regardless of upstream server status. With buffering enabled, this makes sense, since nginx wants to aggressively buffer the request body so it can re-send it if needed.

Refined solution idea: Disable buffering. Unfortunately, while setting "proxy_request_buffering off" and "proxy_http_version 1.1" does disable buffering, it doesn't disable nginx from immediately telling the client "100 Continue". Moreover, nginx only tries one upstream server before giving up, probably because it has no buffered copy of the request body to send to the next server on behalf of the client. Yet if nginx delayed sending "100 Continue" back to the client, it could have taken a little bit more time to find a viable upstream server.

I did some digging before bringing this topic up, and I find a proposed patch (http://mailman.nginx.org/pipermail/nginx-devel/2016-August/008736.html), a request in the forum (https://forum.nginx.org/read.php?2,212533,212533#msg-212533), and a trac ticket (https://trac.nginx.org/nginx/ticket/493) all to disable automatic handling of the 100-continue mechanism. The trac ticket was closed because unbuffered upload was not supported yet, the patch was rejected because it sounded like it was the other side's problem to solve, and finally the forum request was rejected because nginx was "designed as [an] accelerator to minimize backend interaction with a client".

As to that last quoted part, I agree! I'd rather have nginx figure things out than to have the client finagle with the backend server too much. So here's what I think should happen. First, the client's Expect header should not get directly passed on to the upstream server nor should nginx ignore the header entirely (i.e., keep these things the same as they are today). Instead, with unbuffered upload, an upstream block with multiple servers, and proxy_next_upstream set to try another server when one fails:

1. Client sends request with "Expect: 100-continue" to nginx
2. Nginx receives request but does not respond with anything yet
3. Nginx tries the first eligible server by sending a proxied request with "Expect: 100-continue" (not passthrough; this is nginx's own logic and this part exists today as far as I can tell)
4. If the server responds "100 Continue" to nginx *then* nginx responds "100 Continue" to the client and the unbuffered upload proceeds to that server
5. If instead the server fails in a way that proxy_next_upstream is configured to handle, then nginx still doesn't respond to the client, and now tries to reach the next eligible server instead.
6. This process proceeds until a server willing to accept the request is found or all servers have been tried and none are available, at which point nginx sends an appropriate non-100 response to the client (502/503/504).

(Caveat: If an upstream server fails *after* already accepting a request with "100 Continue", then nginx still has to give up since there's no buffering.)

Thoughts? I know there are other ways to solve this problem (e.g. S3-style multipart uploads), but there is a convenience to the "Expect: 100-continue" mechanism and it is pretty widely supported. I don't think this goes against the grain of what nginx is trying to be, especially since unbuffered uploads are supported now.

Thanks for your consideration,
Kristian Bolino