Hello!

On Fri, May 25, 2018 at 09:58:37AM -0400, python wrote:

> While client uploads content into permanent temp file ngx worker doesn't
> process any requests until its finished.
>
> Its an issue in case of large files (GBs) uploading since number of parallel
> uploads are limited by number of workers.

This can happen if the client upload bandwidth is larger than the
available disk bandwidth. Normally this does not happens with
typical clients, but may happen in various unbalanced
configurations. In particular, often such problems are reported
when testing uploads on a localhost.

> https://www.nginx.com/blog/thread-pools-boost-performance-9x/ advises to use
> aio_write:
> "At the moment, offloading to thread pools is implemented only for three
> essential operations: the read() syscall on most operating systems,
> sendfile() on Linux, and aio_write() on Linux which is used when writing
> some temporary files such as those for the cache".
>
> I tried to configure aio_write, but looks that approach won't work for
> client body upload.
>
> 1. How to configure thread pool usage for client body upload?

As of now, thread pools cannot be used for client body upload.

> 2. Could you please suggest any other approaches for non-blocking client
> body upload?

See https://trac.nginx.org/nginx/ticket/1431#comment:1 for a
detailed explanation on how things are expected to work and how
nginx worker can be monopolized, as well as some suggestsions on
how to mitigate the issue.

In the particular case of writing client body to a temporary file,
consider tuning "client_body_buffer_size" to a large enough value
(http://nginx.org/r/client_body_buffer_size), as well as using
smaller receive buffers. You can tune receive buffers using the
"rcvbuf=<n>" parameter of the "listen" directive, see
http://nginx.org/r/listen.

For large uploads you may also consider using
"proxy_request_buffering off", see
http://nginx.org/r/proxy_request_buffering.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx