Welcome! Log In Create A New Profile

Advanced

Re: Client body upload fully occupies worker

Maxim Dounin
May 28, 2018 10:26AM
Hello!

On Fri, May 25, 2018 at 09:58:37AM -0400, python wrote:

> While client uploads content into permanent temp file ngx worker doesn't
> process any requests until its finished.
>
> Its an issue in case of large files (GBs) uploading since number of parallel
> uploads are limited by number of workers.

This can happen if the client upload bandwidth is larger than the
available disk bandwidth. Normally this does not happens with
typical clients, but may happen in various unbalanced
configurations. In particular, often such problems are reported
when testing uploads on a localhost.

> https://www.nginx.com/blog/thread-pools-boost-performance-9x/ advises to use
> aio_write:
> "At the moment, offloading to thread pools is implemented only for three
> essential operations: the read() syscall on most operating systems,
> sendfile() on Linux, and aio_write() on Linux which is used when writing
> some temporary files such as those for the cache".
>
> I tried to configure aio_write, but looks that approach won't work for
> client body upload.
>
> 1. How to configure thread pool usage for client body upload?

As of now, thread pools cannot be used for client body upload.

> 2. Could you please suggest any other approaches for non-blocking client
> body upload?

See https://trac.nginx.org/nginx/ticket/1431#comment:1 for a
detailed explanation on how things are expected to work and how
nginx worker can be monopolized, as well as some suggestsions on
how to mitigate the issue.

In the particular case of writing client body to a temporary file,
consider tuning "client_body_buffer_size" to a large enough value
(http://nginx.org/r/client_body_buffer_size), as well as using
smaller receive buffers. You can tune receive buffers using the
"rcvbuf=<n>" parameter of the "listen" directive, see
http://nginx.org/r/listen.

For large uploads you may also consider using
"proxy_request_buffering off", see
http://nginx.org/r/proxy_request_buffering.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Client body upload fully occupies worker

python May 25, 2018 09:58AM

Re: Client body upload fully occupies worker

Maxim Dounin May 28, 2018 10:26AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 71
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready