Hello!

On Tue, Jun 16, 2020 at 03:42:04AM -0400, shaharmor wrote:

> I noticed that worker_connections while is defined as the maximum number of
> connections per worker, nginx pre-allocated enough memory to handle all
> possible worker_connections, even before they are actually needed.
>
> For example, setting worker_connections to 10485760 causes nginx to take
> 4.3GB of memory upon init.
>
> Is this how its supposed to be?

Yes.

> Is there a way to tell nginx to only allocate memory as needed?

No.

Connection structures are small and cannot be freed, so nginx
allocates them all on startup. This way it avoids memory
fragmentation, and makes connnection management easier and faster.

Note well that you cannot really use connection structures in
nginx without corresponding kernel structures such as sockets and
appropriate buffers, so using arbitrary large worker_connections
value does not make sense. If you are seeing that it takes
significant amount of memory, most likely you've configured it
incorrectly.

For example, some relevant numbers can be seen in the following
blog post by WhatsApp:

https://blog.whatsapp.com/1-million-is-so-2011

Numbers in suggest that about 64G of memory is needed to handle 2
millions of connections. Scaling this to 10 millions as in your
example gives about 256G, so 4G for worker_connections shouldn't
be noticeable.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx