Welcome! Log In Create A New Profile

Advanced

Re: Possible memory leak?

Maxim Dounin
March 12, 2019 09:58PM
Hello!

On Tue, Mar 12, 2019 at 02:09:06PM -0400, wkbrad wrote:

> First of all, thanks so much for your insights into this and being patient
> with me. :) I'm just trying to understand the issue and what can be done
> about it.
>
> Can you explain to me what you mean by this?
> > you can configure system allocator to use mmap()
>
> I'm not a C programmer so correct me if I'm wrong, but doesn't the Nginx
> code determine which memory allocator it uses?

Normally C programs use malloc() / free() functions as provided by
system libc library to allocate memory. While it is possible for
an application to provide its own implementation of these
functions, this is something rarely used in practice.

> If not can you point me to an article that describes how to do that as I
> would like to test it?

For details on how to control system allocator on Linux, please
refer to the mallopt(3) manpage, notably the
MALLOC_MMAP_THRESHOLD_ environment variable. Web version is
available here:

http://man7.org/linux/man-pages/man3/mallopt.3.html

Please refer to the M_MMAP_THRESHOLD description in the same man
page for details on what it does and various implications.

Using a values less than NGX_CYCLE_POOL_SIZE (16k by default)
should help to move all configuration-related allocations into
mmap(), so these can be freed independently. Alternatively,
recompiling nginx with NGX_CYCLE_POOL_SIZE set to a value larger
than 128k (default mmap() threshold) should have similar
effect.

Note though that there may be other limiting factors,
such as MALLOC_MMAP_MAX_, which limits maximum number of mmap()
allocations to 65536 by default.

You can also play with different allocators by using the
LD_PRELOAD environment variable, see for example jemalloc's wiki
here:

https://github.com/jemalloc/jemalloc/wiki/Getting-Started

> Also, you seem to be saying that Nginx IS attempting to free the memory but
> is not able to due to the way the OS is allocating memory or refusing to
> release the memory. I've tested this in several Linux distros, kernels, and
> Nginx versions and I see the same behavior in all of them. Do you know of
> an OS or specific distro where Nginx can release the old memory allocations
> correctly? I would like to test that too. :)

Any Linux distro can be tuned so freed memory will be returned to
the system, see above. And for example on FreeBSD, which uses
jemalloc as a system allocator, unused memory is properly returned
to the system out of the box (though can be seen in virtual
address space occupied by the process, since the allocator uses
madvise() to make the memory as unused instead of unmapping a
mapping).

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Possible memory leak?

wkbrad February 28, 2019 01:43PM

Re: Possible memory leak?

Maxim Dounin February 28, 2019 03:08PM

Re: Possible memory leak?

wkbrad February 28, 2019 03:54PM

Re: Possible memory leak?

Maxim Dounin February 28, 2019 04:50PM

Re: Possible memory leak?

wkbrad February 28, 2019 05:06PM

RE: Possible memory leak?

Reinis Rozitis February 28, 2019 05:28PM

Re: Possible memory leak?

Alceu R. de Freitas Jr. via nginx February 28, 2019 06:02PM

Re: RE: Possible memory leak?

wkbrad February 28, 2019 06:44PM

Re: Possible memory leak?

wkbrad March 07, 2019 01:33PM

Re: Possible memory leak?

Anoop Alias March 07, 2019 08:06PM

Re: Possible memory leak?

wkbrad March 07, 2019 09:35PM

Re: Possible memory leak?

Anoop Alias March 07, 2019 10:10PM

Re: Possible memory leak?

wkbrad March 08, 2019 10:39AM

Re: Possible memory leak?

Anoop Alias March 08, 2019 06:54PM

Re: Possible memory leak?

gariac March 08, 2019 01:10AM

Re: Possible memory leak?

wkbrad March 08, 2019 10:42AM

Re: Possible memory leak?

gariac March 08, 2019 08:08PM

Re: Possible memory leak?

wkbrad March 11, 2019 04:37PM

Re: Possible memory leak?

Anoop Alias March 12, 2019 09:56AM

Re: Possible memory leak?

Maxim Dounin March 12, 2019 10:40AM

Re: Possible memory leak?

Anoop Alias March 12, 2019 11:00AM

Re: Possible memory leak?

Maxim Dounin March 12, 2019 11:48AM

Re: Possible memory leak?

wkbrad March 12, 2019 02:09PM

Re: Possible memory leak?

Maxim Dounin March 12, 2019 09:58PM

Re: Possible memory leak?

Peter Booth via nginx March 13, 2019 01:44AM

Re: Possible memory leak?

Anoop Alias March 13, 2019 01:54AM

Re: Possible memory leak?

gariac March 13, 2019 01:58AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 107
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready