I am using NGINX version 1.2.5. Under load serving requests, I encountered memory loss. I found three memory leaks which I fixed in my code. The leaks were in the response handling code where memory was not freed properly especially if NGX_AGAIN was encountered due to SSL back pressure.

However, it prompted this curious situation when the memory leak was happening. I typically reload NGINX on a regular basis to grab new configuration that I posted on an hourly basis. I have one master and four workers. When the memory leak was happening, and memory was depleted on my linux box, the reload would fail at the end of two days since there wasn't enough memory left to start a new worker and remove the old one. That I understand, but what confuses me, is why on every reload, since the old worker was removed, the memory leak wouldn't be masked since a process is removed and its memory should have been returned to Linux. When I perform an NGINX restart, that would return all leaked memory. The difference that I gather is that on a restart the master worker is also brought down. But isn't every worker running with its own memory scope?

Can someone explain to me how memory is handled differently in a reload versus a restart that can explain this behavior?

Thank you.

Without seeing your patches and holding it against the 1.7 branch, which has several patches already in this area, its impossible to say anything.

---
nginx for Windows http://nginx-win.ecsds.eu/

Here is some more information I got, maybe it can she some light.

I perform a service nginx start

sh-4.1# service nginx status
nginx (pid 19610 19609 19608 19607 19593) is running...
sh-4.1# ps -ef | grep nginx
root 19593 1 7 16:24 ? 00:00:04 nginx: master process /usr/local/bin/nginx -c /etc/nginx/nginx.conf
nobody 19607 19593 0 16:24 ? 00:00:00 nginx: worker process
nobody 19608 19593 0 16:24 ? 00:00:00 nginx: worker process
nobody 19609 19593 0 16:24 ? 00:00:00 nginx: worker process
nobody 19610 19593 0 16:24 ? 00:00:00 nginx: worker process
root 19653 19083 0 16:25 pts/1 00:00:00 grep nginx
sh-4.1#


In the logs, I print out the PID where the memory allocator is initialized:

sh-4.1# grep 19571 *
error.log:2014-06-04 16:24:36.535 +0000 [error] ngx_http_url_filtering_init(cloud_addons/ngx_http_url_filtering_module.c:603): Allocating the allocator: PID of this process: 19571

I looked up 19571 ands it does not exist anymore. So i reran this and monitored the processes and discovered that 19571 is nginx coming up initially before the master:


root 19571 19570 88 16:41 pts/1 00:00:01 /usr/local/bin/nginx -c /etc/nginx/nginx.conf

If I issue a reload:

service nginx reload

error.log:2014-06-04 16:27:36.103 +0000 [error] ngx_http_url_filtering_init(cloud_addons/ngx_http_url_filtering_module.c:603): Allocating the allocator: PID of this process: 19593

This is the master now that is allocating a new allocator.

When I issue a service nginx stop, which is similar to a restart:

error.log:2014-06-04 16:31:59.682 +0000 [error]
ngx_http_url_filtering_exit(cloud_addons/ngx_http_url_filtering_module.c:680):
Dellocating the allocator: PID of this process: 19593

This "cloud_addons/ngx_http_url_filtering_module.c" is an addon which can cause the 'bug', can you try without any addons? or with the mainline version? alot has changed since 1.2.5

---
nginx for Windows http://nginx-win.ecsds.eu/