Hi,

we've seen the following happen a few times, and I'd like to know who is
sending the SIGKILL, and if I can do anything about that.

We run a lengthy update process in a shutdown function after the response
has been flushed with fastcgi_finish_request(). It usually takes 6-10s, but
under high load, it can take 40s or more. Sometimes, after roughly 30s (sometimes
more, sometimes less), the process performing the update receives a SIGKILL.
While we can usually recover from this (by simply restarting the update), the
SIGKILL may just hit us while we own the shared memory lock to update APC. This
lock is _not_ freed when the process dies, so that all further attempts to
access APC block.

In the PHP error_log, the typical order of events is:

[26-Feb-2014 15:02:01] update start (PID 27582)
[26-Feb-2014 15:02:01] WARNING: [pool php] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 48 total children
[26-Feb-2014 15:02:02] WARNING: [pool php] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 0 idle, and 53 total children
[...] more of the same

[26-Feb-2014 15:02:08] WARNING: [pool php] server reached pm.max_children setting (80), consider raising it
[26-Feb-2014 15:02:29] WARNING: [pool php] child 27582 exited on signal 9 (SIGKILL) after 103.188030 seconds from start


So, my question is, who is sending a SIGKILL to the process, why is
it done, and can I influence this behaviour in any way?

As far as I can tell, the SIGKILL is not caused by exceeding the
PHP max_execution_time (the overall CPU time used is smaller than
the configured limit) or the cpu time ulimit (set to unlimited).


thanks,


rainer

--

---
You received this message because you are subscribed to the Google Groups "highload-php-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to highload-php-en+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

If I think correctly, the Master Process is terminating child due to the lack of response from particular child.



What’s your server’s load? Did you consider horizontal scaling? Using more workers than CPU cores is not a good practice.



From: highload-php-en@googlegroups.com [mailto:highload-php-en@googlegroups..com] On Behalf Of rainer.canavan@sevenval.com
Sent: Wednesday, March 19, 2014 8:19 PM
To: highload-php-en@googlegroups.com
Subject: SIGKILL while performing lengthy task in shutdown function



Hi,

we've seen the following happen a few times, and I'd like to know who is
sending the SIGKILL, and if I can do anything about that.

We run a lengthy update process in a shutdown function after the response
has been flushed with fastcgi_finish_request(). It usually takes 6-10s, but
under high load, it can take 40s or more. Sometimes, after roughly 30s (sometimes
more, sometimes less), the process performing the update receives a SIGKILL..
While we can usually recover from this (by simply restarting the update), the
SIGKILL may just hit us while we own the shared memory lock to update APC. This
lock is _not_ freed when the process dies, so that all further attempts to
access APC block.

In the PHP error_log, the typical order of events is:

[26-Feb-2014 15:02:01] update start (PID 27582)
[26-Feb-2014 15:02:01] WARNING: [pool php] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 48 total children
[26-Feb-2014 15:02:02] WARNING: [pool php] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 0 idle, and 53 total children

[...] more of the same


[26-Feb-2014 15:02:08] WARNING: [pool php] server reached pm.max_children setting (80), consider raising it
[26-Feb-2014 15:02:29] WARNING: [pool php] child 27582 exited on signal 9 (SIGKILL) after 103.188030 seconds from start


So, my question is, who is sending a SIGKILL to the process, why is
it done, and can I influence this behaviour in any way?

As far as I can tell, the SIGKILL is not caused by exceeding the
PHP max_execution_time (the overall CPU time used is smaller than
the configured limit) or the cpu time ulimit (set to unlimited).


thanks,


rainer

--

---
You received this message because you are subscribed to the Google Groups "highload-php-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to highload-php-en+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "highload-php-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to highload-php-en+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.