Welcome! Log In Create A New Profile

Advanced

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Phillip Oldham
December 09, 2009 04:14AM
miradev wrote:
> We've been running nginx + spawn-fcgi for 6 months without any issues. Until one morning I come in and all our sites are down because there were no php-cgi processes running. Obviously I'm keen to not have this happen again. I could make use of something like monit to monitor the php-cgi processes, but even a few minutes down time (waiting for a monit cycle) is not good.
>
> The only article on this I've been able to find so far is this one: http://blog.taragana.com/index.php/archive/how-to-stop-crashing-hanging-of-php-cgi-spawn-fcgi-with-nginx-lighttpd/
>
I had a very similar problem when using lighttpd+spawn-fcgi. We had
moved from Apache about 3 years ago because of load issues with mod_php
to lighttpd+spawnfcgi+php. This worked fine for a while, but as our load
increased we started to see PHP being unresponsive, and lighty returning
503s.

After some digging I found the cause was the way spawnfcgi handled
incomming requests in conjunction with PHP's internal
PHP_FCGI_MAX_REQUESTS setting. This setting is set to 250 by default in
PHP, and when this limit is hit PHP recycles the child thread. During
this recycle spawnfcgi would continue to handle requests, pass them to
the recycling thread, get no response, and in turn lighty would return a
503. IIRC this would also cause the recycling thread to die, slowly
killing off all the threads and PHP completely.

Playing with PHP_FCGI_MAX_REQUESTS helped in the short-term but we
continued to experience the problem, so when I heard of nginx I quickly
switched to nginx+php-fcgi (using the -b switch). Since then I've yet to
see an issue, and we've had ~100 production sites running on that stack
for over 2 years. Since starting to use supervisord we've begun the
process of moving the rest of our ~500 domains/sites over to to that stack.

Using PHP `-b` in my experience is rock-solid. Running it under
supervisord means that if it dies it is brought back up straight away.
This does not affect nginx, and you can always show a nice error page
should this case happen - I remember that if php went down hard under
mod_php it would cause apache to fail also. Bad times.

My suggestion is to get rid of spawnfcgi and install supervisord. Here's
the supervisord config I use on my dev machine:

[fcgi-program:php]
user=nginx
command=/usr/bin/php-cgi -c /etc/php5/php.ini -b /tmp/php.socket
socket=unix:///tmp/php.socket
autostart=true
startsecs=5
startretries=3
stopwaitsecs=10
environment=PHP_FCGI_MAX_REQUESTS=50,PHP_FCGI_CHILDREN=4

Some on the board suggest having nginx running under supervisord also,
but I'm a little worried about that approach; if supervisord goes down
for any reason on a live server everything goes down, whereas if nginx
is kept separate and supervisord goes down nginx can continue to serve
requests (albeit error pages) while supervisord comes back up and brings
your app back online (php, database, memcached, etc).

Our production runs:
supervisord > php-fcgi, mysqld, memcached, and a number of python
"services" (our webapp layer)
monit > nginx, supervisord, sshd, other "system" level important daemons
which means if supervisord goes down for any reason, it'll only be down
for 1min until monit can bring it back up. Since memcached is killed our
webapp is brought back up in a "clean" state. Users are asked to log in
again, but this rarely (or, rather, is yet to) happens and is a mild
inconvenience for them.

Give it a try. All these technologies are highly configurable and there
are many permutations. To find your optimal configuration is going to
take experimentation. It's not like Apache where there's only one
"right" way to do things; you've got more control - make use of it!

--

*Phillip B Oldham*
ActivityHQ
phill@activityhq.com <mailto:phill@theactivitypeople.co.uk>

------------------------------------------------------------------------

*Policies*

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.

------------------------------------------------------------------------
_______________________________________________
nginx mailing list
nginx@nginx.org
http://nginx.org/mailman/listinfo/nginx
Subject Author Posted

Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

miradev December 08, 2009 03:17AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 08, 2009 06:20AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

miradev December 08, 2009 06:29AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 08, 2009 06:36AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Grzegorz Nosek December 08, 2009 06:48AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 08, 2009 06:56AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Grzegorz Nosek December 08, 2009 07:00AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Grzegorz Nosek December 08, 2009 07:06AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 08, 2009 07:06AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

nerdgrind December 08, 2009 04:43PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 08, 2009 05:10PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

nerdgrind December 08, 2009 05:31PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 08, 2009 05:56PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

nerdgrind December 08, 2009 09:21PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 08, 2009 09:34PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

nerdgrind December 08, 2009 11:35PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 08, 2009 11:52PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Denis Filimonov December 09, 2009 01:08AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Grzegorz Nosek December 09, 2009 03:00AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

nerdgrind December 09, 2009 03:27AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

nerdgrind December 08, 2009 07:46PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

dannynoonan March 06, 2010 12:58AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Glen Lumanau December 08, 2009 08:08PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Jim Ohlstein December 08, 2009 08:18PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Jim Ohlstein December 08, 2009 08:27PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 08, 2009 08:20PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Igor Sysoev December 09, 2009 04:06AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Phillip Oldham December 09, 2009 04:14AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 09, 2009 07:26AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 09, 2009 07:38AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Igor Sysoev December 09, 2009 08:00AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Phillip Oldham December 09, 2009 04:14AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Grzegorz Nosek December 09, 2009 04:46AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Phillip Oldham December 09, 2009 04:58AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

nerdgrind December 09, 2009 04:54AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 09, 2009 07:26AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

miradev December 09, 2009 07:24AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

nerdgrind December 09, 2009 02:54PM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 09, 2009 07:42AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

Akins, Brian December 09, 2009 10:56AM

Re: Fast CGI (spawn-fcgi / php-cgi) crashes/dies/hangs

mike December 09, 2009 01:18PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 310
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready