Welcome! Log In Create A New Profile

Advanced

Intermittent problems with child workers exiting with SIGSEGV

Posted by alexd 
Hi,

I have successfully installed php-fpm using the version available in
PHP core via SVN. However now I'm getting intermittent 502 bad gateway
problems in Nginx. Taking a look at php-fpm.log shows the following
happening at the time of each 502:

Jan 23 17:30:45.254989 [WARNING] [pool default] child 18390 exited on
signal 11 SIGSEGV after 8.940000 seconds from start
Jan 23 17:30:45.254989 [NOTICE] [pool default] child 18402 started

This doesn't happen on every page load - if I restart php-fpm then
everything's fine for 10-20 page refreshes then I only get 502s until
I believe php-fpm spots the issue and restarts. Then the pattern
repeats.

I'm a bit stumped - it feels more like a memory issue than a
configuration one but the box (256mb Debian slice) shows plenty of
free memory at all times. I tried moving child workers from 50 down to
12 but that didn't help either.

(I know that similar SIGSEGV problems have been reported before but
they seem to relate to older codebases and/or configuration issues.)

Any help would be much appreciated, and do let me know if you want me
to paste any configuration files etc.

Many thanks,

Alex
Hi!
Your log just says that the process got SIGSEGV but the reason can be
very different.
In very many cases this is not fpm issue at all: bugs in extensions,
accelerators etc.
The only way to find the reason is to provide a backtrace: you have to
collect cores and get the trace with gdb or just run fpm under gdb.

On Sat, Jan 23, 2010 at 8:48 PM, alexd <alexander.dean@gmail.com> wrote:
> Hi,
>
> I have successfully installed php-fpm using the version available in
> PHP core via SVN. However now I'm getting intermittent 502 bad gateway
> problems in Nginx. Taking a look at php-fpm.log shows the following
> happening at the time of each 502:
>
> Jan 23 17:30:45.254989 [WARNING] [pool default] child 18390 exited on
> signal 11 SIGSEGV after 8.940000 seconds from start
> Jan 23 17:30:45.254989 [NOTICE] [pool default] child 18402 started
>
> This doesn't happen on every page load - if I restart php-fpm then
> everything's fine for 10-20 page refreshes then I only get 502s until
> I believe php-fpm spots the issue and restarts. Then the pattern
> repeats.
>
> I'm a bit stumped - it feels more like a memory issue than a
> configuration one but the box (256mb Debian slice) shows plenty of
> free memory at all times. I tried moving child workers from 50 down to
> 12 but that didn't help either.
>
> (I know that similar SIGSEGV problems have been reported before but
> they seem to relate to older codebases and/or configuration issues.)
>
> Any help would be much appreciated, and do let me know if you want me
> to paste any configuration files etc.
>
> Many thanks,
>
> Alex
>



--

wbr,
fisher
Re: Intermittent problems with child workers exiting with SIGSEGV
January 23, 2010 03:50PM
Recycling in only 9 seconds seems pretty quick.

Sent from my iPhone

On Jan 23, 2010, at 9:48 AM, alexd <alexander.dean@gmail.com> wrote:

> Hi,
>
> I have successfully installed php-fpm using the version available in
> PHP core via SVN. However now I'm getting intermittent 502 bad gateway
> problems in Nginx. Taking a look at php-fpm.log shows the following
> happening at the time of each 502:
>
> Jan 23 17:30:45.254989 [WARNING] [pool default] child 18390 exited on
> signal 11 SIGSEGV after 8.940000 seconds from start
> Jan 23 17:30:45.254989 [NOTICE] [pool default] child 18402 started
>
> This doesn't happen on every page load - if I restart php-fpm then
> everything's fine for 10-20 page refreshes then I only get 502s until
> I believe php-fpm spots the issue and restarts. Then the pattern
> repeats.
>
> I'm a bit stumped - it feels more like a memory issue than a
> configuration one but the box (256mb Debian slice) shows plenty of
> free memory at all times. I tried moving child workers from 50 down to
> 12 but that didn't help either.
>
> (I know that similar SIGSEGV problems have been reported before but
> they seem to relate to older codebases and/or configuration issues.)
>
> Any help would be much appreciated, and do let me know if you want me
> to paste any configuration files etc.
>
> Many thanks,
>
> Alex
Re: Intermittent problems with child workers exiting with SIGSEGV
January 23, 2010 04:18PM
>Recycling in only 9 seconds seems pretty quick.

It looks like he is receiving a SIGSEGV which would be a segmentation fault,
so it wouldn't be related to recycling. I haven't tried the 5.3.x code, but
it appears that it is probably something in your application code that is
buggy.

On Sat, Jan 23, 2010 at 3:47 PM, Michael Shadle <mike503@gmail.com> wrote:

> Recycling in only 9 seconds seems pretty quick.
>
> Sent from my iPhone
>
>
> On Jan 23, 2010, at 9:48 AM, alexd <alexander.dean@gmail.com> wrote:
>
> Hi,
>>
>> I have successfully installed php-fpm using the version available in
>> PHP core via SVN. However now I'm getting intermittent 502 bad gateway
>> problems in Nginx. Taking a look at php-fpm.log shows the following
>> happening at the time of each 502:
>>
>> Jan 23 17:30:45.254989 [WARNING] [pool default] child 18390 exited on
>> signal 11 SIGSEGV after 8.940000 seconds from start
>> Jan 23 17:30:45.254989 [NOTICE] [pool default] child 18402 started
>>
>> This doesn't happen on every page load - if I restart php-fpm then
>> everything's fine for 10-20 page refreshes then I only get 502s until
>> I believe php-fpm spots the issue and restarts. Then the pattern
>> repeats.
>>
>> I'm a bit stumped - it feels more like a memory issue than a
>> configuration one but the box (256mb Debian slice) shows plenty of
>> free memory at all times. I tried moving child workers from 50 down to
>> 12 but that didn't help either.
>>
>> (I know that similar SIGSEGV problems have been reported before but
>> they seem to relate to older codebases and/or configuration issues.)
>>
>> Any help would be much appreciated, and do let me know if you want me
>> to paste any configuration files etc.
>>
>> Many thanks,
>>
>> Alex
>>
>
Re: Intermittent problems with child workers exiting with SIGSEGV
January 23, 2010 04:24PM
This is true. I was getting confused with SIGCHLD. Some people have
posted on here with very quick recycle rates with SIGCHLDs...

On Sat, Jan 23, 2010 at 1:17 PM, Davy Campano <dcampano@gmail.com> wrote:
>>Recycling in only 9 seconds seems pretty quick.
> It looks like he is receiving a SIGSEGV which would be a segmentation fault,
> so it wouldn't be related to recycling.  I haven't tried the 5.3.x code, but
> it appears that it is probably something in your application code that is
> buggy.
>
> On Sat, Jan 23, 2010 at 3:47 PM, Michael Shadle <mike503@gmail.com> wrote:
>>
>> Recycling in only 9 seconds seems pretty quick.
>>
>> Sent from my iPhone
>>
>> On Jan 23, 2010, at 9:48 AM, alexd <alexander.dean@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I have successfully installed php-fpm using the version available in
>>> PHP core via SVN. However now I'm getting intermittent 502 bad gateway
>>> problems in Nginx. Taking a look at php-fpm.log shows the following
>>> happening at the time of each 502:
>>>
>>> Jan 23 17:30:45.254989 [WARNING] [pool default] child 18390 exited on
>>> signal 11 SIGSEGV after 8.940000 seconds from start
>>> Jan 23 17:30:45.254989 [NOTICE] [pool default] child 18402 started
>>>
>>> This doesn't happen on every page load - if I restart php-fpm then
>>> everything's fine for 10-20 page refreshes then I only get 502s until
>>> I believe php-fpm spots the issue and restarts. Then the pattern
>>> repeats.
>>>
>>> I'm a bit stumped - it feels more like a memory issue than a
>>> configuration one but the box (256mb Debian slice) shows plenty of
>>> free memory at all times. I tried moving child workers from 50 down to
>>> 12 but that didn't help either.
>>>
>>> (I know that similar SIGSEGV problems have been reported before but
>>> they seem to relate to older codebases and/or configuration issues.)
>>>
>>> Any help would be much appreciated, and do let me know if you want me
>>> to paste any configuration files etc.
>>>
>>> Many thanks,
>>>
>>> Alex
>
>
Re: Intermittent problems with child workers exiting with SIGSEGV
January 23, 2010 04:26PM
Alex,

Are you running any other extensions? Probably the best place to get it
figured out though is by debugging it with gdb as Alexey mentioned. The
good part is that it sounds like it is quickly reproducible.

On Sat, Jan 23, 2010 at 4:21 PM, Michael Shadle <mike503@gmail.com> wrote:

> This is true. I was getting confused with SIGCHLD. Some people have
> posted on here with very quick recycle rates with SIGCHLDs...
>
> On Sat, Jan 23, 2010 at 1:17 PM, Davy Campano <dcampano@gmail.com> wrote:
> >>Recycling in only 9 seconds seems pretty quick.
> > It looks like he is receiving a SIGSEGV which would be a segmentation
> fault,
> > so it wouldn't be related to recycling. I haven't tried the 5.3.x code,
> but
> > it appears that it is probably something in your application code that is
> > buggy.
> >
> > On Sat, Jan 23, 2010 at 3:47 PM, Michael Shadle <mike503@gmail.com>
> wrote:
> >>
> >> Recycling in only 9 seconds seems pretty quick.
> >>
> >> Sent from my iPhone
> >>
> >> On Jan 23, 2010, at 9:48 AM, alexd <alexander.dean@gmail.com> wrote:
> >>
> >>> Hi,
> >>>
> >>> I have successfully installed php-fpm using the version available in
> >>> PHP core via SVN. However now I'm getting intermittent 502 bad gateway
> >>> problems in Nginx. Taking a look at php-fpm.log shows the following
> >>> happening at the time of each 502:
> >>>
> >>> Jan 23 17:30:45.254989 [WARNING] [pool default] child 18390 exited on
> >>> signal 11 SIGSEGV after 8.940000 seconds from start
> >>> Jan 23 17:30:45.254989 [NOTICE] [pool default] child 18402 started
> >>>
> >>> This doesn't happen on every page load - if I restart php-fpm then
> >>> everything's fine for 10-20 page refreshes then I only get 502s until
> >>> I believe php-fpm spots the issue and restarts. Then the pattern
> >>> repeats.
> >>>
> >>> I'm a bit stumped - it feels more like a memory issue than a
> >>> configuration one but the box (256mb Debian slice) shows plenty of
> >>> free memory at all times. I tried moving child workers from 50 down to
> >>> 12 but that didn't help either.
> >>>
> >>> (I know that similar SIGSEGV problems have been reported before but
> >>> they seem to relate to older codebases and/or configuration issues.)
> >>>
> >>> Any help would be much appreciated, and do let me know if you want me
> >>> to paste any configuration files etc.
> >>>
> >>> Many thanks,
> >>>
> >>> Alex
> >
> >
>
Thanks for everybody's suggestions!

I'm running a pretty straightforward PHP (./configure --enable-fpm --
with-libevent=/usr/local/lib \
--enable-mysql --with-mysql=/usr/local/mysql --enable-pdo_mysql), so I
don't think the problem lies with my extensions. I'm not running any
accelerators or similar.

The problem is happening with a pretty simple MaxMind script too:
<?php
include("/var/www/lib/MaxMind/geoip.inc");
$gi = geoip_open("/var/www/lib/MaxMind/data/
GeoIP.dat",GEOIP_STANDARD);
$country = geoip_country_name_by_addr($gi, "80.24.24.24");
echo "You're currently in: " . $country;
geoip_close($gi);
?>

If I run this from the command line (php -f geotest.php) then I can
run it ad infinitum without any errors.

To debug this I tried the following:

1. Reduced the number of worker processes to 1 (after doing this every
2nd reload of the page would be a 502).

2. Stopped php-fpm then ran the commands:
sudo -i
gdb php-fpm
set args --fpm-config=/etc/php/php-fpm.conf
run

3. In another window, ps -ef | grep fpm to find the child worker PID
and then back in the gdb window:
attach 19561
continue

4. Then I go and reload the page in the browser, get a 502 and then in
the gdb window I see:

Reading symbols from /lib/libnss_nis.so.2...done.
Loaded symbols for /lib/libnss_nis.so.2
0x00007f24337ada20 in __accept_nocancel () from /lib/libpthread.so.0
(gdb) continue
Continuing.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f243462a6e0 (LWP 19561)]
fpm_status_increment_accepted_conn (shm=0x223ad90) at /home/admin/
sources/php_5_3_fpm/sapi/fpm/fpm/fpm_status.c:77
77 status.accepted_conn++;
(gdb)

I hope this gives some more information!

Thanks,

Alex

On Jan 23, 9:24 pm, Davy Campano <dcamp...@gmail.com> wrote:
> Alex,
>
> Are you running any other extensions?  Probably the best place to get it
> figured out though is by debugging it with gdb as Alexey mentioned.  The
> good part is that it sounds like it is quickly reproducible.
>
> On Sat, Jan 23, 2010 at 4:21 PM, Michael Shadle <mike...@gmail.com> wrote:
> > This is true. I was getting confused with SIGCHLD. Some people have
> > posted on here with very quick recycle rates with SIGCHLDs...
>
> > On Sat, Jan 23, 2010 at 1:17 PM, Davy Campano <dcamp...@gmail.com> wrote:
> > >>Recycling in only 9 seconds seems pretty quick.
> > > It looks like he is receiving a SIGSEGV which would be a segmentation
> > fault,
> > > so it wouldn't be related to recycling.  I haven't tried the 5.3.x code,
> > but
> > > it appears that it is probably something in your application code that is
> > > buggy.
>
> > > On Sat, Jan 23, 2010 at 3:47 PM, Michael Shadle <mike...@gmail.com>
> > wrote:
>
> > >> Recycling in only 9 seconds seems pretty quick.
>
> > >> Sent from my iPhone
>
> > >> On Jan 23, 2010, at 9:48 AM, alexd <alexander.d...@gmail.com> wrote:
>
> > >>> Hi,
>
> > >>> I have successfully installed php-fpm using the version available in
> > >>> PHP core via SVN. However now I'm getting intermittent 502 bad gateway
> > >>> problems in Nginx. Taking a look at php-fpm.log shows the following
> > >>> happening at the time of each 502:
>
> > >>> Jan 23 17:30:45.254989 [WARNING] [pool default] child 18390 exited on
> > >>> signal 11 SIGSEGV after 8.940000 seconds from start
> > >>> Jan 23 17:30:45.254989 [NOTICE] [pool default] child 18402 started
>
> > >>> This doesn't happen on every page load - if I restart php-fpm then
> > >>> everything's fine for 10-20 page refreshes then I only get 502s until
> > >>> I believe php-fpm spots the issue and restarts. Then the pattern
> > >>> repeats.
>
> > >>> I'm a bit stumped - it feels more like a memory issue than a
> > >>> configuration one but the box (256mb Debian slice) shows plenty of
> > >>> free memory at all times. I tried moving child workers from 50 down to
> > >>> 12 but that didn't help either.
>
> > >>> (I know that similar SIGSEGV problems have been reported before but
> > >>> they seem to relate to older codebases and/or configuration issues.)
>
> > >>> Any help would be much appreciated, and do let me know if you want me
> > >>> to paste any configuration files etc.
>
> > >>> Many thanks,
>
> > >>> Alex
>
>
Here's some more from gdb in case it's helpful:

(gdb) bt
#0 fpm_status_increment_accepted_conn (shm=0x223ad90) at /home/admin/
sources/php_5_3_fpm/sapi/fpm/fpm/fpm_status.c:77
#1 0x0000000000716b17 in fcgi_accept_request (req=0x7fff74a5c3e0) at /
home/admin/sources/php_5_3_fpm/sapi/fpm/fpm/fastcgi.c:1024
#2 0x0000000000719e7d in main (argc=0, argv=0x7fff74a5e900) at /home/
admin/sources/php_5_3_fpm/sapi/fpm/fpm/fpm_main.c:1763
(gdb) frame 0
#0 fpm_status_increment_accepted_conn (shm=0x223ad90) at /home/admin/
sources/php_5_3_fpm/sapi/fpm/fpm/fpm_status.c:77
77 status.accepted_conn++;
(gdb) info locals
No locals.
(gdb) frame 1
#1 0x0000000000716b17 in fcgi_accept_request (req=0x7fff74a5c3e0) at /
home/admin/sources/php_5_3_fpm/sapi/fpm/fpm/fastcgi.c:1024
1024
fpm_request_reading_headers();
(gdb) info locals
fds = {fd = 0, events = 0, revents = 0}
ret = <value optimized out>
(gdb) frame 2
#2 0x0000000000719e7d in main (argc=0, argv=0x7fff74a5e900) at /home/
admin/sources/php_5_3_fpm/sapi/fpm/fpm/fpm_main.c:1763
1763 while (fcgi_accept_request(&request) >= 0) {
(gdb) info locals
__bailout = {{__jmpbuf = {0, 4374544114463408116, 4331968,
140735150418264, 2, 0, -4374286089079357452, 4374543706499710964},
__mask_was_saved = 0,
__saved_mask = {__val = {140735150417248, 140735150417640,
139793442966288, 0, 139793474440592, 139793474443664, 139793459194794,
139793443016664,
139793459187896, 4294967296, 4294967449, 0, 139793474452696,
140735150417696, 140735150417640, 2972705047}}}}
exit_status = 0
c = <value optimized out>
file_handle = {type = ZEND_HANDLE_MAPPED, filename = 0x235a940 "",
opened_path = 0x0, handle = {fd = 37154672, fp = 0x236ef70, stream = {
handle = 0x236ef70, isatty = 0, mmap = {len = 353, pos = 0, map
= 0x7f2434518000, buf = 0x7f2434518000 <Address 0x7f2434518000 out of
bounds>,
old_handle = 0x2397b10, old_closer = 0x6a22b0
<zend_stream_stdio_closer>}, reader = 0x6a28d0
<zend_stream_stdio_reader>,
fsizer = 0x6a27f0 <zend_stream_stdio_fsizer>, closer = 0x6a2840
<zend_stream_mmap_closer>}}, free_filename = 0 '\0'}
orig_optind = 1
orig_optarg = 0x0
ini_entries_len = 37320688
max_requests = 500
requests = 1
fcgi_fd = 0
request = {listen_socket = 0, fd = 5, id = 1, keep = 0, closed = 1,
in_len = 0, in_pad = 0, out_hdr = 0x0, out_pos = 0x7fff74a5c410
"\001\006",
out_buf = "\001\006\000\001\000S\005\000X-Powered-By: PHP/5.3.2-dev\r
\nContent-type: text/html\r\n\r\n You're currently in: Spain
\000\000\000\000\000\001\003\000\001\000\b", '\0' <repeats 8089
times>, reserved = '\0' <repeats 15 times>, env = 0x0}
fpm_config = 0x7fff74a5fa86 "/etc/php/php-fpm.conf"
(gdb) frame 0
#0 fpm_status_increment_accepted_conn (shm=0x223ad90) at /home/admin/
sources/php_5_3_fpm/sapi/fpm/fpm/fpm_status.c:77
77 status.accepted_conn++;


On Jan 24, 3:21 am, alexd <alexander.d...@gmail.com> wrote:
> Thanks for everybody's suggestions!
>
> I'm running a pretty straightforward PHP (./configure --enable-fpm --
> with-libevent=/usr/local/lib \
> --enable-mysql --with-mysql=/usr/local/mysql --enable-pdo_mysql), so I
> don't think the problem lies with my extensions. I'm not running any
> accelerators or similar.
>
> The problem is happening with a pretty simple MaxMind script too:
> <?php
> include("/var/www/lib/MaxMind/geoip.inc");
> $gi = geoip_open("/var/www/lib/MaxMind/data/
> GeoIP.dat",GEOIP_STANDARD);
> $country = geoip_country_name_by_addr($gi, "80.24.24.24");
> echo "You're currently in: " . $country;
> geoip_close($gi);
> ?>
>
> If I run this from the command line (php -f geotest.php) then I can
> run it ad infinitum without any errors.
>
> To debug this I tried the following:
>
> 1. Reduced the number of worker processes to 1 (after doing this every
> 2nd reload of the page would be a 502).
>
> 2. Stopped php-fpm then ran the commands:
> sudo -i
> gdb php-fpm
> set args --fpm-config=/etc/php/php-fpm.conf
> run
>
> 3. In another window, ps -ef | grep fpm to find the child worker PID
> and then back in the gdb window:
> attach 19561
> continue
>
> 4. Then I go and reload the page in the browser, get a 502 and then in
> the gdb window I see:
>
> Reading symbols from /lib/libnss_nis.so.2...done.
> Loaded symbols for /lib/libnss_nis.so.2
> 0x00007f24337ada20 in __accept_nocancel () from /lib/libpthread.so.0
> (gdb) continue
> Continuing.
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7f243462a6e0 (LWP 19561)]
> fpm_status_increment_accepted_conn (shm=0x223ad90) at /home/admin/
> sources/php_5_3_fpm/sapi/fpm/fpm/fpm_status.c:77
> 77              status.accepted_conn++;
> (gdb)
>
> I hope this gives some more information!
>
> Thanks,
>
> Alex
>
> On Jan 23, 9:24 pm, Davy Campano <dcamp...@gmail.com> wrote:
>
> > Alex,
>
> > Are you running any other extensions?  Probably the best place to get it
> > figured out though is by debugging it with gdb as Alexey mentioned.  The
> > good part is that it sounds like it is quickly reproducible.
>
> > On Sat, Jan 23, 2010 at 4:21 PM, Michael Shadle <mike...@gmail.com> wrote:
> > > This is true. I was getting confused with SIGCHLD. Some people have
> > > posted on here with very quick recycle rates with SIGCHLDs...
>
> > > On Sat, Jan 23, 2010 at 1:17 PM, Davy Campano <dcamp...@gmail.com> wrote:
> > > >>Recycling in only 9 seconds seems pretty quick.
> > > > It looks like he is receiving a SIGSEGV which would be a segmentation
> > > fault,
> > > > so it wouldn't be related to recycling.  I haven't tried the 5.3.x code,
> > > but
> > > > it appears that it is probably something in your application code that is
> > > > buggy.
>
> > > > On Sat, Jan 23, 2010 at 3:47 PM, Michael Shadle <mike...@gmail.com>
> > > wrote:
>
> > > >> Recycling in only 9 seconds seems pretty quick.
>
> > > >> Sent from my iPhone
>
> > > >> On Jan 23, 2010, at 9:48 AM, alexd <alexander.d...@gmail.com> wrote:
>
> > > >>> Hi,
>
> > > >>> I have successfully installed php-fpm using the version available in
> > > >>> PHP core via SVN. However now I'm getting intermittent 502 bad gateway
> > > >>> problems in Nginx. Taking a look at php-fpm.log shows the following
> > > >>> happening at the time of each 502:
>
> > > >>> Jan 23 17:30:45.254989 [WARNING] [pool default] child 18390 exited on
> > > >>> signal 11 SIGSEGV after 8.940000 seconds from start
> > > >>> Jan 23 17:30:45.254989 [NOTICE] [pool default] child 18402 started
>
> > > >>> This doesn't happen on every page load - if I restart php-fpm then
> > > >>> everything's fine for 10-20 page refreshes then I only get 502s until
> > > >>> I believe php-fpm spots the issue and restarts. Then the pattern
> > > >>> repeats.
>
> > > >>> I'm a bit stumped - it feels more like a memory issue than a
> > > >>> configuration one but the box (256mb Debian slice) shows plenty of
> > > >>> free memory at all times. I tried moving child workers from 50 down to
> > > >>> 12 but that didn't help either.
>
> > > >>> (I know that similar SIGSEGV problems have been reported before but
> > > >>> they seem to relate to older codebases and/or configuration issues.)
>
> > > >>> Any help would be much appreciated, and do let me know if you want me
> > > >>> to paste any configuration files etc.
>
> > > >>> Many thanks,
>
> > > >>> Alex
>
>
On 01/24/2010 06:28 AM, alexd wrote:
> Here's some more from gdb in case it's helpful:

Not really helpful as the compiler has optimized everything.
Please rebuild PHP with --enable-debug (this adds -g3 -O0 to CFLAGS automatically)
and regenerate the backtrace.

Did you mean you can reproduce it with this particular script _only_ or with any script?
It would be also nice to have a short reproduce script which doesn't use any includes etc.

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime statistics for PHP
The same. But if I choose released PHP source + fpm patchset, it works
correctly.
Might be some bugs in the difference between released source and SVN.

2010/1/24 alexd <alexander.dean@gmail.com>:
> Hi,
>
> I have successfully installed php-fpm using the version available in
> PHP core via SVN. However now I'm getting intermittent 502 bad gateway
> problems in Nginx. Taking a look at php-fpm.log shows the following
> happening at the time of each 502:
>
> Jan 23 17:30:45.254989 [WARNING] [pool default] child 18390 exited on
> signal 11 SIGSEGV after 8.940000 seconds from start
> Jan 23 17:30:45.254989 [NOTICE] [pool default] child 18402 started
>
> This doesn't happen on every page load - if I restart php-fpm then
> everything's fine for 10-20 page refreshes then I only get 502s until
> I believe php-fpm spots the issue and restarts. Then the pattern
> repeats.
>
> I'm a bit stumped - it feels more like a memory issue than a
> configuration one but the box (256mb Debian slice) shows plenty of
> free memory at all times. I tried moving child workers from 50 down to
> 12 but that didn't help either.
>
> (I know that similar SIGSEGV problems have been reported before but
> they seem to relate to older codebases and/or configuration issues.)
>
> Any help would be much appreciated, and do let me know if you want me
> to paste any configuration files etc.
>
> Many thanks,
>
> Alex
>



--
Ren Xiaolei
Good to know that I'm not the only one experiencing this! (Antony I'm
having a few 502s on other pages but this is the only predictable/
repeatable one.)

Here is the backtrace with PHP debug enabled:

Program received signal SIGSEGV, Segmentation fault.
0x0000000000890c1f in fpm_status_increment_accepted_conn
(shm=0x2be1170) at /home/admin/sources/php_5_3_fpm/sapi/fpm/fpm/
fpm_status.c:75
75 status = *(struct fpm_status_s *)shm->mem;
(gdb) bt
#0 0x0000000000890c1f in fpm_status_increment_accepted_conn
(shm=0x2be1170) at /home/admin/sources/php_5_3_fpm/sapi/fpm/fpm/
fpm_status.c:75
#1 0x000000000088e71c in fpm_request_reading_headers () at /home/
admin/sources/php_5_3_fpm/sapi/fpm/fpm/fpm_request.c:45
#2 0x0000000000886558 in fcgi_accept_request (req=0x7fff8cb26040) at /
home/admin/sources/php_5_3_fpm/sapi/fpm/fpm/fastcgi.c:1024
#3 0x000000000088cb00 in main (argc=2, argv=0x7fff8cb28298) at /home/
admin/sources/php_5_3_fpm/sapi/fpm/fpm/fpm_main.c:1763
(gdb) frame 0
#0 0x0000000000890c1f in fpm_status_increment_accepted_conn
(shm=0x2be1170) at /home/admin/sources/php_5_3_fpm/sapi/fpm/fpm/
fpm_status.c:75
75 status = *(struct fpm_status_s *)shm->mem;
(gdb) info locals
status = {pm = -1934467744, idle = 32767, active = -481036337, total =
549755813, accepted_conn = 0, last_update = {tv_sec = 140715178422312,
tv_usec = 1172860}}
(gdb) frame 1
#1 0x000000000088e71c in fpm_request_reading_headers () at /home/
admin/sources/php_5_3_fpm/sapi/fpm/fpm/fpm_request.c:45
45 fpm_status_increment_accepted_conn(fpm_status_shm);
(gdb) info locals
slot = (struct fpm_shm_slot_s *) 0x7fface397000
(gdb) frame 2
#2 0x0000000000886558 in fcgi_accept_request (req=0x7fff8cb26040) at /
home/admin/sources/php_5_3_fpm/sapi/fpm/fpm/fastcgi.c:1024
1024
fpm_request_reading_headers();
(gdb) info locals
fds = {fd = -1934467264, events = 32767, revents = 0}
ret = 0

If anyone wants to recreate my issue, I think all the information
needed is above, part from perhaps the include files for my PHP
script:

http://geolite.maxmind.com/download/geoip/api/php/geoip.inc
http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat..gz

I appreciate everyone's efforts to help! I'm going to try one of the
two other installation routes now as I really need to get PHP-FPM
working asap...

Thanks,

Alex

On Jan 26, 10:01 am, 任晓磊 <julycl...@gmail.com> wrote:
> The same. But if I choose released PHP source + fpm patchset, it works
> correctly.
> Might be some bugs in the difference between released source and SVN.
>
> 2010/1/24 alexd <alexander.d...@gmail.com>:
>
>
>
> > Hi,
>
> > I have successfully installed php-fpm using the version available in
> > PHP core via SVN. However now I'm getting intermittent 502 bad gateway
> > problems in Nginx. Taking a look at php-fpm.log shows the following
> > happening at the time of each 502:
>
> > Jan 23 17:30:45.254989 [WARNING] [pool default] child 18390 exited on
> > signal 11 SIGSEGV after 8.940000 seconds from start
> > Jan 23 17:30:45.254989 [NOTICE] [pool default] child 18402 started
>
> > This doesn't happen on every page load - if I restart php-fpm then
> > everything's fine for 10-20 page refreshes then I only get 502s until
> > I believe php-fpm spots the issue and restarts. Then the pattern
> > repeats.
>
> > I'm a bit stumped - it feels more like a memory issue than a
> > configuration one but the box (256mb Debian slice) shows plenty of
> > free memory at all times. I tried moving child workers from 50 down to
> > 12 but that didn't help either.
>
> > (I know that similar SIGSEGV problems have been reported before but
> > they seem to relate to older codebases and/or configuration issues.)
>
> > Any help would be much appreciated, and do let me know if you want me
> > to paste any configuration files etc.
>
> > Many thanks,
>
> > Alex
>
> --
> Ren Xiaolei
On 01/26/2010 06:45 PM, alexd wrote:
> Good to know that I'm not the only one experiencing this! (Antony I'm
> having a few 502s on other pages but this is the only predictable/
> repeatable one.)

Try with latest SVN checkout, I can see that Jerome has fixed something similar quite recently.

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime statistics for PHP
Thanks Antony,

I tried the new version from SVN but I just got a different type of
error (No input file specified rather than 502s).

I've gone back to using spawn-fcgi - took 5 mins to setup and is rock
solid, no 502s in sight. I'll check back in on PHP-FPM in a few
months, see where it's got to in stability terms.

Thanks for everyone's help,

Alex

On Jan 27, 10:12 am, Antony Dovgal <t...@daylessday.org> wrote:
> On 01/26/2010 06:45 PM, alexd wrote:
>
> > Good to know that I'm not the only one experiencing this! (Antony I'm
> > having a few 502s on other pages but this is the only predictable/
> > repeatable one.)
>
> Try with latest SVN checkout, I can see that Jerome has fixed something similar quite recently.
>
> --
> Wbr,
> Antony Dovgal
> ---http://pinba.org- realtime statistics for PHP
Re: Intermittent problems with child workers exiting with SIGSEGV
January 27, 2010 01:56PM
do you need 5.3?

5.2 is rock solid with fpm, for sure.

On Wed, Jan 27, 2010 at 9:29 AM, alexd <alexander.dean@gmail.com> wrote:
> Thanks Antony,
>
> I tried the new version from SVN but I just got a different type of
> error (No input file specified rather than 502s).
>
> I've gone back to using spawn-fcgi - took 5 mins to setup and is rock
> solid, no 502s in sight. I'll check back in on PHP-FPM in a few
> months, see where it's got to in stability terms.
>
> Thanks for everyone's help,
>
> Alex
>
> On Jan 27, 10:12 am, Antony Dovgal <t...@daylessday.org> wrote:
>> On 01/26/2010 06:45 PM, alexd wrote:
>>
>> > Good to know that I'm not the only one experiencing this! (Antony I'm
>> > having a few 502s on other pages but this is the only predictable/
>> > repeatable one.)
>>
>> Try with latest SVN checkout, I can see that Jerome has fixed something similar quite recently.
>>
>> --
>> Wbr,
>> Antony Dovgal
>> ---http://pinba.org- realtime statistics for PHP
>
which revision of SVN did you use ?


tony did just correct a (stupid) bug on the status module I added recently.
You should try this.

2010/1/27 alexd <alexander.dean@gmail.com>

> Thanks Antony,
>
> I tried the new version from SVN but I just got a different type of
> error (No input file specified rather than 502s).
>
> I've gone back to using spawn-fcgi - took 5 mins to setup and is rock
> solid, no 502s in sight. I'll check back in on PHP-FPM in a few
> months, see where it's got to in stability terms.
>
> Thanks for everyone's help,
>
> Alex
>
> On Jan 27, 10:12 am, Antony Dovgal <t...@daylessday.org> wrote:
> > On 01/26/2010 06:45 PM, alexd wrote:
> >
> > > Good to know that I'm not the only one experiencing this! (Antony I'm
> > > having a few 502s on other pages but this is the only predictable/
> > > repeatable one.)
> >
> > Try with latest SVN checkout, I can see that Jerome has fixed something
> similar quite recently.
> >
> > --
> > Wbr,
> > Antony Dovgal
> > ---http://pinba.org- realtime statistics for PHP
>
Re: Intermittent problems with child workers exiting with SIGSEGV
January 28, 2010 09:30AM
What does this new status module do? Is there any documentation somewhere
for it?

Also, Alex, the "no input file specified" occurs in Nginx when Nginx cannot
find the php page that it is trying to find, usually due to this variable
not being set or pointing to the wrong path:

fastcgi_param SCRIPT_FILENAME /var/www/html$fastcgi_script_name;

Davy

2010/1/28 Jérôme Loyet <ml@fatbsd.com>

> which revision of SVN did you use ?
>
>
> tony did just correct a (stupid) bug on the status module I added recently.
> You should try this.
>
> 2010/1/27 alexd <alexander.dean@gmail.com>
>
> Thanks Antony,
>>
>> I tried the new version from SVN but I just got a different type of
>> error (No input file specified rather than 502s).
>>
>> I've gone back to using spawn-fcgi - took 5 mins to setup and is rock
>> solid, no 502s in sight. I'll check back in on PHP-FPM in a few
>> months, see where it's got to in stability terms.
>>
>> Thanks for everyone's help,
>>
>> Alex
>>
>> On Jan 27, 10:12 am, Antony Dovgal <t...@daylessday.org> wrote:
>> > On 01/26/2010 06:45 PM, alexd wrote:
>> >
>> > > Good to know that I'm not the only one experiencing this! (Antony I'm
>> > > having a few 502s on other pages but this is the only predictable/
>> > > repeatable one.)
>> >
>> > Try with latest SVN checkout, I can see that Jerome has fixed something
>> similar quite recently.
>> >
>> > --
>> > Wbr,
>> > Antony Dovgal
>> > ---http://pinba.org- realtime statistics for PHP
>>
>
>
2010/1/28 Davy Campano <dcampano@gmail.com>
>
> What does this new status module do?  Is there any documentation somewhere for it?

Check the conf file
(http://svn.php.net/viewvc/php/php-src/branches/PHP_5_3_FPM/sapi/fpm/php-fpm.conf.in?revision=292487&view=markup)
and search for status.

> Also, Alex, the "no input file specified" occurs in Nginx when Nginx cannot find the php page that it is trying to find, usually due to this variable not being set or pointing to the wrong path:
> fastcgi_param  SCRIPT_FILENAME  /var/www/html$fastcgi_script_name;
> Davy
>
> 2010/1/28 Jérôme Loyet <ml@fatbsd.com>
>>
>> which revision of SVN did you use ?
>>
>> tony did just correct a (stupid) bug on the status module I added recently. You should try this.
>>
>> 2010/1/27 alexd <alexander.dean@gmail.com>
>>>
>>> Thanks Antony,
>>>
>>> I tried the new version from SVN but I just got a different type of
>>> error (No input file specified rather than 502s).
>>>
>>> I've gone back to using spawn-fcgi - took 5 mins to setup and is rock
>>> solid, no 502s in sight. I'll check back in on PHP-FPM in a few
>>> months, see where it's got to in stability terms.
>>>
>>> Thanks for everyone's help,
>>>
>>> Alex
>>>
>>> On Jan 27, 10:12 am, Antony Dovgal <t...@daylessday.org> wrote:
>>> > On 01/26/2010 06:45 PM, alexd wrote:
>>> >
>>> > > Good to know that I'm not the only one experiencing this! (Antony I'm
>>> > > having a few 502s on other pages but this is the only predictable/
>>> > > repeatable one.)
>>> >
>>> > Try with latest SVN checkout, I can see that Jerome has fixed something similar quite recently.
>>> >
>>> > --
>>> > Wbr,
>>> > Antony Dovgal
>>> > ---http://pinba.org- realtime statistics for PHP
>>
>
Hello

I am useing the svn version:
svn co http://svn.php.net/repository/php/php-src/branches/PHP_5_3_FPM
php_5_3_fpm
Checked out revision 294158.


I am getting those errors:

Jan 28 19:27:03.862673 [DEBUG] pid 22254, fpm_got_signal(), line 48:
received SIGCHLD
Jan 28 19:27:03.862873 [WARNING] pid 22254, fpm_children_bury(), line
249: [pool testPool] child 22276 exited on signal 11 SIGSEGV after
63.943302 seconds from start
Jan 28 19:27:03.867909 [NOTICE] pid 22254, fpm_children_make(), line
403: [pool testPool] child 22365 started
Jan 28 19:27:35.022235 [DEBUG] pid 22254, fpm_got_signal(), line 48:
received SIGCHLD
Jan 28 19:27:35.022277 [WARNING] pid 22254, fpm_children_bury(), line
249: [pool testPool] child 22279 exited on signal 11 SIGSEGV after
95.099939 seconds from start
Jan 28 19:27:35.024085 [NOTICE] pid 22254, fpm_children_make(), line
403: [pool testPool] child 22391 started
Jan 28 19:27:36.345317 [DEBUG] pid 22254, fpm_got_signal(), line 48:
received SIGCHLD
Jan 28 19:27:36.345355 [WARNING] pid 22254, fpm_children_bury(), line
249: [pool testPool] child 22280 exited on signal 11 SIGSEGV after
96.422132 seconds from start
Jan 28 19:27:36.346552 [NOTICE] pid 22254, fpm_children_make(), line
403: [pool testPool] child 22396 started
Jan 28 19:27:36.501724 [WARNING] pid 22254, fpm_stdio_child_said(),
line 154: [pool testPool] child 22281 said into stderr: "zend_mm_heap
corrupted"
Jan 28 19:27:36.501789 [DEBUG] pid 22254, fpm_got_signal(), line 48:
received SIGCHLD
Jan 28 19:27:36.501813 [WARNING] pid 22254, fpm_children_bury(), line
249: [pool testPool] child 22281 exited with code 1 after 96.577647
seconds from start
Jan 28 19:27:36.503316 [NOTICE] pid 22254, fpm_children_make(), line
403: [pool testPool] child 22397 started
Jan 28 19:27:39.235049 [DEBUG] pid 22254, fpm_got_signal(), line 48:
received SIGCHLD
Jan 28 19:27:39.235147 [WARNING] pid 22254, fpm_children_bury(), line
249: [pool testPool] child 22282 exited on signal 11 SIGSEGV after
99.310080 seconds from start
Jan 28 19:27:39.239045 [NOTICE] pid 22254, fpm_children_make(), line
403: [pool testPool] child 22398 started



On 28 Jan., 16:52, Jérôme Loyet <m...@fatbsd.com> wrote:
> 2010/1/28 Davy Campano <dcamp...@gmail.com>
>
>
>
> > What does this new status module do?  Is there any documentation somewhere for it?
>
> Check the conf file
> (http://svn.php.net/viewvc/php/php-src/branches/PHP_5_3_FPM/sapi/fpm/p...)
> and search for status.
>
>
>
> > Also, Alex, the "no input file specified" occurs in Nginx when Nginx cannot find the php page that it is trying to find, usually due to this variable not being set or pointing to the wrong path:
> > fastcgi_param  SCRIPT_FILENAME  /var/www/html$fastcgi_script_name;
> > Davy
>
> > 2010/1/28 Jérôme Loyet <m...@fatbsd.com>
>
> >> which revision of SVN did you use ?
>
> >> tony did just correct a (stupid) bug on the status module I added recently. You should try this.
>
> >> 2010/1/27 alexd <alexander.d...@gmail.com>
>
> >>> Thanks Antony,
>
> >>> I tried the new version from SVN but I just got a different type of
> >>> error (No input file specified rather than 502s).
>
> >>> I've gone back to using spawn-fcgi - took 5 mins to setup and is rock
> >>> solid, no 502s in sight. I'll check back in on PHP-FPM in a few
> >>> months, see where it's got to in stability terms.
>
> >>> Thanks for everyone's help,
>
> >>> Alex
>
> >>> On Jan 27, 10:12 am, Antony Dovgal <t...@daylessday.org> wrote:
> >>> > On 01/26/2010 06:45 PM, alexd wrote:
>
> >>> > > Good to know that I'm not the only one experiencing this! (Antony I'm
> >>> > > having a few 502s on other pages but this is the only predictable/
> >>> > > repeatable one.)
>
> >>> > Try with latest SVN checkout, I can see that Jerome has fixed something similar quite recently.
>
> >>> > --
> >>> > Wbr,
> >>> > Antony Dovgal
> >>> > ---http://pinba.org-realtime statistics for PHP
ca you send to me the source of the PHP file which is causing this ?
Can you try compile fpm in debug mode and enable coredumps ?

Thanks
++ Jerome

2010/1/28 fo3-nik5 <fo3nik5@gmail.com>:
> Hello
>
> I am useing the svn version:
> svn co http://svn.php.net/repository/php/php-src/branches/PHP_5_3_FPM
> php_5_3_fpm
> Checked out revision 294158.
>
>
> I am getting those errors:
>
> Jan 28 19:27:03.862673 [DEBUG] pid 22254, fpm_got_signal(), line 48:
> received SIGCHLD
> Jan 28 19:27:03.862873 [WARNING] pid 22254, fpm_children_bury(), line
> 249: [pool testPool] child 22276 exited on signal 11 SIGSEGV after
> 63.943302 seconds from start
> Jan 28 19:27:03.867909 [NOTICE] pid 22254, fpm_children_make(), line
> 403: [pool testPool] child 22365 started
> Jan 28 19:27:35.022235 [DEBUG] pid 22254, fpm_got_signal(), line 48:
> received SIGCHLD
> Jan 28 19:27:35.022277 [WARNING] pid 22254, fpm_children_bury(), line
> 249: [pool testPool] child 22279 exited on signal 11 SIGSEGV after
> 95.099939 seconds from start
> Jan 28 19:27:35.024085 [NOTICE] pid 22254, fpm_children_make(), line
> 403: [pool testPool] child 22391 started
> Jan 28 19:27:36.345317 [DEBUG] pid 22254, fpm_got_signal(), line 48:
> received SIGCHLD
> Jan 28 19:27:36.345355 [WARNING] pid 22254, fpm_children_bury(), line
> 249: [pool testPool] child 22280 exited on signal 11 SIGSEGV after
> 96.422132 seconds from start
> Jan 28 19:27:36.346552 [NOTICE] pid 22254, fpm_children_make(), line
> 403: [pool testPool] child 22396 started
> Jan 28 19:27:36.501724 [WARNING] pid 22254, fpm_stdio_child_said(),
> line 154: [pool testPool] child 22281 said into stderr: "zend_mm_heap
> corrupted"
> Jan 28 19:27:36.501789 [DEBUG] pid 22254, fpm_got_signal(), line 48:
> received SIGCHLD
> Jan 28 19:27:36.501813 [WARNING] pid 22254, fpm_children_bury(), line
> 249: [pool testPool] child 22281 exited with code 1 after 96.577647
> seconds from start
> Jan 28 19:27:36.503316 [NOTICE] pid 22254, fpm_children_make(), line
> 403: [pool testPool] child 22397 started
> Jan 28 19:27:39.235049 [DEBUG] pid 22254, fpm_got_signal(), line 48:
> received SIGCHLD
> Jan 28 19:27:39.235147 [WARNING] pid 22254, fpm_children_bury(), line
> 249: [pool testPool] child 22282 exited on signal 11 SIGSEGV after
> 99.310080 seconds from start
> Jan 28 19:27:39.239045 [NOTICE] pid 22254, fpm_children_make(), line
> 403: [pool testPool] child 22398 started
>
>
>
> On 28 Jan., 16:52, Jérôme Loyet <m...@fatbsd.com> wrote:
>> 2010/1/28 Davy Campano <dcamp...@gmail.com>
>>
>>
>>
>> > What does this new status module do?  Is there any documentation somewhere for it?
>>
>> Check the conf file
>> (http://svn.php.net/viewvc/php/php-src/branches/PHP_5_3_FPM/sapi/fpm/p....)
>> and search for status.
>>
>>
>>
>> > Also, Alex, the "no input file specified" occurs in Nginx when Nginx cannot find the php page that it is trying to find, usually due to this variable not being set or pointing to the wrong path:
>> > fastcgi_param  SCRIPT_FILENAME  /var/www/html$fastcgi_script_name;
>> > Davy
>>
>> > 2010/1/28 Jérôme Loyet <m...@fatbsd.com>
>>
>> >> which revision of SVN did you use ?
>>
>> >> tony did just correct a (stupid) bug on the status module I added recently. You should try this.
>>
>> >> 2010/1/27 alexd <alexander.d...@gmail.com>
>>
>> >>> Thanks Antony,
>>
>> >>> I tried the new version from SVN but I just got a different type of
>> >>> error (No input file specified rather than 502s).
>>
>> >>> I've gone back to using spawn-fcgi - took 5 mins to setup and is rock
>> >>> solid, no 502s in sight. I'll check back in on PHP-FPM in a few
>> >>> months, see where it's got to in stability terms.
>>
>> >>> Thanks for everyone's help,
>>
>> >>> Alex
>>
>> >>> On Jan 27, 10:12 am, Antony Dovgal <t...@daylessday.org> wrote:
>> >>> > On 01/26/2010 06:45 PM, alexd wrote:
>>
>> >>> > > Good to know that I'm not the only one experiencing this! (Antony I'm
>> >>> > > having a few 502s on other pages but this is the only predictable/
>> >>> > > repeatable one.)
>>
>> >>> > Try with latest SVN checkout, I can see that Jerome has fixed something similar quite recently.
>>
>> >>> > --
>> >>> > Wbr,
>> >>> > Antony Dovgal
>> >>> > ---http://pinba.org-realtime statistics for PHP
>
On 01/28/2010 09:31 PM, fo3-nik5 wrote:
> Hello
>
> I am useing the svn version:
> svn co http://svn.php.net/repository/php/php-src/branches/PHP_5_3_FPM
> php_5_3_fpm
> Checked out revision 294158.

svn up

It was my fault, as usual =|

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime statistics for PHP
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 141
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready