Welcome! Log In Create A New Profile

Advanced

Nginx + Php-fpm = Intermittent 502 Bad Gateway errors

Posted by empire29 
Nginx + Php-fpm = Intermittent 502 Bad Gateway errors
April 02, 2010 10:23AM
I am running Nginx and proxying php requests via FastCGI to PHP-FPM for processing. I will randomly receive 502 Bad Gateway error pages - I can reproduce this issue by clicking around my PHP websites very rapidly/refreshing a page for a minute or two. When I get the 502 error page all I have to do is refresh the browser and the page refreshes properly.

Here is my setup:

[code]
nginx/0.7.64
PHP 5.3.2 (fpm-fcgi) (built: Apr 1 2010 06:42:04)
Ubuntu 9.10 (Latest 2.6 Paravirt)
[/code]


I compiled PHP-FPM using this ./configure directive


[code]
./configure --enable-fpm --sysconfdir=/etc/php5/conf.d --with-config-file-path=/etc/php5/conf.d/php.ini --with-zlib --with-openssl --enable-zip --enable-exif --enable-ftp --enable-mbstring --enable-mbregex --enable-soap --enable-sockets --disable-cgi --with-curl --with-curlwrappers --with-gd --with-mcrypt --enable-memcache --with-mhash --with-jpeg-dir=/usr/local/lib --with-mysql=/usr/bin/mysql --with-mysqli=/usr/bin/mysql_config --enable-pdo --with-pdo-mysql=/usr/bin/mysql --with-pdo-sqlite --with-pspell --with-snmp --with-sqlite --with-tidy --with-xmlrpc --with-xsl
[/code]


My php-fpm.conf looks like this (the relevant parts):

[code]
...
<value name="pm">
<value name="max_children">3</value>
...
<value name="request_terminate_timeout">60s</value>
<value name="request_slowlog_timeout">30s</value>
<value name="slowlog">/var/log/php-fpm.log.slow</value>
<value name="rlimit_files">1024</value>
<value name="rlimit_core">0</value>
<value name="chroot"></value>
<value name="chdir"></value>
<value name="catch_workers_output">yes</value>
<value name="max_requests">500</value>
...
[/code]

I've tried increasing the max_children to 10 and it makes no difference. I've also tried setting it to 'dynamic' and setting max_children to 50, and start_server to '5' without any difference.

I have tried using both 1 and 5 nginx worker processes.

My fastcgi_params conf looks like:

[code]
fastcgi_connect_timeout 60;
fastcgi_send_timeout 180;
fastcgi_read_timeout 180;
fastcgi_buffer_size 128k;
fastcgi_buffers 4 256k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
fastcgi_intercept_errors on;

fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;

fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SERVER_PROTOCOL $server_protocol;

fastcgi_param GATEWAY_INTERFACE CGI/1.1;
fastcgi_param SERVER_SOFTWARE nginx/$nginx_version;

fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
fastcgi_param REDIRECT_STATUS 200;
[/code]

Nginx logs the error as:

[code]
[error] 3947#0: *10530 connect() failed (111: Connection refused) while connecting to upstream, client: 68.40.xxx.xxx, server: www.domain.com, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "www.domain.com"
[/code]


PHP-FPM logs the follow at the time of the error:

[code]
[NOTICE] pid 17161, fpm_unix_init_main(), line 255: getrlimit(nofile): max:1024, cur:1024
[NOTICE] pid 17161, fpm_event_init_main(), line 93: libevent: using epoll
[NOTICE] pid 17161, fpm_init(), line 50: fpm is running, pid 17161
[DEBUG] pid 17161, fpm_children_make(), line 403: [pool default] child 17162 started
[DEBUG] pid 17161, fpm_children_make(), line 403: [pool default] child 17163 started
[DEBUG] pid 17161, fpm_children_make(), line 403: [pool default] child 17164 started
[NOTICE] pid 17161, fpm_event_loop(), line 111: ready to handle connections
[/code]

My CPU usage maxes out around 10-15% when I recreate the issue. My Free mem (free -m) is > 130MB

I had this intermittent 502 Bad Gateway issue when in was using php5-cgi to service my php requests as well. Does anyone know how to fix this?

get, get, get! shouldget.com
Gordon Pettey
Re: Nginx + Php-fpm = Intermittent 502 Bad Gateway errors
April 02, 2010 12:14PM
I had this issue a while back. The solution was screwing around with
fastcgi_buffers and fastcgi_buffer_size. I ended up with
fastcgi_buffers 256 16k;
fastcgi_buffer_size 32k;

On Fri, Apr 2, 2010 at 8:23 AM, empire29 <nginx-forum@nginx.us> wrote:
> I am running Nginx and proxying php requests via FastCGI to PHP-FPM for processing. I will randomly receive 502 Bad Gateway error pages - I can reproduce this issue by clicking around my PHP websites very rapidly/refreshing a page for a minute or two. When I get the 502 error page all I have to do is refresh the browser and the page refreshes properly.
>
> Here is my setup:
>
>
> nginx/0.7.64
> PHP 5.3.2 (fpm-fcgi) (built: Apr 1 2010 06:42:04)
> Ubuntu 9.10 (Latest 2.6 Paravirt)
>
>
>
> I compiled PHP-FPM using this ./configure directive
>
>
>
> ./configure --enable-fpm --sysconfdir=/etc/php5/conf.d --with-config-file-path=/etc/php5/conf.d/php.ini --with-zlib --with-openssl --enable-zip --enable-exif --enable-ftp --enable-mbstring --enable-mbregex --enable-soap --enable-sockets --disable-cgi --with-curl --with-curlwrappers --with-gd --with-mcrypt --enable-memcache --with-mhash --with-jpeg-dir=/usr/local/lib --with-mysql=/usr/bin/mysql --with-mysqli=/usr/bin/mysql_config --enable-pdo --with-pdo-mysql=/usr/bin/mysql --with-pdo-sqlite --with-pspell --with-snmp --with-sqlite --with-tidy --with-xmlrpc --with-xsl
>
>
>
> My php-fpm.conf looks like this (the relevant parts):
>
>
> ...
>
>     3
>     ...
>     60s
>       30s
>       /var/log/php-fpm.log.slow
>       1024
>       0
>
>
>       yes
>       500
> ...
>
>
> I've tried increasing the max_children to 10 and it makes no difference. I've also tried setting it to 'dynamic' and setting max_children to 50, and start_server to '5' without any difference.
>
> I have tried using both 1 and 5 nginx worker processes.
>
> My fastcgi_params conf looks like:
>
>
> fastcgi_connect_timeout 60;
> fastcgi_send_timeout 180;
> fastcgi_read_timeout 180;
> fastcgi_buffer_size 128k;
> fastcgi_buffers 4 256k;
> fastcgi_busy_buffers_size 256k;
> fastcgi_temp_file_write_size 256k;
> fastcgi_intercept_errors on;
>
> fastcgi_param  QUERY_STRING       $query_string;
> fastcgi_param  REQUEST_METHOD     $request_method;
> fastcgi_param  CONTENT_TYPE       $content_type;
> fastcgi_param  CONTENT_LENGTH     $content_length;
>
> fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
> fastcgi_param  REQUEST_URI        $request_uri;
> fastcgi_param  DOCUMENT_URI       $document_uri;
> fastcgi_param  DOCUMENT_ROOT      $document_root;
> fastcgi_param  SERVER_PROTOCOL    $server_protocol;
>
> fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
> fastcgi_param  SERVER_SOFTWARE    nginx/$nginx_version;
>
> fastcgi_param  REMOTE_ADDR        $remote_addr;
> fastcgi_param  REMOTE_PORT        $remote_port;
> fastcgi_param  SERVER_ADDR        $server_addr;
> fastcgi_param  SERVER_PORT        $server_port;
> fastcgi_param  SERVER_NAME        $server_name;
> fastcgi_param  REDIRECT_STATUS    200;
>
>
> Nginx logs the error as:
>
>
>  3947#0: *10530 connect() failed (111: Connection refused) while connecting to upstream, client: 68.40.xxx.xxx, server: www.domain.com, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "www.domain.com"
>
>
>
> PHP-FPM logs the follow at the time of the error:
>
>
>  pid 17161, fpm_unix_init_main(), line 255: getrlimit(nofile): max:1024, cur:1024
>  pid 17161, fpm_event_init_main(), line 93: libevent: using epoll
>  pid 17161, fpm_init(), line 50: fpm is    running, pid 17161
>  pid 17161, fpm_children_make(), line 403:  child 17162 started
>  pid 17161, fpm_children_make(), line   403:  child 17163    started
>  pid 17161,  fpm_children_make(), line 403:  child 17164 started
>  pid 17161, fpm_event_loop(), line 111: ready to handle connections
>
>
> My CPU usage maxes out around 10-15% when I recreate the issue. My Free mem (free -m) is > 130MB
>
> I had this intermittent 502 Bad Gateway issue when in was using php5-cgi to service my php requests as well. Does anyone know how to fix this?
>
> Posted at Nginx Forum: http://forum.nginx.org/read.php?3,70566,70566#msg-70566
>
>


--
To unsubscribe, reply using "remove me" as the subject.
Re: Nginx + Php-fpm = Intermittent 502 Bad Gateway errors
April 06, 2010 10:50AM
I am also getting this, but decided to go back to 5.2.10, from Ubuntu
Karmic. Running into other errors now.

On Apr 2, 12:13 pm, Gordon Pettey <petteyg...@gmail.com> wrote:
> I had this issue a while back. The solution was screwing around with
> fastcgi_buffers and fastcgi_buffer_size. I ended up with
>         fastcgi_buffers                 256 16k;
>         fastcgi_buffer_size             32k;
>
> On Fri, Apr 2, 2010 at 8:23 AM, empire29 <nginx-fo...@nginx.us> wrote:
> > I am running Nginx and proxying php requests via FastCGI to PHP-FPM for processing. I will randomly receive 502 Bad Gateway error pages - I can reproduce this issue by clicking around my PHP websites very rapidly/refreshing a page for a minute or two. When I get the 502 error page all I have to do is refresh the browser and the page refreshes properly.
>
> > Here is my setup:
>
> > nginx/0.7.64
> > PHP 5.3.2 (fpm-fcgi) (built: Apr 1 2010 06:42:04)
> > Ubuntu 9.10 (Latest 2.6 Paravirt)
>
> > I compiled PHP-FPM using this ./configure directive
>
> > ./configure --enable-fpm --sysconfdir=/etc/php5/conf.d --with-config-file-path=/etc/php5/conf.d/php.ini --with-zlib --with-openssl --enable-zip --enable-exif --enable-ftp --enable-mbstring --enable-mbregex --enable-soap --enable-sockets --disable-cgi --with-curl --with-curlwrappers --with-gd --with-mcrypt --enable-memcache --with-mhash --with-jpeg-dir=/usr/local/lib --with-mysql=/usr/bin/mysql --with-mysqli=/usr/bin/mysql_config --enable-pdo --with-pdo-mysql=/usr/bin/mysql --with-pdo-sqlite --with-pspell --with-snmp --with-sqlite --with-tidy --with-xmlrpc --with-xsl
>
> > My php-fpm.conf looks like this (the relevant parts):
>
> > ...
>
> >     3
> >     ...
> >     60s
> >       30s
> >       /var/log/php-fpm.log.slow
> >       1024
> >       0
>
> >       yes
> >       500
> > ...
>
> > I've tried increasing the max_children to 10 and it makes no difference.. I've also tried setting it to 'dynamic' and setting max_children to 50, and start_server to '5' without any difference.
>
> > I have tried using both 1 and 5 nginx worker processes.
>
> > My fastcgi_params conf looks like:
>
> > fastcgi_connect_timeout 60;
> > fastcgi_send_timeout 180;
> > fastcgi_read_timeout 180;
> > fastcgi_buffer_size 128k;
> > fastcgi_buffers 4 256k;
> > fastcgi_busy_buffers_size 256k;
> > fastcgi_temp_file_write_size 256k;
> > fastcgi_intercept_errors on;
>
> > fastcgi_param  QUERY_STRING       $query_string;
> > fastcgi_param  REQUEST_METHOD     $request_method;
> > fastcgi_param  CONTENT_TYPE       $content_type;
> > fastcgi_param  CONTENT_LENGTH     $content_length;
>
> > fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
> > fastcgi_param  REQUEST_URI        $request_uri;
> > fastcgi_param  DOCUMENT_URI       $document_uri;
> > fastcgi_param  DOCUMENT_ROOT      $document_root;
> > fastcgi_param  SERVER_PROTOCOL    $server_protocol;
>
> > fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
> > fastcgi_param  SERVER_SOFTWARE    nginx/$nginx_version;
>
> > fastcgi_param  REMOTE_ADDR        $remote_addr;
> > fastcgi_param  REMOTE_PORT        $remote_port;
> > fastcgi_param  SERVER_ADDR        $server_addr;
> > fastcgi_param  SERVER_PORT        $server_port;
> > fastcgi_param  SERVER_NAME        $server_name;
> > fastcgi_param  REDIRECT_STATUS    200;
>
> > Nginx logs the error as:
>
> >  3947#0: *10530 connect() failed (111: Connection refused) while connecting to upstream, client: 68.40.xxx.xxx, server:www.domain.com, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "www.domain.com"
>
> > PHP-FPM logs the follow at the time of the error:
>
> >  pid 17161, fpm_unix_init_main(), line 255: getrlimit(nofile): max:1024, cur:1024
> >  pid 17161, fpm_event_init_main(), line 93: libevent: using epoll
> >  pid 17161, fpm_init(), line 50: fpm is    running, pid 17161
> >  pid 17161, fpm_children_make(), line 403:  child 17162 started
> >  pid 17161, fpm_children_make(), line   403:  child 17163    started
> >  pid 17161,  fpm_children_make(), line 403:  child 17164 started
> >  pid 17161, fpm_event_loop(), line 111: ready to handle connections
>
> > My CPU usage maxes out around 10-15% when I recreate the issue. My Free mem (free -m) is > 130MB
>
> > I had this intermittent 502 Bad Gateway issue when in was using php5-cgi to service my php requests as well. Does anyone know how to fix this?
>
> > Posted at Nginx Forum:http://forum.nginx.org/read.php?3,70566,70566#msg-70566
>
>


--
To unsubscribe, reply using "remove me" as the subject.
Re: Nginx + Php-fpm = Intermittent 502 Bad Gateway errors
April 06, 2010 10:55AM
FYI - I neglected to mention that I was originally runinng php-fpm under supervisor - because of this, i had to turn deamonized=off in the php-fpm config.

I ended up removing php-fpm from supervisor and running it normally via an /etc/init.d script at startup and everything seems to be working well now. I think im going to change over to using a unix socket instead of a ip:port setup though.

get, get, get! shouldget.com
On Sat, Apr 3, 2010 at 12:13 AM, Gordon Pettey <petteyg359@gmail.com> wrote:
> I had this issue a while back. The solution was screwing around with
> fastcgi_buffers and fastcgi_buffer_size. I ended up with
>        fastcgi_buffers                 256 16k;
>        fastcgi_buffer_size             32k;

Increasing buffer size is one of works to do.
You might try to increase fastcgi_read_timeout , optimize your
database queries, etc, in order to decrease the time costed by waiting
of fastcgi process.

The best situation:
nginx reads request from client to a big buffer, then connect to a
free fastcgi server process, sent the whole buffer to it. Then,
fastcgi process queries the database in a very short time, and send
the result to a buffer of nginx, then go on sleeping. Actually, the
time of a cycle of fastcgi working should be as short as possible.

--
Ren Xiaolei


--
To unsubscribe, reply using "remove me" as the subject.
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 138
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready