Welcome! Log In Create A New Profile

Advanced

Debugging 504 Gateway Timeout and its actual cause and solution

January 04, 2016 08:34AM
Hi,

We are running following stack on our web server Varnish + Nginx + FastCGI (php-fpm) on RHEL 6.6

Its a dynamic website with different result sets everytime and has around 2 million url's indexed with Google.
* Its running on nginx/1.5.12 and PHP 5.3.3 (Will be upgraded to latest nginx and PHP soon)
* Nginx connects to php-fpm running locally on same server on port 9000

We are getting 504 Gateway timeout intermittently on some pages which we are unable to resolve. The URL's which give 504 works fine after sometime.
We get to know about 504 from our logs and we haven't been able to replicate this as it randomly happens on any URL and works after sometime.

I have had couple of discussions with developer but as per him the underlying php script hardly does anything and it should not take this long (120 seconds) but still it is giving 504 Gateway timeout.

Need to establish where exactly the issue occurs :
* Is it a problem with Nginx ?
* Is it a problem with php-fpm ?
* Is it a problem with underlying php scripts ?
* Is it possible that nginx is not able to connect to php-fpm ?
* Would it resolve if we use Unix socket instead of TCP/IP connection to ?

The URL times out after 120 seconds with 504

Below is the error seen :
2016/01/04 17:29:20 [error] 1070#0: *196333149 upstream timed out (110: Connection timed out) while connecting to upstream, client: 66.249.74.95, server: x.x.x.x, request: "GET /Some/url HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "example.com"


Earlier with fastcgi_connect_timeout of 150 seconds - it used to give at 502 status code after 63 seconds with default net.ipv4.tcp_syn_retries = 5 on RHEL 6.6 ; afterwards we set net.ipv4.tcp_syn_retries = 6 and then it started giving 502 after 127 seconds.

Once I set fastcgi_connect_timeout = 120 it started giving 504 status code. I understand fastcgi_connect_timeout with such high value is not good.

Need to findout why exactly we are getting 504 (I know its timeout but the cause is unknown). Need to get to the root cause to fix it permanently.

How do I confirm where exactly the issue is ?

If its poorly written code then I need to inform the developer that 504 is happening due to issue in php code and not due to nginx or php-fpm and if its due to Nginx or Php-fpm then need to fix that.



Thanks in Advance!
Subject Author Posted

Debugging 504 Gateway Timeout and its actual cause and solution

Keyur January 04, 2016 08:34AM

Re: Debugging 504 Gateway Timeout and its actual cause and solution

Keyur January 04, 2016 08:53AM

Re: Debugging 504 Gateway Timeout and its actual cause and solution

Richard Stanway January 04, 2016 09:12AM

Re: Debugging 504 Gateway Timeout and its actual cause and solution

Keyur January 06, 2016 02:56AM

Re: Debugging 504 Gateway Timeout and its actual cause and solution

Maxim Dounin January 08, 2016 11:36AM

Re: Debugging 504 Gateway Timeout and its actual cause and solution

Keyur January 11, 2016 12:51AM

Re: Debugging 504 Gateway Timeout and its actual cause and solution

itpp2012 January 04, 2016 09:34AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 66
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready