Thanks Richard & itpp2015 for your response.

Further update :

There are 2 cases :

1. 504 @ 120 seconds coming with below mentioned error :

2016/01/05 03:50:54 [error] 1070#0: *201650845 upstream timed out (110: Connection timed out) while connecting to upstream, client: 66.249.74.99, server: x.x.x.x, request: "GET /some/url HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "example.com"

2. 504 @ 300 seconds coming with below mentioned error :

2016/01/05 00:51:43 [error] 1067#0: *200656359 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 115.112.161.9, server: 192.168.12.101, request: "GET /some/url HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "example.com"

* No errors found in php-fpm logs.
* Number of php-fpm processes were also normal. Backend doesn't look overloaded as other requests were served out fine at the same time.

* Only one php-fpm pool is being used. One php-fpm master (parent) process and other slave (child) processes are usually at normal range only when 5xx are observed. There is no significant growth in number of php-fpm processes and even if grows then server has enough capacity to fork new ones and serve the request.