on normal time ,php-fpm has 30+ process. all request finished in 10
seconds. (i execute commands "netstat -nap|wc -l" ,it show 23000,
and "netstat -nap|grep 9000|wc -l" shows 7600)
on busy time ,php-fpm has 40+ process(with a lot of concurrent request) ,
lots of requests get 499 response code( i know because i set php-fpm to
terminate the request executing over 10 seconds) ,according nginx acess
log, and almost all of them dosen't have a $upstream_response_time, but
only $request_time. the server has a load average of 5-9,cpu usage at
30%, no %system busy ,no $io busy. execute "netstat -nap|wc -l" ,it
show 43000, and "netstat -nap|grep 9000|wc -l" shows 18000+

why on busy time ,the php-fpm dosen't response any thing to nginx server
,but the server seems not busy ?



System setup:
RHEL 5.4
nginx 0.8.45
php.5.3.6 with php-fpm .

nginx.conf:


log_format main
'$remote_addr\t$remote_user\t$time_local\t$request\t$status\t$body_bytes_sent\t$http_referer\t$http_user_agent\t$content_length\t$request_time\t$upstream_response_time\t$sent_http_content_type\t$sent_http_transfer_encoding\t$http_m\t$request_body';

server_names_hash_bucket_size 128;
client_header_buffer_size 128k;
large_client_header_buffers 8 128k;
client_max_body_size 200m;
client_body_buffer_size 128k;

proxy_connect_timeout 600;
proxy_read_timeout 600;
proxy_send_timeout 600;
proxy_buffer_size 16k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;

fastcgi_connect_timeout 60;
fastcgi_send_timeout 60;
fastcgi_read_timeout 60;
fastcgi_buffer_size 64k;
fastcgi_buffers 4 64k;
fastcgi_busy_buffers_size 128k;
fastcgi_temp_file_write_size 128k;
fastcgi_intercept_errors on;

php-fpm.conf:
pm = dynamic
pm.max_children = 300
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 15
pm.max_requests = 2000

request_terminate_timeout = 10s
request_slowlog_timeout =2s

--

---
You received this message because you are subscribed to the Google Groups "highload-php-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to highload-php-en+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

On 2013-10-22 07:31, billy wrote:
> on normal time ,php-fpm has 30+ process. all request finished in 10 seconds. (i execute commands "netstat -nap|wc -l" ,it show 23000, and "netstat -nap|grep 9000|wc -l" shows 7600)
> on busy time ,php-fpm has 40+ process(with a lot of concurrent request) , lots of requests get 499 response code( i know because i set php-fpm to terminate the request executing over 10 seconds) ,according nginx acess log, and almost all of them dosen't have a $upstream_response_time, but only $request_time. the server has a load average of 5-9,cpu usage at 30%, no %system busy ,no $io busy. execute "netstat -nap|wc -l" ,it show 43000, and "netstat -nap|grep 9000|wc -l" shows 18000+
>
> why on busy time ,the php-fpm dosen't response any thing to nginx server ,but the server seems not busy ?

Seems to me that PHP scripts wait for some external resource - a DB or an other network service.
The scripts themselves aren't busy, but they still time out since that external service is answering slow.

What do the scripts do?
Can you do some profiling to see where exactly PHP scripts spend time?

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime profiling for PHP

--

---
You received this message because you are subscribed to the Google Groups "highload-php-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to highload-php-en+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

On 2013-10-25 10:16, billy wrote:
> yeah, our php script did do a lot of network request and parse the result to finish our application logic .
> but even if php is waiting for network resource ,why not php-fpm spawn additional new processes to answer incoming request ? as it seems the system could afford creating more php process ?

How exactly would that help you?
You get the timeouts not because there are not enough PHP processes, but because those PHP processes take too much time.

(Keep this on the list, pls).

> On Tuesday, October 22, 2013 4:31:39 PM UTC+8, Antony Dovgal wrote:
>
> On 2013-10-22 07:31, billy wrote:
> > on normal time ,php-fpm has 30+ process. all request finished in 10 seconds. (i execute commands "netstat -nap|wc -l" ,it show 23000, and "netstat -nap|grep 9000|wc -l" shows 7600)
> > on busy time ,php-fpm has 40+ process(with a lot of concurrent request) , lots of requests get 499 response code( i know because i set php-fpm to terminate the request executing over 10 seconds) ,according nginx acess log, and almost all of them dosen't have a $upstream_response_time, but only $request_time. the server has a load average of 5-9,cpu usage at 30%, no %system busy ,no $io busy. execute "netstat -nap|wc -l" ,it show 43000, and "netstat -nap|grep 9000|wc -l" shows 18000+
> >
> > why on busy time ,the php-fpm dosen't response any thing to nginx server ,but the server seems not busy ?
>
> Seems to me that PHP scripts wait for some external resource - a DB or an other network service.
> The scripts themselves aren't busy, but they still time out since that external service is answering slow.
>
> What do the scripts do?
> Can you do some profiling to see where exactly PHP scripts spend time?
>
> --
> Wbr,
> Antony Dovgal
> ---
> http://pinba.org - realtime profiling for PHP
>


--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime profiling for PHP

--

---
You received this message because you are subscribed to the Google Groups "highload-php-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to highload-php-en+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

because from the nginx access log , i found the upstream server in fact
did not respond the nginx before the time out occurs . so i doubt there
is no free php process to take that process .

i mean when one php process hangs on to process time consuming work (such
as network requests), fpm could spawn a new process to taken the new
incoming requests, so the incoming requests couldn't be queued .




On Fri, Oct 25, 2013 at 3:52 PM, Antony Dovgal <tony@daylessday.org> wrote:

> On 2013-10-25 10:16, billy wrote:
>
>> yeah, our php script did do a lot of network request and parse the result
>> to finish our application logic .
>> but even if php is waiting for network resource ,why not php-fpm spawn
>> additional new processes to answer incoming request ? as it seems the
>> system could afford creating more php process ?
>>
>
> How exactly would that help you?
> You get the timeouts not because there are not enough PHP processes, but
> because those PHP processes take too much time.
>
> (Keep this on the list, pls).
>
> On Tuesday, October 22, 2013 4:31:39 PM UTC+8, Antony Dovgal wrote:
>>
>> On 2013-10-22 07:31, billy wrote:
>> > on normal time ,php-fpm has 30+ process. all request finished in
>> 10 seconds. (i execute commands "netstat -nap|wc -l" ,it show 23000,
>> and "netstat -nap|grep 9000|wc -l" shows 7600)
>> > on busy time ,php-fpm has 40+ process(with a lot of concurrent
>> request) , lots of requests get 499 response code( i know because i set
>> php-fpm to terminate the request executing over 10 seconds) ,according
>> nginx acess log, and almost all of them dosen't have a
>> $upstream_response_time, but only $request_time. the server has a load
>> average of 5-9,cpu usage at 30%, no %system busy ,no $io busy. execute
>> "netstat -nap|wc -l" ,it show 43000, and "netstat -nap|grep 9000|wc -l"
>> shows 18000+
>> >
>> > why on busy time ,the php-fpm dosen't response any thing to nginx
>> server ,but the server seems not busy ?
>>
>> Seems to me that PHP scripts wait for some external resource - a DB
>> or an other network service.
>> The scripts themselves aren't busy, but they still time out since
>> that external service is answering slow.
>>
>> What do the scripts do?
>> Can you do some profiling to see where exactly PHP scripts spend time?
>>
>> --
>> Wbr,
>> Antony Dovgal
>> ---
>> http://pinba.org - realtime profiling for PHP
>>
>>
>
> --
> Wbr,
> Antony Dovgal
> ---
> http://pinba.org - realtime profiling for PHP
>

--

---
You received this message because you are subscribed to the Google Groups "highload-php-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to highload-php-en+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.