Hi All,

Looking for some general guidance on php-fpm settings for a very busy site
3+mm page views a day.

We have 4 php app servers and they are getting killed during busy time, I'm
going to add 2 more, but started looking at php-fpm logs and seeing things
about warnings that php-fpm is busy, that I should add more min/max / start
servers, so I'm sure our configuration can be improved.

Any suggestions as to:

1) use static or dynamic?

2) If static, what should max children be set to?

3) if dynamic, what are ideal parameters for the dynamic stuff for very busy
situations?

Thanks

2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> Hi All,
> Looking for some general guidance on php-fpm settings for a very busy site
> 3+mm page views a day.
> We have 4 php app servers and they are getting killed during busy time, I'm
> going to add 2 more, but started looking at php-fpm logs and seeing things
> about warnings that php-fpm is busy, that I should add more min/max / start
> servers, so I'm sure our configuration can be improved.
> Any suggestions as to:
> 1) use static or dynamic?

depends

> 2) If static, what should max children be set to?

42 ?

> 3) if dynamic, what are ideal parameters for the dynamic stuff for very busy
> situations?

More seriously it's complicated to find the best conf. If it's a very
busy site, I recommand to :
1- 1 or 2 servers for running nginx or any other reverse proxy
2- several servers to run php-fpm and only php-fpm.
3- use static or dynamic with max_spare_children = max_children (you
want to be ready to handle the heavy load).
4- set max_children to something which sounds great to you at the
first place and adjust after depending on the load and the usage of
the server. If you have 4GB of RAM and each php-fpm process uses on
average 32MB you can set max children to 4096/32 - 10%. But you will
never find the right and perfect solution at the first try.

> Thanks

Thanks, I'm still trying to understand what the bottleneck is.

What is the highest error reporting level: debug?

Thanks

2010/10/7 Jérôme Loyet <ml@fatbsd.com>

> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> > Hi All,
> > Looking for some general guidance on php-fpm settings for a very busy
> site
> > 3+mm page views a day.
> > We have 4 php app servers and they are getting killed during busy time,
> I'm
> > going to add 2 more, but started looking at php-fpm logs and seeing
> things
> > about warnings that php-fpm is busy, that I should add more min/max /
> start
> > servers, so I'm sure our configuration can be improved.
> > Any suggestions as to:
> > 1) use static or dynamic?
>
> depends
>
> > 2) If static, what should max children be set to?
>
> 42 ?
>
> > 3) if dynamic, what are ideal parameters for the dynamic stuff for very
> busy
> > situations?
>
> More seriously it's complicated to find the best conf. If it's a very
> busy site, I recommand to :
> 1- 1 or 2 servers for running nginx or any other reverse proxy
> 2- several servers to run php-fpm and only php-fpm.
> 3- use static or dynamic with max_spare_children = max_children (you
> want to be ready to handle the heavy load).
> 4- set max_children to something which sounds great to you at the
> first place and adjust after depending on the load and the usage of
> the server. If you have 4GB of RAM and each php-fpm process uses on
> average 32MB you can set max children to 4096/32 - 10%. But you will
> never find the right and perfect solution at the first try.
>
> > Thanks
>

2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> Thanks, I'm still trying to understand what the bottleneck is.
> What is the highest error reporting level: debug?

yes but notice should be more appropriate


> Thanks
>
> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>
>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> > Hi All,
>> > Looking for some general guidance on php-fpm settings for a very busy
>> > site
>> > 3+mm page views a day.
>> > We have 4 php app servers and they are getting killed during busy time,
>> > I'm
>> > going to add 2 more, but started looking at php-fpm logs and seeing
>> > things
>> > about warnings that php-fpm is busy, that I should add more min/max /
>> > start
>> > servers, so I'm sure our configuration can be improved.
>> > Any suggestions as to:
>> > 1) use static or dynamic?
>>
>> depends
>>
>> > 2) If static, what should max children be set to?
>>
>> 42 ?
>>
>> > 3) if dynamic, what are ideal parameters for the dynamic stuff for very
>> > busy
>> > situations?
>>
>> More seriously it's complicated to find the best conf. If it's a very
>> busy site, I recommand to :
>> 1- 1 or 2 servers for running nginx or any other reverse proxy
>> 2- several servers to run php-fpm and only php-fpm.
>> 3- use static or dynamic with max_spare_children = max_children (you
>> want to be ready to handle the heavy load).
>> 4- set max_children to something which sounds great to you at the
>> first place and adjust after depending on the load and the usage of
>> the server. If you have 4GB of RAM and each php-fpm process uses on
>> average 32MB you can set max children to 4096/32 - 10%. But you will
>> never find the right and perfect solution at the first try.
>>
>> > Thanks
>
>

2010/10/7 Jérôme Loyet <ml@fatbsd.com>

> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> > Hi All,
> > Looking for some general guidance on php-fpm settings for a very busy
> site
> > 3+mm page views a day.
> > We have 4 php app servers and they are getting killed during busy time,
> I'm
> > going to add 2 more, but started looking at php-fpm logs and seeing
> things
> > about warnings that php-fpm is busy, that I should add more min/max /
> start
> > servers, so I'm sure our configuration can be improved.
> > Any suggestions as to:
> > 1) use static or dynamic?
>
> depends
>
> > 2) If static, what should max children be set to?
>
> 42 ?
>
> > 3) if dynamic, what are ideal parameters for the dynamic stuff for very
> busy
> > situations?
>
> More seriously it's complicated to find the best conf. If it's a very
> busy site, I recommand to :
> 1- 1 or 2 servers for running nginx or any other reverse proxy
>

-- we have one Nginx web server running, works great, delivers all of our
static content, barely a bump on the cpu utilizaiton, passes all php
requests to app farm


> 2- several servers to run php-fpm and only php-fpm.
>

-- we currently have 3 dedicated php-fpm servers, the 4th runs both the
php-fpm service and a read only slave db, I am about to provision 2 more
servers, will take the one that is currently running the db as well out of
rotation.


> 3- use static or dynamic with max_spare_children = max_children (you
> want to be ready to handle the heavy load).
>

-- I don't actually mind keeping a static number of servers ready, have
upped it to 100 from previous 50 in dynamic configuration as I don't exactly
understand (I thought I did) the dynamic stuff. Will review it again to try
and get a better handle on it.



> 4- set max_children to something which sounds great to you at the
> first place and adjust after depending on the load and the usage of
> the server. If you have 4GB of RAM and each php-fpm process uses on
> average 32MB you can set max children to 4096/32 - 10%. But you will
> never find the right and perfect solution at the first try.
>

-- How can I "easily" determine how much RAM on average the php-fpm
processes use?

>
> > Thanks
>

Why would notice be more appropriate? Right now I'm trying to get as much
information out of the system as possible. Is "debug" inclusive of notice?

2010/10/7 Jérôme Loyet <ml@fatbsd.com>

> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> > Thanks, I'm still trying to understand what the bottleneck is.
> > What is the highest error reporting level: debug?
>
> yes but notice should be more appropriate
>
>
> > Thanks
> >
> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >>
> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >> > Hi All,
> >> > Looking for some general guidance on php-fpm settings for a very busy
> >> > site
> >> > 3+mm page views a day.
> >> > We have 4 php app servers and they are getting killed during busy
> time,
> >> > I'm
> >> > going to add 2 more, but started looking at php-fpm logs and seeing
> >> > things
> >> > about warnings that php-fpm is busy, that I should add more min/max /
> >> > start
> >> > servers, so I'm sure our configuration can be improved.
> >> > Any suggestions as to:
> >> > 1) use static or dynamic?
> >>
> >> depends
> >>
> >> > 2) If static, what should max children be set to?
> >>
> >> 42 ?
> >>
> >> > 3) if dynamic, what are ideal parameters for the dynamic stuff for
> very
> >> > busy
> >> > situations?
> >>
> >> More seriously it's complicated to find the best conf. If it's a very
> >> busy site, I recommand to :
> >> 1- 1 or 2 servers for running nginx or any other reverse proxy
> >> 2- several servers to run php-fpm and only php-fpm.
> >> 3- use static or dynamic with max_spare_children = max_children (you
> >> want to be ready to handle the heavy load).
> >> 4- set max_children to something which sounds great to you at the
> >> first place and adjust after depending on the load and the usage of
> >> the server. If you have 4GB of RAM and each php-fpm process uses on
> >> average 32MB you can set max children to 4096/32 - 10%. But you will
> >> never find the right and perfect solution at the first try.
> >>
> >> > Thanks
> >
> >
>

2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> Why would notice be more appropriate?  Right now I'm trying to get as much
> information out of the system as possible.  Is "debug" inclusive of notice?

yes

>
> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>
>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> > Thanks, I'm still trying to understand what the bottleneck is.
>> > What is the highest error reporting level: debug?
>>
>> yes but notice should be more appropriate
>>
>>
>> > Thanks
>> >
>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >>
>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >> > Hi All,
>> >> > Looking for some general guidance on php-fpm settings for a very busy
>> >> > site
>> >> > 3+mm page views a day.
>> >> > We have 4 php app servers and they are getting killed during busy
>> >> > time,
>> >> > I'm
>> >> > going to add 2 more, but started looking at php-fpm logs and seeing
>> >> > things
>> >> > about warnings that php-fpm is busy, that I should add more min/max /
>> >> > start
>> >> > servers, so I'm sure our configuration can be improved.
>> >> > Any suggestions as to:
>> >> > 1) use static or dynamic?
>> >>
>> >> depends
>> >>
>> >> > 2) If static, what should max children be set to?
>> >>
>> >> 42 ?
>> >>
>> >> > 3) if dynamic, what are ideal parameters for the dynamic stuff for
>> >> > very
>> >> > busy
>> >> > situations?
>> >>
>> >> More seriously it's complicated to find the best conf. If it's a very
>> >> busy site, I recommand to :
>> >> 1- 1 or 2 servers for running nginx or any other reverse proxy
>> >> 2- several servers to run php-fpm and only php-fpm.
>> >> 3- use static or dynamic with max_spare_children = max_children (you
>> >> want to be ready to handle the heavy load).
>> >> 4- set max_children to something which sounds great to you at the
>> >> first place and adjust after depending on the load and the usage of
>> >> the server. If you have 4GB of RAM and each php-fpm process uses on
>> >> average 32MB you can set max children to 4096/32 - 10%. But you will
>> >> never find the right and perfect solution at the first try.
>> >>
>> >> > Thanks
>> >
>> >
>
>

I thought notice was inclusive of debug, not the other way around?
Debug is a higher level of log output, whereas notice is not.

2010/10/7 Jérôme Loyet <ml@fatbsd.com>

> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> > Why would notice be more appropriate? Right now I'm trying to get as
> much
> > information out of the system as possible. Is "debug" inclusive of
> notice?
>
> yes
>
> >
> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >>
> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >> > Thanks, I'm still trying to understand what the bottleneck is.
> >> > What is the highest error reporting level: debug?
> >>
> >> yes but notice should be more appropriate
> >>
> >>
> >> > Thanks
> >> >
> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >> >>
> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >> >> > Hi All,
> >> >> > Looking for some general guidance on php-fpm settings for a very
> busy
> >> >> > site
> >> >> > 3+mm page views a day.
> >> >> > We have 4 php app servers and they are getting killed during busy
> >> >> > time,
> >> >> > I'm
> >> >> > going to add 2 more, but started looking at php-fpm logs and seeing
> >> >> > things
> >> >> > about warnings that php-fpm is busy, that I should add more min/max
> /
> >> >> > start
> >> >> > servers, so I'm sure our configuration can be improved.
> >> >> > Any suggestions as to:
> >> >> > 1) use static or dynamic?
> >> >>
> >> >> depends
> >> >>
> >> >> > 2) If static, what should max children be set to?
> >> >>
> >> >> 42 ?
> >> >>
> >> >> > 3) if dynamic, what are ideal parameters for the dynamic stuff for
> >> >> > very
> >> >> > busy
> >> >> > situations?
> >> >>
> >> >> More seriously it's complicated to find the best conf. If it's a very
> >> >> busy site, I recommand to :
> >> >> 1- 1 or 2 servers for running nginx or any other reverse proxy
> >> >> 2- several servers to run php-fpm and only php-fpm.
> >> >> 3- use static or dynamic with max_spare_children = max_children (you
> >> >> want to be ready to handle the heavy load).
> >> >> 4- set max_children to something which sounds great to you at the
> >> >> first place and adjust after depending on the load and the usage of
> >> >> the server. If you have 4GB of RAM and each php-fpm process uses on
> >> >> average 32MB you can set max children to 4096/32 - 10%. But you will
> >> >> never find the right and perfect solution at the first try.
> >> >>
> >> >> > Thanks
> >> >
> >> >
> >
> >
>

Could someone make sure I understand, debug will give me the highest level
of error information, correct? this is what I'm looking for.

I just got this error:


Oct 07 12:41:38.287422 [ERROR] pid 27916,
fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
retrieve process activity of one or more child(ren). Will try again later.


What does that mean??


On Thu, Oct 7, 2010 at 1:28 PM, Jonathan Langevin <intel352@gmail.com>wrote:

> I thought notice was inclusive of debug, not the other way around?
> Debug is a higher level of log output, whereas notice is not.
>
>
> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>
>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> > Why would notice be more appropriate? Right now I'm trying to get as
>> much
>> > information out of the system as possible. Is "debug" inclusive of
>> notice?
>>
>> yes
>>
>> >
>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >>
>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >> > Thanks, I'm still trying to understand what the bottleneck is.
>> >> > What is the highest error reporting level: debug?
>> >>
>> >> yes but notice should be more appropriate
>> >>
>> >>
>> >> > Thanks
>> >> >
>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >> >>
>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >> >> > Hi All,
>> >> >> > Looking for some general guidance on php-fpm settings for a very
>> busy
>> >> >> > site
>> >> >> > 3+mm page views a day.
>> >> >> > We have 4 php app servers and they are getting killed during busy
>> >> >> > time,
>> >> >> > I'm
>> >> >> > going to add 2 more, but started looking at php-fpm logs and
>> seeing
>> >> >> > things
>> >> >> > about warnings that php-fpm is busy, that I should add more
>> min/max /
>> >> >> > start
>> >> >> > servers, so I'm sure our configuration can be improved.
>> >> >> > Any suggestions as to:
>> >> >> > 1) use static or dynamic?
>> >> >>
>> >> >> depends
>> >> >>
>> >> >> > 2) If static, what should max children be set to?
>> >> >>
>> >> >> 42 ?
>> >> >>
>> >> >> > 3) if dynamic, what are ideal parameters for the dynamic stuff for
>> >> >> > very
>> >> >> > busy
>> >> >> > situations?
>> >> >>
>> >> >> More seriously it's complicated to find the best conf. If it's a
>> very
>> >> >> busy site, I recommand to :
>> >> >> 1- 1 or 2 servers for running nginx or any other reverse proxy
>> >> >> 2- several servers to run php-fpm and only php-fpm.
>> >> >> 3- use static or dynamic with max_spare_children = max_children (you
>> >> >> want to be ready to handle the heavy load).
>> >> >> 4- set max_children to something which sounds great to you at the
>> >> >> first place and adjust after depending on the load and the usage of
>> >> >> the server. If you have 4GB of RAM and each php-fpm process uses on
>> >> >> average 32MB you can set max children to 4096/32 - 10%. But you will
>> >> >> never find the right and perfect solution at the first try.
>> >> >>
>> >> >> > Thanks
>> >> >
>> >> >
>> >
>> >
>>
>
>

The number of processes available were not the number expected.

What is your FPM config? Mainly we'll need to see your static/dynamic
process settings.

On Thu, Oct 7, 2010 at 1:46 PM, Ilan Berkner <iberkner@gmail.com> wrote:

> Could someone make sure I understand, debug will give me the highest level
> of error information, correct? this is what I'm looking for.
>
> I just got this error:
>
>
> Oct 07 12:41:38.287422 [ERROR] pid 27916,
> fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
> retrieve process activity of one or more child(ren). Will try again later..
>
>
> What does that mean??
>
>
> On Thu, Oct 7, 2010 at 1:28 PM, Jonathan Langevin <intel352@gmail.com>wrote:
>
>> I thought notice was inclusive of debug, not the other way around?
>> Debug is a higher level of log output, whereas notice is not.
>>
>>
>> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>
>>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>> > Why would notice be more appropriate? Right now I'm trying to get as
>>> much
>>> > information out of the system as possible. Is "debug" inclusive of
>>> notice?
>>>
>>> yes
>>>
>>> >
>>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>> >>
>>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>> >> > Thanks, I'm still trying to understand what the bottleneck is.
>>> >> > What is the highest error reporting level: debug?
>>> >>
>>> >> yes but notice should be more appropriate
>>> >>
>>> >>
>>> >> > Thanks
>>> >> >
>>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>> >> >>
>>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>> >> >> > Hi All,
>>> >> >> > Looking for some general guidance on php-fpm settings for a very
>>> busy
>>> >> >> > site
>>> >> >> > 3+mm page views a day.
>>> >> >> > We have 4 php app servers and they are getting killed during busy
>>> >> >> > time,
>>> >> >> > I'm
>>> >> >> > going to add 2 more, but started looking at php-fpm logs and
>>> seeing
>>> >> >> > things
>>> >> >> > about warnings that php-fpm is busy, that I should add more
>>> min/max /
>>> >> >> > start
>>> >> >> > servers, so I'm sure our configuration can be improved.
>>> >> >> > Any suggestions as to:
>>> >> >> > 1) use static or dynamic?
>>> >> >>
>>> >> >> depends
>>> >> >>
>>> >> >> > 2) If static, what should max children be set to?
>>> >> >>
>>> >> >> 42 ?
>>> >> >>
>>> >> >> > 3) if dynamic, what are ideal parameters for the dynamic stuff
>>> for
>>> >> >> > very
>>> >> >> > busy
>>> >> >> > situations?
>>> >> >>
>>> >> >> More seriously it's complicated to find the best conf. If it's a
>>> very
>>> >> >> busy site, I recommand to :
>>> >> >> 1- 1 or 2 servers for running nginx or any other reverse proxy
>>> >> >> 2- several servers to run php-fpm and only php-fpm.
>>> >> >> 3- use static or dynamic with max_spare_children = max_children
>>> (you
>>> >> >> want to be ready to handle the heavy load).
>>> >> >> 4- set max_children to something which sounds great to you at the
>>> >> >> first place and adjust after depending on the load and the usage of
>>> >> >> the server. If you have 4GB of RAM and each php-fpm process uses on
>>> >> >> average 32MB you can set max children to 4096/32 - 10%. But you
>>> will
>>> >> >> never find the right and perfect solution at the first try.
>>> >> >>
>>> >> >> > Thanks
>>> >> >
>>> >> >
>>> >
>>> >
>>>
>>
>>
>

I was previously using dynamic with the settings shown, have now switched to
static, trying to determine "ideal" configuration...

any help would be appreciated.


pm = static
pm.max_children = 100

;pm = dynamic
;pm_max_children = 50
;pm.start_servers =25
;pm.min_spare_servers = 10
;pm.max_spare_servers = 35
;pm.max_requests = 500


On Thu, Oct 7, 2010 at 1:50 PM, Jonathan Langevin <intel352@gmail.com>wrote:

> The number of processes available were not the number expected.
>
> What is your FPM config? Mainly we'll need to see your static/dynamic
> process settings.
>
>
> On Thu, Oct 7, 2010 at 1:46 PM, Ilan Berkner <iberkner@gmail.com> wrote:
>
>> Could someone make sure I understand, debug will give me the highest level
>> of error information, correct? this is what I'm looking for.
>>
>> I just got this error:
>>
>>
>> Oct 07 12:41:38.287422 [ERROR] pid 27916,
>> fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
>> retrieve process activity of one or more child(ren). Will try again later.
>>
>>
>> What does that mean??
>>
>>
>> On Thu, Oct 7, 2010 at 1:28 PM, Jonathan Langevin <intel352@gmail.com>wrote:
>>
>>> I thought notice was inclusive of debug, not the other way around?
>>> Debug is a higher level of log output, whereas notice is not.
>>>
>>>
>>> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>>
>>>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>>> > Why would notice be more appropriate? Right now I'm trying to get as
>>>> much
>>>> > information out of the system as possible. Is "debug" inclusive of
>>>> notice?
>>>>
>>>> yes
>>>>
>>>> >
>>>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>>> >>
>>>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>>> >> > Thanks, I'm still trying to understand what the bottleneck is.
>>>> >> > What is the highest error reporting level: debug?
>>>> >>
>>>> >> yes but notice should be more appropriate
>>>> >>
>>>> >>
>>>> >> > Thanks
>>>> >> >
>>>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>>> >> >>
>>>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>>> >> >> > Hi All,
>>>> >> >> > Looking for some general guidance on php-fpm settings for a very
>>>> busy
>>>> >> >> > site
>>>> >> >> > 3+mm page views a day.
>>>> >> >> > We have 4 php app servers and they are getting killed during
>>>> busy
>>>> >> >> > time,
>>>> >> >> > I'm
>>>> >> >> > going to add 2 more, but started looking at php-fpm logs and
>>>> seeing
>>>> >> >> > things
>>>> >> >> > about warnings that php-fpm is busy, that I should add more
>>>> min/max /
>>>> >> >> > start
>>>> >> >> > servers, so I'm sure our configuration can be improved.
>>>> >> >> > Any suggestions as to:
>>>> >> >> > 1) use static or dynamic?
>>>> >> >>
>>>> >> >> depends
>>>> >> >>
>>>> >> >> > 2) If static, what should max children be set to?
>>>> >> >>
>>>> >> >> 42 ?
>>>> >> >>
>>>> >> >> > 3) if dynamic, what are ideal parameters for the dynamic stuff
>>>> for
>>>> >> >> > very
>>>> >> >> > busy
>>>> >> >> > situations?
>>>> >> >>
>>>> >> >> More seriously it's complicated to find the best conf. If it's a
>>>> very
>>>> >> >> busy site, I recommand to :
>>>> >> >> 1- 1 or 2 servers for running nginx or any other reverse proxy
>>>> >> >> 2- several servers to run php-fpm and only php-fpm.
>>>> >> >> 3- use static or dynamic with max_spare_children = max_children
>>>> (you
>>>> >> >> want to be ready to handle the heavy load).
>>>> >> >> 4- set max_children to something which sounds great to you at the
>>>> >> >> first place and adjust after depending on the load and the usage
>>>> of
>>>> >> >> the server. If you have 4GB of RAM and each php-fpm process uses
>>>> on
>>>> >> >> average 32MB you can set max children to 4096/32 - 10%. But you
>>>> will
>>>> >> >> never find the right and perfect solution at the first try.
>>>> >> >>
>>>> >> >> > Thanks
>>>> >> >
>>>> >> >
>>>> >
>>>> >
>>>>
>>>
>>>
>>
>

Ilan, at the time that you received that error in your logs, you were
running a static configuration?

On Thu, Oct 7, 2010 at 1:56 PM, Ilan Berkner <iberkner@gmail.com> wrote:

> I was previously using dynamic with the settings shown, have now switched
> to static, trying to determine "ideal" configuration...
>
> any help would be appreciated.
>
>
> pm = static
> pm.max_children = 100
>
> ;pm = dynamic
> ;pm_max_children = 50
> ;pm.start_servers =25
> ;pm.min_spare_servers = 10
> ;pm.max_spare_servers = 35
> ;pm.max_requests = 500
>
>
> On Thu, Oct 7, 2010 at 1:50 PM, Jonathan Langevin <intel352@gmail.com>wrote:
>
>> The number of processes available were not the number expected.
>>
>> What is your FPM config? Mainly we'll need to see your static/dynamic
>> process settings.
>>
>>
>> On Thu, Oct 7, 2010 at 1:46 PM, Ilan Berkner <iberkner@gmail.com> wrote:
>>
>>> Could someone make sure I understand, debug will give me the highest
>>> level of error information, correct? this is what I'm looking for.
>>>
>>> I just got this error:
>>>
>>>
>>> Oct 07 12:41:38.287422 [ERROR] pid 27916,
>>> fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
>>> retrieve process activity of one or more child(ren). Will try again later.
>>>
>>>
>>> What does that mean??
>>>
>>>
>>> On Thu, Oct 7, 2010 at 1:28 PM, Jonathan Langevin <intel352@gmail.com>wrote:
>>>
>>>> I thought notice was inclusive of debug, not the other way around?
>>>> Debug is a higher level of log output, whereas notice is not.
>>>>
>>>>
>>>> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>>>
>>>>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>>>> > Why would notice be more appropriate? Right now I'm trying to get as
>>>>> much
>>>>> > information out of the system as possible. Is "debug" inclusive of
>>>>> notice?
>>>>>
>>>>> yes
>>>>>
>>>>> >
>>>>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>>>> >>
>>>>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>>>> >> > Thanks, I'm still trying to understand what the bottleneck is.
>>>>> >> > What is the highest error reporting level: debug?
>>>>> >>
>>>>> >> yes but notice should be more appropriate
>>>>> >>
>>>>> >>
>>>>> >> > Thanks
>>>>> >> >
>>>>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>>>> >> >>
>>>>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>>>> >> >> > Hi All,
>>>>> >> >> > Looking for some general guidance on php-fpm settings for a
>>>>> very busy
>>>>> >> >> > site
>>>>> >> >> > 3+mm page views a day.
>>>>> >> >> > We have 4 php app servers and they are getting killed during
>>>>> busy
>>>>> >> >> > time,
>>>>> >> >> > I'm
>>>>> >> >> > going to add 2 more, but started looking at php-fpm logs and
>>>>> seeing
>>>>> >> >> > things
>>>>> >> >> > about warnings that php-fpm is busy, that I should add more
>>>>> min/max /
>>>>> >> >> > start
>>>>> >> >> > servers, so I'm sure our configuration can be improved.
>>>>> >> >> > Any suggestions as to:
>>>>> >> >> > 1) use static or dynamic?
>>>>> >> >>
>>>>> >> >> depends
>>>>> >> >>
>>>>> >> >> > 2) If static, what should max children be set to?
>>>>> >> >>
>>>>> >> >> 42 ?
>>>>> >> >>
>>>>> >> >> > 3) if dynamic, what are ideal parameters for the dynamic stuff
>>>>> for
>>>>> >> >> > very
>>>>> >> >> > busy
>>>>> >> >> > situations?
>>>>> >> >>
>>>>> >> >> More seriously it's complicated to find the best conf. If it's a
>>>>> very
>>>>> >> >> busy site, I recommand to :
>>>>> >> >> 1- 1 or 2 servers for running nginx or any other reverse proxy
>>>>> >> >> 2- several servers to run php-fpm and only php-fpm.
>>>>> >> >> 3- use static or dynamic with max_spare_children = max_children
>>>>> (you
>>>>> >> >> want to be ready to handle the heavy load).
>>>>> >> >> 4- set max_children to something which sounds great to you at the
>>>>> >> >> first place and adjust after depending on the load and the usage
>>>>> of
>>>>> >> >> the server. If you have 4GB of RAM and each php-fpm process uses
>>>>> on
>>>>> >> >> average 32MB you can set max children to 4096/32 - 10%. But you
>>>>> will
>>>>> >> >> never find the right and perfect solution at the first try.
>>>>> >> >>
>>>>> >> >> > Thanks
>>>>> >> >
>>>>> >> >
>>>>> >
>>>>> >
>>>>>
>>>>
>>>>
>>>
>>
>

yes

On Thu, Oct 7, 2010 at 2:15 PM, Jonathan Langevin <intel352@gmail.com>wrote:

> Ilan, at the time that you received that error in your logs, you were
> running a static configuration?
>
> On Thu, Oct 7, 2010 at 1:56 PM, Ilan Berkner <iberkner@gmail.com> wrote:
>
>> I was previously using dynamic with the settings shown, have now switched
>> to static, trying to determine "ideal" configuration...
>>
>> any help would be appreciated.
>>
>>
>> pm = static
>> pm.max_children = 100
>>
>> ;pm = dynamic
>> ;pm_max_children = 50
>> ;pm.start_servers =25
>> ;pm.min_spare_servers = 10
>> ;pm.max_spare_servers = 35
>> ;pm.max_requests = 500
>>
>>
>> On Thu, Oct 7, 2010 at 1:50 PM, Jonathan Langevin <intel352@gmail.com>wrote:
>>
>>> The number of processes available were not the number expected.
>>>
>>> What is your FPM config? Mainly we'll need to see your static/dynamic
>>> process settings.
>>>
>>>
>>> On Thu, Oct 7, 2010 at 1:46 PM, Ilan Berkner <iberkner@gmail.com> wrote:
>>>
>>>> Could someone make sure I understand, debug will give me the highest
>>>> level of error information, correct? this is what I'm looking for.
>>>>
>>>> I just got this error:
>>>>
>>>>
>>>> Oct 07 12:41:38.287422 [ERROR] pid 27916,
>>>> fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
>>>> retrieve process activity of one or more child(ren). Will try again later.
>>>>
>>>>
>>>> What does that mean??
>>>>
>>>>
>>>> On Thu, Oct 7, 2010 at 1:28 PM, Jonathan Langevin <intel352@gmail.com>wrote:
>>>>
>>>>> I thought notice was inclusive of debug, not the other way around?
>>>>> Debug is a higher level of log output, whereas notice is not.
>>>>>
>>>>>
>>>>> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>>>>
>>>>>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>>>>> > Why would notice be more appropriate? Right now I'm trying to get
>>>>>> as much
>>>>>> > information out of the system as possible. Is "debug" inclusive of
>>>>>> notice?
>>>>>>
>>>>>> yes
>>>>>>
>>>>>> >
>>>>>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>>>>> >>
>>>>>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>>>>> >> > Thanks, I'm still trying to understand what the bottleneck is.
>>>>>> >> > What is the highest error reporting level: debug?
>>>>>> >>
>>>>>> >> yes but notice should be more appropriate
>>>>>> >>
>>>>>> >>
>>>>>> >> > Thanks
>>>>>> >> >
>>>>>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>>>>> >> >>
>>>>>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>>>>> >> >> > Hi All,
>>>>>> >> >> > Looking for some general guidance on php-fpm settings for a
>>>>>> very busy
>>>>>> >> >> > site
>>>>>> >> >> > 3+mm page views a day.
>>>>>> >> >> > We have 4 php app servers and they are getting killed during
>>>>>> busy
>>>>>> >> >> > time,
>>>>>> >> >> > I'm
>>>>>> >> >> > going to add 2 more, but started looking at php-fpm logs and
>>>>>> seeing
>>>>>> >> >> > things
>>>>>> >> >> > about warnings that php-fpm is busy, that I should add more
>>>>>> min/max /
>>>>>> >> >> > start
>>>>>> >> >> > servers, so I'm sure our configuration can be improved.
>>>>>> >> >> > Any suggestions as to:
>>>>>> >> >> > 1) use static or dynamic?
>>>>>> >> >>
>>>>>> >> >> depends
>>>>>> >> >>
>>>>>> >> >> > 2) If static, what should max children be set to?
>>>>>> >> >>
>>>>>> >> >> 42 ?
>>>>>> >> >>
>>>>>> >> >> > 3) if dynamic, what are ideal parameters for the dynamic stuff
>>>>>> for
>>>>>> >> >> > very
>>>>>> >> >> > busy
>>>>>> >> >> > situations?
>>>>>> >> >>
>>>>>> >> >> More seriously it's complicated to find the best conf. If it's a
>>>>>> very
>>>>>> >> >> busy site, I recommand to :
>>>>>> >> >> 1- 1 or 2 servers for running nginx or any other reverse proxy
>>>>>> >> >> 2- several servers to run php-fpm and only php-fpm.
>>>>>> >> >> 3- use static or dynamic with max_spare_children = max_children
>>>>>> (you
>>>>>> >> >> want to be ready to handle the heavy load).
>>>>>> >> >> 4- set max_children to something which sounds great to you at
>>>>>> the
>>>>>> >> >> first place and adjust after depending on the load and the usage
>>>>>> of
>>>>>> >> >> the server. If you have 4GB of RAM and each php-fpm process uses
>>>>>> on
>>>>>> >> >> average 32MB you can set max children to 4096/32 - 10%. But you
>>>>>> will
>>>>>> >> >> never find the right and perfect solution at the first try.
>>>>>> >> >>
>>>>>> >> >> > Thanks
>>>>>> >> >
>>>>>> >> >
>>>>>> >
>>>>>> >
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> Could someone make sure I understand, debug will give me the highest level
> of error information, correct? this is what I'm looking for.
> I just got this error:
>
> Oct 07 12:41:38.287422 [ERROR] pid 27916,
> fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
> retrieve process activity of one or more child(ren). Will try again later..
>
> What does that mean??

it means that the parent process has not been able to retrieve
consistent informations about its children (if each child is idle or
active and if idle + active = total_children).
If the static PM is used, it just means that for once the status page
won't be up to date, so there is nothing to care about.
If the dynamic PM is used, it means that the parent won't do anything
until it has consistent informations.

So if the errors triggers once this not a problem. If it happens every
second (the frequency of fpm_pctl_perform_idle_server_maintenance())
it might be a problem.

>
> On Thu, Oct 7, 2010 at 1:28 PM, Jonathan Langevin <intel352@gmail.com>
> wrote:
>>
>> I thought notice was inclusive of debug, not the other way around?
>> Debug is a higher level of log output, whereas notice is not.
>>
>> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>>
>>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>> > Why would notice be more appropriate?  Right now I'm trying to get as
>>> > much
>>> > information out of the system as possible.  Is "debug" inclusive of
>>> > notice?
>>>
>>> yes
>>>
>>> >
>>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>> >>
>>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>> >> > Thanks, I'm still trying to understand what the bottleneck is.
>>> >> > What is the highest error reporting level: debug?
>>> >>
>>> >> yes but notice should be more appropriate
>>> >>
>>> >>
>>> >> > Thanks
>>> >> >
>>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>> >> >>
>>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>>> >> >> > Hi All,
>>> >> >> > Looking for some general guidance on php-fpm settings for a very
>>> >> >> > busy
>>> >> >> > site
>>> >> >> > 3+mm page views a day.
>>> >> >> > We have 4 php app servers and they are getting killed during busy
>>> >> >> > time,
>>> >> >> > I'm
>>> >> >> > going to add 2 more, but started looking at php-fpm logs and
>>> >> >> > seeing
>>> >> >> > things
>>> >> >> > about warnings that php-fpm is busy, that I should add more
>>> >> >> > min/max /
>>> >> >> > start
>>> >> >> > servers, so I'm sure our configuration can be improved.
>>> >> >> > Any suggestions as to:
>>> >> >> > 1) use static or dynamic?
>>> >> >>
>>> >> >> depends
>>> >> >>
>>> >> >> > 2) If static, what should max children be set to?
>>> >> >>
>>> >> >> 42 ?
>>> >> >>
>>> >> >> > 3) if dynamic, what are ideal parameters for the dynamic stuff
>>> >> >> > for
>>> >> >> > very
>>> >> >> > busy
>>> >> >> > situations?
>>> >> >>
>>> >> >> More seriously it's complicated to find the best conf. If it's a
>>> >> >> very
>>> >> >> busy site, I recommand to :
>>> >> >> 1- 1 or 2 servers for running nginx or any other reverse proxy
>>> >> >> 2- several servers to run php-fpm and only php-fpm.
>>> >> >> 3- use static or dynamic with max_spare_children = max_children
>>> >> >> (you
>>> >> >> want to be ready to handle the heavy load).
>>> >> >> 4- set max_children to something which sounds great to you at the
>>> >> >> first place and adjust after depending on the load and the usage of
>>> >> >> the server. If you have 4GB of RAM and each php-fpm process uses on
>>> >> >> average 32MB you can set max children to 4096/32 - 10%. But you
>>> >> >> will
>>> >> >> never find the right and perfect solution at the first try.
>>> >> >>
>>> >> >> > Thanks
>>> >> >
>>> >> >
>>> >
>>> >
>>
>
>

FYI - I am seeing this every so often:

Oct 08 08:48:14.087065 [ERROR] pid 1549,
fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
retrieve process activity of one or more child(ren). Will try again later.

Oct 08 10:22:01.829260 [ERROR] pid 1549,
fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
retrieve process activity of one or more child(ren). Will try again later.

Oct 08 10:46:36.296591 [ERROR] pid 1549,
fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
retrieve process activity of one or more child(ren). Will try again later.

Oct 08 11:53:41.484788 [ERROR] pid 1549,
fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
retrieve process activity of one or more child(ren). Will try again later.

Oct 08 12:35:35.574208 [ERROR] pid 1549,
fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
retrieve process activity of one or more child(ren). Will try again later.


2010/10/7 Jérôme Loyet <ml@fatbsd.com>

> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> > Could someone make sure I understand, debug will give me the highest
> level
> > of error information, correct? this is what I'm looking for.
> > I just got this error:
> >
> > Oct 07 12:41:38.287422 [ERROR] pid 27916,
> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable
> to
> > retrieve process activity of one or more child(ren). Will try again
> later.
> >
> > What does that mean??
>
> it means that the parent process has not been able to retrieve
> consistent informations about its children (if each child is idle or
> active and if idle + active = total_children).
> If the static PM is used, it just means that for once the status page
> won't be up to date, so there is nothing to care about.
> If the dynamic PM is used, it means that the parent won't do anything
> until it has consistent informations.
>
> So if the errors triggers once this not a problem. If it happens every
> second (the frequency of fpm_pctl_perform_idle_server_maintenance())
> it might be a problem.
>
> >
> > On Thu, Oct 7, 2010 at 1:28 PM, Jonathan Langevin <intel352@gmail.com>
> > wrote:
> >>
> >> I thought notice was inclusive of debug, not the other way around?
> >> Debug is a higher level of log output, whereas notice is not.
> >>
> >> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >>>
> >>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >>> > Why would notice be more appropriate? Right now I'm trying to get as
> >>> > much
> >>> > information out of the system as possible. Is "debug" inclusive of
> >>> > notice?
> >>>
> >>> yes
> >>>
> >>> >
> >>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >>> >>
> >>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >>> >> > Thanks, I'm still trying to understand what the bottleneck is.
> >>> >> > What is the highest error reporting level: debug?
> >>> >>
> >>> >> yes but notice should be more appropriate
> >>> >>
> >>> >>
> >>> >> > Thanks
> >>> >> >
> >>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >>> >> >>
> >>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >>> >> >> > Hi All,
> >>> >> >> > Looking for some general guidance on php-fpm settings for a
> very
> >>> >> >> > busy
> >>> >> >> > site
> >>> >> >> > 3+mm page views a day.
> >>> >> >> > We have 4 php app servers and they are getting killed during
> busy
> >>> >> >> > time,
> >>> >> >> > I'm
> >>> >> >> > going to add 2 more, but started looking at php-fpm logs and
> >>> >> >> > seeing
> >>> >> >> > things
> >>> >> >> > about warnings that php-fpm is busy, that I should add more
> >>> >> >> > min/max /
> >>> >> >> > start
> >>> >> >> > servers, so I'm sure our configuration can be improved.
> >>> >> >> > Any suggestions as to:
> >>> >> >> > 1) use static or dynamic?
> >>> >> >>
> >>> >> >> depends
> >>> >> >>
> >>> >> >> > 2) If static, what should max children be set to?
> >>> >> >>
> >>> >> >> 42 ?
> >>> >> >>
> >>> >> >> > 3) if dynamic, what are ideal parameters for the dynamic stuff
> >>> >> >> > for
> >>> >> >> > very
> >>> >> >> > busy
> >>> >> >> > situations?
> >>> >> >>
> >>> >> >> More seriously it's complicated to find the best conf. If it's a
> >>> >> >> very
> >>> >> >> busy site, I recommand to :
> >>> >> >> 1- 1 or 2 servers for running nginx or any other reverse proxy
> >>> >> >> 2- several servers to run php-fpm and only php-fpm.
> >>> >> >> 3- use static or dynamic with max_spare_children = max_children
> >>> >> >> (you
> >>> >> >> want to be ready to handle the heavy load).
> >>> >> >> 4- set max_children to something which sounds great to you at the
> >>> >> >> first place and adjust after depending on the load and the usage
> of
> >>> >> >> the server. If you have 4GB of RAM and each php-fpm process uses
> on
> >>> >> >> average 32MB you can set max children to 4096/32 - 10%. But you
> >>> >> >> will
> >>> >> >> never find the right and perfect solution at the first try.
> >>> >> >>
> >>> >> >> > Thanks
> >>> >> >
> >>> >> >
> >>> >
> >>> >
> >>
> >
> >
>

2010/10/8 Ilan Berkner <iberkner@gmail.com>:
> FYI - I am seeing this every so often:
> Oct 08 08:48:14.087065 [ERROR] pid 1549,
> fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
> retrieve process activity of one or more child(ren). Will try again later..
> Oct 08 10:22:01.829260 [ERROR] pid 1549,
> fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
> retrieve process activity of one or more child(ren). Will try again later..
> Oct 08 10:46:36.296591 [ERROR] pid 1549,
> fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
> retrieve process activity of one or more child(ren). Will try again later..
> Oct 08 11:53:41.484788 [ERROR] pid 1549,
> fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
> retrieve process activity of one or more child(ren). Will try again later..
> Oct 08 12:35:35.574208 [ERROR] pid 1549,
> fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable to
> retrieve process activity of one or more child(ren). Will try again later..
>

it's not so often, this function is called every seconds and you have
errors with several minutes between each. But you should open a bug
report so that we look for a correction.

> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>>
>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> > Could someone make sure I understand, debug will give me the highest
>> > level
>> > of error information, correct? this is what I'm looking for.
>> > I just got this error:
>> >
>> > Oct 07 12:41:38.287422 [ERROR] pid 27916,
>> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable
>> > to
>> > retrieve process activity of one or more child(ren). Will try again
>> > later.
>> >
>> > What does that mean??
>>
>> it means that the parent process has not been able to retrieve
>> consistent informations about its children (if each child is idle or
>> active and if idle + active = total_children).
>> If the static PM is used, it just means that for once the status page
>> won't be up to date, so there is nothing to care about.
>> If the dynamic PM is used, it means that the parent won't do anything
>> until it has consistent informations.
>>
>> So if the errors triggers once this not a problem. If it happens every
>> second (the frequency of fpm_pctl_perform_idle_server_maintenance())
>> it might be a problem.
>>
>> >
>> > On Thu, Oct 7, 2010 at 1:28 PM, Jonathan Langevin <intel352@gmail.com>
>> > wrote:
>> >>
>> >> I thought notice was inclusive of debug, not the other way around?
>> >> Debug is a higher level of log output, whereas notice is not.
>> >>
>> >> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >>>
>> >>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >>> > Why would notice be more appropriate?  Right now I'm trying to get
>> >>> > as
>> >>> > much
>> >>> > information out of the system as possible.  Is "debug" inclusive of
>> >>> > notice?
>> >>>
>> >>> yes
>> >>>
>> >>> >
>> >>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >>> >>
>> >>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >>> >> > Thanks, I'm still trying to understand what the bottleneck is.
>> >>> >> > What is the highest error reporting level: debug?
>> >>> >>
>> >>> >> yes but notice should be more appropriate
>> >>> >>
>> >>> >>
>> >>> >> > Thanks
>> >>> >> >
>> >>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >>> >> >>
>> >>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >>> >> >> > Hi All,
>> >>> >> >> > Looking for some general guidance on php-fpm settings for a
>> >>> >> >> > very
>> >>> >> >> > busy
>> >>> >> >> > site
>> >>> >> >> > 3+mm page views a day.
>> >>> >> >> > We have 4 php app servers and they are getting killed during
>> >>> >> >> > busy
>> >>> >> >> > time,
>> >>> >> >> > I'm
>> >>> >> >> > going to add 2 more, but started looking at php-fpm logs and
>> >>> >> >> > seeing
>> >>> >> >> > things
>> >>> >> >> > about warnings that php-fpm is busy, that I should add more
>> >>> >> >> > min/max /
>> >>> >> >> > start
>> >>> >> >> > servers, so I'm sure our configuration can be improved.
>> >>> >> >> > Any suggestions as to:
>> >>> >> >> > 1) use static or dynamic?
>> >>> >> >>
>> >>> >> >> depends
>> >>> >> >>
>> >>> >> >> > 2) If static, what should max children be set to?
>> >>> >> >>
>> >>> >> >> 42 ?
>> >>> >> >>
>> >>> >> >> > 3) if dynamic, what are ideal parameters for the dynamic stuff
>> >>> >> >> > for
>> >>> >> >> > very
>> >>> >> >> > busy
>> >>> >> >> > situations?
>> >>> >> >>
>> >>> >> >> More seriously it's complicated to find the best conf. If it's a
>> >>> >> >> very
>> >>> >> >> busy site, I recommand to :
>> >>> >> >> 1- 1 or 2 servers for running nginx or any other reverse proxy
>> >>> >> >> 2- several servers to run php-fpm and only php-fpm.
>> >>> >> >> 3- use static or dynamic with max_spare_children = max_children
>> >>> >> >> (you
>> >>> >> >> want to be ready to handle the heavy load).
>> >>> >> >> 4- set max_children to something which sounds great to you at
>> >>> >> >> the
>> >>> >> >> first place and adjust after depending on the load and the usage
>> >>> >> >> of
>> >>> >> >> the server. If you have 4GB of RAM and each php-fpm process uses
>> >>> >> >> on
>> >>> >> >> average 32MB you can set max children to 4096/32 - 10%. But you
>> >>> >> >> will
>> >>> >> >> never find the right and perfect solution at the first try.
>> >>> >> >>
>> >>> >> >> > Thanks
>> >>> >> >
>> >>> >> >
>> >>> >
>> >>> >
>> >>
>> >
>> >
>
>

I understand what you mean, in terms of that its not often and it does
appear to be working...

Am happy to open a bug report, how / where do I open a bug report and what
information is required?

Right now I'm running as static with max children set to 100. Do you
recommend switching to dynamic, if so, what would be good settings for a
busy site? Each box has 4GB and strictly runs php-fpm, can add more RAM if
needed, but there seems to be between 500MB and 1GB free at all times on
theses systems, so I think I'm OK with RAM for now.

2010/10/8 Jérôme Loyet <ml@fatbsd.com>

> 2010/10/8 Ilan Berkner <iberkner@gmail.com>:
> > FYI - I am seeing this every so often:
> > Oct 08 08:48:14.087065 [ERROR] pid 1549,
> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable
> to
> > retrieve process activity of one or more child(ren). Will try again
> later.
> > Oct 08 10:22:01.829260 [ERROR] pid 1549,
> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable
> to
> > retrieve process activity of one or more child(ren). Will try again
> later.
> > Oct 08 10:46:36.296591 [ERROR] pid 1549,
> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable
> to
> > retrieve process activity of one or more child(ren). Will try again
> later.
> > Oct 08 11:53:41.484788 [ERROR] pid 1549,
> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable
> to
> > retrieve process activity of one or more child(ren). Will try again
> later.
> > Oct 08 12:35:35.574208 [ERROR] pid 1549,
> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable
> to
> > retrieve process activity of one or more child(ren). Will try again
> later.
> >
>
> it's not so often, this function is called every seconds and you have
> errors with several minutes between each. But you should open a bug
> report so that we look for a correction.
>
> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >>
> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >> > Could someone make sure I understand, debug will give me the highest
> >> > level
> >> > of error information, correct? this is what I'm looking for.
> >> > I just got this error:
> >> >
> >> > Oct 07 12:41:38.287422 [ERROR] pid 27916,
> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
> unable
> >> > to
> >> > retrieve process activity of one or more child(ren). Will try again
> >> > later.
> >> >
> >> > What does that mean??
> >>
> >> it means that the parent process has not been able to retrieve
> >> consistent informations about its children (if each child is idle or
> >> active and if idle + active = total_children).
> >> If the static PM is used, it just means that for once the status page
> >> won't be up to date, so there is nothing to care about.
> >> If the dynamic PM is used, it means that the parent won't do anything
> >> until it has consistent informations.
> >>
> >> So if the errors triggers once this not a problem. If it happens every
> >> second (the frequency of fpm_pctl_perform_idle_server_maintenance())
> >> it might be a problem.
> >>
> >> >
> >> > On Thu, Oct 7, 2010 at 1:28 PM, Jonathan Langevin <intel352@gmail.com
> >
> >> > wrote:
> >> >>
> >> >> I thought notice was inclusive of debug, not the other way around?
> >> >> Debug is a higher level of log output, whereas notice is not.
> >> >>
> >> >> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >> >>>
> >> >>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >> >>> > Why would notice be more appropriate? Right now I'm trying to get
> >> >>> > as
> >> >>> > much
> >> >>> > information out of the system as possible. Is "debug" inclusive
> of
> >> >>> > notice?
> >> >>>
> >> >>> yes
> >> >>>
> >> >>> >
> >> >>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >> >>> >>
> >> >>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >> >>> >> > Thanks, I'm still trying to understand what the bottleneck is..
> >> >>> >> > What is the highest error reporting level: debug?
> >> >>> >>
> >> >>> >> yes but notice should be more appropriate
> >> >>> >>
> >> >>> >>
> >> >>> >> > Thanks
> >> >>> >> >
> >> >>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >> >>> >> >>
> >> >>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >> >>> >> >> > Hi All,
> >> >>> >> >> > Looking for some general guidance on php-fpm settings for a
> >> >>> >> >> > very
> >> >>> >> >> > busy
> >> >>> >> >> > site
> >> >>> >> >> > 3+mm page views a day.
> >> >>> >> >> > We have 4 php app servers and they are getting killed during
> >> >>> >> >> > busy
> >> >>> >> >> > time,
> >> >>> >> >> > I'm
> >> >>> >> >> > going to add 2 more, but started looking at php-fpm logs and
> >> >>> >> >> > seeing
> >> >>> >> >> > things
> >> >>> >> >> > about warnings that php-fpm is busy, that I should add more
> >> >>> >> >> > min/max /
> >> >>> >> >> > start
> >> >>> >> >> > servers, so I'm sure our configuration can be improved.
> >> >>> >> >> > Any suggestions as to:
> >> >>> >> >> > 1) use static or dynamic?
> >> >>> >> >>
> >> >>> >> >> depends
> >> >>> >> >>
> >> >>> >> >> > 2) If static, what should max children be set to?
> >> >>> >> >>
> >> >>> >> >> 42 ?
> >> >>> >> >>
> >> >>> >> >> > 3) if dynamic, what are ideal parameters for the dynamic
> stuff
> >> >>> >> >> > for
> >> >>> >> >> > very
> >> >>> >> >> > busy
> >> >>> >> >> > situations?
> >> >>> >> >>
> >> >>> >> >> More seriously it's complicated to find the best conf. If it's
> a
> >> >>> >> >> very
> >> >>> >> >> busy site, I recommand to :
> >> >>> >> >> 1- 1 or 2 servers for running nginx or any other reverse proxy
> >> >>> >> >> 2- several servers to run php-fpm and only php-fpm.
> >> >>> >> >> 3- use static or dynamic with max_spare_children =
> max_children
> >> >>> >> >> (you
> >> >>> >> >> want to be ready to handle the heavy load).
> >> >>> >> >> 4- set max_children to something which sounds great to you at
> >> >>> >> >> the
> >> >>> >> >> first place and adjust after depending on the load and the
> usage
> >> >>> >> >> of
> >> >>> >> >> the server. If you have 4GB of RAM and each php-fpm process
> uses
> >> >>> >> >> on
> >> >>> >> >> average 32MB you can set max children to 4096/32 - 10%. But
> you
> >> >>> >> >> will
> >> >>> >> >> never find the right and perfect solution at the first try.
> >> >>> >> >>
> >> >>> >> >> > Thanks
> >> >>> >> >
> >> >>> >> >
> >> >>> >
> >> >>> >
> >> >>
> >> >
> >> >
> >
> >
>

2010/10/8 Ilan Berkner <iberkner@gmail.com>:
> I understand what you mean, in terms of that its not often and it does
> appear to be working...
> Am happy to open a bug report, how / where do I open a bug report and what
> information is required?

Report here http://bugs.php.net/report.php but read first
http://bugs.php.net/how-to-report.php

> Right now I'm running as static with max children set to 100.  Do you
> recommend switching to dynamic, if so, what would be good settings for a
> busy site?  Each box has 4GB and strictly runs php-fpm, can add more RAM if
> needed, but there seems to be between 500MB and 1GB free at all times on
> theses systems, so I think I'm OK with RAM for now.

do you monitor PHP-FPM activity through its status page ? Look how
php-fpm is used in order to determine if max_children is set right.
After having several days of data you'll be able to tweak and see how
it goes ... and start again ... :)

>
> 2010/10/8 Jérôme Loyet <ml@fatbsd.com>
>>
>> 2010/10/8 Ilan Berkner <iberkner@gmail.com>:
>> > FYI - I am seeing this every so often:
>> > Oct 08 08:48:14.087065 [ERROR] pid 1549,
>> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable
>> > to
>> > retrieve process activity of one or more child(ren). Will try again
>> > later.
>> > Oct 08 10:22:01.829260 [ERROR] pid 1549,
>> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable
>> > to
>> > retrieve process activity of one or more child(ren). Will try again
>> > later.
>> > Oct 08 10:46:36.296591 [ERROR] pid 1549,
>> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable
>> > to
>> > retrieve process activity of one or more child(ren). Will try again
>> > later.
>> > Oct 08 11:53:41.484788 [ERROR] pid 1549,
>> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable
>> > to
>> > retrieve process activity of one or more child(ren). Will try again
>> > later.
>> > Oct 08 12:35:35.574208 [ERROR] pid 1549,
>> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www] unable
>> > to
>> > retrieve process activity of one or more child(ren). Will try again
>> > later.
>> >
>>
>> it's not so often, this function is called every seconds and you have
>> errors with several minutes between each. But you should open a bug
>> report so that we look for a correction.
>>
>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >>
>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >> > Could someone make sure I understand, debug will give me the highest
>> >> > level
>> >> > of error information, correct? this is what I'm looking for.
>> >> > I just got this error:
>> >> >
>> >> > Oct 07 12:41:38.287422 [ERROR] pid 27916,
>> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
>> >> > unable
>> >> > to
>> >> > retrieve process activity of one or more child(ren). Will try again
>> >> > later.
>> >> >
>> >> > What does that mean??
>> >>
>> >> it means that the parent process has not been able to retrieve
>> >> consistent informations about its children (if each child is idle or
>> >> active and if idle + active = total_children).
>> >> If the static PM is used, it just means that for once the status page
>> >> won't be up to date, so there is nothing to care about.
>> >> If the dynamic PM is used, it means that the parent won't do anything
>> >> until it has consistent informations.
>> >>
>> >> So if the errors triggers once this not a problem. If it happens every
>> >> second (the frequency of fpm_pctl_perform_idle_server_maintenance())
>> >> it might be a problem.
>> >>
>> >> >
>> >> > On Thu, Oct 7, 2010 at 1:28 PM, Jonathan Langevin
>> >> > <intel352@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> I thought notice was inclusive of debug, not the other way around?
>> >> >> Debug is a higher level of log output, whereas notice is not.
>> >> >>
>> >> >> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >> >>>
>> >> >>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >> >>> > Why would notice be more appropriate?  Right now I'm trying to
>> >> >>> > get
>> >> >>> > as
>> >> >>> > much
>> >> >>> > information out of the system as possible.  Is "debug" inclusive
>> >> >>> > of
>> >> >>> > notice?
>> >> >>>
>> >> >>> yes
>> >> >>>
>> >> >>> >
>> >> >>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >> >>> >>
>> >> >>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >> >>> >> > Thanks, I'm still trying to understand what the bottleneck is.
>> >> >>> >> > What is the highest error reporting level: debug?
>> >> >>> >>
>> >> >>> >> yes but notice should be more appropriate
>> >> >>> >>
>> >> >>> >>
>> >> >>> >> > Thanks
>> >> >>> >> >
>> >> >>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >> >>> >> >>
>> >> >>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >> >>> >> >> > Hi All,
>> >> >>> >> >> > Looking for some general guidance on php-fpm settings for a
>> >> >>> >> >> > very
>> >> >>> >> >> > busy
>> >> >>> >> >> > site
>> >> >>> >> >> > 3+mm page views a day.
>> >> >>> >> >> > We have 4 php app servers and they are getting killed
>> >> >>> >> >> > during
>> >> >>> >> >> > busy
>> >> >>> >> >> > time,
>> >> >>> >> >> > I'm
>> >> >>> >> >> > going to add 2 more, but started looking at php-fpm logs
>> >> >>> >> >> > and
>> >> >>> >> >> > seeing
>> >> >>> >> >> > things
>> >> >>> >> >> > about warnings that php-fpm is busy, that I should add more
>> >> >>> >> >> > min/max /
>> >> >>> >> >> > start
>> >> >>> >> >> > servers, so I'm sure our configuration can be improved.
>> >> >>> >> >> > Any suggestions as to:
>> >> >>> >> >> > 1) use static or dynamic?
>> >> >>> >> >>
>> >> >>> >> >> depends
>> >> >>> >> >>
>> >> >>> >> >> > 2) If static, what should max children be set to?
>> >> >>> >> >>
>> >> >>> >> >> 42 ?
>> >> >>> >> >>
>> >> >>> >> >> > 3) if dynamic, what are ideal parameters for the dynamic
>> >> >>> >> >> > stuff
>> >> >>> >> >> > for
>> >> >>> >> >> > very
>> >> >>> >> >> > busy
>> >> >>> >> >> > situations?
>> >> >>> >> >>
>> >> >>> >> >> More seriously it's complicated to find the best conf. If
>> >> >>> >> >> it's a
>> >> >>> >> >> very
>> >> >>> >> >> busy site, I recommand to :
>> >> >>> >> >> 1- 1 or 2 servers for running nginx or any other reverse
>> >> >>> >> >> proxy
>> >> >>> >> >> 2- several servers to run php-fpm and only php-fpm.
>> >> >>> >> >> 3- use static or dynamic with max_spare_children =
>> >> >>> >> >> max_children
>> >> >>> >> >> (you
>> >> >>> >> >> want to be ready to handle the heavy load).
>> >> >>> >> >> 4- set max_children to something which sounds great to you at
>> >> >>> >> >> the
>> >> >>> >> >> first place and adjust after depending on the load and the
>> >> >>> >> >> usage
>> >> >>> >> >> of
>> >> >>> >> >> the server. If you have 4GB of RAM and each php-fpm process
>> >> >>> >> >> uses
>> >> >>> >> >> on
>> >> >>> >> >> average 32MB you can set max children to 4096/32 - 10%. But
>> >> >>> >> >> you
>> >> >>> >> >> will
>> >> >>> >> >> never find the right and perfect solution at the first try.
>> >> >>> >> >>
>> >> >>> >> >> > Thanks
>> >> >>> >> >
>> >> >>> >> >
>> >> >>> >
>> >> >>> >
>> >> >>
>> >> >
>> >> >
>> >
>> >
>
>

At this time, I do not monitor php-fpm via its status page, excellent
suggestion, I'll see what I can do to turn that on.

I submitted the bug: 53028

What's a good article on how to enable the php-fpm status page via Nginx, I
remember reading about it somewhere.

Thanks

2010/10/8 Jérôme Loyet <ml@fatbsd.com>

> 2010/10/8 Ilan Berkner <iberkner@gmail.com>:
> > I understand what you mean, in terms of that its not often and it does
> > appear to be working...
> > Am happy to open a bug report, how / where do I open a bug report and
> what
> > information is required?
>
> Report here http://bugs.php.net/report.php but read first
> http://bugs.php.net/how-to-report.php
>
> > Right now I'm running as static with max children set to 100. Do you
> > recommend switching to dynamic, if so, what would be good settings for a
> > busy site? Each box has 4GB and strictly runs php-fpm, can add more RAM
> if
> > needed, but there seems to be between 500MB and 1GB free at all times on
> > theses systems, so I think I'm OK with RAM for now.
>
> do you monitor PHP-FPM activity through its status page ? Look how
> php-fpm is used in order to determine if max_children is set right.
> After having several days of data you'll be able to tweak and see how
> it goes ... and start again ... :)
>
> >
> > 2010/10/8 Jérôme Loyet <ml@fatbsd.com>
> >>
> >> 2010/10/8 Ilan Berkner <iberkner@gmail.com>:
> >> > FYI - I am seeing this every so often:
> >> > Oct 08 08:48:14.087065 [ERROR] pid 1549,
> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
> unable
> >> > to
> >> > retrieve process activity of one or more child(ren). Will try again
> >> > later.
> >> > Oct 08 10:22:01.829260 [ERROR] pid 1549,
> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
> unable
> >> > to
> >> > retrieve process activity of one or more child(ren). Will try again
> >> > later.
> >> > Oct 08 10:46:36.296591 [ERROR] pid 1549,
> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
> unable
> >> > to
> >> > retrieve process activity of one or more child(ren). Will try again
> >> > later.
> >> > Oct 08 11:53:41.484788 [ERROR] pid 1549,
> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
> unable
> >> > to
> >> > retrieve process activity of one or more child(ren). Will try again
> >> > later.
> >> > Oct 08 12:35:35.574208 [ERROR] pid 1549,
> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
> unable
> >> > to
> >> > retrieve process activity of one or more child(ren). Will try again
> >> > later.
> >> >
> >>
> >> it's not so often, this function is called every seconds and you have
> >> errors with several minutes between each. But you should open a bug
> >> report so that we look for a correction.
> >>
> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >> >>
> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >> >> > Could someone make sure I understand, debug will give me the
> highest
> >> >> > level
> >> >> > of error information, correct? this is what I'm looking for.
> >> >> > I just got this error:
> >> >> >
> >> >> > Oct 07 12:41:38.287422 [ERROR] pid 27916,
> >> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
> >> >> > unable
> >> >> > to
> >> >> > retrieve process activity of one or more child(ren). Will try again
> >> >> > later.
> >> >> >
> >> >> > What does that mean??
> >> >>
> >> >> it means that the parent process has not been able to retrieve
> >> >> consistent informations about its children (if each child is idle or
> >> >> active and if idle + active = total_children).
> >> >> If the static PM is used, it just means that for once the status page
> >> >> won't be up to date, so there is nothing to care about.
> >> >> If the dynamic PM is used, it means that the parent won't do anything
> >> >> until it has consistent informations.
> >> >>
> >> >> So if the errors triggers once this not a problem. If it happens
> every
> >> >> second (the frequency of fpm_pctl_perform_idle_server_maintenance())
> >> >> it might be a problem.
> >> >>
> >> >> >
> >> >> > On Thu, Oct 7, 2010 at 1:28 PM, Jonathan Langevin
> >> >> > <intel352@gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> I thought notice was inclusive of debug, not the other way around?
> >> >> >> Debug is a higher level of log output, whereas notice is not.
> >> >> >>
> >> >> >> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >> >> >>>
> >> >> >>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >> >> >>> > Why would notice be more appropriate? Right now I'm trying to
> >> >> >>> > get
> >> >> >>> > as
> >> >> >>> > much
> >> >> >>> > information out of the system as possible. Is "debug"
> inclusive
> >> >> >>> > of
> >> >> >>> > notice?
> >> >> >>>
> >> >> >>> yes
> >> >> >>>
> >> >> >>> >
> >> >> >>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >> >> >>> >>
> >> >> >>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >> >> >>> >> > Thanks, I'm still trying to understand what the bottleneck
> is.
> >> >> >>> >> > What is the highest error reporting level: debug?
> >> >> >>> >>
> >> >> >>> >> yes but notice should be more appropriate
> >> >> >>> >>
> >> >> >>> >>
> >> >> >>> >> > Thanks
> >> >> >>> >> >
> >> >> >>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
> >> >> >>> >> >>
> >> >> >>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
> >> >> >>> >> >> > Hi All,
> >> >> >>> >> >> > Looking for some general guidance on php-fpm settings for
> a
> >> >> >>> >> >> > very
> >> >> >>> >> >> > busy
> >> >> >>> >> >> > site
> >> >> >>> >> >> > 3+mm page views a day.
> >> >> >>> >> >> > We have 4 php app servers and they are getting killed
> >> >> >>> >> >> > during
> >> >> >>> >> >> > busy
> >> >> >>> >> >> > time,
> >> >> >>> >> >> > I'm
> >> >> >>> >> >> > going to add 2 more, but started looking at php-fpm logs
> >> >> >>> >> >> > and
> >> >> >>> >> >> > seeing
> >> >> >>> >> >> > things
> >> >> >>> >> >> > about warnings that php-fpm is busy, that I should add
> more
> >> >> >>> >> >> > min/max /
> >> >> >>> >> >> > start
> >> >> >>> >> >> > servers, so I'm sure our configuration can be improved.
> >> >> >>> >> >> > Any suggestions as to:
> >> >> >>> >> >> > 1) use static or dynamic?
> >> >> >>> >> >>
> >> >> >>> >> >> depends
> >> >> >>> >> >>
> >> >> >>> >> >> > 2) If static, what should max children be set to?
> >> >> >>> >> >>
> >> >> >>> >> >> 42 ?
> >> >> >>> >> >>
> >> >> >>> >> >> > 3) if dynamic, what are ideal parameters for the dynamic
> >> >> >>> >> >> > stuff
> >> >> >>> >> >> > for
> >> >> >>> >> >> > very
> >> >> >>> >> >> > busy
> >> >> >>> >> >> > situations?
> >> >> >>> >> >>
> >> >> >>> >> >> More seriously it's complicated to find the best conf. If
> >> >> >>> >> >> it's a
> >> >> >>> >> >> very
> >> >> >>> >> >> busy site, I recommand to :
> >> >> >>> >> >> 1- 1 or 2 servers for running nginx or any other reverse
> >> >> >>> >> >> proxy
> >> >> >>> >> >> 2- several servers to run php-fpm and only php-fpm.
> >> >> >>> >> >> 3- use static or dynamic with max_spare_children =
> >> >> >>> >> >> max_children
> >> >> >>> >> >> (you
> >> >> >>> >> >> want to be ready to handle the heavy load).
> >> >> >>> >> >> 4- set max_children to something which sounds great to you
> at
> >> >> >>> >> >> the
> >> >> >>> >> >> first place and adjust after depending on the load and the
> >> >> >>> >> >> usage
> >> >> >>> >> >> of
> >> >> >>> >> >> the server. If you have 4GB of RAM and each php-fpm process
> >> >> >>> >> >> uses
> >> >> >>> >> >> on
> >> >> >>> >> >> average 32MB you can set max children to 4096/32 - 10%. But
> >> >> >>> >> >> you
> >> >> >>> >> >> will
> >> >> >>> >> >> never find the right and perfect solution at the first try.
> >> >> >>> >> >>
> >> >> >>> >> >> > Thanks
> >> >> >>> >> >
> >> >> >>> >> >
> >> >> >>> >
> >> >> >>> >
> >> >> >>
> >> >> >
> >> >> >
> >> >
> >> >
> >
> >
>

Ok, I got status working:

accepted conn: 72 pool: www process manager: static idle processes: 99
active processes: 1 total processes: 100

which data points matter to me from the above information?

On Fri, Oct 8, 2010 at 2:42 PM, Ilan Berkner <iberkner@gmail.com> wrote:

> At this time, I do not monitor php-fpm via its status page, excellent
> suggestion, I'll see what I can do to turn that on.
>
> I submitted the bug: 53028
>
> What's a good article on how to enable the php-fpm status page via Nginx, I
> remember reading about it somewhere.
>
> Thanks
>
> 2010/10/8 Jérôme Loyet <ml@fatbsd.com>
>
>> 2010/10/8 Ilan Berkner <iberkner@gmail.com>:
>> > I understand what you mean, in terms of that its not often and it does
>> > appear to be working...
>> > Am happy to open a bug report, how / where do I open a bug report and
>> what
>> > information is required?
>>
>> Report here http://bugs.php.net/report.php but read first
>> http://bugs.php.net/how-to-report.php
>>
>> > Right now I'm running as static with max children set to 100. Do you
>> > recommend switching to dynamic, if so, what would be good settings for a
>> > busy site? Each box has 4GB and strictly runs php-fpm, can add more RAM
>> if
>> > needed, but there seems to be between 500MB and 1GB free at all times on
>> > theses systems, so I think I'm OK with RAM for now.
>>
>> do you monitor PHP-FPM activity through its status page ? Look how
>> php-fpm is used in order to determine if max_children is set right.
>> After having several days of data you'll be able to tweak and see how
>> it goes ... and start again ... :)
>>
>> >
>> > 2010/10/8 Jérôme Loyet <ml@fatbsd.com>
>> >>
>> >> 2010/10/8 Ilan Berkner <iberkner@gmail.com>:
>> >> > FYI - I am seeing this every so often:
>> >> > Oct 08 08:48:14.087065 [ERROR] pid 1549,
>> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
>> unable
>> >> > to
>> >> > retrieve process activity of one or more child(ren). Will try again
>> >> > later.
>> >> > Oct 08 10:22:01.829260 [ERROR] pid 1549,
>> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
>> unable
>> >> > to
>> >> > retrieve process activity of one or more child(ren). Will try again
>> >> > later.
>> >> > Oct 08 10:46:36.296591 [ERROR] pid 1549,
>> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
>> unable
>> >> > to
>> >> > retrieve process activity of one or more child(ren). Will try again
>> >> > later.
>> >> > Oct 08 11:53:41.484788 [ERROR] pid 1549,
>> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
>> unable
>> >> > to
>> >> > retrieve process activity of one or more child(ren). Will try again
>> >> > later.
>> >> > Oct 08 12:35:35.574208 [ERROR] pid 1549,
>> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
>> unable
>> >> > to
>> >> > retrieve process activity of one or more child(ren). Will try again
>> >> > later.
>> >> >
>> >>
>> >> it's not so often, this function is called every seconds and you have
>> >> errors with several minutes between each. But you should open a bug
>> >> report so that we look for a correction.
>> >>
>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >> >>
>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >> >> > Could someone make sure I understand, debug will give me the
>> highest
>> >> >> > level
>> >> >> > of error information, correct? this is what I'm looking for.
>> >> >> > I just got this error:
>> >> >> >
>> >> >> > Oct 07 12:41:38.287422 [ERROR] pid 27916,
>> >> >> > fpm_pctl_perform_idle_server_maintenance(), line 347: [pool www]
>> >> >> > unable
>> >> >> > to
>> >> >> > retrieve process activity of one or more child(ren). Will try
>> again
>> >> >> > later.
>> >> >> >
>> >> >> > What does that mean??
>> >> >>
>> >> >> it means that the parent process has not been able to retrieve
>> >> >> consistent informations about its children (if each child is idle or
>> >> >> active and if idle + active = total_children).
>> >> >> If the static PM is used, it just means that for once the status
>> page
>> >> >> won't be up to date, so there is nothing to care about.
>> >> >> If the dynamic PM is used, it means that the parent won't do
>> anything
>> >> >> until it has consistent informations.
>> >> >>
>> >> >> So if the errors triggers once this not a problem. If it happens
>> every
>> >> >> second (the frequency of fpm_pctl_perform_idle_server_maintenance())
>> >> >> it might be a problem.
>> >> >>
>> >> >> >
>> >> >> > On Thu, Oct 7, 2010 at 1:28 PM, Jonathan Langevin
>> >> >> > <intel352@gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> I thought notice was inclusive of debug, not the other way
>> around?
>> >> >> >> Debug is a higher level of log output, whereas notice is not.
>> >> >> >>
>> >> >> >> 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >> >> >>>
>> >> >> >>> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >> >> >>> > Why would notice be more appropriate? Right now I'm trying to
>> >> >> >>> > get
>> >> >> >>> > as
>> >> >> >>> > much
>> >> >> >>> > information out of the system as possible. Is "debug"
>> inclusive
>> >> >> >>> > of
>> >> >> >>> > notice?
>> >> >> >>>
>> >> >> >>> yes
>> >> >> >>>
>> >> >> >>> >
>> >> >> >>> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >> >> >>> >>
>> >> >> >>> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >> >> >>> >> > Thanks, I'm still trying to understand what the bottleneck
>> is.
>> >> >> >>> >> > What is the highest error reporting level: debug?
>> >> >> >>> >>
>> >> >> >>> >> yes but notice should be more appropriate
>> >> >> >>> >>
>> >> >> >>> >>
>> >> >> >>> >> > Thanks
>> >> >> >>> >> >
>> >> >> >>> >> > 2010/10/7 Jérôme Loyet <ml@fatbsd.com>
>> >> >> >>> >> >>
>> >> >> >>> >> >> 2010/10/7 Ilan Berkner <iberkner@gmail.com>:
>> >> >> >>> >> >> > Hi All,
>> >> >> >>> >> >> > Looking for some general guidance on php-fpm settings
>> for a
>> >> >> >>> >> >> > very
>> >> >> >>> >> >> > busy
>> >> >> >>> >> >> > site
>> >> >> >>> >> >> > 3+mm page views a day.
>> >> >> >>> >> >> > We have 4 php app servers and they are getting killed
>> >> >> >>> >> >> > during
>> >> >> >>> >> >> > busy
>> >> >> >>> >> >> > time,
>> >> >> >>> >> >> > I'm
>> >> >> >>> >> >> > going to add 2 more, but started looking at php-fpm logs
>> >> >> >>> >> >> > and
>> >> >> >>> >> >> > seeing
>> >> >> >>> >> >> > things
>> >> >> >>> >> >> > about warnings that php-fpm is busy, that I should add
>> more
>> >> >> >>> >> >> > min/max /
>> >> >> >>> >> >> > start
>> >> >> >>> >> >> > servers, so I'm sure our configuration can be improved..
>> >> >> >>> >> >> > Any suggestions as to:
>> >> >> >>> >> >> > 1) use static or dynamic?
>> >> >> >>> >> >>
>> >> >> >>> >> >> depends
>> >> >> >>> >> >>
>> >> >> >>> >> >> > 2) If static, what should max children be set to?
>> >> >> >>> >> >>
>> >> >> >>> >> >> 42 ?
>> >> >> >>> >> >>
>> >> >> >>> >> >> > 3) if dynamic, what are ideal parameters for the dynamic
>> >> >> >>> >> >> > stuff
>> >> >> >>> >> >> > for
>> >> >> >>> >> >> > very
>> >> >> >>> >> >> > busy
>> >> >> >>> >> >> > situations?
>> >> >> >>> >> >>
>> >> >> >>> >> >> More seriously it's complicated to find the best conf. If
>> >> >> >>> >> >> it's a
>> >> >> >>> >> >> very
>> >> >> >>> >> >> busy site, I recommand to :
>> >> >> >>> >> >> 1- 1 or 2 servers for running nginx or any other reverse
>> >> >> >>> >> >> proxy
>> >> >> >>> >> >> 2- several servers to run php-fpm and only php-fpm.
>> >> >> >>> >> >> 3- use static or dynamic with max_spare_children =
>> >> >> >>> >> >> max_children
>> >> >> >>> >> >> (you
>> >> >> >>> >> >> want to be ready to handle the heavy load).
>> >> >> >>> >> >> 4- set max_children to something which sounds great to you
>> at
>> >> >> >>> >> >> the
>> >> >> >>> >> >> first place and adjust after depending on the load and the
>> >> >> >>> >> >> usage
>> >> >> >>> >> >> of
>> >> >> >>> >> >> the server. If you have 4GB of RAM and each php-fpm
>> process
>> >> >> >>> >> >> uses
>> >> >> >>> >> >> on
>> >> >> >>> >> >> average 32MB you can set max children to 4096/32 - 10%.
>> But
>> >> >> >>> >> >> you
>> >> >> >>> >> >> will
>> >> >> >>> >> >> never find the right and perfect solution at the first
>> try.
>> >> >> >>> >> >>
>> >> >> >>> >> >> > Thanks
>> >> >> >>> >> >
>> >> >> >>> >> >
>> >> >> >>> >
>> >> >> >>> >
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >
>> >> >
>> >
>> >
>>
>
>