Hi

I manage an associative hosting and I think to use PHP-FPM.
We have 700 websites in our server today.
I ask me a few questions (like pm.start_servers). Some websites have
very few visitors. I dont want overloading the server with a bad
configuration.
Can you help me for define a website configuration example ?

Thanks
Romain.

Hello Romain,

This is, currently, this actual problem with FPM : Shared hosting
environment.

There is a patch called "ondemand mode" that you can find somewhere on
the PHP bugtracker, which allows you to specify pm.start_servers=0

Without using this patch, there is no actual solution for you, indeed,
700 pools, even if you specify start_servers=1, it will be 700 PHP
processes ! I hope you have a 32GB RAM server... !

The other solution is to run several website in the same pool, or even,
all website in the same pool... But this is not, imho, an actual
solution because it's not secured at all and not the way it is designed
to be used.

If you have some programming skills, I encourage you to take part in the
development of the "ondemand" patch. It has been tested and was, if I
remember well, nearly finished, but I think it was for an old FPM
version (in comparision to the current version) so the main task would
be to adapt it to the last stable FPM version.


Hope it'll help you.

Regards,
Troll

On 07/08/2011 03:45 PM, Romain wrote:
> Hi
>
> I manage an associative hosting and I think to use PHP-FPM.
> We have 700 websites in our server today.
> I ask me a few questions (like pm.start_servers). Some websites have
> very few visitors. I dont want overloading the server with a bad
> configuration.
> Can you help me for define a website configuration example ?
>
> Thanks
> Romain.

>
>
>
> If you have some programming skills, I encourage you to take part in the
> development of the "ondemand" patch. It has been tested and was, if I
> remember well, nearly finished, but I think it was for an old FPM version
> (in comparision to the current version) so the main task would be to adapt
> it to the last stable FPM version.
>
>
Do you have a link to the ondemand patch?

Cheers,

--
David Coallier,
CTO Orchestra Platform Ltd, https://orchestra.io
@davidcoallier, skype://david.coallier

> Hello Romain,
>
> This is, currently, this actual problem with FPM : Shared hosting
> environment.
>
> There is a patch called "ondemand mode" that you can find somewhere on
> the PHP bugtracker, which allows you to specify pm.start_servers=0
>
> Without using this patch, there is no actual solution for you, indeed,
> 700 pools, even if you specify start_servers=1, it will be 700 PHP
> processes ! I hope you have a 32GB RAM server... !
>

Indeed, I tried this once - and it's a disaster.

I *love* PHP-FPM, but it's not useful for situations with lots of
different pools on one host. Currently.

But what else is there?
FastCGI is OK, but needs open_basedir and thus is slow.

mod_php is all with the same user..


What do people use for a server with 500 vhosts?

CGI/FastCGI with up to 1000 vhosts : http://www.legtux.org/phpinfo.php
It works better than mod_php as administrators.

Romain

Le 08/07/2011 16:34, rainer@ultra-secure.de a écrit :
>> Hello Romain,
>>
>> This is, currently, this actual problem with FPM : Shared hosting
>> environment.
>>
>> There is a patch called "ondemand mode" that you can find somewhere on
>> the PHP bugtracker, which allows you to specify pm.start_servers=0
>>
>> Without using this patch, there is no actual solution for you, indeed,
>> 700 pools, even if you specify start_servers=1, it will be 700 PHP
>> processes ! I hope you have a 32GB RAM server... !
>>
> Indeed, I tried this once - and it's a disaster.
>
> I *love* PHP-FPM, but it's not useful for situations with lots of
> different pools on one host. Currently.
>
> But what else is there?
> FastCGI is OK, but needs open_basedir and thus is slow.
>
> mod_php is all with the same user..
>
>
> What do people use for a server with 500 vhosts?
>
>
>
>

Hi,

Thanks you for your help. I will do research on "ondemand".

For resume, today with the last stable FPM version, it is not possible
to used for several websites.

Therefore we must used mod_php in prefork configuration for Apache ? No
other solution ? FastCGI is realy slow ?

Thanks
Romain.


Le 08/07/2011 16:21, Troll a écrit :
> Hello Romain,
>
> This is, currently, this actual problem with FPM : Shared hosting
> environment.
>
> There is a patch called "ondemand mode" that you can find somewhere on
> the PHP bugtracker, which allows you to specify pm.start_servers=0
>
> Without using this patch, there is no actual solution for you, indeed,
> 700 pools, even if you specify start_servers=1, it will be 700 PHP
> processes ! I hope you have a 32GB RAM server... !
>
> The other solution is to run several website in the same pool, or
> even, all website in the same pool... But this is not, imho, an actual
> solution because it's not secured at all and not the way it is
> designed to be used.
>
> If you have some programming skills, I encourage you to take part in
> the development of the "ondemand" patch. It has been tested and was,
> if I remember well, nearly finished, but I think it was for an old FPM
> version (in comparision to the current version) so the main task would
> be to adapt it to the last stable FPM version.
>
>
> Hope it'll help you.
>
> Regards,
> Troll
>
> On 07/08/2011 03:45 PM, Romain wrote:
>> Hi
>>
>> I manage an associative hosting and I think to use PHP-FPM.
>> We have 700 websites in our server today.
>> I ask me a few questions (like pm.start_servers). Some websites have
>> very few visitors. I dont want overloading the server with a bad
>> configuration.
>> Can you help me for define a website configuration example ?
>>
>> Thanks
>> Romain.

> FastCGI is realy slow ?
FPM IS FastCgi ! "F.P.M." stands for "FastCGI Process Manager"

CGI is really slow. FastCGI, as its name indicates... Is not slow. CGI
is slow for the simple reason you have to completely launch PHP and all
its libraries & etc. ... whenever FastCGI launches "hanging" PHP
processes that are waiting for requests to spend their time doing
something more useful than "hanging" in memory :)

But this is the very fact why there is currently a problem with running
many websites with FastCGI : it's memory consuming if you have so many
hanging processes waiting for queries (as a php process is ~ 15MB, and
even more when parsing a wordpress website ! (~35MB !) ).


(note : "hanging" here does not mean the unix "hang" signal, huh, it
just means hanging as anything in real life)

If you find anything useful about the "ondemand" patch to adapt to newer
versions, or if you find an alternative solution, don't hesitate to post
it on the list.

I am personally interested in it, and I am almost certain many others too.

Regards,


Troll

On 07/08/2011 10:43 PM, Romain wrote:
> Hi,
>
> Thanks you for your help. I will do research on "ondemand".
>
> For resume, today with the last stable FPM version, it is not possible
> to used for several websites.
>
> Therefore we must used mod_php in prefork configuration for Apache ?
> No other solution ? FastCGI is realy slow ?
>
> Thanks
> Romain.
>
>
> Le 08/07/2011 16:21, Troll a écrit :
>> Hello Romain,
>>
>> This is, currently, this actual problem with FPM : Shared hosting
>> environment.
>>
>> There is a patch called "ondemand mode" that you can find somewhere
>> on the PHP bugtracker, which allows you to specify pm.start_servers=0
>>
>> Without using this patch, there is no actual solution for you,
>> indeed, 700 pools, even if you specify start_servers=1, it will be
>> 700 PHP processes ! I hope you have a 32GB RAM server... !
>>
>> The other solution is to run several website in the same pool, or
>> even, all website in the same pool... But this is not, imho, an
>> actual solution because it's not secured at all and not the way it is
>> designed to be used.
>>
>> If you have some programming skills, I encourage you to take part in
>> the development of the "ondemand" patch. It has been tested and was,
>> if I remember well, nearly finished, but I think it was for an old
>> FPM version (in comparision to the current version) so the main task
>> would be to adapt it to the last stable FPM version.
>>
>>
>> Hope it'll help you.
>>
>> Regards,
>> Troll
>>
>> On 07/08/2011 03:45 PM, Romain wrote:
>>> Hi
>>>
>>> I manage an associative hosting and I think to use PHP-FPM.
>>> We have 700 websites in our server today.
>>> I ask me a few questions (like pm.start_servers). Some websites have
>>> very few visitors. I dont want overloading the server with a bad
>>> configuration.
>>> Can you help me for define a website configuration example ?
>>>
>>> Thanks
>>> Romain.

2011/7/8 David Coallier <david@orchestra.io>:
>>
>>
>> If you have some programming skills, I encourage you to take part in the
>> development of the "ondemand" patch. It has been tested and was, if I
>> remember well, nearly finished, but I think it was for an old FPM version
>> (in comparision to the current version) so the main task would be to adapt
>> it to the last stable FPM version.
>>
>
> Do you have a link to the ondemand patch?

yes! https://bugs.php.net/bug.php?id=52569

the patch have been updated for the last sources 2 days ago. You're
welcome to test it (even if there's still a bug I have to figure out).

> Cheers,
>
> --
> David Coallier,
> CTO Orchestra Platform Ltd, https://orchestra.io
> @davidcoallier, skype://david.coallier
>

for the record,

I've just submit a small fix to the patch in order to correct the known bug..

++ jerome

PS: the patch can be found in https://bugs.php.net/bug.php?id=52569.

Also note it should be applied on sources fetch from SVN:
fpm-ondemand.v8.patch on svn.php.net/viewvc/php/php-src/branches/PHP_5_4
fpm-ondemand.v8-5.3.patch on svn.php.net/viewvc/php/php-src/branches/PHP_5_3


Le 9 juillet 2011 01:20, Jérôme Loyet <ml@fatbsd.com> a écrit :
> 2011/7/8 David Coallier <david@orchestra.io>:
>>>
>>>
>>> If you have some programming skills, I encourage you to take part in the
>>> development of the "ondemand" patch. It has been tested and was, if I
>>> remember well, nearly finished, but I think it was for an old FPM version
>>> (in comparision to the current version) so the main task would be to adapt
>>> it to the last stable FPM version.
>>>
>>
>> Do you have a link to the ondemand patch?
>
> yes! https://bugs.php.net/bug.php?id=52569
>
> the patch have been updated for the last sources 2 days ago. You're
> welcome to test it (even if there's still a bug I have to figure out).
>
>> Cheers,
>>
>> --
>> David Coallier,
>> CTO Orchestra Platform Ltd, https://orchestra.io
>> @davidcoallier, skype://david.coallier
>>
>

The patch has now been tested & validated ?

It's been a long time this feature request/patch has been waiting for an
interest (or maybe simply spare time) from the main tree developers, do
you know why it has not been integrated yet in the main tree ?

Everyone knows that this feature is expected from many many users in
order to deploy FPM in shared hosting environment, I can't understand
why it has not been put onto the priority list.


By the way :

> CGI/FastCGI with up to 1000 vhosts : http://www.legtux.org/phpinfo.php
> It works better than mod_php as administrators.
>
> Romain
Yes, but certainly not using a single pool / website. Maybe using only a
couple of pools with each of them handling hundreds of website (which
then is similar to the use of mod_php, in a certain way). Or maybe using
ondemand patch :)

If the patch has been updated to the last sources, I'll certainly test
it. I have to update my (compiled from sources) php version.

But does the code directly taken from the SVN repository is stable enough ?


Regards,
Troll


On 07/09/2011 02:26 AM, Jérôme Loyet wrote:
> for the record,
>
> I've just submit a small fix to the patch in order to correct the known bug.
>
> ++ jerome
>
> PS: the patch can be found in https://bugs.php.net/bug.php?id=52569.
>
> Also note it should be applied on sources fetch from SVN:
> fpm-ondemand.v8.patch on svn.php.net/viewvc/php/php-src/branches/PHP_5_4
> fpm-ondemand.v8-5.3.patch on svn.php.net/viewvc/php/php-src/branches/PHP_5_3
>
>
> Le 9 juillet 2011 01:20, Jérôme Loyet<ml@fatbsd.com> a écrit :
>> 2011/7/8 David Coallier<david@orchestra.io>:
>>>>
>>>> If you have some programming skills, I encourage you to take part in the
>>>> development of the "ondemand" patch. It has been tested and was, if I
>>>> remember well, nearly finished, but I think it was for an old FPM version
>>>> (in comparision to the current version) so the main task would be to adapt
>>>> it to the last stable FPM version.
>>>>
>>> Do you have a link to the ondemand patch?
>> yes! https://bugs.php.net/bug.php?id=52569
>>
>> the patch have been updated for the last sources 2 days ago. You're
>> welcome to test it (even if there's still a bug I have to figure out).
>>
>>> Cheers,
>>>
>>> --
>>> David Coallier,
>>> CTO Orchestra Platform Ltd, https://orchestra.io
>>> @davidcoallier, skype://david.coallier
>>>

Hi thanks for info,
can you describe how it works please - does it changes environment/uid
for the current processes, or it just starts the new process as CGI
when nginx gets a request?
For the latter case, it may be easier just to set php as CGI, for the
large number of sites when each requests starts a new process there
won't be a benefit from FPM.

Grigori

On 9 июл, 03:26, Jérôme Loyet <m...@fatbsd.com> wrote:
> for the record,
>
> I've just submit a small fix to the patch in order to correct the known bug.
>
> ++ jerome
>
> PS: the patch can be found inhttps://bugs.php.net/bug.php?id=52569.

No, setting up CGI is not the same. Your CGI process will end once the
query is over (stop me if I'm wrong). Not your fastcgi process, as I
explained in a previous reply.

To make it simple, "ondemand" mode is just the same as "dynamic" mode
but you can state start_servers=0 instead of a minimum start_servers=1
with dynamic mode.

Regards,
Troll

On 07/09/2011 10:29 AM, grigori wrote:
> Hi thanks for info,
> can you describe how it works please - does it changes environment/uid
> for the current processes, or it just starts the new process as CGI
> when nginx gets a request?
> For the latter case, it may be easier just to set php as CGI, for the
> large number of sites when each requests starts a new process there
> won't be a benefit from FPM.
>
> Grigori
>
> On 9 июл, 03:26, Jérôme Loyet<m...@fatbsd.com> wrote:
>> for the record,
>>
>> I've just submit a small fix to the patch in order to correct the known bug.
>>
>> ++ jerome
>>
>> PS: the patch can be found inhttps://bugs.php.net/bug.php?id=52569.

2011/7/9 Troll <trollofdarkness@gmail.com>:
> The patch has now been tested & validated ?

I've tested it on my own and it needs to be tesed in order to be validated.

>
> It's been a long time this feature request/patch has been waiting for an
> interest (or maybe simply spare time) from the main tree developers, do you
> know why it has not been integrated yet in the main tree ?

simply because it's not been tested. Help us testing it (and fix it if
needed) and it integration into the sources will be much faster.

>
> Everyone knows that this feature is expected from many many users in order
> to deploy FPM in shared hosting environment, I can't understand why it has
> not been put onto the priority list.

It's all a matter of time. Don't forget it's opensource and based on
voluntary participation

>
>
> By the way :
>
> CGI/FastCGI with up to 1000 vhosts : http://www.legtux.org/phpinfo.php
> It works better than mod_php as administrators.
>
> Romain
>
> Yes, but certainly not using a single pool / website. Maybe using only a
> couple of pools with each of them handling hundreds of website (which then
> is similar to the use of mod_php, in a certain way). Or maybe using ondemand
> patch :)
>
> If the patch has been updated to the last sources, I'll certainly test it.. I
> have to update my (compiled from sources) php version.
>
> But does the code directly taken from the SVN repository is stable enough ?
>
>
> Regards,
> Troll
>
>
> On 07/09/2011 02:26 AM, Jérôme Loyet wrote:
>
> for the record,
>
> I've just submit a small fix to the patch in order to correct the known bug.
>
> ++ jerome
>
> PS: the patch can be found in https://bugs.php.net/bug.php?id=52569.
>
> Also note it should be applied on sources fetch from SVN:
> fpm-ondemand.v8.patch on svn.php.net/viewvc/php/php-src/branches/PHP_5_4
> fpm-ondemand.v8-5.3.patch on
> svn.php.net/viewvc/php/php-src/branches/PHP_5_3
>
>
> Le 9 juillet 2011 01:20, Jérôme Loyet <ml@fatbsd.com> a écrit :
>
> 2011/7/8 David Coallier <david@orchestra.io>:
>
> If you have some programming skills, I encourage you to take part in the
> development of the "ondemand" patch. It has been tested and was, if I
> remember well, nearly finished, but I think it was for an old FPM version
> (in comparision to the current version) so the main task would be to adapt
> it to the last stable FPM version.
>
> Do you have a link to the ondemand patch?
>
> yes! https://bugs.php.net/bug.php?id=52569
>
> the patch have been updated for the last sources 2 days ago. You're
> welcome to test it (even if there's still a bug I have to figure out).
>
> Cheers,
>
> --
> David Coallier,
> CTO Orchestra Platform Ltd, https://orchestra.io
> @davidcoallier, skype://david.coallier
>
>

2011/7/9 Troll <trollofdarkness@gmail.com>:
> No, setting up CGI is not the same. Your CGI process will end once the query
> is over (stop me if I'm wrong). Not your fastcgi process, as I explained in
> a previous reply.

You are right. CGI process are spawn for each request and can only
served one request. FCGI has been invented to handle this "bug". FCGI
is a loop version of CGI. It means that a process launched once and
serves many requests sequentially.

>
> To make it simple, "ondemand" mode is just the same as "dynamic" mode but
> you can state start_servers=0 instead of a minimum start_servers=1 with
> dynamic mode.

The "ondemand" mode can have from 0 process to max_children. When a
request arrives, it no process is available (0 process or all
processes are serving a request), a new process is created in order to
serve the incoming request. After then, processes stays awake to
server the next comming requests for a defined amount of time
(pm.process_idle_timeout parameter). For exemple, if a process has
done nothing in the last 30s, it's killed.:

1- The request R1 arrives
2- "ondemand" PM forks the process P1
3- the process P1 serves the request P1
4- 3s waiting
5- The request R2 arrives
6- the process P1 serves the request P2
7- 30s waiting
8- the process P1 is killed
9-wait 10 minutes
10- go to step 1

To sum up, the "ondemand" mode is almost the same concept as CGI but
it anticipates incoming requests by not stopping processes right after
it serves its first request. It's optimization. And generaly, when a
request arrives, there good chances that anothers will come in the
near future (someone who is browsing the main page, will certainly
browse sub pages serveral seconds after).

hope it answers your questions

++ Jerome

>
> Regards,
> Troll
>
> On 07/09/2011 10:29 AM, grigori wrote:
>>
>> Hi thanks for info,
>> can you describe how it works please - does it changes environment/uid
>> for the current processes, or it just starts the new process as CGI
>> when nginx gets a request?
>> For the latter case, it may be easier just to set php as CGI, for the
>> large number of sites when each requests starts a new process there
>> won't be a benefit from FPM.
>>
>> Grigori
>>
>> On 9 июл, 03:26, Jérôme Loyet<m...@fatbsd.com>  wrote:
>>>
>>> for the record,
>>>
>>> I've just submit a small fix to the patch in order to correct the known
>>> bug.
>>>
>>> ++ jerome
>>>
>>> PS: the patch can be found inhttps://bugs.php.net/bug.php?id=52569.
>

> It's all a matter of time. Don't forget it's opensource and based on
> voluntary participation
Yes, I am fully aware of that. But there has been many developments on
FPM since the time "ondemand" patch has been released for the first
time, what I was saying is that I think FPM developers should try to
integrate this feature before other ones...


By the way, that remembers me that I've never posted my small patch
(used on my own server for month without any problems) for handling
overall number of processes... I should do it that day, before I forget !

Regards,
Troll

On 07/09/2011 12:21 PM, Jérôme Loyet wrote:
> 2011/7/9 Troll<trollofdarkness@gmail.com>:
>> The patch has now been tested& validated ?
> I've tested it on my own and it needs to be tesed in order to be validated.
>
>> It's been a long time this feature request/patch has been waiting for an
>> interest (or maybe simply spare time) from the main tree developers, do you
>> know why it has not been integrated yet in the main tree ?
> simply because it's not been tested. Help us testing it (and fix it if
> needed) and it integration into the sources will be much faster.
>
>> Everyone knows that this feature is expected from many many users in order
>> to deploy FPM in shared hosting environment, I can't understand why it has
>> not been put onto the priority list.
> It's all a matter of time. Don't forget it's opensource and based on
> voluntary participation
>
>>
>> By the way :
>>
>> CGI/FastCGI with up to 1000 vhosts : http://www.legtux.org/phpinfo.php
>> It works better than mod_php as administrators.
>>
>> Romain
>>
>> Yes, but certainly not using a single pool / website. Maybe using only a
>> couple of pools with each of them handling hundreds of website (which then
>> is similar to the use of mod_php, in a certain way). Or maybe using ondemand
>> patch :)
>>
>> If the patch has been updated to the last sources, I'll certainly test it. I
>> have to update my (compiled from sources) php version.
>>
>> But does the code directly taken from the SVN repository is stable enough ?
>>
>>
>> Regards,
>> Troll
>>
>>
>> On 07/09/2011 02:26 AM, Jérôme Loyet wrote:
>>
>> for the record,
>>
>> I've just submit a small fix to the patch in order to correct the known bug.
>>
>> ++ jerome
>>
>> PS: the patch can be found in https://bugs.php.net/bug.php?id=52569.
>>
>> Also note it should be applied on sources fetch from SVN:
>> fpm-ondemand.v8.patch on svn.php.net/viewvc/php/php-src/branches/PHP_5_4
>> fpm-ondemand.v8-5.3.patch on
>> svn.php.net/viewvc/php/php-src/branches/PHP_5_3
>>
>>
>> Le 9 juillet 2011 01:20, Jérôme Loyet<ml@fatbsd.com> a écrit :
>>
>> 2011/7/8 David Coallier<david@orchestra.io>:
>>
>> If you have some programming skills, I encourage you to take part in the
>> development of the "ondemand" patch. It has been tested and was, if I
>> remember well, nearly finished, but I think it was for an old FPM version
>> (in comparision to the current version) so the main task would be to adapt
>> it to the last stable FPM version.
>>
>> Do you have a link to the ondemand patch?
>>
>> yes! https://bugs.php.net/bug.php?id=52569
>>
>> the patch have been updated for the last sources 2 days ago. You're
>> welcome to test it (even if there's still a bug I have to figure out).
>>
>> Cheers,
>>
>> --
>> David Coallier,
>> CTO Orchestra Platform Ltd, https://orchestra.io
>> @davidcoallier, skype://david.coallier
>>
>>

So, for 700 sites from which 200 are accessed once per minute, we'll
have 200 process starts/ends per minute.

For 200 sites accessed 3 times per minute, we'll need
200 resident processes (+ all other starting/ending).

If we set the max_child 100 (eating 3Gb RAM), what happends?
Will we have starts/ends for each second request to be searved with a
right environment?
Or will the new request wait for 30 sec till some process expires to
have another born?
Or will it spawn 500th process into swap?

There does not seem to be an easy answer to the TS situation.


On 9 июл, 13:32, Jérôme Loyet <m...@fatbsd.com> wrote:
> 2011/7/9 Troll <trollofdarkn...@gmail.com>:
>
> > No, setting up CGI is not the same. Your CGI process will end once the query
> > is over (stop me if I'm wrong). Not your fastcgi process, as I explained in
> > a previous reply.
>
> You are right. CGI process are spawn for each request and can only
> served one request. FCGI has been invented to handle this "bug". FCGI
> is a loop version of CGI. It means that a process launched once and
> serves many requests sequentially.
>
>
>
> > To make it simple, "ondemand" mode is just the same as "dynamic" mode but
> > you can state start_servers=0 instead of a minimum start_servers=1 with
> > dynamic mode.
>
> The "ondemand" mode can have from 0 process to max_children. When a
> request arrives, it no process is available (0 process or all
> processes are serving a request), a new process is created in order to
> serve the incoming request. After then, processes stays awake to
> server the next comming requests for a defined amount of time
> (pm.process_idle_timeout parameter). For exemple, if a process has
> done nothing in the last 30s, it's killed.:
>
> 1- The request R1 arrives
> 2- "ondemand" PM forks the process P1
> 3- the process P1 serves the request P1
> 4- 3s waiting
> 5- The request R2 arrives
> 6- the process P1 serves the request P2
> 7- 30s waiting
> 8- the process P1 is killed
> 9-wait 10 minutes
> 10- go to step 1
>
> To sum up, the "ondemand" mode is almost the same concept as CGI but
> it anticipates incoming requests by not stopping processes right after
> it serves its first request. It's optimization. And generaly, when a
> request arrives, there good chances that anothers will come in the
> near future (someone who is browsing the main page, will certainly
> browse sub pages serveral seconds after).
>
> hope it answers your questions
>
> ++ Jerome
>
>
>
>
>
>
>
>
>
> > Regards,
> > Troll
>
> > On 07/09/2011 10:29 AM, grigori wrote:
>
> >> Hi thanks for info,
> >> can you describe how it works please - does it changes environment/uid
> >> for the current processes, or it just starts the new process as CGI
> >> when nginx gets a request?
> >> For the latter case, it may be easier just to set php as CGI, for the
> >> large number of sites when each requests starts a new process there
> >> won't be a benefit from FPM.
>
> >> Grigori
>
> >> On 9 июл, 03:26, Jérôme Loyet<m...@fatbsd.com>  wrote:
>
> >>> for the record,
>
> >>> I've just submit a small fix to the patch in order to correct the known
> >>> bug.
>
> >>> ++ jerome
>
> >>> PS: the patch can be found inhttps://bugs.php.net/bug.php?id=52569.

By the way, about testing the patch, is the code directly taken from the
SVN repository (svn.php.net/viewvc/php/php-src/branches/PHP_5_3) stable
enough ?

I'd like to test it & help its development, but currently don't have any
test server, only a production one :)

Regards,
Troll

On 07/09/2011 12:21 PM, Jérôme Loyet wrote:
> 2011/7/9 Troll<trollofdarkness@gmail.com>:
>> The patch has now been tested& validated ?
> I've tested it on my own and it needs to be tesed in order to be validated.
>
>> It's been a long time this feature request/patch has been waiting for an
>> interest (or maybe simply spare time) from the main tree developers, do you
>> know why it has not been integrated yet in the main tree ?
> simply because it's not been tested. Help us testing it (and fix it if
> needed) and it integration into the sources will be much faster.
>
>> Everyone knows that this feature is expected from many many users in order
>> to deploy FPM in shared hosting environment, I can't understand why it has
>> not been put onto the priority list.
> It's all a matter of time. Don't forget it's opensource and based on
> voluntary participation
>
>>
>> By the way :
>>
>> CGI/FastCGI with up to 1000 vhosts : http://www.legtux.org/phpinfo.php
>> It works better than mod_php as administrators.
>>
>> Romain
>>
>> Yes, but certainly not using a single pool / website. Maybe using only a
>> couple of pools with each of them handling hundreds of website (which then
>> is similar to the use of mod_php, in a certain way). Or maybe using ondemand
>> patch :)
>>
>> If the patch has been updated to the last sources, I'll certainly test it. I
>> have to update my (compiled from sources) php version.
>>
>> But does the code directly taken from the SVN repository is stable enough ?
>>
>>
>> Regards,
>> Troll
>>
>>
>> On 07/09/2011 02:26 AM, Jérôme Loyet wrote:
>>
>> for the record,
>>
>> I've just submit a small fix to the patch in order to correct the known bug.
>>
>> ++ jerome
>>
>> PS: the patch can be found in https://bugs.php.net/bug.php?id=52569.
>>
>> Also note it should be applied on sources fetch from SVN:
>> fpm-ondemand.v8.patch on svn.php.net/viewvc/php/php-src/branches/PHP_5_4
>> fpm-ondemand.v8-5.3.patch on
>> svn.php.net/viewvc/php/php-src/branches/PHP_5_3
>>
>>
>> Le 9 juillet 2011 01:20, Jérôme Loyet<ml@fatbsd.com> a écrit :
>>
>> 2011/7/8 David Coallier<david@orchestra.io>:
>>
>> If you have some programming skills, I encourage you to take part in the
>> development of the "ondemand" patch. It has been tested and was, if I
>> remember well, nearly finished, but I think it was for an old FPM version
>> (in comparision to the current version) so the main task would be to adapt
>> it to the last stable FPM version.
>>
>> Do you have a link to the ondemand patch?
>>
>> yes! https://bugs.php.net/bug.php?id=52569
>>
>> the patch have been updated for the last sources 2 days ago. You're
>> welcome to test it (even if there's still a bug I have to figure out).
>>
>> Cheers,
>>
>> --
>> David Coallier,
>> CTO Orchestra Platform Ltd, https://orchestra.io
>> @davidcoallier, skype://david.coallier
>>
>>

If you have 200 websites accessed 1 time/minute, it will depend on your
configuration :

expiration time < 1 min -> processes will die and then be reborn next minute
expiration time > 1 min -> processes will never end because they will
receive a request before the expiration time, and it will reset the
expiration time (since we're talking here about spare time).

So in any case, imho, you will get a high number of php processes in
memory. But try to do that with CGI and I think your server will just
fall down (the hard disk at first maybe) since it would be launching
around 200 new process more or less constantly.

If we compare to the behaviour mod_php would have : if max_client < 200,
then some requests will queue, the response time will be higher than
with a fpm use, but less memory would be used (not really sure in fact,
because if we then compare apache vs. nginx, you will in fact get 200
apache2 process which will get each own some memoery, whereas nginx is
totally able to handle 200 requests with a single process (at most two)
which will consume less memory (I don't remember my test figures but it
was about 20MB at max if I remember well, something like that).
if max_client > 200, you'll certainly get as many apache2 process as you
would have got of php processes.

Maybe I'm wrong, but I think this is what would happen in such a
situation. :)

Regards,
Troll

On 07/09/2011 01:42 PM, grigori wrote:
> So, for 700 sites from which 200 are accessed once per minute, we'll
> have 200 process starts/ends per minute.
>
> For 200 sites accessed 3 times per minute, we'll need
> 200 resident processes (+ all other starting/ending).
>
> If we set the max_child 100 (eating 3Gb RAM), what happends?
> Will we have starts/ends for each second request to be searved with a
> right environment?
> Or will the new request wait for 30 sec till some process expires to
> have another born?
> Or will it spawn 500th process into swap?
>
> There does not seem to be an easy answer to the TS situation.
>
>
> On 9 июл, 13:32, Jérôme Loyet<m...@fatbsd.com> wrote:
>> 2011/7/9 Troll<trollofdarkn...@gmail.com>:
>>
>>> No, setting up CGI is not the same. Your CGI process will end once the query
>>> is over (stop me if I'm wrong). Not your fastcgi process, as I explained in
>>> a previous reply.
>> You are right. CGI process are spawn for each request and can only
>> served one request. FCGI has been invented to handle this "bug". FCGI
>> is a loop version of CGI. It means that a process launched once and
>> serves many requests sequentially.
>>
>>
>>
>>> To make it simple, "ondemand" mode is just the same as "dynamic" mode but
>>> you can state start_servers=0 instead of a minimum start_servers=1 with
>>> dynamic mode.
>> The "ondemand" mode can have from 0 process to max_children. When a
>> request arrives, it no process is available (0 process or all
>> processes are serving a request), a new process is created in order to
>> serve the incoming request. After then, processes stays awake to
>> server the next comming requests for a defined amount of time
>> (pm.process_idle_timeout parameter). For exemple, if a process has
>> done nothing in the last 30s, it's killed.:
>>
>> 1- The request R1 arrives
>> 2- "ondemand" PM forks the process P1
>> 3- the process P1 serves the request P1
>> 4- 3s waiting
>> 5- The request R2 arrives
>> 6- the process P1 serves the request P2
>> 7- 30s waiting
>> 8- the process P1 is killed
>> 9-wait 10 minutes
>> 10- go to step 1
>>
>> To sum up, the "ondemand" mode is almost the same concept as CGI but
>> it anticipates incoming requests by not stopping processes right after
>> it serves its first request. It's optimization. And generaly, when a
>> request arrives, there good chances that anothers will come in the
>> near future (someone who is browsing the main page, will certainly
>> browse sub pages serveral seconds after).
>>
>> hope it answers your questions
>>
>> ++ Jerome
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>> Regards,
>>> Troll
>>> On 07/09/2011 10:29 AM, grigori wrote:
>>>> Hi thanks for info,
>>>> can you describe how it works please - does it changes environment/uid
>>>> for the current processes, or it just starts the new process as CGI
>>>> when nginx gets a request?
>>>> For the latter case, it may be easier just to set php as CGI, for the
>>>> large number of sites when each requests starts a new process there
>>>> won't be a benefit from FPM.
>>>> Grigori
>>>> On 9 июл, 03:26, Jérôme Loyet<m...@fatbsd.com> wrote:
>>>>> for the record,
>>>>> I've just submit a small fix to the patch in order to correct the known
>>>>> bug.
>>>>> ++ jerome
>>>>> PS: the patch can be found inhttps://bugs.php.net/bug.php?id=52569.

2011/7/9 Troll <trollofdarkness@gmail.com>:
> By the way, about testing the patch, is the code directly taken from the SVN
> repository (svn.php.net/viewvc/php/php-src/branches/PHP_5_3) stable enough ?

5.3 branch should only receives bugfixes so there is no reason not to
be stable enough.

But as it's not been released yet (we are in a RC stage), I can't guaranty it.

>
> I'd like to test it & help its development, but currently don't have any
> test server, only a production one :)

you are the only one who can tell if you can test new stuff on your
production server :) I can't make that call for you

>
> Regards,
> Troll
>
> On 07/09/2011 12:21 PM, Jérôme Loyet wrote:
>>
>> 2011/7/9 Troll<trollofdarkness@gmail.com>:
>>>
>>> The patch has now been tested&  validated ?
>>
>> I've tested it on my own and it needs to be tesed in order to be
>> validated.
>>
>>> It's been a long time this feature request/patch has been waiting for an
>>> interest (or maybe simply spare time) from the main tree developers, do
>>> you
>>> know why it has not been integrated yet in the main tree ?
>>
>> simply because it's not been tested. Help us testing it (and fix it if
>> needed) and it integration into the sources will be much faster.
>>
>>> Everyone knows that this feature is expected from many many users in
>>> order
>>> to deploy FPM in shared hosting environment, I can't understand why it
>>> has
>>> not been put onto the priority list.
>>
>> It's all a matter of time. Don't forget it's opensource and based on
>> voluntary participation
>>
>>>
>>> By the way :
>>>
>>> CGI/FastCGI with up to 1000 vhosts : http://www.legtux.org/phpinfo.php
>>> It works better than mod_php as administrators.
>>>
>>> Romain
>>>
>>> Yes, but certainly not using a single pool / website. Maybe using only a
>>> couple of pools with each of them handling hundreds of website (which
>>> then
>>> is similar to the use of mod_php, in a certain way). Or maybe using
>>> ondemand
>>> patch :)
>>>
>>> If the patch has been updated to the last sources, I'll certainly test
>>> it. I
>>> have to update my (compiled from sources) php version.
>>>
>>> But does the code directly taken from the SVN repository is stable enough
>>> ?
>>>
>>>
>>> Regards,
>>> Troll
>>>
>>>
>>> On 07/09/2011 02:26 AM, Jérôme Loyet wrote:
>>>
>>> for the record,
>>>
>>> I've just submit a small fix to the patch in order to correct the known
>>> bug.
>>>
>>> ++ jerome
>>>
>>> PS: the patch can be found in https://bugs.php.net/bug.php?id=52569.
>>>
>>> Also note it should be applied on sources fetch from SVN:
>>> fpm-ondemand.v8.patch  on svn.php.net/viewvc/php/php-src/branches/PHP_5_4
>>> fpm-ondemand.v8-5.3.patch  on
>>> svn.php.net/viewvc/php/php-src/branches/PHP_5_3
>>>
>>>
>>> Le 9 juillet 2011 01:20, Jérôme Loyet<ml@fatbsd.com>  a écrit :
>>>
>>> 2011/7/8 David Coallier<david@orchestra.io>:
>>>
>>> If you have some programming skills, I encourage you to take part in the
>>> development of the "ondemand" patch. It has been tested and was, if I
>>> remember well, nearly finished, but I think it was for an old FPM version
>>> (in comparision to the current version) so the main task would be to
>>> adapt
>>> it to the last stable FPM version.
>>>
>>> Do you have a link to the ondemand patch?
>>>
>>> yes! https://bugs.php.net/bug.php?id=52569
>>>
>>> the patch have been updated for the last sources 2 days ago. You're
>>> welcome to test it (even if there's still a bug I have to figure out).
>>>
>>> Cheers,
>>>
>>> --
>>> David Coallier,
>>> CTO Orchestra Platform Ltd, https://orchestra.io
>>> @davidcoallier, skype://david.coallier
>>>
>>>
>

On 9 июл, 14:52, Troll <trollofdarkn...@gmail.com> wrote:
> If you have 200 websites accessed 1 time/minute, it will depend on your
> configuration :
>
> expiration time < 1 min -> processes will die and then be reborn next minute
> expiration time > 1 min -> processes will never end because they will
> receive a request before the expiration time, and it will reset the
> expiration time (since we're talking here about spare time).

Which means the rarely-served sites get deadlocked by the more popular
ones.


> So in any case, imho, you will get a high number of php processes in
> memory. But try to do that with CGI and I think your server will just
> fall down (the hard disk at first maybe) since it would be launching
> around 200 new process more or less constantly.

With CGI and mod_php you get predictable results at least, without
race conditions.


> then some requests will queue, the response time will be higher than
> with a fpm use, but less memory would be used (not really sure in fact,
> because if we then compare apache vs. nginx, you will in fact get 200
> apache2 process
the requests will either be queued, or we get 200 processes;
in reality, we don't run more then 100 children in apache

> which will get each own some memoery,
but will switch context without restarting


The patch solves a problem for the non-visited sites only.
With almost any load like one request per minute for hundreds of sites
the economy on re-starting may take you into a race condition with
locking parts of the sites out, the small expiration time makes you a
CGI server.