Welcome! Log In Create A New Profile

Advanced

How to change php5-fpm owner for each virtual host?

Posted by Ismet Togay 
Ismet Togay
How to change php5-fpm owner for each virtual host?
April 05, 2010 09:06AM
Hi all,

I use nginx 0.8.35 and php 5.3.2 with php5-fpm. I have many virtual
hosts and everything works well, except ONE thing:
All php5-fpm processes have the same owner(namely, www-data) no matter
what the virtual host is. What can I do to make each process work
having the user/group related with that domain?

################################
Here is an example;

domainA.com is in /var/www/domainA.com/public_html where domainA.com
folder (and all its content) is owned by userA and groupA.
domainB.com is in /var/www/domainB.com/public_html where domainB.com
folder (and all its content) is owned by userB and groupB.

So, the owners of php5-fpm processes who are responsible to run php
scripts for domainA and domainB should be userA and userB accordingly,
in order to access and modify the folder contents properly. But how???
################################

Thanks in advance...

ismet


--
To unsubscribe, reply using "remove me" as the subject.
Jérôme Loyet
Re: How to change php5-fpm owner for each virtual host?
April 05, 2010 09:28AM
Hi

for each worker/pool/virtual host (call it whatever you want) you can
set a a user/group to it.

<section name="pool">
<value name="name">site1</value>
<value name="user">site1</value>
<value name="group">www</value>
....
</section>
<section name="pool">
<value name="name">site2</value>
<value name="user">site2</value>
<value name="group">www</value>
....
</section>

hope it helps
++ Jerome

2010/4/5 Ismet Togay <ismet.togay@gmail.com>:
> Hi all,
>
> I use nginx 0.8.35 and php 5.3.2 with php5-fpm. I have many virtual
> hosts and everything works well, except ONE thing:
> All php5-fpm processes have the same owner(namely, www-data) no matter
> what the virtual host is. What can I do to make each process work
> having the user/group related with that domain?
>
> ################################
> Here is an example;
>
> domainA.com is in /var/www/domainA.com/public_html where domainA.com
> folder (and all its content) is owned by userA and groupA.
> domainB.com is in /var/www/domainB.com/public_html where domainB.com
> folder (and all its content) is owned by userB and groupB.
>
> So, the owners of php5-fpm processes who are responsible to run php
> scripts for domainA and domainB should be userA and userB accordingly,
> in order to access and modify the folder contents properly. But how???
> ################################
>
> Thanks in advance...
>
> ismet
>
>
> --
> To unsubscribe, reply using "remove me" as the subject.
>
2010/4/5 Jérôme Loyet <ml@fatbsd.com>:
> Hi
>
> for each worker/pool/virtual host (call it whatever you want) you can
> set a a user/group to it.
If we do so, the amount of processes is fixed to each pool, and cannot
be shared by others.
Many people want a fastcgi_params to pass the user/group to the
process, setuid, fufill request and go back to root listening.
--
Ren Xiaolei


--
To unsubscribe, reply using "remove me" as the subject.
Jérôme Loyet
Re: How to change php5-fpm owner for each virtual host?
April 07, 2010 06:28AM
2010/4/7 任晓磊 <julyclyde@gmail.com>:
> 2010/4/5 Jérôme Loyet <ml@fatbsd.com>:
>> Hi
>>
>> for each worker/pool/virtual host (call it whatever you want) you can
>> set a a user/group to it.
> If we do so, the amount of processes is fixed to each pool, and cannot
> be shared by others.

It's how FPM works.

> Many people want a fastcgi_params to pass the user/group to the
> process, setuid, fufill request and go back to root listening.

It's technically possible but it'll cost security as all processes
will be uid/euid root by default. When a fastcgi request is handle by
a process, it'll have to dynamically determine which euid/egid to
switch to, then switch to this user, execute code which is not secure
and then go back to root euid. Imagine what can do the following PHP
code in this case:

<?php
posix_seteuid(0);
posix_kill(-9,-1);
?>

It's technically possible but it cost too much security!

++ Jerome


--
To unsubscribe, reply using "remove me" as the subject.
Jérôme Loyet
Re: How to change php5-fpm owner for each virtual host?
April 07, 2010 11:34AM
Le 7 avril 2010 12:26, Jérôme Loyet <ml@fatbsd.com> a écrit :
> 2010/4/7 任晓磊 <julyclyde@gmail.com>:
>> 2010/4/5 Jérôme Loyet <ml@fatbsd.com>:
>>> Hi
>>>
>>> for each worker/pool/virtual host (call it whatever you want) you can
>>> set a a user/group to it.
>> If we do so, the amount of processes is fixed to each pool, and cannot
>> be shared by others.
>
> It's how FPM works.
>
>> Many people want a fastcgi_params to pass the user/group to the
>> process, setuid, fufill request and go back to root listening.
>
> It's technically possible but it'll cost security as all processes
> will be uid/euid root by default. When a fastcgi request is handle by
> a process, it'll have to dynamically determine which euid/egid to
> switch to, then switch to this user, execute code which is not secure
> and then go back to root euid. Imagine what can do the following PHP
> code in this case:
>
> <?php
>  posix_seteuid(0);
>  posix_kill(-9,-1);
> ?>
>
> It's technically possible but it cost too much security!
>

I tried to implement it quickly (and dirty) on my box. What did I do:
1- I remove the seuid and setgid function call from the code so that
each children stays owned by root
2- before reading the fastcgi headers from a request, I call
setegid(0) and seteuid(0) to become root before handling the request
3- after having reading the fastcgi headers, I call setegid(1021) and
seteuid(1021) (1021 is a local user) so that all children execution
will be made as my local user and not root

I recompile fpm and run it. Then I test it by using this simple php script:

<?php
echo "uid=".posix_getuid();
echo "<br />gid=".posix_getgid();
echo "<br />euid=".posix_geteuid();
echo "<br />egid=".posix_getegid();
echo "<br /><code>";
echo file_get_contents('/root/.profile');
echo "</code><br />";

posix_setegid(0);
posix_seteuid(0);

echo "<br /><code>";
echo file_get_contents('/root/.profile');
echo "</code><br />";
phpinfo();
?>

It show:
uid=0
gid=0
euid=1021
egid=1021

which tells me that the effective user id is my local user but that my
real user id is root (so as the group id).
Then I tried to read a file my local user is not supposed to have access to:

echo file_get_contents('/root/.profile');

I got in return: Warning: file_get_contents(/root/.profile): failed to
open stream: Permission denied in /home/www/test/docs/test.php on line
7

It's the normal behaviour as I'm identified as my local user.

Now I call the functions posix_setegid(0) and posix_seteuid(0) to
become root and try again to access the file I'm not supposed to have
rights on...

echo file_get_contents('/root/.profile');

# $OpenBSD: dot.profile,v 1.8 2009/05/06 22:02:05 millert Exp $ # #
sh/ksh initialization
....


Ohhhh surprise I gain root access ! To prevent that you have to
disable the posix_seteuid and posix_setegid and you'll move the
security layer to PHP instead of your kernel !

The need is good but it won't be possible ... instead you have another
idea to implement what you want :)

++ jerome


--
To unsubscribe, reply using "remove me" as the subject.
On 04/07/2010 12:26 PM, Jérôme Loyet wrote:
> 2010/4/7 任晓磊<julyclyde@gmail.com>:
>
>> 2010/4/5 Jérôme Loyet<ml@fatbsd.com>:
>>
>>> Hi
>>>
>>> for each worker/pool/virtual host (call it whatever you want) you can
>>> set a a user/group to it.
>>>
>> If we do so, the amount of processes is fixed to each pool, and cannot
>> be shared by others.
>>
> It's how FPM works.
>
>
>> Many people want a fastcgi_params to pass the user/group to the
>> process, setuid, fufill request and go back to root listening.
>>
> It's technically possible but it'll cost security as all processes
> will be uid/euid root by default. When a fastcgi request is handle by
> a process, it'll have to dynamically determine which euid/egid to
> switch to, then switch to this user, execute code which is not secure
> and then go back to root euid.
How would you go "back to root euid"? AFAIK once you dropped the
privileges you are not longer allowed to use the setgid()/setuid()
family of functions so you can never become root again.

> Imagine what can do the following PHP
> code in this case:
>
> <?php
> posix_seteuid(0);
> posix_kill(-9,-1);
> ?>
>
>
Since this piece is called with non-root privileges would it just result
in an error? From the documentation of posix_seteuid():
"This is a privileged function and needs appropriate privileges (usually
root) on the system to be able to perform this function."

> It's technically possible but it cost too much security!
>
> ++ Jerome
>
>
>



--
To unsubscribe, reply using "remove me" as the subject.
Re: How to change php5-fpm owner for each virtual host?
April 07, 2010 12:10PM
On Apr 7, 2010, at 5:11 AM, "Dennis J." <djacobfeuerborn@gmail.com>
wrote:

> How would you go "back to root euid"? AFAIK once you dropped the
> privileges you are not longer allowed to use the setgid()/setuid()
> family of functions so you can never become root again.

Wouldn't that require a forked process? In which case it seems like
the same thing as suphp for apache, which was neat and all but forking
a new php-cgi process per request couldn't scale at all for me.

I don't know the technical differences and options but Jerome and I
had talked about it because it would be the ultimate thing but the
only way to drop privs is to fork a child process (I think?)



--
To unsubscribe, reply using "remove me" as the subject.
Jérôme Loyet
Re: How to change php5-fpm owner for each virtual host?
April 07, 2010 12:36PM
2010/4/7 Dennis J. <djacobfeuerborn@gmail.com>:
> On 04/07/2010 12:26 PM, Jérôme Loyet wrote:
>>
>> 2010/4/7 任晓磊<julyclyde@gmail.com>:
>>
>>>
>>> 2010/4/5 Jérôme Loyet<ml@fatbsd.com>:
>>>
>>>>
>>>> Hi
>>>>
>>>> for each worker/pool/virtual host (call it whatever you want) you can
>>>> set a a user/group to it.
>>>>
>>>
>>> If we do so, the amount of processes is fixed to each pool, and cannot
>>> be shared by others.
>>>
>>
>> It's how FPM works.
>>
>>
>>>
>>> Many people want a fastcgi_params to pass the user/group to the
>>> process, setuid, fufill request and go back to root listening.
>>>
>>
>> It's technically possible but it'll cost security as all processes
>> will be uid/euid root by default. When a fastcgi request is handle by
>> a process, it'll have to dynamically determine which euid/egid to
>> switch to, then switch to this user, execute code which is not secure
>> and then go back to root euid.
>
> How would you go "back to root euid"? AFAIK once you dropped the privileges
> you are not longer allowed to use the setgid()/setuid() family of functions
> so you can never become root again.

I'm talking about setEuid and setEgid not setuid and setgid. I'm
changing the effective user so that I can move back to root.

The only way to secure is to fork for each request and end the child
at the end of the request which will be a performence disaster.

++ Jerome

>
>>  Imagine what can do the following PHP
>> code in this case:
>>
>> <?php
>>   posix_seteuid(0);
>>   posix_kill(-9,-1);
>> ?>
>>
>>
>
> Since this piece is called with non-root privileges would it just result in
> an error? From the documentation of posix_seteuid():
> "This is a privileged function and needs appropriate privileges (usually
> root) on the system to be able to perform this function."

google setuid and seteuid for example. See this for details:
http://unixpapa.com/incnote/setuid.html

test it yourself

the program is run as root

seteduid(1021); // become user 1021
seteduid(0); //move back to root
seteuiod(4242); // become user 4242

>
>> It's technically possible but it cost too much security!
>>
>> ++ Jerome
>>
>>
>>
>
>
>
> --
> To unsubscribe, reply using "remove me" as the subject.
>
On 04/07/2010 06:34 PM, Jérôme Loyet wrote:
> 2010/4/7 Dennis J.<djacobfeuerborn@gmail.com>:
>
>> On 04/07/2010 12:26 PM, Jérôme Loyet wrote:
>>
>>> 2010/4/7 任晓磊<julyclyde@gmail.com>:
>>>
>>>
>>>> 2010/4/5 Jérôme Loyet<ml@fatbsd.com>:
>>>>
>>>>
>>>>> Hi
>>>>>
>>>>> for each worker/pool/virtual host (call it whatever you want) you can
>>>>> set a a user/group to it.
>>>>>
>>>>>
>>>> If we do so, the amount of processes is fixed to each pool, and cannot
>>>> be shared by others.
>>>>
>>>>
>>> It's how FPM works.
>>>
>>>
>>>
>>>> Many people want a fastcgi_params to pass the user/group to the
>>>> process, setuid, fufill request and go back to root listening.
>>>>
>>>>
>>> It's technically possible but it'll cost security as all processes
>>> will be uid/euid root by default. When a fastcgi request is handle by
>>> a process, it'll have to dynamically determine which euid/egid to
>>> switch to, then switch to this user, execute code which is not secure
>>> and then go back to root euid.
>>>
>> How would you go "back to root euid"? AFAIK once you dropped the privileges
>> you are not longer allowed to use the setgid()/setuid() family of functions
>> so you can never become root again.
>>
> I'm talking about setEuid and setEgid not setuid and setgid. I'm
> changing the effective user so that I can move back to root.
>
Didn't see the "e" in there but that is basically my point. Either you
drop privileges completely so you can't move back which would give you
security but only allows for one request per process or you use
seteuid() which allows re-use of the process but kills security.

> The only way to secure is to fork for each request and end the child
> at the end of the request which will be a performence disaster.
>
>
Have you benchmarked this? You'd still be pre-forking the processes and
only "consume" them when an actual request needs to be processed so
performance wise you'd end up like this:

fastest: normal fpm (process pool with process reuse)
?: setuid() fpm (process pool without process reuse)
slowest: php-cgi (processes get spawned on demand)

I.e. you basically end up with the same model as Apache with the mpm
worker processes. You have the processes waiting so when a request comes
you can immediately handle it. This should work relatively well until
your req/s is > then fork-rate/s.

I've been trying to come up with a solution for this as well but so far
I didn't find the time to dive into the FPM code. Would you mind posting
your patch to the list?

Regards,
Dennis


--
To unsubscribe, reply using "remove me" as the subject.
Jérôme Loyet
Re: How to change php5-fpm owner for each virtual host?
April 09, 2010 02:16AM
2010/4/7 Dennis J. <djacobfeuerborn@gmail.com>:
> On 04/07/2010 06:34 PM, Jérôme Loyet wrote:
>>
>> 2010/4/7 Dennis J.<djacobfeuerborn@gmail.com>:
>>
>>>
>>> On 04/07/2010 12:26 PM, Jérôme Loyet wrote:
>>>
>>>>
>>>> 2010/4/7 任晓磊<julyclyde@gmail.com>:
>>>>
>>>>
>>>>>
>>>>> 2010/4/5 Jérôme Loyet<ml@fatbsd.com>:
>>>>>
>>>>>
>>>>>>
>>>>>> Hi
>>>>>>
>>>>>> for each worker/pool/virtual host (call it whatever you want) you can
>>>>>> set a a user/group to it.
>>>>>>
>>>>>>
>>>>>
>>>>> If we do so, the amount of processes is fixed to each pool, and cannot
>>>>> be shared by others.
>>>>>
>>>>>
>>>>
>>>> It's how FPM works.
>>>>
>>>>
>>>>
>>>>>
>>>>> Many people want a fastcgi_params to pass the user/group to the
>>>>> process, setuid, fufill request and go back to root listening.
>>>>>
>>>>>
>>>>
>>>> It's technically possible but it'll cost security as all processes
>>>> will be uid/euid root by default. When a fastcgi request is handle by
>>>> a process, it'll have to dynamically determine which euid/egid to
>>>> switch to, then switch to this user, execute code which is not secure
>>>> and then go back to root euid.
>>>>
>>>
>>> How would you go "back to root euid"? AFAIK once you dropped the
>>> privileges
>>> you are not longer allowed to use the setgid()/setuid() family of
>>> functions
>>> so you can never become root again.
>>>
>>
>> I'm talking about setEuid and setEgid not setuid and setgid. I'm
>> changing the effective user so that I can move back to root.
>>
>
> Didn't see the "e" in there but that is basically my point. Either you drop
> privileges completely so you can't move back which would give you security
> but only allows for one request per process or you use seteuid() which
> allows re-use of the process but kills security.
>
>> The only way to secure is to fork for each request and end the child
>> at the end of the request which will be a performence disaster.
>>
>>
>
> Have you benchmarked this? You'd still be pre-forking the processes and only
> "consume" them when an actual request needs to be processed so performance
> wise you'd end up like this:
>
> fastest: normal fpm (process pool with process reuse)
> ?:  setuid() fpm (process pool without process reuse)
> slowest: php-cgi (processes get spawned on demand)
>
> I.e. you basically end up with the same model as Apache with the mpm worker
> processes. You have the processes waiting so when a request comes you can
> immediately handle it. This should work relatively well until your req/s is
>> then fork-rate/s.
>
> I've been trying to come up with a solution for this as well but so far I
> didn't find the time to dive into the FPM code. Would you mind posting your
> patch to the list?

FPM has two forking models:
1 - static: FPM forks N processes at startup and setuid them. Each
child is calling the accept() function to wait for connections. The
kernel is doing the "load balancing" between the children. When a
child dies (for any reasons), the FPM master process restarts it. In
this case you have, for each pool, a fixed number of children
dedicated to a pool.

2 - dynamic: FPM forks N processes at startup and setuid them. Each
child is calling the accept() function to wait for connections. The
kernel is doing the "load balancing" between the children. When a
child dies (for any reasons), the FPM master process restarts it. In
parallel, the FPM Master process is constantly watching, for each
pool, the number of idle children and forks some new of there is not
enough and kill some if there is too much (with min_spare_servers and
max_spare_servers). It's exactly the same model as apache.

We can imagine another model: "ondemand".

The master process will listen on all sockets (as it does now). But it
won't fork any children. It will setup an event (libevent) on READ on
each socket. When a connection is waiting, it'll be notfied by
libevent and it'll be able to fork a child for this task. When the
child finishes its processing, it stays open for a timeout for
incoming new connections. If no connections comes within the timeout,
the master process will kill it and setup a new event on the socket to
fork a new child at the next request.

What is the complexity ? What is the load on the master process ? What
are the performances on this ?


BTW, the dirty patch I used to test the setEuid function is the
following (be careful I told you it was dirty):

++ Jerome

Index: fastcgi.c
===================================================================
--- fastcgi.c (revision 297735)
+++ fastcgi.c (working copy)
@@ -982,9 +982,13 @@

fpm_request_accepting();

+ setegid(0);
+ seteuid(0);
FCGI_LOCK(req->listen_socket);
req->fd =
accept(listen_socket, (struct sockaddr *)&sa, &len);
FCGI_UNLOCK(req->listen_socket);
+ setegid(1021);
+ seteuid(1021);
if (req->fd >= 0 && allowed_clients) {
int n = 0;
int allowed = 0;


Index: fpm_unix.c
===================================================================
--- fpm_unix.c (revision 297735)
+++ fpm_unix.c (working copy)
@@ -184,7 +184,7 @@
chdir("/");
}

- if (is_root) {
+ if (0 && is_root) {
if (wp->set_gid) {
if (0 > setgid(wp->set_gid)) {
zlog(ZLOG_STUFF, ZLOG_SYSERROR, "[pool
%s] setgid(%d) failed", wp->config->name, wp->set_gid);
Re: How to change php5-fpm owner for each virtual host?
April 09, 2010 02:34AM
2010/4/8 Jérôme Loyet <ml@fatbsd.com>:

> The master process will listen on all sockets (as it does now). But it
> won't fork any children. It will setup an event (libevent) on READ on
> each socket. When a connection is waiting, it'll be notfied by
> libevent and it'll be able to fork a child for this task. When the
> child finishes its processing, it stays open for a timeout for
> incoming new connections. If no connections comes within the timeout,
> the master process will kill it and setup a new event on the socket to
> fork a new child at the next request.
>
> What is the complexity ? What is the load on the master process ? What
> are the performances on this ?

The one way I could see this as being slightly less overhead is having
an idle timeout - so it forks a child as $user, and when it is done,
it hangs around for a bit until it times out. That way, it's like a
reusable child, and any site with a decent load would spike the number
of children up and maintain it (somewhat like setting the dynamic
manager w/ a small amount of children to start, but max_children being
high and letting it spike dynamically higher...)

Same concept I suppose just with the starting amount of children set
to 0 in this case...


--
To unsubscribe, reply using "remove me" as the subject.
Jérôme Loyet
Re: How to change php5-fpm owner for each virtual host?
April 09, 2010 02:38AM
2010/4/9 Michael Shadle <mike503@gmail.com>:
> 2010/4/8 Jérôme Loyet <ml@fatbsd.com>:
>
>> The master process will listen on all sockets (as it does now). But it
>> won't fork any children. It will setup an event (libevent) on READ on
>> each socket. When a connection is waiting, it'll be notfied by
>> libevent and it'll be able to fork a child for this task. When the
>> child finishes its processing, it stays open for a timeout for
>> incoming new connections. If no connections comes within the timeout,
>> the master process will kill it and setup a new event on the socket to
>> fork a new child at the next request.
>>
>> What is the complexity ? What is the load on the master process ? What
>> are the performances on this ?
>
> The one way I could see this as being slightly less overhead is having
> an idle timeout - so it forks a child as $user, and when it is done,
> it hangs around for a bit until it times out. That way, it's like a
> reusable child, and any site with a decent load would spike the number
> of children up and maintain it (somewhat like setting the dynamic
> manager w/ a small amount of children to start, but max_children being
> high and letting it spike dynamically higher...)
>
> Same concept I suppose just with the starting amount of children set
> to 0 in this case...

Yes but technicaly it's not that simple. The classic way to handle
connections makes you having at least one child waiting for a
connection. If we don't wan't any dedicaded child, we have to makes it
different and it'll be much more complicated.

You should read technical documentation on the subject:
man socket
man bind
man listen
man accept
http://www.monkey.org/~provos/libevent/doxygen-1.4.10/

and so on


--
To unsubscribe, reply using "remove me" as the subject.
Re: How to change php5-fpm owner for each virtual host?
April 09, 2010 02:42AM
2010/4/8 Jérôme Loyet <ml@fatbsd.com>:

> Yes but technicaly it's not that simple. The classic way to handle
> connections makes you having at least one child waiting for a
> connection. If we don't wan't any dedicaded child, we have to makes it
> different and it'll be much more complicated.
>
> You should read technical documentation on the subject:
> man socket
> man bind
> man listen
> man accept
> http://www.monkey.org/~provos/libevent/doxygen-1.4.10/
>
> and so on

oh no, I'm not saying it's simple. It's the "dream" we all have, but
the reality is it probably wouldn't scale well. :) I'm not encouraging
it, was just throwing out random ideas/commentary. I think the
realistic suggestion people will have to be told is "setup one dynamic
pool for each user, with one process to start, and set max_children
appropriately for each pool" and let the dynamic process manager
handle it.


--
To unsubscribe, reply using "remove me" as the subject.
Jérôme Loyet
Re: How to change php5-fpm owner for each virtual host?
April 09, 2010 03:04AM
2010/4/9 Michael Shadle <mike503@gmail.com>:
> 2010/4/8 Jérôme Loyet <ml@fatbsd.com>:
>
>> Yes but technicaly it's not that simple. The classic way to handle
>> connections makes you having at least one child waiting for a
>> connection. If we don't wan't any dedicaded child, we have to makes it
>> different and it'll be much more complicated.
>>
>> You should read technical documentation on the subject:
>> man socket
>> man bind
>> man listen
>> man accept
>> http://www.monkey.org/~provos/libevent/doxygen-1.4.10/
>>
>> and so on
>
> oh no, I'm not saying it's simple. It's the "dream" we all have, but
> the reality is it probably wouldn't scale well. :) I'm not encouraging
> it, was just throwing out random ideas/commentary. I think the
> realistic suggestion people will have to be told is "setup one dynamic
> pool for each user, with one process to start, and set max_children
> appropriately for each pool" and let the dynamic process manager
> handle it.

yes but if you have 1000 pools doing nothing almost all the time,
you'll have 1000 processes doing nothing ... it's a waste of
ressources


--
To unsubscribe, reply using "remove me" as the subject.
Re: How to change php5-fpm owner for each virtual host?
April 09, 2010 03:24AM
2010/4/9 Jérôme Loyet <ml@fatbsd.com>:

> yes but if you have 1000 pools doing nothing almost all the time,
> you'll have 1000 processes doing nothing ... it's a waste of
> ressources

exactly. I'm thinking of how to address shared hosts with lots of
users. without the "dream" approach that's the best they can do, short
of just running one pool for all users like standard the apache module
would.


--
To unsubscribe, reply using "remove me" as the subject.
> FPM has two forking models:
> 1 - static: FPM forks N processes at startup and setuid them. Each
> child is calling the accept() function to wait for connections. The
> kernel is doing the "load balancing" between the children. When a
> child dies (for any reasons), the FPM master process restarts it. In
> this case you have, for each pool, a fixed number of children
> dedicated to a pool.
>
> 2 - dynamic: FPM forks N processes at startup and setuid them. Each
> child is calling the accept() function to wait for connections. The
> kernel is doing the "load balancing" between the children. When a
> child dies (for any reasons), the FPM master process restarts it. In
> parallel, the FPM Master process is constantly watching, for each
> pool, the number of idle children and forks some new of there is not
> enough and kill some if there is too much (with min_spare_servers and
> max_spare_servers). It's exactly the same model as apache.
>
> We can imagine another model: "ondemand".
>
> The master process will listen on all sockets (as it does now). But it
> won't fork any children. It will setup an event (libevent) on READ on
> each socket. When a connection is waiting, it'll be notfied by
> libevent and it'll be able to fork a child for this task. When the
> child finishes its processing, it stays open for a timeout for
> incoming new connections. If no connections comes within the timeout,
> the master process will kill it and setup a new event on the socket to
> fork a new child at the next request.
>
> What is the complexity ? What is the load on the master process ? What
> are the performances on this ?
>
>
What I'm wondering is if the "

FPM forks N processes at startup and setuid them

" thing can be improved on.
What if you would fork say 100 workers as root and then wait with the
setuid() until you actually get a request? The worker could then handle
the process with the dynamic uid and terminate and the master process
could fork a new worker process in its place. That way you would get the
benefit of having a pool of worker processes that could be used for any
requests that come in but still handle each request with a dedicated uid.
You could of course "cache" the process once it's finished rather than
immediately letting it die as described by you above but I see this as a
separate issue independent of the fork/setuid one which I think is the
big hurdle we need to get over first before we can think about caching
processes.

> BTW, the dirty patch I used to test the setEuid function is the
> following (be careful I told you it was dirty):
>
I don't plan to use it verbatim just to point me to the right places in
the FPM code. I hope I get some time on the weekend to experiment with
this and if I do I will post my results on the list.
Thanks!

Regards,
Dennis


--
To unsubscribe, reply using "remove me" as the subject.
Jérôme Loyet
Re: How to change php5-fpm owner for each virtual host?
April 10, 2010 05:14AM
2010/4/9 Dennis J. <djacobfeuerborn@gmail.com>:
>
>> FPM has two forking models:
>> 1 - static: FPM forks N processes at startup and setuid them. Each
>> child is calling the accept() function to wait for connections. The
>> kernel is doing the "load balancing" between the children. When a
>> child dies (for any reasons), the FPM master process restarts it. In
>> this case you have, for each pool, a fixed number of children
>> dedicated to a pool.
>>
>> 2 - dynamic: FPM forks N processes at startup and setuid them. Each
>> child is calling the accept() function to wait for connections. The
>> kernel is doing the "load balancing" between the children. When a
>> child dies (for any reasons), the FPM master process restarts it. In
>> parallel, the FPM Master process is constantly watching, for each
>> pool, the number of idle children and forks some new of there is not
>> enough and kill some if there is too much (with min_spare_servers and
>> max_spare_servers). It's exactly the same model as apache.
>>
>> We can imagine another model: "ondemand".
>>
>> The master process will listen on all sockets (as it does now). But it
>> won't fork any children. It will setup an event (libevent) on READ on
>> each socket. When a connection is waiting, it'll be notfied by
>> libevent and it'll be able to fork a child for this task. When the
>> child finishes its processing, it stays open for a timeout for
>> incoming new connections. If no connections comes within the timeout,
>> the master process will kill it and setup a new event on the socket to
>> fork a new child at the next request.
>>
>> What is the complexity ? What is the load on the master process ? What
>> are the performances on this ?
>>
>>
>
> What I'm wondering is if the "
>
> FPM forks N processes at startup and setuid them
>
> " thing can be improved on.
> What if you would fork say 100 workers as root and then wait with the
> setuid() until you actually get a request? The worker could then handle the
> process with the dynamic uid and terminate and the master process could fork
> a new worker process in its place. That way you would get the benefit of
> having a pool of worker processes that could be used for any requests that
> come in but still handle each request with a dedicated uid.
> You could of course "cache" the process once it's finished rather than
> immediately letting it die as described by you above but I see this as a
> separate issue independent of the fork/setuid one which I think is the big
> hurdle we need to get over first before we can think about caching
> processes.
>
>> BTW, the dirty patch I used to test the setEuid function is the
>> following (be careful I told you it was dirty):
>>
>
> I don't plan to use it verbatim just to point me to the right places in the
> FPM code. I hope I get some time on the weekend to experiment with this and
> if I do I will post my results on the list.
> Thanks!
>

I start working on a new process management style (after static and
dynamic). It's call "ondemand". It's like dynamic have
start_servers=0, max_spare_servers=0 and min_spare_servers=0. No
children are forked. When a request arrives, the master process forks
a child to handle the request. If a child is idle for more than X
seconds (10 in the following patch), it's been killed by the master
process.

You can try the following patch: http://pastebin.com/EbfX23Pv

++ Jerome


--
To unsubscribe, reply using "remove me" as the subject.
Jérôme Loyet
Re: How to change php5-fpm owner for each virtual host?
April 10, 2010 05:48AM
Le 10 avril 2010 11:13, Jérôme Loyet <ml@fatbsd.com> a écrit :
> 2010/4/9 Dennis J. <djacobfeuerborn@gmail.com>:
>>
>>> FPM has two forking models:
>>> 1 - static: FPM forks N processes at startup and setuid them. Each
>>> child is calling the accept() function to wait for connections. The
>>> kernel is doing the "load balancing" between the children. When a
>>> child dies (for any reasons), the FPM master process restarts it. In
>>> this case you have, for each pool, a fixed number of children
>>> dedicated to a pool.
>>>
>>> 2 - dynamic: FPM forks N processes at startup and setuid them. Each
>>> child is calling the accept() function to wait for connections. The
>>> kernel is doing the "load balancing" between the children. When a
>>> child dies (for any reasons), the FPM master process restarts it. In
>>> parallel, the FPM Master process is constantly watching, for each
>>> pool, the number of idle children and forks some new of there is not
>>> enough and kill some if there is too much (with min_spare_servers and
>>> max_spare_servers). It's exactly the same model as apache.
>>>
>>> We can imagine another model: "ondemand".
>>>
>>> The master process will listen on all sockets (as it does now). But it
>>> won't fork any children. It will setup an event (libevent) on READ on
>>> each socket. When a connection is waiting, it'll be notfied by
>>> libevent and it'll be able to fork a child for this task. When the
>>> child finishes its processing, it stays open for a timeout for
>>> incoming new connections. If no connections comes within the timeout,
>>> the master process will kill it and setup a new event on the socket to
>>> fork a new child at the next request.
>>>
>>> What is the complexity ? What is the load on the master process ? What
>>> are the performances on this ?
>>>
>>>
>>
>> What I'm wondering is if the "
>>
>> FPM forks N processes at startup and setuid them
>>
>> " thing can be improved on.
>> What if you would fork say 100 workers as root and then wait with the
>> setuid() until you actually get a request? The worker could then handle the
>> process with the dynamic uid and terminate and the master process could fork
>> a new worker process in its place. That way you would get the benefit of
>> having a pool of worker processes that could be used for any requests that
>> come in but still handle each request with a dedicated uid.
>> You could of course "cache" the process once it's finished rather than
>> immediately letting it die as described by you above but I see this as a
>> separate issue independent of the fork/setuid one which I think is the big
>> hurdle we need to get over first before we can think about caching
>> processes.
>>
>>> BTW, the dirty patch I used to test the setEuid function is the
>>> following (be careful I told you it was dirty):
>>>
>>
>> I don't plan to use it verbatim just to point me to the right places in the
>> FPM code. I hope I get some time on the weekend to experiment with this and
>> if I do I will post my results on the list.
>> Thanks!
>>
>
> I start working on a new process management style (after static and
> dynamic). It's call "ondemand". It's like dynamic have
> start_servers=0, max_spare_servers=0 and min_spare_servers=0. No
> children are forked. When a request arrives, the master process forks
> a child to handle the request. If a child is idle for more than X
> seconds (10 in the following patch), it's been killed by the master
> process.
>
> You can try the following patch: http://pastebin.com/EbfX23Pv

I forgot to say the patch is buggy at some points (double fork
sometimes, timeout on wrong time value, ... but the main part is there
and is working)

>
> ++ Jerome
>


--
To unsubscribe, reply using "remove me" as the subject.
Jérôme Loyet
Re: How to change php5-fpm owner for each virtual host?
April 10, 2010 07:24AM
Le 10 avril 2010 11:47, Jérôme Loyet <ml@fatbsd.com> a écrit :
> Le 10 avril 2010 11:13, Jérôme Loyet <ml@fatbsd.com> a écrit :
>> 2010/4/9 Dennis J. <djacobfeuerborn@gmail.com>:
>>>
>>>> FPM has two forking models:
>>>> 1 - static: FPM forks N processes at startup and setuid them. Each
>>>> child is calling the accept() function to wait for connections. The
>>>> kernel is doing the "load balancing" between the children. When a
>>>> child dies (for any reasons), the FPM master process restarts it. In
>>>> this case you have, for each pool, a fixed number of children
>>>> dedicated to a pool.
>>>>
>>>> 2 - dynamic: FPM forks N processes at startup and setuid them. Each
>>>> child is calling the accept() function to wait for connections. The
>>>> kernel is doing the "load balancing" between the children. When a
>>>> child dies (for any reasons), the FPM master process restarts it. In
>>>> parallel, the FPM Master process is constantly watching, for each
>>>> pool, the number of idle children and forks some new of there is not
>>>> enough and kill some if there is too much (with min_spare_servers and
>>>> max_spare_servers). It's exactly the same model as apache.
>>>>
>>>> We can imagine another model: "ondemand".
>>>>
>>>> The master process will listen on all sockets (as it does now). But it
>>>> won't fork any children. It will setup an event (libevent) on READ on
>>>> each socket. When a connection is waiting, it'll be notfied by
>>>> libevent and it'll be able to fork a child for this task. When the
>>>> child finishes its processing, it stays open for a timeout for
>>>> incoming new connections. If no connections comes within the timeout,
>>>> the master process will kill it and setup a new event on the socket to
>>>> fork a new child at the next request.
>>>>
>>>> What is the complexity ? What is the load on the master process ? What
>>>> are the performances on this ?
>>>>
>>>>
>>>
>>> What I'm wondering is if the "
>>>
>>> FPM forks N processes at startup and setuid them
>>>
>>> " thing can be improved on.
>>> What if you would fork say 100 workers as root and then wait with the
>>> setuid() until you actually get a request? The worker could then handle the
>>> process with the dynamic uid and terminate and the master process could fork
>>> a new worker process in its place. That way you would get the benefit of
>>> having a pool of worker processes that could be used for any requests that
>>> come in but still handle each request with a dedicated uid.
>>> You could of course "cache" the process once it's finished rather than
>>> immediately letting it die as described by you above but I see this as a
>>> separate issue independent of the fork/setuid one which I think is the big
>>> hurdle we need to get over first before we can think about caching
>>> processes.
>>>
>>>> BTW, the dirty patch I used to test the setEuid function is the
>>>> following (be careful I told you it was dirty):
>>>>
>>>
>>> I don't plan to use it verbatim just to point me to the right places in the
>>> FPM code. I hope I get some time on the weekend to experiment with this and
>>> if I do I will post my results on the list.
>>> Thanks!
>>>
>>
>> I start working on a new process management style (after static and
>> dynamic). It's call "ondemand". It's like dynamic have
>> start_servers=0, max_spare_servers=0 and min_spare_servers=0. No
>> children are forked. When a request arrives, the master process forks
>> a child to handle the request. If a child is idle for more than X
>> seconds (10 in the following patch), it's been killed by the master
>> process.
>>
>> You can try the following patch: http://pastebin.com/EbfX23Pv
>
> I forgot to say the patch is buggy at some points (double fork
> sometimes, timeout on wrong time value, ... but the main part is there
> and is working)

new revision with some corrections: http://pastebin.com/dXV0knVM

>
>>
>> ++ Jerome
>>
>


--
To unsubscribe, reply using "remove me" as the subject.
Jérôme Loyet
Re: How to change php5-fpm owner for each virtual host?
November 11, 2010 07:46PM
I'm reopening this thread as some work have been done since. There is
a PHP bug report which is related to this "ondemand" process manager:
http://bugs.php.net/bug.php?id=52569

The patch included won't apply on current snapshot as many improvement
have been done in FPM since. It will appy on revision 303365.

If you have any questions, please ask.

++ jerome

Le 10 avril 2010 13:21, Jérôme Loyet <ml@fatbsd.com> a écrit :
> Le 10 avril 2010 11:47, Jérôme Loyet <ml@fatbsd.com> a écrit :
>> Le 10 avril 2010 11:13, Jérôme Loyet <ml@fatbsd.com> a écrit :
>>> 2010/4/9 Dennis J. <djacobfeuerborn@gmail.com>:
>>>>
>>>>> FPM has two forking models:
>>>>> 1 - static: FPM forks N processes at startup and setuid them. Each
>>>>> child is calling the accept() function to wait for connections. The
>>>>> kernel is doing the "load balancing" between the children. When a
>>>>> child dies (for any reasons), the FPM master process restarts it. In
>>>>> this case you have, for each pool, a fixed number of children
>>>>> dedicated to a pool.
>>>>>
>>>>> 2 - dynamic: FPM forks N processes at startup and setuid them. Each
>>>>> child is calling the accept() function to wait for connections. The
>>>>> kernel is doing the "load balancing" between the children. When a
>>>>> child dies (for any reasons), the FPM master process restarts it. In
>>>>> parallel, the FPM Master process is constantly watching, for each
>>>>> pool, the number of idle children and forks some new of there is not
>>>>> enough and kill some if there is too much (with min_spare_servers and
>>>>> max_spare_servers). It's exactly the same model as apache.
>>>>>
>>>>> We can imagine another model: "ondemand".
>>>>>
>>>>> The master process will listen on all sockets (as it does now). But it
>>>>> won't fork any children. It will setup an event (libevent) on READ on
>>>>> each socket. When a connection is waiting, it'll be notfied by
>>>>> libevent and it'll be able to fork a child for this task. When the
>>>>> child finishes its processing, it stays open for a timeout for
>>>>> incoming new connections. If no connections comes within the timeout,
>>>>> the master process will kill it and setup a new event on the socket to
>>>>> fork a new child at the next request.
>>>>>
>>>>> What is the complexity ? What is the load on the master process ? What
>>>>> are the performances on this ?
>>>>>
>>>>>
>>>>
>>>> What I'm wondering is if the "
>>>>
>>>> FPM forks N processes at startup and setuid them
>>>>
>>>> " thing can be improved on.
>>>> What if you would fork say 100 workers as root and then wait with the
>>>> setuid() until you actually get a request? The worker could then handle the
>>>> process with the dynamic uid and terminate and the master process could fork
>>>> a new worker process in its place. That way you would get the benefit of
>>>> having a pool of worker processes that could be used for any requests that
>>>> come in but still handle each request with a dedicated uid.
>>>> You could of course "cache" the process once it's finished rather than
>>>> immediately letting it die as described by you above but I see this as a
>>>> separate issue independent of the fork/setuid one which I think is the big
>>>> hurdle we need to get over first before we can think about caching
>>>> processes.
>>>>
>>>>> BTW, the dirty patch I used to test the setEuid function is the
>>>>> following (be careful I told you it was dirty):
>>>>>
>>>>
>>>> I don't plan to use it verbatim just to point me to the right places in the
>>>> FPM code. I hope I get some time on the weekend to experiment with this and
>>>> if I do I will post my results on the list.
>>>> Thanks!
>>>>
>>>
>>> I start working on a new process management style (after static and
>>> dynamic). It's call "ondemand". It's like dynamic have
>>> start_servers=0, max_spare_servers=0 and min_spare_servers=0. No
>>> children are forked. When a request arrives, the master process forks
>>> a child to handle the request. If a child is idle for more than X
>>> seconds (10 in the following patch), it's been killed by the master
>>> process.
>>>
>>> You can try the following patch: http://pastebin.com/EbfX23Pv
>>
>> I forgot to say the patch is buggy at some points (double fork
>> sometimes, timeout on wrong time value, ... but the main part is there
>> and is working)
>
> new revision with some corrections: http://pastebin.com/dXV0knVM
>
>>
>>>
>>> ++ Jerome
>>>
>>
>
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 249
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 500 on July 15, 2024
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready