Right now it's one single large patch.

I'm wondering if it makes sense to actually patch PHP, and take all
the files that got patched and pull those out into a tarball.

Then, instead of a patch, people can check out (or unpack a tarball)
the files ON TOP of the PHP distribution files.

By doing this, we can break up the patch into the individual files and
get it into something like github or I could setup an svn or launchpad
project for it, and people can contribute to individual files, add new
files, etc.

Anyone have any thoughts on that? Currently it's somewhat of a PITA to
manage changes to one single large file as a patch.

I will note that this effort may become futile depending on if it gets
put into PECL or portions merged into PHP itself; I am stuck between
looking at trying to advance the project independently or waiting to
hear back on integration status.

In the meantime though, I thought this would be a good forum to bring it up in.

I'm liking Git but I'm sure a lot of people are not ready to migrade
from SVN yet.

Michael Shadle wrote:
> Right now it's one single large patch.
>
> I'm wondering if it makes sense to actually patch PHP, and take all
> the files that got patched and pull those out into a tarball.
>
> Then, instead of a patch, people can check out (or unpack a tarball)
> the files ON TOP of the PHP distribution files.
>
> By doing this, we can break up the patch into the individual files and
> get it into something like github or I could setup an svn or launchpad
> project for it, and people can contribute to individual files, add new
> files, etc.
>
> Anyone have any thoughts on that? Currently it's somewhat of a PITA to
> manage changes to one single large file as a patch.
>
> I will note that this effort may become futile depending on if it gets
> put into PECL or portions merged into PHP itself; I am stuck between
> looking at trying to advance the project independently or waiting to
> hear back on integration status.
>
> In the meantime though, I thought this would be a good forum to bring it up in.
>

Free hosted services are a bonus in my book.

Besides, the contribution model probably works better with a git/bzr
vs. an svn model.

On Tue, Jun 30, 2009 at 5:28 PM, Chris
Cortese<cortese.consulting@gmail.com> wrote:
>
> I'm liking Git but I'm sure a lot of people are not ready to migrade from
> SVN yet.
>
> Michael Shadle wrote:
>>
>> Right now it's one single large patch.
>>
>> I'm wondering if it makes sense to actually patch PHP, and take all
>> the files that got patched and pull those out into a tarball.
>>
>> Then, instead of a patch, people can check out (or unpack a tarball)
>> the files ON TOP of the PHP distribution files.
>>
>> By doing this, we can break up the patch into the individual files and
>> get it into something like github or I could setup an svn or launchpad
>> project for it, and people can contribute to individual files, add new
>> files, etc.
>>
>> Anyone have any thoughts on that? Currently it's somewhat of a PITA to
>> manage changes to one single large file as a patch.
>>
>> I will note that this effort may become futile depending on if it gets
>> put into PECL or portions merged into PHP itself; I am stuck between
>> looking at trying to advance the project independently or waiting to
>> hear back on integration status.
>>
>> In the meantime though, I thought this would be a good forum to bring it
>> up in.
>>
>
>

I do like github, and of course ohloh. UI there is nicer. Though I'm
sure in time the UIs will look nothing like they do now.

Faster you get it up there the faster we can spin PR on it to help the
cause. ;-)


On Jun 30, 8:36

On 01.07.2009 03:24, Michael Shadle wrote:
> Anyone have any thoughts on that? Currently it's somewhat of a PITA to
> manage changes to one single large file as a patch.

I believe the best choice would be to forget that silly GPL business and actually incorporate FPM into PHP.
It's been discussed many times (even in person) and both Dmitry Stogov (maintainer of sapi/fcgi in PHP)
and Andrey do realize that this is what should eventually be done.

Looking at the current situation, I feel that Andrey is not really interested in maintaining/developing
the patch anymore (or just doesn't have much time, the result is the same), so I think it's time to change
the license and continue the development in the official PHP repository.
It is especially good time also because 5.4 (no way to include it in 5.3, too late) development cycle
has just started and we have a LOT of time for testing and discussions.

I remember only one _REAL_ obstacle for that - FPM required (does it still?) a patched libevent version
and that is, of course, should be fixed - either by using libev or by pushing the patch upstream.

Anyway, I know that the patch (especially one that comes with developers who would like to maintain it)
would be very welcome in PHP, so please no need for external repositories, patching etc.

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime statistics for PHP

Yeah. I'm awaiting a reply for the next steps from those guys. I've
been in contact with all of them and they were happy to see the
license change.

One thing though that i personally think might be good is to keep the
manager code out of php itself. That way additional features can be
added at a faster pace than waiting around for a new php build (didn't
they say 5.3 has been in the works for 2 years?)

If php core could add in the necessary stuff to allow an external
module to do the work required (maybe in pecl?) then any bugs or
features can be addressed more rapidly assuming it is within the scope
of the manager and not something that requires php core changes.

The manager can easily be distributed via pecl and distributions can
add it as a package just like they do with other pecl modules. So it
will still be extremely easy to adopt php-fpm, much easier than
patching.

Have you checked the wishlist on the wiki? There's still a lot of
desired features out there and that's what I've been looking to
cultivate. It will be a while before it's in php core most likely and
in the meantime I would like to keep things moving.

I hold your opinion in the highest regard though (second to andrei's)
and one reason I'm lagging a little is due to waiting to hear back
from the php team, as this project may fold in to php core and/or pecl
anyway. But in the meantime i would like to continue development on
it... I'm going to ask people to make donations (i'm pledging a bunch
too) so we can pay if needed for certain things to get done. Might as
well have them ready before php takes it over right? People can
contribute just in general or ask that it goes toward a specific
feature.

Sent from my iPhone

On Jul 2, 2009, at 7:22 AM, Antony Dovgal <tony@daylessday.org> wrote:

>
> On 01.07.2009 03:24, Michael Shadle wrote:
>> Anyone have any thoughts on that? Currently it's somewhat of a PITA
>> to
>> manage changes to one single large file as a patch.
>
> I believe the best choice would be to forget that silly GPL business
> and actually incorporate FPM into PHP.
> It's been discussed many times (even in person) and both Dmitry
> Stogov (maintainer of sapi/fcgi in PHP)
> and Andrey do realize that this is what should eventually be done.
>
> Looking at the current situation, I feel that Andrey is not really
> interested in maintaining/developing
> the patch anymore (or just doesn't have much time, the result is the
> same), so I think it's time to change
> the license and continue the development in the official PHP
> repository.
> It is especially good time also because 5.4 (no way to include it in
> 5.3, too late) development cycle
> has just started and we have a LOT of time for testing and
> discussions.
>
> I remember only one _REAL_ obstacle for that - FPM required (does it
> still?) a patched libevent version
> and that is, of course, should be fixed - either by using libev or
> by pushing the patch upstream.
>
> Anyway, I know that the patch (especially one that comes with
> developers who would like to maintain it)
> would be very welcome in PHP, so please no need for external
> repositories, patching etc.
>
> --
> Wbr,
> Antony Dovgal
> ---
> http://pinba.org - realtime statistics for PHP

On 02.07.2009 21:21, Michael Shadle wrote:
> One thing though that i personally think might be good is to keep the
> manager code out of php itself. That way additional features can be
> added at a faster pace than waiting around for a new php build (didn't
> they say 5.3 has been in the works for 2 years?)

Yes, major versions do take a long time to be stabilized.
But you don't have to wait for another major release to work on the manager code.
Also I don't see how you're going to get the patch into PHP and in
the same time keep a part out of it, is that what you meant?

> If php core could add in the necessary stuff to allow an external
> module to do the work required (maybe in pecl?)

PECL is an extension repository.
You can put a SAPI there (I believe), but it's completely different story and it's just
not that convenient for users at the moment (if you have any ideas how to improve it -
don't hesitate to share them).

> Have you checked the wishlist on the wiki? There's still a lot of
> desired features out there and that's what I've been looking to
> cultivate. It will be a while before it's in php core most likely and
> in the meantime I would like to keep things moving.

Yes, those are exactly the reasons why Andrey put it on hold.
I'd say just get it in the core and keep adding features, or you will never get there.

> I hold your opinion in the highest regard though (second to andrei's)
> and one reason I'm lagging a little is due to waiting to hear back
> from the php team, as this project may fold in to php core and/or pecl
> anyway. But in the meantime i would like to continue development on
> it...

Putting it into the core also has another advantage - more attention and more developers,
so you won't have to work on it alone.

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime statistics for PHP

On Thu, Jul 2, 2009 at 10:41 AM, Antony Dovgal<tony@daylessday.org> wrote:

> Yes, major versions do take a long time to be stabilized.
> But you don't have to wait for another major release to work on the manager code.
> Also I don't see how you're going to get the patch into PHP and in
> the same time keep a part out of it, is that what you meant?

Yes, I'm looking at it from a two part angle. There's a management
component - the config file and its syntax and telling PHP-FPM how to
behave, and then the PHP core functionality, i.e. things like the
request_execution_timeout or whatever which forcibly kill the process,
that probably must live inside of PHP. So certain hooks inside of PHP
must exist, but perhaps it could live in PHP and the FPM portion can
be split off so it can be maintained, not as a SAPI but more as
something taking advantage of those hooks or APIs (basically
"extending" the FastCGI SAPI, that may get all the current PHP-FPM
functionality sans configuration file)

Approaching it from a more modular angle than bolting the two
together. Then if you have a fix or enhancement you need to wait for
5.4.1, 5.4.2, etc... seems like it might be a good idea to split it
up. We could change the config file syntax to be more like nginx's (I
think Andrei wanted to anyway) and add in more functionality without
needing to wait for PHP itself.

> PECL is an extension repository.
> You can put a SAPI there (I believe), but it's completely different story and it's just
> not that convenient for users at the moment (if you have any ideas how to improve it -
> don't hesitate to share them).

(See above, Rasmus thinks it can be PECL-able) - we could also have a
committer or two to PECL, and then have the code be on github or
something and use that to continue improvements, and then when they
are "bought off" then commit it to PECL. To me, I am looking at it as
an "extension" that allows for taking advantage of the aforementioned
hooks that will be put into PHP for fastcgi management.

> Yes, those are exactly the reasons why Andrey put it on hold.
> I'd say just get it in the core and keep adding features, or you will never get there.

Getting it into core will be slow, but I am bugging those guys every
so often to keep it on their radar. At the moment the ball is in their
court to get back to me on what they think the next steps are. But in
the meantime I'd like to keep development moving. They can take it
whenever they're ready, but maybe we can add some functionality in the
meantime.

> Putting it into the core also has another advantage - more attention and more developers,
> so you won't have to work on it alone.

Yeah - see my above note.

Also, you should try to get pinba in! I can see it going hand in hand
with PHP-FPM. All about monitoring PHP, managing processes, etc.
Although pinba requires too many external dependencies right now (a
specialized MySQL UDF right?) if that could be generalized to an API,
perhaps it could be something else that the FPM / manager interface
can provide, is reporting into the PHP processes themselves, via
RESTful calls, or some other API calls. (RESTful would be great, then
anyone can write a client, and doesn't libevent have an embedded HTTP
interface in it too?)

Part of my wishlist includes basic monitoring of PHP-FPM already...

On 02.07.2009 21:54, Michael Shadle wrote:
> Yes, I'm looking at it from a two part angle. There's a management
> component - the config file and its syntax and telling PHP-FPM how to
> behave, and then the PHP core functionality, i.e. things like the
> request_execution_timeout or whatever which forcibly kill the process,
> that probably must live inside of PHP. So certain hooks inside of PHP
> must exist, but perhaps it could live in PHP and the FPM portion can
> be split off so it can be maintained, not as a SAPI but more as
> something taking advantage of those hooks or APIs (basically
> "extending" the FastCGI SAPI, that may get all the current PHP-FPM
> functionality sans configuration file)

Well, I have to admit I still don't see it, but I guess you know what you're talking about.

> Approaching it from a more modular angle than bolting the two
> together. Then if you have a fix or enhancement you need to wait for
> 5.4.1, 5.4.2, etc...

And you'll still have to wait for FPM x.y.z, x.y.z+1 etc =)
I think you exaggerate it a bit - nobody expects a release for each fix
and bugfix releases are done quite often.

> Also, you should try to get pinba in!

I can put the extension in PECL, no problem.
But there is not much point in this while building a MySQL plugin requires so much magic.

> I can see it going hand in hand
> with PHP-FPM. All about monitoring PHP, managing processes, etc.
> Although pinba requires too many external dependencies right now (a
> specialized MySQL UDF right?)

A plugin/storage engine, yes.
I'm still hoping to see the build process improved soonish, it will make things soo much easier.

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime statistics for PHP

On Thu, Jul 2, 2009 at 11:13 AM, Antony Dovgal<tony@daylessday.org> wrote:
>
> On 02.07.2009 21:54, Michael Shadle wrote:
>> Yes, I'm looking at it from a two part angle. There's a management
>> component - the config file and its syntax and telling PHP-FPM how to
>> behave, and then the PHP core functionality, i.e. things like the
>> request_execution_timeout or whatever which forcibly kill the process,
>> that probably must live inside of PHP. So certain hooks inside of PHP
>> must exist, but perhaps it could live in PHP and the FPM portion can
>> be split off so it can be maintained, not as a SAPI but more as
>> something taking advantage of those hooks or APIs (basically
>> "extending" the FastCGI SAPI, that may get all the current PHP-FPM
>> functionality sans configuration file)
>
> Well, I have to admit I still don't see it, but I guess you know what you're talking about.

You actually know more than I do about this. You actually create PHP modules :)

>> Approaching it from a more modular angle than bolting the two
>> together. Then if you have a fix or enhancement you need to wait for
>> 5.4.1, 5.4.2, etc...
>
> And you'll still have to wait for FPM x.y.z, x.y.z+1 etc =)
> I think you exaggerate it a bit - nobody expects a release for each fix
> and bugfix releases are done quite often.

Yes, but if FPM is in PECL/externally managed, PECL changes occur
anytime. A PHP core change does not. Thousands of bugs and
enhancements are always floating around there, unless it's a security
risk, they won't roll out a new version of PHP core just for some FPM
enhancements.

That's why adding in FPM hooks and the other enhancements Andrei made
into PHP core might be awesome, but giving it API that can be extended
via PECL might be a good idea. Agility to change the manager portion
without asking the PHP team to make a release just for the manager...

On 03.07.2009 00:53, Michael Shadle wrote:
> Yes, but if FPM is in PECL/externally managed, PECL changes occur
> anytime. A PHP core change does not.

You can change something in the core anytime you want (if it's not in a freeze, which happens quite rarely).
IMO bugfix releases are done in reasonable time (several months), so I don't see any reasons
to be afraid of the core just because you'll have to follow its release cycle.

I can easily imagine a release done just because of some critical bug in a
certain (popular) SAPI, if that's what's worrying you.
As for non-critical bugs - well, they're called that for a reason, so you'll have to wait, yes.

I don't recall having any problems just because I have to follow the core release cycle.
Yes, it's nice to have your own cycle, but it's really not something uber-critical, believe me.

> That's why adding in FPM hooks and the other enhancements Andrei made
> into PHP core might be awesome, but giving it API that can be extended
> via PECL might be a good idea. Agility to change the manager portion
> without asking the PHP team to make a release just for the manager...

I agree, that would be nice.
But the fact is that there is no such API and there is no easy way to
install a SAPI from PECL at the moment, and creating them would take a lot of time.
That's why I want just start the process and see if we can make it good enough for _users_;
we can take care of _maintainers_ and their worries later.

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime statistics for PHP

On Thu, Jul 2, 2009 at 2:20 PM, Antony Dovgal<tony@daylessday.org> wrote:

> You can change something in the core anytime you want (if it's not in a freeze, which happens quite rarely).

Yes, but you have to wait for an RC at least for it to be rolled
officially by the PHP team.

> IMO bugfix releases are done in reasonable time (several months), so I don't see any reasons
> to be afraid of the core just because you'll have to follow its release cycle.

I'm not afraid of the core. The core requires necessary changes for
sure. But I think we can advance things quicker independent of the
core otherwise.

> I can easily imagine a release done just because of some critical bug in a
> certain (popular) SAPI, if that's what's worrying you.
> As for non-critical bugs - well, they're called that for a reason, so you'll have to wait, yes.

PHP-FPM is lacking a lot of enhancements though. The major one -
adaptive spawning - is key. Monitoring would be good too, and possibly
the capability to change at runtime the amount of processes or create
new process groups etc... the possibilities are endless once a
protocol/remote management capability is created. Check the wishlist
(that last piece isn't in there but very well could be :))

> I don't recall having any problems just because I have to follow the core release cycle.
> Yes, it's nice to have your own cycle, but it's really not something uber-critical, believe me.

I'm looking at it from the end-user perspective. How often do PHP RC's
come out? Say we get a specific feature included in PHP-FPM. We'll be
sitting around idling waiting for a new RC to get rolled, and even
then, the RC will not be adopted by any distros...

PECL has the capability to move faster and distros pick up PECL
modules separately too. Due to the nature of PHP-FPM running in
fastcgi pools too, it will only load it on startup, so the overhead
would be minimal too, I assume.

> I agree, that would be nice.
> But the fact is that there is no such API and there is no easy way to
> install a SAPI from PECL at the moment, and creating them would take a lot of time.
> That's why I want just start the process and see if we can make it good enough for _users_;
> we can take care of _maintainers_ and their worries later.

Well, adding in PHP-FPM as it currently stands would only require (in
my mind) a few new API hooks that a PECL module could speak to.

I see it being more difficult and odd to ask PHP users who want
PHP-FPM to have a secondary configuration file in /etc/php-fpm.conf
with a different syntax than people are normally used to (XML syntax,
but not really true XML functionality) as opposed to the normal
php.ini file.

On 03.07.2009 01:56, Michael Shadle wrote:
> I'm looking at it from the end-user perspective. How often do PHP RC's
> come out?

Often enough, as I said, once in several months.

> Say we get a specific feature included in PHP-FPM. We'll be
> sitting around idling waiting for a new RC to get rolled, and even
> then, the RC will not be adopted by any distros...

Hah! Distros will not adopt a _release_ for years, there are still several shipping 5.1.6.
And distros certainly do not ship third-party extensions (PECL included), you can forget
about FPM-enabled PHP packages if it goes somewhere outside the core.

> I see it being more difficult and odd to ask PHP users who want
> PHP-FPM to have a secondary configuration file in /etc/php-fpm.conf
> with a different syntax than people are normally used to (XML syntax,
> but not really true XML functionality) as opposed to the normal
> php.ini file.

Well, that's not a problem, I think - different SAPI, different configuration.
You can't fit all your FPM settings into php.ini anyways.

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime statistics for PHP

On Fri, Jul 3, 2009 at 12:50 AM, Antony Dovgal<tony@daylessday.org> wrote:

>> I'm looking at it from the end-user perspective. How often do PHP RC's
>> come out?
>
> Often enough, as I said, once in several months.

I'm ready to get releases out faster than that if possible. As long as
we have coders and funding to pay for whatever is needed to get things
jump started...

>> Say we get a specific feature included in PHP-FPM. We'll be
>> sitting around idling waiting for a new RC to get rolled, and even
>> then, the RC will not be adopted by any distros...
>
> Hah! Distros will not adopt a _release_ for years, there are still several shipping 5.1.6.
> And distros certainly do not ship third-party extensions (PECL included), you can forget
> about FPM-enabled PHP packages if it goes somewhere outside the core.

Ubuntu has a stripped down php5 build and then extensions. However,
now that I'm looking maybe you're right and I was wrong. They're part
of PHP core just separated out, not PECL ones. Doh. I guess all of
those are just things that could have been compiled in.

But that is one reason I don't want to tie things to a release
necessarily. Once the hooks are in and into a distro (after years)
then you can use a distro-supplied PHP with the external FPM
management tools. :)

>> I see it being more difficult and odd to ask PHP users who want
>> PHP-FPM to have a secondary configuration file in /etc/php-fpm.conf
>> with a different syntax than people are normally used to (XML syntax,
>> but not really true XML functionality) as opposed to the normal
>> php.ini file.
>
> Well, that's not a problem, I think - different SAPI, different configuration.
> You can't fit all your FPM settings into php.ini anyways.

Exactly. That's why I think it will be weird. Normal PHP config is the
ini file but then the FPM is XML...

When it comes down to it, I should probably just listen to you. You
actually do development and have much more experience with how things
work in this arena.

I'm just waiting around to hear back from the PHP folks, and hopefully
in the meantime getting something solid out there for each release and
enhance it.

Anyone who has the skills to attack the wishlist, let me know. Ideally
you'll do it to help advance the project but I will pump some money
into the project if I have to.

On 03.07.2009 12:05, Michael Shadle wrote:
> I'm ready to get releases out faster than that if possible. As long as
> we have coders and funding to pay for whatever is needed to get things
> jump started...

Faster than that might be quite annoying for end users.
A SAPI is pretty critical part of PHP and upgrading PHP once a week is a lot of job,
even if you have only a couple of servers.

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime statistics for PHP

On Fri, Jul 3, 2009 at 1:12 AM, Antony Dovgal<tony@daylessday.org> wrote:

> Faster than that might be quite annoying for end users.
> A SAPI is pretty critical part of PHP and upgrading PHP once a week is a lot of job,
> even if you have only a couple of servers.

Agreed.

Although I would not consider it production, it would be like any
project (the FPM portion) - with a git tree or something of the sort
being in constant flux, and the PECL module host the stable or at
least RC-style code.

I am not intending on releasing 1.0, 1.1, 1.2, 1.3 each week. But
continuous development and if people want to try out new features they
can, contributions welcome, etc.

However that is operating under the assumption that this is split
apart into two different portions.