Welcome! Log In Create A New Profile

Advanced

Should PHP Workers Always Equal Number Of CPUs

Posted by Anonymous User 
Anonymous User
Should PHP Workers Always Equal Number Of CPUs
February 20, 2012 01:46AM
Howdy,

We are using php-fpm, it is going well, and now at the point of
optimization. We are running a 4 core VM with 1024mb of memory. Our
work load is CPU bound (vmstat 1) shows 0 idle and 0 wait. The php
workers are eating all the cpu, nginx is barely even using a percent.

I have it setup in static worker mode, and have benchmarked 4, 8, 12,
16, 32, 64 workers. Very oddly, the more workers (i.e. 16, 32, 64) the
worst results in terms of requests per second from siege and also
blitz.io. Honestly its quite baffling. Shouldn't more workers provider
better performance? Honestly, the sweet spot seems to be 6 or 8
workers. The only thing I can think of is with the higher number of
workers, its actually doing more cpu context switches and thus slower.
If this is case, why would anybody run more than N number of CPU PHP
workers?

Thanks for the explanation.
Maciej Lisiewski
Re: Should PHP Workers Always Equal Number Of CPUs
February 20, 2012 02:22AM
> We are using php-fpm, it is going well, and now at the point of
> optimization. We are running a 4 core VM with 1024mb of memory. Our
> work load is CPU bound (vmstat 1) shows 0 idle and 0 wait. The php
> workers are eating all the cpu, nginx is barely even using a percent.
>
> I have it setup in static worker mode, and have benchmarked 4, 8, 12,
> 16, 32, 64 workers. Very oddly, the more workers (i.e. 16, 32, 64) the
> worst results in terms of requests per second from siege and also
> blitz.io. Honestly its quite baffling. Shouldn't more workers provider
> better performance? Honestly, the sweet spot seems to be 6 or 8
> workers. The only thing I can think of is with the higher number of
> workers, its actually doing more cpu context switches and thus slower.
> If this is case, why would anybody run more than N number of CPU PHP
> workers?

First of all increasing number workers increases performance in one
case only: if the current number isn't able to use 100% cpu(s) because
it's waiting for something.
Having the number of workers = number of cores (or just slightly more)
potentially works best in a single, not very likely, scenario: when
there is no wait for anything outside php worker - no database queries,
cache fetch requests, no file access etc. It's exactly the case of a
simple 'hello world' php code used for testing.
In real life it's very likely half (or more) of the execution time is
spent waiting for external data source. If you haven't done it yet
repeat your tests with a real world app.

If that are in fact results from tests run on real life app it's quite
possible the factor distorting the results is RAM or lack of it to be
precise - with just 1GB on that vps, which is most likely running mail
server, mysql and whatever else came default with the distro, you are
left with not a whole lot of free memory - maybe as little as 500MB -
and that's about enough for around 6-8 workers running something
moderately heavy, like for example default install (no add-ons) of
Drupal 7 or Wordpress (30MB for worker and about the same for app).
Beyond that welcome to swapping hell and serious performance
degradation.

Having more workers (within reason) than the minimum required to tap
100% of cpu is recommended because it will reduce the negative effects
of having a couple workers stuck on a long running requests (long db
queries etc) at the same time - it will reduce general performance very
slightly, but at the same time smooth the spikes over.

--
Maciej Lisiewski
Jérôme Loyet
Re: Should PHP Workers Always Equal Number Of CPUs
February 20, 2012 02:24AM
2012/2/20 SocketFail <justin@pagelines.com>
>
> Howdy,
>
> We are using php-fpm, it is going well, and now at the point of
> optimization. We are running a 4 core VM with 1024mb of memory. Our
> work load is CPU bound (vmstat 1) shows 0 idle and 0 wait. The php
> workers are eating all the cpu, nginx is barely even using a percent.
>
> I have it setup in static worker mode, and have benchmarked 4, 8, 12,
> 16, 32, 64 workers. Very oddly, the more workers (i.e. 16, 32, 64) the
> worst results in terms of requests per second from siege and also
> blitz.io. Honestly its quite baffling. Shouldn't more workers provider
> better performance? Honestly, the sweet spot seems to be 6 or 8
> workers. The only thing I can think of is with the higher number of
> workers, its actually doing more cpu context switches and thus slower.
> If this is case, why would anybody run more than N number of CPU PHP
> workers?

it depends on what's your PHP code is doing. If your code does only
CPU calculation, you should not have more PHP processes than CPU
numbers. But if you PHP code is doing other stuff it's wise to have
more PHP processes than the number of CPU you have available. For
exemple, if your script is calling a remote MySQL server, the PHP
process will just wait while the MySQL server is calculating its
answer. In this case if you have the same number of CPU than PHP
processes (and if you don't have anything else on the server), you
have one CPU doing nothing but waiting for a socket.

So, depending on your code, you should have the PHP number of
processes from N+20% to M/m. N is the number of CPU, M is the
available memory for PHP and m is the average memory of one PHP
process.

You'll have to find the limit :)

>
> Thanks for the explanation.
Anonymous User
Re: Should PHP Workers Always Equal Number Of CPUs
February 20, 2012 02:53AM
Hello guys thanks for the speedy responses.

The test we are running is a default install of Wordpress 3.3.1 and the 2011 theme. So there of course will be MySQL queries and what not, but according to vmstat there is 0% idle and 0% wait when running a siege test like:

siege -c 10 -b -t30s http://www.mydomain.com

Basically, increasing the number of workers from 6 to 32, did not improve the number of requests per second. Seems to just be maxing out the CPU's 100%. This VM is just running php+php-fpm, no mail (postfix), no nginx (other server), nothing but PHP crunching.
Antony Dovgal
Re: Should PHP Workers Always Equal Number Of CPUs
February 20, 2012 03:26AM
On 02/20/2012 11:22 AM, Jérôme Loyet wrote:
> So, depending on your code, you should have the PHP number of
> processes from N+20% to M/m. N is the number of CPU, M is the
> available memory for PHP and m is the average memory of one PHP
> process.

You lose nothing if the number of workers is as high as your memory allows.
It's even better to have some free workers waiting for the job to appear than
to have a job waiting for a worker to finish the previous one.

We use the following config:
the number of workers ~= RAM/(average size of PHP process determined empirically * 1.2 (just to be safe))

And yes, it's a static number.

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime profiling for PHP
Jörg Wiegleb
Re: Should PHP Workers Always Equal Number Of CPUs
February 21, 2012 02:02AM
the switch "pm" can be set to "static|dynamic|ondemand". (e.g. linux-box: pool folder - /etc/php5/fpm/pool.d)

pm = ondemand

all other settings will be done by switch "ONDEMAND" and dont need to be set, because they are set automatically.

pm.max_children = 10
pm.start_servers = 4
pm.min_spare_servers = 2
pm.max_spare_servers = 6
;pm.process_idle_timeout = 10s;
;pm.max_requests = 1000

--

Am Sonntag, 19. Februar 2012, 22:03:09 schrieb SocketFail:
> Howdy,
>
> We are using php-fpm, it is going well, and now at the point of
> optimization. We are running a 4 core VM with 1024mb of memory. Our
> work load is CPU bound (vmstat 1) shows 0 idle and 0 wait. The php
> workers are eating all the cpu, nginx is barely even using a percent.
>
> I have it setup in static worker mode, and have benchmarked 4, 8, 12,
> 16, 32, 64 workers. Very oddly, the more workers (i.e. 16, 32, 64) the
> worst results in terms of requests per second from siege and also
> blitz.io. Honestly its quite baffling. Shouldn't more workers provider
> better performance? Honestly, the sweet spot seems to be 6 or 8
> workers. The only thing I can think of is with the higher number of
> workers, its actually doing more cpu context switches and thus slower.
> If this is case, why would anybody run more than N number of CPU PHP
> workers?
>
> Thanks for the explanation.
Re: Should PHP Workers Always Equal Number Of CPUs
March 07, 2016 05:44AM
Ah, so increasing the number workers only increases the performance in one case. I thought it would increase performance overall.

Thanks for clarifying.
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 77
Record Number of Users: 6 on August 08, 2016
Record Number of Guests: 386 on August 02, 2016
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready