Howdy,

We are using php-fpm, it is going well, and now at the point of
optimization. We are running a 4 core VM with 1024mb of memory. Our
work load is CPU bound (vmstat 1) shows 0 idle and 0 wait. The php
workers are eating all the cpu, nginx is barely even using a percent.

I have it setup in static worker mode, and have benchmarked 4, 8, 12,
16, 32, 64 workers. Very oddly, the more workers (i.e. 16, 32, 64) the
worst results in terms of requests per second from siege and also
blitz.io. Honestly its quite baffling. Shouldn't more workers provider
better performance? Honestly, the sweet spot seems to be 6 or 8
workers. The only thing I can think of is with the higher number of
workers, its actually doing more cpu context switches and thus slower.
If this is case, why would anybody run more than N number of CPU PHP
workers?

Thanks for the explanation.

> We are using php-fpm, it is going well, and now at the point of
> optimization. We are running a 4 core VM with 1024mb of memory. Our
> work load is CPU bound (vmstat 1) shows 0 idle and 0 wait. The php
> workers are eating all the cpu, nginx is barely even using a percent.
>
> I have it setup in static worker mode, and have benchmarked 4, 8, 12,
> 16, 32, 64 workers. Very oddly, the more workers (i.e. 16, 32, 64) the
> worst results in terms of requests per second from siege and also
> blitz.io. Honestly its quite baffling. Shouldn't more workers provider
> better performance? Honestly, the sweet spot seems to be 6 or 8
> workers. The only thing I can think of is with the higher number of
> workers, its actually doing more cpu context switches and thus slower.
> If this is case, why would anybody run more than N number of CPU PHP
> workers?

First of all increasing number workers increases performance in one
case only: if the current number isn't able to use 100% cpu(s) because
it's waiting for something.
Having the number of workers = number of cores (or just slightly more)
potentially works best in a single, not very likely, scenario: when
there is no wait for anything outside php worker - no database queries,
cache fetch requests, no file access etc. It's exactly the case of a
simple 'hello world' php code used for testing.
In real life it's very likely half (or more) of the execution time is
spent waiting for external data source. If you haven't done it yet
repeat your tests with a real world app.

If that are in fact results from tests run on real life app it's quite
possible the factor distorting the results is RAM or lack of it to be
precise - with just 1GB on that vps, which is most likely running mail
server, mysql and whatever else came default with the distro, you are
left with not a whole lot of free memory - maybe as little as 500MB -
and that's about enough for around 6-8 workers running something
moderately heavy, like for example default install (no add-ons) of
Drupal 7 or Wordpress (30MB for worker and about the same for app).
Beyond that welcome to swapping hell and serious performance
degradation.

Having more workers (within reason) than the minimum required to tap
100% of cpu is recommended because it will reduce the negative effects
of having a couple workers stuck on a long running requests (long db
queries etc) at the same time - it will reduce general performance very
slightly, but at the same time smooth the spikes over.

--
Maciej Lisiewski

2012/2/20 SocketFail <justin@pagelines.com>
>
> Howdy,
>
> We are using php-fpm, it is going well, and now at the point of
> optimization. We are running a 4 core VM with 1024mb of memory. Our
> work load is CPU bound (vmstat 1) shows 0 idle and 0 wait. The php
> workers are eating all the cpu, nginx is barely even using a percent.
>
> I have it setup in static worker mode, and have benchmarked 4, 8, 12,
> 16, 32, 64 workers. Very oddly, the more workers (i.e. 16, 32, 64) the
> worst results in terms of requests per second from siege and also
> blitz.io. Honestly its quite baffling. Shouldn't more workers provider
> better performance? Honestly, the sweet spot seems to be 6 or 8
> workers. The only thing I can think of is with the higher number of
> workers, its actually doing more cpu context switches and thus slower.
> If this is case, why would anybody run more than N number of CPU PHP
> workers?

it depends on what's your PHP code is doing. If your code does only
CPU calculation, you should not have more PHP processes than CPU
numbers. But if you PHP code is doing other stuff it's wise to have
more PHP processes than the number of CPU you have available. For
exemple, if your script is calling a remote MySQL server, the PHP
process will just wait while the MySQL server is calculating its
answer. In this case if you have the same number of CPU than PHP
processes (and if you don't have anything else on the server), you
have one CPU doing nothing but waiting for a socket.

So, depending on your code, you should have the PHP number of
processes from N+20% to M/m. N is the number of CPU, M is the
available memory for PHP and m is the average memory of one PHP
process.

You'll have to find the limit :)

>
> Thanks for the explanation.

Hello guys thanks for the speedy responses.

The test we are running is a default install of Wordpress 3.3.1 and the 2011 theme. So there of course will be MySQL queries and what not, but according to vmstat there is 0% idle and 0% wait when running a siege test like:

siege -c 10 -b -t30s http://www.mydomain.com

Basically, increasing the number of workers from 6 to 32, did not improve the number of requests per second. Seems to just be maxing out the CPU's 100%. This VM is just running php+php-fpm, no mail (postfix), no nginx (other server), nothing but PHP crunching.

On 02/20/2012 11:22 AM, Jérôme Loyet wrote:
> So, depending on your code, you should have the PHP number of
> processes from N+20% to M/m. N is the number of CPU, M is the
> available memory for PHP and m is the average memory of one PHP
> process.

You lose nothing if the number of workers is as high as your memory allows.
It's even better to have some free workers waiting for the job to appear than
to have a job waiting for a worker to finish the previous one.

We use the following config:
the number of workers ~= RAM/(average size of PHP process determined empirically * 1.2 (just to be safe))

And yes, it's a static number.

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime profiling for PHP

the switch "pm" can be set to "static|dynamic|ondemand". (e.g. linux-box: pool folder - /etc/php5/fpm/pool.d)

pm = ondemand

all other settings will be done by switch "ONDEMAND" and dont need to be set, because they are set automatically.

pm.max_children = 10
pm.start_servers = 4
pm.min_spare_servers = 2
pm.max_spare_servers = 6
;pm.process_idle_timeout = 10s;
;pm.max_requests = 1000

--

Am Sonntag, 19. Februar 2012, 22:03:09 schrieb SocketFail:
> Howdy,
>
> We are using php-fpm, it is going well, and now at the point of
> optimization. We are running a 4 core VM with 1024mb of memory. Our
> work load is CPU bound (vmstat 1) shows 0 idle and 0 wait. The php
> workers are eating all the cpu, nginx is barely even using a percent.
>
> I have it setup in static worker mode, and have benchmarked 4, 8, 12,
> 16, 32, 64 workers. Very oddly, the more workers (i.e. 16, 32, 64) the
> worst results in terms of requests per second from siege and also
> blitz.io. Honestly its quite baffling. Shouldn't more workers provider
> better performance? Honestly, the sweet spot seems to be 6 or 8
> workers. The only thing I can think of is with the higher number of
> workers, its actually doing more cpu context switches and thus slower.
> If this is case, why would anybody run more than N number of CPU PHP
> workers?
>
> Thanks for the explanation.

Ah, so increasing the number workers only increases the performance in one case. I thought it would increase performance overall.

Thanks for clarifying.

This is indeed a concept that's been baffling me as well. I think that both the terminology used in PHP FPM (configuration vs PHP FPM status page values) as well the absence of any proper manual beyond what's already present in the pool files make things even more confusing.

After a lot of trial and error on high traffic websites running the top 2 CMSs (WordPress & Joomla), averaging from 1500 to 5000 concurrent visitors (Google Analytics), here's what I have experienced so far...

Assuming a highly tuned setup on Ubuntu 14.04 using Varnish 4.x, Nginx 1.10.x (with FastCGI caching on), PHP5-FPM 5.5.x (with APCu & Opcache) & MySQL 5.5 (using InnoDB), there ARE times when you have valid traffic which can't all be cached in Varnish and Nginx, especially when you have a refresh TTL of 1 to 3 mins because the site is a major news portal which needs to be updated constantly.

Varnish & Nginx can do one heck of a job allowing only unique traffic passing through to PHP5-FPM for processing MySQL can be properly tuned to avoid overloading CPUs and the system overall. The only bottleneck seems to be PHP5-FPM. If you go by most tutorials available online, suggesting that the right pm.max_children value should be calculated from available RAM and the average or top memory consumed by each PHP process, you'll probably kill your server, even if its a 32 or 40 CPU beast, whenever your cache is warming up, with all CPUs maxed out and your website simply crashing. I suppose that calculation works on short lived processes that are constantly refreshed. On CPU intensive processes, it just doesn't cut it.

Again a reminder that I'm not talking about sites hosted on small VMs with less than 8GBs of RAM. And since I understand that "high traffic" is a relative concept to most people, I'm clarifying that I'm talking about websites hosted on a single or dual dedicated server setup (web/db) that do dozens of millions of page views per month or say a minimum of 1 million absolute unique users per month.

By setting pm.max_children to a value slightly lower or equal to the number of your CPUs (in such scenarios) appears to actually properly *contain* CPU overloading from PHP5-FPM (something like this http://jmp.sh/lzka4CE - notice that at traffic peaks total CPU never exceeds around 80-85%). Especially when using "ondemand" as "pm", which is subsequently controlled by pm.max_children, pm.process_idle_timeout & pm.max_requests. Configuring pm.max_children to a value of 20 to 24 on a 24 CPU thread server, pm.process_idle_timeout to 1-3 secs and pm.max_requests to something between 200-500, seems to hit a sweet spot. Adjusting pm.max_requests too much doesn't seem to make a huge difference to be honest. However you can see dramatically different results when you change pm.process_idle_timeout from say to 3 to 2 secs.

If your experience falls into similar traffic patterns, I'd really like your input on the matter.

---
Maker of Engintron - https://engintron.com
The easy way to integrate Nginx on cPanel as a reverse caching proxy that actually works for any type of website (even forums or e-shops).