Should PHP Workers Always Equal Number Of CPUs

[SocketFail]

Howdy,

We are using php-fpm, it is going well, and now at the point of
optimization. We are running a 4 core VM with 1024mb of memory. Our
work load is CPU bound (vmstat 1) shows 0 idle and 0 wait. The php
workers are eating all the cpu, nginx is barely even using a percent.

I have it setup in static worker mode, and have benchmarked 4, 8, 12,
16, 32, 64 workers. Very oddly, the more workers (i.e. 16, 32, 64) the
worst results in terms of requests per second from siege and also
blitz.io. Honestly its quite baffling. Shouldn't more workers provider
better performance? Honestly, the sweet spot seems to be 6 or 8
workers. The only thing I can think of is with the higher number of
workers, its actually doing more cpu context switches and thus slower.
If this is case, why would anybody run more than N number of CPU PHP
workers?

Thanks for the explanation.

[Maciej Lisiewski]

First of all increasing number workers increases performance in one
case only: if the current number isn't able to use 100% cpu(s) because
it's waiting for something.
Having the number of workers = number of cores (or just slightly more)
potentially works best in a single, not very likely, scenario: when
there is no wait for anything outside php worker - no database queries,
cache fetch requests, no file access etc. It's exactly the case of a
simple 'hello world' php code used for testing.
In real life it's very likely half (or more) of the execution time is
spent waiting for external data source. If you haven't done it yet
repeat your tests with a real world app.

If that are in fact results from tests run on real life app it's quite
possible the factor distorting the results is RAM or lack of it to be
precise - with just 1GB on that vps, which is most likely running mail
server, mysql and whatever else came default with the distro, you are
left with not a whole lot of free memory - maybe as little as 500MB -
and that's about enough for around 6-8 workers running something
moderately heavy, like for example default install (no add-ons) of
Drupal 7 or Wordpress (30MB for worker and about the same for app).
Beyond that welcome to swapping hell and serious performance
degradation.

Having more workers (within reason) than the minimum required to tap
100% of cpu is recommended because it will reduce the negative effects
of having a couple workers stuck on a long running requests (long db
queries etc) at the same time - it will reduce general performance very
slightly, but at the same time smooth the spikes over.

--
Maciej Lisiewski

[Jérôme Loyet]

2012/2/20 SocketFail <jus...@pagelines.com>

it depends on what's your PHP code is doing. If your code does only
CPU calculation, you should not have more PHP processes than CPU
numbers. But if you PHP code is doing other stuff it's wise to have
more PHP processes than the number of CPU you have available. For
exemple, if your script is calling a remote MySQL server, the PHP
process will just wait while the MySQL server is calculating its
answer. In this case if you have the same number of CPU than PHP
processes (and if you don't have anything else on the server), you
have one CPU doing nothing but waiting for a socket.

So, depending on your code, you should have the PHP number of
processes from N+20% to M/m. N is the number of CPU, M is the
available memory for PHP and m is the average memory of one PHP
process.

You'll have to find the limit :)

>
> Thanks for the explanation.

[Antony Dovgal]

On 02/20/2012 11:22 AM, J�r�me Loyet wrote:
> So, depending on your code, you should have the PHP number of
> processes from N+20% to M/m. N is the number of CPU, M is the
> available memory for PHP and m is the average memory of one PHP
> process.

You lose nothing if the number of workers is as high as your memory allows.
It's even better to have some free workers waiting for the job to appear than
to have a job waiting for a worker to finish the previous one.

We use the following config:
the number of workers ~= RAM/(average size of PHP process determined empirically * 1.2 (just to be safe))

And yes, it's a static number.

--
Wbr,
Antony Dovgal
---
http://pinba.org - realtime profiling for PHP

[Jörg Wiegleb]

the switch "pm" can be set to "static|dynamic|ondemand". (e.g. linux-box: pool folder - /etc/php5/fpm/pool.d)

pm = ondemand

all other settings will be done by switch "ONDEMAND" and dont need to be set, because they are set automatically.

pm.max_children = 10
pm.start_servers = 4
pm.min_spare_servers = 2
pm.max_spare_servers = 6
;pm.process_idle_timeout = 10s;
;pm.max_requests = 1000

--