Hi,

I have un-imagineable load spike in the server. Load is above 100 always and my top command shows

4821 www 1 70 0 149M 45540K CPU1 1 0:40 14.99% php-fpm
4822 www 1 70 0 146M 41836K RUN 0 0:28 14.99% php-fpm
4811 www 1 70 0 141M 46112K accept 0 0:49 13.96% php-fpm
4816 www 1 76 0 138M 43128K accept 0 0:48 10.99% php-fpm
4817 www 1 65 0 138M 29560K accept 1 0:33 9.96% php-fpm
4808 www 1 66 0 142M 50820K accept 1 1:13 8.98% php-fpm
4810 www 1 76 0 143M 43856K accept 0 0:54 8.98% php-fpm
4814 www 1 76 0 143M 51796K accept 0 1:23 6.98% php-fpm
4809 www 1 76 0 138M 40188K accept 0 0:28 6.98% php-fpm
4815 www 1 76 0 138M 33672K accept 0 0:43 5.96% php-fpm
7320 root 1 76 0 18824K 9660K RUN 1 0:02 5.96% htmldoc
4823 www 1 76 0 139M 35148K accept 1 0:41 3.96% php-fpm
4812 www 1 76 0 139M 38156K accept 1 0:41 3.96% php-fpm
4813 www 1 76 0 138M 32352K accept 1 0:39 1.95% php-fpm
4825 www 1 76 0 141M 43608K accept 1 0:50 0.98% php-fpm
4819 www 1 76 0 139M 40664K accept 0 0:46 0.98% php-fpm

Please let me know how to resolve this.

Note: Server has nginx installed with php-fpm.

first you should begin by telling which version you're using, then you
should give your conf file

2010/5/15 ragavendraganesh <nginx-forum@nginx.us>

> Hi,
>
> I have un-imagineable load spike in the server. Load is above 100 always
> and my top command shows
>
> 4821 www 1 70 0 149M 45540K CPU1 1 0:40 14.99% php-fpm
> 4822 www 1 70 0 146M 41836K RUN 0 0:28 14.99% php-fpm
> 4811 www 1 70 0 141M 46112K accept 0 0:49 13.96% php-fpm
> 4816 www 1 76 0 138M 43128K accept 0 0:48 10.99% php-fpm
> 4817 www 1 65 0 138M 29560K accept 1 0:33 9.96% php-fpm
> 4808 www 1 66 0 142M 50820K accept 1 1:13 8.98% php-fpm
> 4810 www 1 76 0 143M 43856K accept 0 0:54 8.98% php-fpm
> 4814 www 1 76 0 143M 51796K accept 0 1:23 6.98% php-fpm
> 4809 www 1 76 0 138M 40188K accept 0 0:28 6.98% php-fpm
> 4815 www 1 76 0 138M 33672K accept 0 0:43 5.96% php-fpm
> 7320 root 1 76 0 18824K 9660K RUN 1 0:02 5.96% htmldoc
> 4823 www 1 76 0 139M 35148K accept 1 0:41 3.96% php-fpm
> 4812 www 1 76 0 139M 38156K accept 1 0:41 3.96% php-fpm
> 4813 www 1 76 0 138M 32352K accept 1 0:39 1.95% php-fpm
> 4825 www 1 76 0 141M 43608K accept 1 0:50 0.98% php-fpm
> 4819 www 1 76 0 139M 40664K accept 0 0:46 0.98% php-fpm
>
> Please let me know how to resolve this.
>
> Note: Server has nginx installed with php-fpm.
>
> Posted at Nginx Forum:
> http://forum.nginx.org/read.php?3,86831,86831#msg-86831
>
>

PHP version i am using in php5

Nginx version used is nginx-0.8.36

Apache version 2

Please let me know if you need any other details and also help me clear this problem.

Thanks for your precious time

If your machine is consistently reporting a load of 100 and you're
actually able to poke around and see that value, you likely have a lot
of processes blocked on slow IO operations.

The first culprits that come to my mind are:
- Is this a virtual machine?
- Are you consistently accessing a network filesystem?
- Are your php scripts themselves consistently spinning off further
network (via curl or otherwise) requests?
- Is there some kind of full-system security/backup scan running all the time?

I don't think anything specific to php-fpm or nginx is the culprit
here, but it is likely either that you are running on underpowered
hardware for your needs or your scripts are just doing things they
probably ought not ;)

On Sat, May 15, 2010 at 5:14 AM, ragavendraganesh <nginx-forum@nginx.us> wrote:
> PHP version i am using in php5
>
> Nginx version used is nginx-0.8.36
>
> Apache version 2
>
> Please let me know if you need any other details and also help me clear this problem.
>
> Thanks for your precious time
>
> Posted at Nginx Forum: http://forum.nginx.org/read.php?3,86831,86928#msg-86928
>
>



--
--
Ammon Lauritzen

Hi,

Thanks for the previous reply. Actually after this clue I have checked with system log and found that there was segmentation fault occurs between kernel and php-fpm.

Can anyone please look in the log below and provide me the root cause of this problem ?


May 15 20:52:40 MKWK001 kernel: JBD: barrier-based sync failed on cciss!c0d1-8 - disabling barriers
May 15 21:53:54 MKWK001 kernel: blocks= 976707632 block_size= 512
May 15 21:53:54 MKWK001 kernel: blocks= 13674214832 block_size= 512
May 15 22:30:21 MKWK001 kernel: php-fpm[23766] general protection ip:73be19 sp:7fff37da1280 error:0 in php-fpm[400000+84a000]
May 15 22:31:22 MKWK001 kernel: php-fpm[23776] general protection ip:73be19 sp:7fff37da1280 error:0 in php-fpm[400000+84a000]
May 15 22:32:24 MKWK001 kernel: php-fpm[25042] general protection ip:73be19 sp:7fff37da1280 error:0 in php-fpm[400000+84a000]
May 15 22:33:26 MKWK001 kernel: php-fpm[25039] general protection ip:73be19 sp:7fff37da1280 error:0 in php-fpm[400000+84a000]
May 15 22:42:21 MKWK001 kernel: php-fpm[24707]: segfault at 0 ip 00007fc9884b67c1 sp 00007fff37d9ed28 error 4 in libc-2.10.2.so[7fc98843d000+14a000]

I think problem is related to php-fpm and kernel which increases the load in the server.

Your help is much valued and appreciated in this regard.