> procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
> r b swpd free buff cache si so bi bo in cs us sy id
> wa st
> 5 21 1165724 542180 97228 261580 0 0 0 3812 12690 10142 11 9
> 38 41 0
> 17 2 1165724 543352 97244 261508 0 0 0 5020 7574 5605 4 5
> 52 38 0
> 3 8 1165724 542296 97248 261332 0 0 0 3492 10717 7997 6 7
> 65 23 0
> 1 9 1165724 545036 97248 261160 0 0 0 3908 6850 4321 3 5 61
> 30 0
> 0 14 1165724 544320 97248 260940 0 0 0 3236 6474 4485 5 5 28
> 62 0
> 0 23 1165724 541884 97260 261432 0 0 0 4212 11362 8626 11 8
> 25 55 0
> 0 14 1165724 541740 97268 261436 0 0 0 3968 6058 4598 4 4 41
> 51 0
> 0 2 1165724 540876 97276 261324 0 0 0 5792 6281 4923 4 5 53
> 38 0
> 1 9 1165716 540732 97296 261260 32 0 32 3780 14479 11158 13 9
> 53 25 0
> 1 9 1165716 541632 97296 261316 0 0 0 4248 6083 3842 1 3 71
> 24 0
> 3 33 1165716 541016 97296 261372 0 0 4 3696 7933 5780 6 9 37
> 49 0
> 2 29 1165716 541864 97300 261488 0 0 4 4128 6081 4607 6 5 40
> 50 0
> 1 45 1165716 538448 97308 261520 0 0 0 3648 12608 10577 11 7
> 27 56 0
> 5 15 1165716 541024 97308 261124 0 0 0 3844 12440 9727 10 9
> 34 47 0

I don't think there's any network problems, it looks much more like your
system is running out of disk IO bandwidth. There's a continuous stream of
data being sent to disk (bo). The actual rate (~4000 blocks/s) isn't that
high if it's purely linear write, but my *guess* is you're getting quite a
bit of random IO going on. That's causing lots of processes to block.

What's probably also happening is that your sshd and vpn server are being
swapped out (swapped = 1.1G, free = 540M), so when you try and connect, they
have to be swapped back in. Because of all the IO being pushed to disk, the
reads are starved, and swapping them in takes a while, causing your
connections to time out.

Basically the question to answer, why and what are your php processing
writing so much to local disk continuously for?

One thing that can help.

echo 1 > /proc/sys/vm/block_dump; sleep 10; echo 0 > /proc/sys/vm/block_dump

Then look at:

dmesg | less

And scroll down till you see stuff like:

[20394434.668776] pdflush(320): WRITE block 15744 on sda5
[20394434.668783] pdflush(320): WRITE block 15752 on sda5
[20394434.689927] pdflush(320): WRITE block 15760 on sda5
[20394434.699997] pdflush(320): WRITE block 173474328 on sda5
[20394434.772645] cleanup(32657): dirtied inode 4909405 (55FD14AE95D) on
sda5
[20394434.772674] cleanup(32657): dirtied inode 4909405 (55FD14AE95D) on
sda5
[20394435.567579] nginx(14952): dirtied inode 3215421100 (0000623907) on
tmpfs
[20394435.603644] kjournald(2281): WRITE block 901856 on sda1
[20394435.603687] kjournald(2281): WRITE block 901864 on sda1
[20394435.603696] kjournald(2281): WRITE block 901872 on sda1

And see if you can find out what files the php processes are writing to.

Rob


_______________________________________________
nginx mailing list
nginx@nginx.org
http://nginx.org/mailman/listinfo/nginx