Welcome! Log In Create A New Profile

Advanced

Nginx fails to connect to PHP-FPM at 1400-1500 rps

Posted by Bogdan Rudas 
Bogdan Rudas
Nginx fails to connect to PHP-FPM at 1400-1500 rps
November 26, 2013 11:39AM
Hello!

I have PHP-FPM 5.4.4 running on Debian 7 and Nginx 1.4.1.
At ~1500rps PHP fails to accept new connections, there are lot of
connection in state SYN_SENT or SYN_RECV to tcp port of PHP. In 'normal'
conditions there are only few (or zero) such connection.

Here is my modification over default Debian's sysctl settings:

net.core.rmem_default=16777216
net.core.netdev_max_backlog=262144
net.core.somaxconn=262144

net.ipv4.tcp_syncookies=1
net.ipv4.tcp_max_orphans=262144
net.ipv4.tcp_max_syn_backlog=262144
net.ipv4.ip_local_port_range=1024 65535
net.ipv4.tcp_tw_reuse=1

Here is what I see in 'strace' of PHP worker process during problem period:

23:50:29.413415 write(67, "\1\6\0\1\3k\5\0X-Powered-By:
PHP/5.4.4-14+deb7u5\r\nAccept-Charset: UTF-8\r\nSet-Cookie:
_session=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT;
path=/\r\nSet-Cookie: _id=230486417; expires=Wed, 19-Nov-2014 19:50:29 GMT;
path=/\r\nExpires: Mon, 26 Jul 1997 05:00:00 GMT"..., 904) = 904
23:50:29.413486 shutdown(67, 1 /* send */) = 0
23:50:29.413530 recvfrom(67, "\1\5\0\1\0\0\0\0", 8, 0, NULL, NULL) = 8
23:50:29.413575 recvfrom(67, "", 8, 0, NULL, NULL) = 0
23:50:29.413681 close(67) = 0
23:50:29.413791 setitimer(ITIMER_PROF, {it_interval={0, 0}, it_value={0,
0}}, NULL) = 0
23:50:29.413848 accept(0, {sa_family=AF_INET, sin_port=htons(48538),
sin_addr=inet_addr("192.168.1.2")}, [16]) = 67
23:50:29.413897 time(NULL) = 1384890629
23:50:29.413932 times({tms_utime=230, tms_stime=102, tms_cutime=0,
tms_cstime=0}) = 1780793347
23:50:29.413992 poll([{fd=67, events=POLLIN}], 1, 5000) = 0 (Timeout)
23:50:34.418253 close(67) = 0
23:50:34.418319 accept(0, {sa_family=AF_INET, sin_port=htons(44562),
sin_addr=inet_addr("192.168.1.2")}, [16]) = 67
23:50:34.418389 time(NULL) = 1384890634
23:50:34.418420 times({tms_utime=230, tms_stime=102, tms_cutime=0,
tms_cstime=0}) = 1780793847
23:50:34.418472 poll([{fd=67, events=POLLIN}], 1, 5000) = 1 ([{fd=67,
revents=POLLIN}])
23:50:34.418525 read(67, "\1\1\0\1\0\10\0\0", 8) = 8


Please, note poll() timeout at 23:50:29.413992

So it look like under some conditions PHP either Linux kernel fails to work
correctly with new TCP connections, which are hangs in SYN_RECV or SYN_SENT
state, while PHP fails to get new connection descriptir to perform accept().

I have rlimit_files=128000 both for master process and pool workers, pm is
static, backlog=512.


--

---
You received this message because you are subscribed to the Google Groups "highload-php-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to highload-php-en+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 77
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready