Hello, I am testing Nginx-1.9.3/1.9.6 in FreeBSD 10.2.

I have set worker process 8 (= my cpu core), and then nginx really spawned 8 worker process.
However, nginx only use one worker process while I running apache benchmark and monitoring from top -P.
I observed that the working worker process had higher PRI, but nginx should balance worker loading in my thought.
Is this right in FreeBSD for some reason like its scheduler? or I have setup something wrong in my nginx.conf?

Below is my scratch from ps -aux while apache benchmark testing.

# ps -aux | grep nginx
ray 4870 49.0 0.2 32344 12676 - R 5:43PM 0:39.52 nginx: worker process (nginx)
root 4869 0.0 0.0 24152 3468 - Is 5:43PM 0:00.00 nginx: master process ./sbin/ngi
ray 4871 0.0 0.1 28248 7560 - I 5:43PM 0:00.01 nginx: worker process (nginx)
ray 4872 0.0 0.1 28248 7560 - I 5:43PM 0:00.01 nginx: worker process (nginx)
ray 4873 0.0 0.1 28248 7560 - I 5:43PM 0:00.01 nginx: worker process (nginx)
ray 4874 0.0 0.1 28248 7560 - I 5:43PM 0:00.01 nginx: worker process (nginx)
ray 4875 0.0 0.1 28248 7560 - I 5:43PM 0:00.01 nginx: worker process (nginx)
ray 4876 0.0 0.1 28248 7560 - I 5:43PM 0:00.01 nginx: worker process (nginx)
ray 4877 0.0 0.1 28248 7560 - I 5:43PM 0:00.01 nginx: worker process (nginx)

You can see only one process working.


In addition, I have tested max connections of Nginx to approve this problem.
Generally, max connections of Nginx should be worker process * worker connection, but I could only reached the limit of worker_connections(set =10000) based on nginx http stub status module:

Active connections: 9985
server accepts handled requests
5205900 5205900 5103413
Reading: 0 Writing: 1 Waiting: 9984

Below is my nginx.conf:

user ray;
worker_processes 8;
worker_rlimit_nofile 1000000;
error_log logs/error.log warn;

events {
use kqueue;
accept_mutex off;
worker_connections 10000;
}

http {
include mime.types;
default_type application/octet-stream;

access_log off;
sendfile on;
tcp_nopush on;
keepalive_timeout 65;
keepalive_requests 10000;


gzip on;
open_file_cache max=200000 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;
open_file_cache_errors on;
server {
listen 80 backlog=4096 reuseport so_keepalive=30::10;
server_name localhost;

#charset koi8-r;

#access_log logs/host.access.log main;

location / {
root /mnt/html;
index index.html index.htm;
}
location /nginx_status {
# Turn on stats
stub_status on;
access_log off;
}

error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /mnt/html;
}
}
}



Edited 1 time(s). Last edit at 11/17/2015 05:05AM by eaglerayp.

https://www.ruby-forum.com/topic/5362288

---
nginx for Windows http://nginx-win.ecsds.eu/

I have tested the configuration of accept_mutex on/off, it cannot solve this problem.
In my observation on "top -P", after started nginx only "one worker process" gets high priority and acts as I only use 1 worker process, seems no any load balance.

Have you tried tools like openload and such? apache benchmark does a burst and not a sustained pull which can easily get stuck on one worker.

---
nginx for Windows http://nginx-win.ecsds.eu/

I haven't try another traffic generate tools.
But, I used ab from 3 different clients, I think that is enough continuous traffic. The only nginx worker can make 1Gb tx traffic.
In Linux environment, we can see that Nginx really do load balance on those workers from top/mpstat monitor.

Is anyone using Nginx on FreeBSD also have this observation?
Does the problem come out because of the kqueue design, process scheduler, monitoring tool on FreeBSD, or it's an issue of Nginx on FreeBSD?

Although FreeBSD still has great performance with this situation on 1Gb NIC (I still don't have 10Gb testing environment), I just want to figure out this phenomenon.



Edited 1 time(s). Last edit at 11/20/2015 04:45AM by eaglerayp.