Dear Crowd,

First of all, a warm THANK YOU for this absolutely terrific piece of software that is going to be to Apache what are Postfix and Qmail to Sendmail! Using Nginx and PHP-FPM, I have been able to reduce my server load by a factor of 10, when compared to my old Apache-php-mod combination.
There is one niggling problem that persist, and where I hope to be able to get some help. Nginx does support SNI, and it DOES work with my setup (details to follow below), but ONLY for a while. Thereafter I receive incorrect certificate messages, as Nginx seems to pick the first available certificate (either the one of the default https server, or the first in line, if there is no default https one). I have tried to disable caches, etc, to no avail. Any thoughts? Here is my configuration, with domains www.synalinq.net and www.synalinq.org as examples:

FreeBSD 8.0, with current updates.
Jailed Ngnix as reverse proxy (called "secure.synalinq.vlan" on 127.0.1.105), doing all the http and https proxying for the application server "www.synalinq.vlan" on 127.0.1.106 in a jail called "www" serving PHP code via Nginx plus PHP-FPM (which works like a charm).
I will therefore ONLY give you the configuration of the Nginx reverse proxy in the "secure.synalinq.vlan" jail.
Port redirection is done via pf, and works fine, and takes requests from outside to port 80 and 443, respectively, and redirects them to the reverse proxy running on port 10080 (http) and 10443 (https). The reverse proxy directs all requests using http, only (!), to the www jail on port 80.
Internal and external name resolution is done by djbDNS, works great, too.
I am serving a few virtual domains on - you guessed it! - one I.P. address, which is why I have built an "include" tree for the vhosts and additional configuration files that would otherwise result in a very long and repetitive main configuration file. This way, additional domains are also much easier to handle.

Here is /usr/local/etc/nginx/nginx.conf
[code]
# general runtime settings
user www;
worker_processes 4;
worker_rlimit_nofile 8192;
events {
worker_connections 4096;
# use optimised polling for FreeBSD
use kqueue;
}


# general server parameters
http {
# MIME types
include mime.types;
default_type application/octet-stream;
index index.html index.htm index.php;

# size limits
client_body_buffer_size 1k;
client_header_buffer_size 1k;
client_max_body_size 10m;
large_client_header_buffers 3 3k;
connection_pool_size 256;
request_pool_size 4k;
server_names_hash_bucket_size 128;

# user limits
# define table called default_limit_zone which uses < 1 MByte to store session information
limit_zone default_limit_zone $binary_remote_addr 1m;
# limit simultaneous connections from one remote address to 10
limit_conn default_limit_zone 10;

# timeouts
client_body_timeout 60;
client_header_timeout 60;
keepalive_timeout 75 20;
send_timeout 60;

# general options
ignore_invalid_headers on;
recursive_error_pages on;
sendfile on;
charset off;

# tcp options
tcp_nodelay on;
tcp_nopush on;

# compression
gzip on;
gzip_buffers 8 32k;
gzip_comp_level 3;
gzip_http_version 1.1;
gzip_min_length 10;
gzip_types text/plain text/css text/xml application/xml application/xml+rss text/javascript application/x-javascript image/png image/png image/x-icon;
gzip_vary on;
gzip_static on;
gzip_proxied any;
gzip_disable "MSIE [1-6]\.";

# log format
log_format main 'From: $remote_addr | Time: $time_local | '
'Cache status: $upstream_cache_status | Cache control: $upstream_http_cache_control | Cache expires: $upstream_http_expires | '
'Request: $request | Status: $status | Body sent: ${body_bytes_sent}b | '
'Requested: ${request_length}b | Total sent: ${bytes_sent}b | Referrer: $http_referer | '
'User agent: $http_user_agent | Request time: ${request_time}ms | Compression: $gzip_ratio ';
access_log /var/log/nginx-access.log main;
error_log /var/log/nginx-error.log crit;

# proxy settings
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_redirect off;
proxy_intercept_errors on;

# proxy buffering settings
proxy_buffering on;
proxy_connect_timeout 90;
# avoid nginx 504 gateway time-out errors
#proxy_send_timeout 90;
proxy_send_timeout 300;
#proxy_read_timeout 120;
proxy_read_timeout 300;
proxy_buffers 32 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
# where to buffer large proxied requests to the filesystem
proxy_temp_path /var/tmp/nginx/proxy_temp 1 2;
# temporary file size, if served content is larger than the proxy buffer (reduces disk I/O)
proxy_max_temp_file_size 1000m;
# spool uploads to disk instead of clobbering downstream servers
client_body_temp_path /var/tmp/nginx/upload_temp 1 2;

# proxy cache settings
# set the proxy cache path and related parameters
# delete cache data, if not requested for 1 day, with maximum size of 128 MByte
proxy_cache_path /var/tmp/nginx/proxy_cache levels=1:2 keys_zone=default_proxy_cache:10m inactive=1d max_size=128m;
proxy_cache_key "$host$uri$is_args$args$cookie_user";
proxy_cache_valid 200 1h;
proxy_cache_methods GET HEAD;
# cache requests giving "202 Accepted", "301 Moved Permanently" and "302 Found" for 10 minutes
proxy_cache_valid 202 301 302 10m;

# include site-specific user limits and proxy cache settings
include include/server/*;

# ssl settings
ssl_protocols SSLv2 SSLv3 TLSv1;
ssl_prefer_server_ciphers on;
ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;


# http web proxy settings
server {
# we keep the backend http server running on its standard port
# and forward requests to port 10080 to port 81 using pf
listen 127.0.1.105:10080 default rcvbuf=8192 sndbuf=16384 backlog=32000 accept_filter=httpready;
# "_" is for handle all hosts that are not described by server_name
server_name secure.synalinq.net;
include include/synalinq.net/http+https/*;
include include/synalinq.net/http/*;
}


# https web proxy settings
server {
# we keep the backend https server running on its standard port
# and forward requests to port 443 to port 10443 using pf
listen 127.0.1.105:10443;
# "_" is for handle all hosts that are not described by server_name
server_name secure.synalinq.net;
# ssl configuration
ssl on;
ssl_certificate /etc/ssl/certs/secure.synalinq.net.crt.pem;
ssl_certificate_key /etc/ssl/private/secure.synalinq.net.key.pem;
keepalive_timeout 70;
include include/synalinq.net/http+https/*;
include include/synalinq.net/https/*;\
}


# virtual hosting
include include/vhosts/*;
}
[/code]

The above core configuration file gets supplemented by include/synalinq.net/http/default as follows, which handles settings specific to the http stanza of the default domain above:
[code]
# handle Drupal's secure areas and redirect to https
location ~ /admin(/.*) {
root /usr/local/www;
rewrite ^(.*) https://www.synalinq.net$1 permanent;
proxy_pass http://127.0.1.106:80;
#proxy_cache synalinq.net_proxy_cache;
break;
}
location ~ /user(.*) {
root /usr/local/www;
rewrite ^(.*) https://www.synalinq.net$1 permanent;
proxy_pass http://127.0.1.106:80;
#proxy_cache synalinq.net_proxy_cache;
break;
}

# handle default location
location / {
root /usr/local/www;
# limit simultaneous connections from one remote address to 10
limit_conn synalinq.net_limit_zone 10;
proxy_pass http://127.0.1.106:80;
proxy_cache synalinq.net_proxy_cache;
}
[/code]

This is common code for both the http and https stanzas of the default domain, sitting in include/synalinq.net/http+https/default:
[code]
# deny some crawlers
if ($http_user_agent ~* (HTTrack|HTMLParser|libwww)) {
return 444;
}

# redirect access denied error pages to the static page /40x.html
# comment out, if your CMS manages its error pages independently
error_page 400 401 402 403 405 406 407 409 410 411 412 413 414 415 416 417 418 422 423 424 425 426 450 /40x.html;
location = /40x.html {
root /usr/local/www;
}

# redirect server error pages to the static page /50x.html
# uncomment out, if your CMS manages its error pages independently
error_page 404 408 500 501 502 503 504 505 506 507 509 510 /50x.html;
location = /50x.html {
root /usr/local/www;
}

# disallow access to sensitive files
location ~* (/\..*|settings\.php$|\.(htaccess|engine|inc|info|install|module|profile|pl|po|sh|.*sql|theme|tpl(\.php)?|xtmpl)$|^(code-style\.pl|Entries.*|Repository|Root|Tag|Template))$ {
deny all;
}

location /status {
stub_status on;
access_log off;
allow 127.0.1.105;
deny all;
}

# required only when using purl, spaces & og for the following drupal
# modules: ajax_comments, watcher and fasttoggle
# the /og path should be modified to match your default for
# og/purl URL for organic groups
#location ~* ^/og {
#rewrite ^/og\-(.*)/ajax_comments/(.*)$ /index.php?q=ajax_comments/$2 last;
#rewrite ^/og\-(.*)/context/ajax-block-view$ /index.php?q=context/ajax-block-view last;
#rewrite ^/og\-(.*)/comment/reply/(.*)\?reload=1$ /index.php?q=comment/reply/$2&reload=1 last;
#rewrite ^/og\-(.*)/node/([0-9]+)/toggle/(.*)$ /index.php?q=node/$2/toggle/$3 last;
#rewrite ^/og\-(.*)/node/([0-9]+)/edit\?(.*)$ /index.php?q=node/$2/edit?$3 last;
#rewrite ^/og\-(.*)/user/([0-9]+)/watcher/toggle/(.*)$ /index.php?q=user/$2/watcher/toggle/$3 last;
#rewrite ^/(.*)$ /index.php?q=$1 last;
#}

# handle static files
location ~* \.(css|gif|flv|ico|js|jpe?g|mp3|mp4|pdf|png|swf|tif|wmv)$ {
access_log off;
proxy_pass http://127.0.1.106:80;
proxy_cache synalinq.net_proxy_cache;
expires 7d;
}

# handle dynamic content
location ~* \.(php) {
proxy_pass http://127.0.1.106:80;
proxy_cache synalinq.net_proxy_cache;
expires 60s;
}
[/code]

Finally, the following include/synalinq.net/https/default.conf handles settings that are only applicable to the https stanza of the default domain in the above configuration file:
[code]
# handle default location
# do not cache by proxy_cache
location / {
root /usr/local/www;
# limit simultaneous connections from one remote address to 10
limit_conn synalinq.net_limit_zone 10;
# continue to www jail, if we deal with Drupal's sensitive locations
if ($request_filename ~ /admin/) {
proxy_pass http://127.0.1.106:80;
break;
}
if ($request_filename ~ /user) {
proxy_pass http://127.0.1.106:80;
break;
}
# for non-sensitive locations, redirect to http
# comment out the rewrite, if using autocomplete in Drupal
rewrite ^/(.*) http://www.synalinq.net$1 permanent;
proxy_pass http://127.0.1.106:80;
break;
}
[/code]

Server-specific settings are included in include/server/synalinq.net:
[code]
# user limits
# define table called default_limit_zone which uses < 1 MByte to store session information
limit_zone synalinq.net_limit_zone $binary_remote_addr 1m;

# proxy cache settings
# set the proxy cache path and related parameters
# delete cache data, if not requested for 1 day, with maximum size of 128 Mbyte
proxy_cache_path /var/tmp/nginx/proxy_cache levels=1:2 keys_zone=synalinq.net_proxy_cache:10m inactive=1d max_size=128m;
[/code]


Virtual domains are handles by include/vhosts/synalinq.org:
[code]
# http web proxy settings
server {
# we keep the backend http server running on its standard port
# and forward requests to port 10080 to port 81 using pf
listen 127.0.1.105:10080;
server_name www.synalinq.org;
root /home/customers/web/sites/synalinq.org/drupal;
# include configuration files
include include/synalinq.org/http+https/*;
include include/synalinq.org/http/*;
}

# https web proxy settings
server {
# we keep the backend https server running on its standard port
# and forward requests to port 443 to port 10443 using pf
# set the default option only for the virtual domain that has
# to provide failsafe SSL services, e.g. www.caribcas.org
#listen 127.0.1.105:10443 default rcvbuf=8192 sndbuf=16384 backlog=32000 accept_filter=httpready;
listen 127.0.1.105:10443;
server_name www.synalinq.org;
root /home/customers/web/sites/synalinq.org/drupal;
# ssl configuration
ssl on;
ssl_certificate /etc/ssl/certs/www.synalinq.org.crt.pem;
ssl_certificate_key /etc/ssl/private/www.synalinq.org.key.pem;
keepalive_timeout 70;
# include configuration files
include include/synalinq.org/http+https/*;
include include/synalinq.org/https/*;
}
[/code]

The respective include/synalinq.org/http/default.org is:
[code]
# handle Drupal's secure areas and redirect to https
location ~ /admin(/.*) {
root /home/customers/web/sites/synalinq.org/drupal;
rewrite ^(.*) https://$host$1 permanent;
proxy_pass http://127.0.1.106:80;
#proxy_cache synalinq.org_proxy_cache;
break;
}
location ~ /user(.*) {
root /home/customers/web/sites/synalinq.org/drupal;
rewrite ^(.*) https://$host$1 permanent;
proxy_pass http://127.0.1.106:80;
#proxy_cache synalinq.org_proxy_cache;
break;
}

# handle default location
location / {
# limit simultaneous connections from one remote address to 10
limit_conn synalinq.org_limit_zone 10;
root /home/customers/web/sites/synalinq.org/drupal;
proxy_pass http://127.0.1.106:80;
proxy_cache synalinq.org_proxy_cache;
}
[/code]

The configuration file with settings common to both http and https for the virtual domain synalinq.org is include/synalinq.org/http+https/default.conf:
[code]
# deny some crawlers
if ($http_user_agent ~* (HTTrack|HTMLParser|libwww)) {
return 444;
}

# redirect access denied error pages to the static page /40x.html
error_page 400 401 402 403 405 406 407 409 410 411 412 413 414 415 416 417 418 422 423 424 425 426 450 /40x.html;
location = /40x.html {
root /usr/local/www;
}

# redirect server error pages to the static page /50x.html
error_page 404 408 500 501 502 503 504 505 506 507 509 510 /50x.html;
location = /50x.html {
root /usr/local/www;
}

# disallow access to sensitive files
location ~* (/\..*|settings\.php$|\.(htaccess|engine|inc|info|install|module|profile|pl|po|sh|.*sql|theme|tpl(\.php)?|xtmpl)$|^(code-style\.pl|Entries.*|Repository|Root|Tag|Template))$ {
deny all;
}

location /status {
stub_status on;
access_log off;
allow 127.0.1.105;
deny all;
}

# required only when using purl, spaces & og for the following drupal
# modules: ajax_comments, watcher and fasttoggle
# the /og path should be modified to match your default for
# og/purl URL for organic groups
#location ~* ^/og {
#rewrite ^/og\-(.*)/ajax_comments/(.*)$ /index.php?q=ajax_comments/$2 last;
#rewrite ^/og\-(.*)/context/ajax-block-view$ /index.php?q=context/ajax-block-view last;
#rewrite ^/og\-(.*)/comment/reply/(.*)\?reload=1$ /index.php?q=comment/reply/$2&reload=1 last;
#rewrite ^/og\-(.*)/node/([0-9]+)/toggle/(.*)$ /index.php?q=node/$2/toggle/$3 last;
#rewrite ^/og\-(.*)/node/([0-9]+)/edit\?(.*)$ /index.php?q=node/$2/edit?$3 last;
#rewrite ^/og\-(.*)/user/([0-9]+)/watcher/toggle/(.*)$ /index.php?q=user/$2/watcher/toggle/$3 last;
#rewrite ^/(.*)$ /index.php?q=$1 last;
#}

# handle static files
location ~* \.(css|gif|flv|ico|js|jpe?g|mp3|mp4|pdf|png|swf|tif|wmv)$ {
access_log off;
proxy_pass http://127.0.1.106:80;
proxy_cache synalinq.org_proxy_cache;
expires 7d;
}

# handle dynamic content
location ~* \.(php) {
proxy_pass http://127.0.1.106:80;
proxy_cache synalinq.org_proxy_cache;
expires 60s;
}
[/code]

Correspondingly, the https stanza for synalinq.org gets supplemented by include/synalinq.org/https/default.conf as follows:
[code]
# handle default location
# do not cache by proxy_cache
location / {
root /home/customers/web/sites/synalinq.org/drupal;
# limit simultaneous connections from one remote address to 10
limit_conn synalinq.org_limit_zone 10;
# continue to www jail, if we deal with Drupal's sensitive locations
if ($request_filename ~ /admin/) {
proxy_pass http://127.0.1.106:80;
break;
}
if ($request_filename ~ /user) {
proxy_pass http://127.0.1.106:80;
break;
}
# for non-sensitive locations, redirect to http
# comment out the rewrite, if using autocomplete in Drupal
rewrite ^/(.*) http://www.synalinq.org$1 permanent;
proxy_pass http://127.0.1.106:80;
break;
}
[/code]

Finally, the synalinq.org-specific server configuration is in include/server/synalinq.org as follows:
[code]
# user limits
# define table called default_limit_zone which uses < 1 MByte to store session information
limit_zone synalinq.org_limit_zone $binary_remote_addr 1m;

# proxy cache settings
# set the proxy cache path and related parameters
# delete cache data, if not requested for 1 day, with maximum size of 128 Mbyte
proxy_cache_path /var/tmp/nginx/proxy_cache levels=1:2 keys_zone=synalinq.org_proxy_cache:10m inactive=1d max_size=128m;
[/code]

As you can see, the virtual domain is set up in parallel to the "default" domain, using a tree of configuration file snippets as includes.

Again, this set up works. But after ten minutes, or even one hour, I get certificate errors, as the browser starts to pick up the wrong certificate, i.e. the one for www.synalinq.org, while browsing www.synalinq.net, or vice versa. I can nail down the certificate use by defining a default domain by saying:
[code]
listen 127.0.1.105:10443 default rcvbuf=8192 sndbuf=16384 backlog=32000 accept_filter=httpready;
[/code]
in the respective https stanza, but then the non-default domains haven even more problems with their own certificates!

Any help would be greatly appreciated! Thanks a lot,

Chris

Dear All,

Any idea anyone regarding this issue? I would really love to use SNI and Nginx properly! Thanks a lot,

Chris