Hi Ppl,

Ok I have two problems, for two days now, I tried everything I can think of, and didn't manage it to get working properly. So I need help.

First problem is with IPs when using proxy_cache/proxy_pass.
Here's my config.
One server, two nginx instances. Backend and Frontend (which I introduced yesterday).

Backend server is configured with fastcgi/php-fpm .. it was receiving hits directly, no caching, but since server started having problems with number of hits, I installed another nginx instance, under /usr/local/nginx-proxy and configured it like so:

server {
listen 91.185.x.x:80;
server_name m.domain.com;

#charset koi8-r;

access_log logs/x.log;
error_log logs/x.error;

location /admin { proxy_pass http://91.185.y.y:8080; }

location / {

error_page 599 = @bypass;
set $no_cache "0";

if ($args ~* uid=-1) {
set $no_cache "1";
}

if ($no_cache ~* "0") {
return 599;
}

if ($args ~* ^(.*)sid=(.+)(&?)$) {
set $args $1;
}

set $my_cache_key "$scheme$host$uri$args $no_cache";

proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Set-Cookie;
proxy_pass_header Set-Cookie;
proxy_pass_header P3P;

proxy_pass http://91.185.y.y:8080;
proxy_cache_key $my_cache_key;
proxy_cache my-cache;
proxy_cache_valid 200 302 10m;
proxy_cache_valid 404 1m;

}

location @bypass {
proxy_pass http://91.185.y.y:8080;
}
}

Sorry for masking ips, I just prefer privacy, so does my boss probably.

m.domain.com points to that .x.x ip, that is where nginx-proxy listens on (frontend). Backend listens on y.y (another ip).. and naturally, I proxy connection to that ip:port. That works as expected!

Ok, here's my backend server configuration now (the server which until yesterday was receiving hits directly:
server {
listen 91.185.y.y:8080;
server_name proxy.m.domain.com m2.domain.com www.m.domain.com

access_log logs/domain.log;
error_log logs/domain.log.error;

location / {
root /sites/m1.domain.com/htdocs;
index index.html index.php;

}

set_real_ip_from 91.185.x.x;
real_ip_header X-Real-IP;

location /admin/ {
root /sites/m1.domain.com/htdocs;
index index.php;
auth_basic "Restricted";
auth_basic_user_file /usr/local/nginx/conf/htpasswd;
}

valid_referers none blocked server_names *.globalmembersarea.com *.domain.com ~\kcore\. ~\another\. 202.87.41.1

if ($invalid_referer) {
rewrite ^/ http://www.gohere.com/ redirect;
}


if ($request_uri ~* "^/view_clip.php$") {
rewrite ^.*$ http://m.domain.com permanent;
location ~ \.php$ {
fastcgi_pass 127.0.0.1:8888;
fastcgi_index index.php;
# fastcgi_pass_header X-Real-IP;

fastcgi_param SCRIPT_FILENAME /sites/m1.domain.com/htdocs$fastcgi_script_name;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param GATEWAY_INTERFACE CGI/1.1;
fastcgi_param SERVER_SOFTWARE nginx/p0rn_0pt1m1z3d;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param MY_R_ADDR $http_x_forwarded_for;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
}

}

Now.. I tried everything, to get real ip to show when I use $_SERVER[X-Real-IP] or $_SERVER[X-Forwarded-For], but no go. Actually.. I'm getting something from X-Forwarded-For, but it reports isps ips, since my users are connecting from mobile, and mobiles usually go through isps proxy, and sometimes it doesn't report anything, and sometimes it reports my proxy ip, not real ip (as [remote_addr] when using without proxying).

Here's print_r of $_server array:

remote addr:91.185.x.x (my frontend nginx ip)
x real ip:
x forwarded:
Array ( [HOSTNAME] => [PATH] => /usr/local/bin:/usr/bin:/bin [TMP] => /tmp [TMPDIR] => /tmp [TEMP] => /tmp [OSTYPE] => [MACHTYPE] => [MALLOC_CHECK_] => 2 [USER] => nobody [HOME] => /nonexistent [FCGI_ROLE] => RESPONDER [SCRIPT_FILENAME] => /sites/m1.domain.com/htdocs/test_remote.php [QUERY_STRING] => [REQUEST_METHOD] => GET [CONTENT_TYPE] => [CONTENT_LENGTH] => [SCRIPT_NAME] => /test_remote.php [REQUEST_URI] => /test_remote.php [DOCUMENT_URI] => /test_remote.php [DOCUMENT_ROOT] => /usr/local/nginx/html [SERVER_PROTOCOL] => HTTP/1.0 [GATEWAY_INTERFACE] => CGI/1.1 [SERVER_SOFTWARE] => nginx/p0rn_0pt1m1z3d [REMOTE_ADDR] => 91.185.y.y [MY_R_ADDR] => [REMOTE_PORT] => 60222 [SERVER_ADDR] => 91.185.y.y [SERVER_PORT] => 8080 [SERVER_NAME] => m2.domain.com [HTTP_HOST] => 91.185.y.y:8080 [HTTP_CONNECTION] => close [HTTP_USER_AGENT] => Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_0 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Mobile/7A341 Safari/528.16 [HTTP_ACCEPT] => text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 [HTTP_ACCEPT_LANGUAGE] => en-us,en;q=0.5 [HTTP_ACCEPT_ENCODING] => gzip,deflate [HTTP_ACCEPT_CHARSET] => ISO-8859-1,utf-8;q=0.7,*;q=0.7 [HTTP_CACHE_CONTROL] => max-age=0 [PHP_SELF] => /test_remote.php [REQUEST_TIME] => 1267899156 )

As you can see... I have no X-Real-IP anywhere.. nor X-Forwarded-For .. in this array.

I have similliar configuration on apache, that is .. apache as backend, and I have X-Real-IP when using mod_rpaf, and my frontend nginx configuration is the same.

I need equivalent of $_SERVER[remote_addr], and I can't make it work. Anyone had experience using two nginx instances as proxy->web server ?
All I could find on net is either nginx->apache or load balancers->nginx ..

2. Second problem
My frontend nginx, hits limit of 100 mb of ram eventually.. after 20-30 minutes of uptime. I limit memory with ulimit -v, this also doesn't happen on other server where I have apache as a backend (which of course isn't the cause, but just so you know).
I raised the limit to 300 mb, to see what will happen.

I'm using nginx version:
nginx/0.7.64

Thanks,



Edited 1 time(s). Last edit at 03/06/2010 01:26PM by 0x90.