Hi,

I'm trying to set up NGinx to act as a load balancer to a few endpoints. For example, https://nlb.com/portal will forward requests to https://p1.com:7443/arcgis or https://p2.esri.com:7443/arcgis and http://nlb.com/portal will forward requests to https://p1.esri.com:7080/arcgis or https://p2.esri.com:7080/arcgis. I'm finding that by adding the different location for the URI, the request actually goes to https://p1.esri.com/arcgis. It seems that by adding a location within proxy_pass, it sends the request to 80/443. By leaving off the /arcgis/ from proxy_pass, I could see about rewriting the URL, but I'm interested in why this doesn't work. My config file is pasted below:


#user nobody;
worker_processes 1;

error_log logs/error.log;
error_log logs/error.log notice;
error_log logs/error.log info;

#pid logs/nginx.pid;


events {
worker_connections 1024;
}


http {
include mime.types;
default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';

access_log logs/access.log main;

sendfile on;
#tcp_nopush on;

#keepalive_timeout 0;
keepalive_timeout 65;

#gzip on;

upstream conHTTP {
server p1.com:7080;
server p2.com:7080;
}

upstream conHTTPS {
server p1.com:7443;
server p2.com:7443;
}

server {
listen 80;
server_name nlb.com;

location /portal {
proxy_pass http://conHTTP/arcgis;
proxy_set_header X-Forwarded-Host nlb.com;
proxy_redirect /arcgis/ http://nlb.com/portal/;
}

error_page 404 /404.html;

# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}

server {
listen 443 ssl;
server_name nlb.com;

ssl_certificate nlb.cer;
ssl_certificate_key nlb.key;

ssl_session_cache shared:SSL:1m;
ssl_session_timeout 5m;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;

location /portal/ {
proxy_pass https://conHTTPS/arcgis/;
proxy_set_header X-Forwarded-Host nlb.com;
proxy_redirect https://conHTTPS/arcgis/ https://nlb.com/portal/;
}
error_page 404 /404.html;

# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}

Thanks for any help.

location /portal {
proxy_pass http://conHTTP/portal;
proxy_set_header X-Forwarded-Host nlb.com;
}

Or;

location /portal {
proxy_pass http://conHTTP;
proxy_set_header X-Forwarded-Host nlb.com;
}

---
nginx for Windows http://nginx-win.ecsds.eu/

Thank you for the response. The problem is that "portal" doesn't exist no the backend servers. The request to http://nlb.com/portal goes to http://p1.com:7080/portal or http://p2.com:7443/portal, which doesn't exist. My attempt at doing:

proxy_pass http://conHTTP/arcgis

was so that the request comes in as http://nlb.com/portal and it gets translated to http://p1.com:7080/arcgis or http://p2.com:7080/arcgis. If the request comes in as https://nlb.com/portal, it goes to https://p1.com:7443/arcgis or https://p2.esri.com:7443/arcgis.

Then you need to do some rewriting, ea.

rewrite /portal/([^/]+) /arcgis/$1 break;
proxy_pass ........

---
nginx for Windows http://nginx-win.ecsds.eu/

Hm, bear with me while I recap the behavior I see. So this is the new config:

#user nobody;
worker_processes 1;

error_log logs/error.log;
error_log logs/error.log notice;
error_log logs/error.log info;

#pid logs/nginx.pid;


events {
worker_connections 1024;
}


http {
include mime.types;
default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';

access_log logs/access.log main;

sendfile on;
#tcp_nopush on;

#keepalive_timeout 0;
keepalive_timeout 65;

#gzip on;

upstream conHTTP {
server p1.com:7080;
server p2.com:7080;
}

upstream conHTTPS {
server p1.com:7443;
server p2.com:7443;
}

server {
listen 80;
server_name nlb.com;

location /portal {
rewrite /portal/([^/]+) /arcgis/$1 break;
proxy_pass http://conHTTP;
proxy_set_header X-Forwarded-Host nlb.com;
}

error_page 404 /404.html;

# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}

server {
listen 443 ssl;
server_name nlb.com;

ssl_certificate nlb.cer;
ssl_certificate_key nlb.key;

ssl_session_cache shared:SSL:1m;
ssl_session_timeout 5m;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;

location /portal {
rewrite /portal/([^/]+) /arcgis/$1 break;
proxy_pass https://conHTTPS;
proxy_set_header X-Forwarded-Host nlb.com;
}
error_page 404 /404.html;

# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}}

When I make a request to nlb.com:

GET /portal/home/ HTTP/1.1
Host: nlb.com
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8

The response returns:

HTTP/1.1 302 Found
Server: nginx/1.9.15
Date: Wed, 04 May 2016 15:30:01 GMT
Location: https://nlb.com/arcgis/home/
Transfer-Encoding: chunked
Connection: keep-alive

Which sends the next request to the URI Location header:

GET /arcgis/home/ HTTP/1.1
Host: dev001316.esri.com
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8

Which returns a 404

HTTP/1.1 404 Not Found
Server: nginx/1.9.15
Date: Wed, 04 May 2016 15:30:01 GMT
Content-Type: text/html
Content-Length: 571
Connection: keep-alive

The Nginx error logs show an error reaching the arcgis/home page on it's on machine instead of the backend servers:

2016/05/04 08:30:42 [error] 4844#3052: *421 "C:\nginx-1.9.15/html/arcgis/home/index.html" is not found (3: The system cannot find the path specified), client: 10.29.78.47, server: nlb.com, request: "GET /arcgis/home/ HTTP/1.1", host: "nlb.com"

I tried to add a proxy_redirect:

proxy_redirect https://nlb.com/arcgis/ https://nlb.com/portal/;

But that had a worse effect. When I make a request to https://nlb.com/portal/home/:

GET /portal/home/ HTTP/1.1
Host: nlb.com
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8

It gets redirected to https://conhttps/arcgis/home/

HTTP/1.1 302 Found
Server: nginx/1.9.15
Date: Wed, 04 May 2016 15:40:35 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Location: https://conHTTPS/arcgis/home/

Any ideas?

Don't use proxy_redirect.
Adjust proxypass / rewrite to make sure they point to folders that exist.

---
nginx for Windows http://nginx-win.ecsds.eu/

I guess my question is why does adding a rewrite rule send the request to the nginx machine instead of using the backend servers? The path exists on the backend servers.

It should not direct to nginx but change the request path and pass that on to the backend(s), in combination with 'root' being set accordingly.

---
nginx for Windows http://nginx-win.ecsds.eu/

That's what I would think! So a configuration like below seems it should work?

upstream conHTTPS {
server p1.com:7443;
server p2.com:7443;
}

location /portal {
rewrite /portal/([^/]+) /arcgis/$1 break;
proxy_pass https://conHTTPS;
proxy_set_header X-Forwarded-Host nlb.com;
}

From what the help you've provided and what I've researched, this should work, but I'm not seeing the expected results.

> "C:\nginx-1.9.15/html/arcgis/home/index.html" is not found

Your 'root' is set wrong, obviously what you want returned doesn't live there.

---
nginx for Windows http://nginx-win.ecsds.eu/

Hm, so I think I can make my question a bit clearer. The configuration below works:

#upstream conHTTPS {
# server p1.com:7443;
# }

location /portal {
proxy_pass https://p1.com:7443/arcgis;
proxy_set_header X-Forwarded-Host nlb.com;
}

In this case, I'm not using an upstream server and am simply saying send requests to https://nlb.com/portal to https://p1.com:7443/arcgis/

However, the following configuration does not work:

upstream conHTTPS {
server p1.com:7443;
}

location /portal {
proxy_pass https://conHTTPS/arcgis;
proxy_set_header X-Forwarded-Host nlb.com;
}

It strips off the port from the upstream server and sends the request from https://nlb.com/portal to https://p1.com/arcgis. That is my main issue.



Edited 1 time(s). Last edit at 05/06/2016 12:22PM by jtquinn617.

Some google results:

https://tenzer.dk/nginx-with-dynamic-upstreams/
http://stackoverflow.com/questions/24350124/port-number-getting-stripped-out-of-url
https://forum.nginx.org/read.php?2,239790,239794#msg-239794

---
nginx for Windows http://nginx-win.ecsds.eu/

https://tenzer.dk/nginx-with-dynamic-upstreams/

For this one, it seems like it discusses how endpoints are cached. The closing words don't follow what I'm trying to do:

"Just to make it clear, this doesn't only affect setups using an ELB as an upstream server, but applies to any configuration where you use a changing DNS record as your upstream server in Nginx."

http://stackoverflow.com/questions/24350124/port-number-getting-stripped-out-of-url

I had taken a look at this before I originally posted, and it doesn't seem relevant. It seems like the fix for this was within the application, not the nginx as a reverse proxy.

https://forum.nginx.org/read.php?2,239790,239794#msg-239794

I'm using 1.9.15 so I'm not sure if this one is relevant.

I've attached a screenshot of the redirect if it helps.

Thanks for doing that search, I'll address each site:

https://tenzer.dk/nginx-with-dynamic-upstreams/

This seems to be relevant to DNS names changing:

"Closing words
Just to make it clear, this doesn't only affect setups using an ELB as an upstream server, but applies to any configuration where you use a changing DNS record as your upstream server in Nginx."

Mine aren't, so I'm not sure that's the isuse.

http://stackoverflow.com/questions/24350124/port-number-getting-stripped-out-of-url

I can't really sort out what the fix was. It sounds like it was addressed within the application and not Nginx.

https://forum.nginx.org/read.php?2,239790,239794#msg-239794

I'm using 1.9.15 so I'm not sure if this is relevant.

Try the direct IP address in upstream, if that works (port is kept) then it is for sure a dns issue.

It might be an application issue, for example you get data on port 8000 and proxy this to 9000 but the application expects data on 8000 so after the first connect the application removes 9000 and re-inserts 8000, this is typical tomcat behavior until you tell the tomcat application data is coming from 9000.

---
nginx for Windows http://nginx-win.ecsds.eu/