proxy_pass help
May 12, 2015 06:39AM
Hi Guy's.

My current config currently looks up id's from a flat file, takes that id number and then passes traffic to the relevant cluster of servers based on that id number.

Specifics of my config file:

upstream cluster01 {
least_conn;
server 10.105.137.13 slow_start=30s;
server 10.105.137.14 slow_start=30s;
server 10.105.137.15 slow_start=30s;
zone cluster01 64k;
}

upstream cluster02 {
least_conn;
server 10.105.137.39 slow_start=30s down;
server 10.105.137.40 slow_start=30s down;
server 10.105.137.42 slow_start=30s down;
zone cluster02 64k;
}


map $siteid $clusterid {
default 1;
include conf.d/cluster.mapping;
}

proxy_pass http://cluster0$clusterid; # detected cluster id is used here



What i would like to do is pass traffic to clusters with different naming conventions but i'm not sure how. The problem i have is that the cluster names don't actually match the ID numbers so i'd like to be able to use any cluster name i want and only look at the specific ID number when i proxy_pass.

I'd like to be able to write a config to look a little something like this:

upstream cluster01_id6 {
least_conn;
server 10.105.137.13 slow_start=30s;
server 10.105.137.14 slow_start=30s;
server 10.105.137.15 slow_start=30s;
zone cluster01_id6 64k;
}

upstream cluster02_id7 {
least_conn;
server 10.105.137.39 slow_start=30s down;
server 10.105.137.40 slow_start=30s down;
server 10.105.137.42 slow_start=30s down;
zone cluster02_id7 64k;
}

map $siteid $clusterid {
default 1;
include conf.d/cluster.mapping;
}

proxy_pass http://*_id$clusterid; # detected cluster id is used here

I understand that the '*_id' syntax is incorrect but this is how i'd like to run it..... So the name of the zone is irrelevant and proxy_pass is just reading the _id? section which directly relates to the id in cluster.mapping but i can name my zones whatever i like. This means my clusters are named correctly and i can name them whatever i want. : )

I hope that's clear enough. If i can't it means i'll have to rename all my clusters so the id's match which i don't really want to do.
Re: proxy_pass help
May 12, 2015 06:53AM
There is one issue here, a file read is only read at startup, any changes won't be passed until a restart.

Second, using such variable stuff is best done with Lua, https://github.com/chaoslawful/lua-nginx-module
discussions here https://groups.google.com/forum/#!forum/openresty-en
for example;

init_by_lua '
dsstr = {}; ds_as = 0;
local file = io.open("conf.d/cluster.mapping", "r");
for i = 1, count, 1 do
ds_line = file:read();
table.insert (dsstr, ds_line);
ds_as = ds_as + 1;
end
io.close (file);
';

To get flat data into an array,

Then you need init_worker_by_lua; with a timer (ngx.timer.at) to periodically check if the flatfile has changed.

Then you need set_by_lua in a location block to set values based on your Lua array.

It might be better to store this data in a database and query that for each request (which can be cached), for example see 'nginx GEOIP-MYSQL with Cache Non-Blocking.conf' in this archive http://nginx-win.ecsds.eu/download/ngxLuaDB-1.1.zip

---
nginx for Windows http://nginx-win.ecsds.eu/
Re: proxy_pass help
May 12, 2015 07:03AM
Thanks.

Looking at the DB would be great as this information is currently stored there now. Caching would be great too incase the DB is down for some reason but is there a way to 'force' a refresh if any of the ID's change for some reason?
Re: proxy_pass help
May 12, 2015 08:56AM
atqtjay Wrote:
----------------------------------------------------> some reason but is there a way to 'force' a refresh if any of the ID's
> change for some reason?

You could make a location block which accepts ID's which should be purged(or refreshed) from cache. And use a script (scheduled or not) to initiate this.

Or use a ngx timer object to refresh the cache once every 5min.

---
nginx for Windows http://nginx-win.ecsds.eu/
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 258
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 500 on July 15, 2024
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready