I'm looking for a way to do consistent hashing without any 3rd-party modules or perl/lua. I came up with the idea of generating a split_clients and list of upstreams via script, so we can add/remove backends without blowing out the cache on each upstream when a backend server is added, removed or otherwise offline.

What I have looks like the config below. The example only includes 16 upstreams for clarity, and is generated by sorting by the SHA1 hash of server names for each upstream bucket along with the bucket name.

Unfortunately, to get an even distribution of requests to upstream buckets with consistent hashing, I am actually going to need at least 4096 upstreams, and the corresponding number of entries in split_clients.

Will 4096 entries in single split_clients block pose a performance issue? Will split_clients have a distribution problem with a small percentage like "0.0244140625%"? How many "buckets" does the hash table for split_clients have (it doesn't seem to be configurable)?

Thanks for any insights. I haven't actually built a test environment for this yet as the setup is quite a bit of work, so I want to find out if I am doing something stupid before committing a lot of time and resources.

upstream hash0 {server 192.168.47.104; server 192.168.47.102 backup;}
upstream hash1 {server 192.168.47.104; server 192.168.47.102 backup;}
upstream hash2 {server 192.168.47.105; server 192.168.47.104 backup;}
upstream hash3 {server 192.168.47.101; server 192.168.47.103 backup;}
upstream hash4 {server 192.168.47.102; server 192.168.47.101 backup;}
upstream hash5 {server 192.168.47.101; server 192.168.47.104 backup;}
upstream hash6 {server 192.168.47.103; server 192.168.47.102 backup;}
upstream hash7 {server 192.168.47.101; server 192.168.47.105 backup;}
upstream hash8 {server 192.168.47.102; server 192.168.47.105 backup;}
upstream hash9 {server 192.168.47.105; server 192.168.47.102 backup;}
upstream hashA {server 192.168.47.105; server 192.168.47.103 backup;}
upstream hashB {server 192.168.47.103; server 192.168.47.105 backup;}
upstream hashC {server 192.168.47.103; server 192.168.47.105 backup;}
upstream hashD {server 192.168.47.103; server 192.168.47.104 backup;}
upstream hashE {server 192.168.47.101; server 192.168.47.105 backup;}
upstream hashF {server 192.168.47.104; server 192.168.47.101 backup;}

split_clients "${scheme}://${host}${request_uri}" $uhash {
6.25% hash0;
6.25% hash1;
6.25% hash2;
6.25% hash3;
6.25% hash4;
6.25% hash5;
6.25% hash6;
6.25% hash7;
6.25% hash8;
6.25% hash9;
6.25% hashA;
6.25% hashB;
6.25% hashC;
6.25% hashD;
6.25% hashE;
* hashF;}

location /foo {
proxy_pass http://$uhash;}