Hi Robert, Thanks for your input on this. I sincerely appreciate it.

At the moment I am trying to get the most out of DNS (on Docker)… as a way to discover services (individual containers and replicas of containers).

If I can solve the issue without having to introduce a new tool or container, I would really like to. In that sense, I am not trying to use the perfect tool for every job… I am trying to maximize the usage of existing tools to minimize the number of different tools. Yes, this is my own design decision, and there is a price for that.

Coming back to consistent hashing, it still does not make sense to me that nginx falls back to round robin for individual hostnames that map to >1 IP, because that breaks the point of consistent hashing any time a node enters/leaves the hostname. Regardless of my own use case, when does this behavior actually make sense?

Your point about dynamically populating the upstream block makes alot of sense. +1 !!
I could actually use that work-around without a new service discovery mechanism. I could actually run a script in the background to get the multiple IP addresses that the hostname maps to… and update the nginx conf accordingly.

Still, nginx already has resolver… and a TTL… so pinging DNS is something nginx is already doing… seems strange I should have to re-invent that wheel.

I wonder if someone would chime in and say that this behavior is a feature (and if so, what is the use case?) or is it a bug (or possibly oversight?)?

Best,
Ian




> ------------------------------
>
> Message: 3
> Date: Mon, 3 Feb 2020 13:28:20 -0800
> From: Robert Paprocki <rpaprocki@fearnothingproductions.net>
> To: nginx@nginx.org
> Subject: Re: upstream hash consistent seems to depend on order of DNS
> entries (Maxim Dounin)
> Message-ID:
> <CAEuShZcwsqndFmQbp-UUmXa3-pMJzKcBf8jNbxgAuYo0chDK7Q@mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
>> In my case, the number of hosts will change over time, and I can?t update
> the nginx config. So I thought it would make sense to use a hostname that
> resolves to many IPs. This would be a scalable solutioin
>
> In that case, it makes sense to use a templating tool to dynamically
> populate the contents of the upstream{} block. Hook it in with your service
> discovery/registration system.
>
> On Mon, Feb 3, 2020 at 12:59 PM Ian Morris Nieves <imnieves@gmail.com>
> wrote:
>
>> Hi Maxim,
>>
>> Thank you for your response.
>> OK, I understand what you have said.
>>
>> This seems slightly strange to me. We both understand the purpose/point
>> of hashing (consistent or not consistent). Why does it then make sense to
>> fall-back to round robin?
>>
>> In my case, the number of hosts will change over time, and I can?t update
>> the nginx config. So I thought it would make sense to use a hostname that
>> resolves to many IPs. This would be a scalable solutioin.
>>
>> I would think that the more correct solution would be to hash the IP
>> addresses that all the names resolve to?. not the names themselves.
>>
>> I must be missing/mis-understanding something. Does the current
>> implementation/solution make sense to you?
>>
>> Best,
>> Ian
>>
>>>
>>> Message: 2
>>> Date: Mon, 3 Feb 2020 16:05:38 +0300
>>> From: Maxim Dounin <mdounin@mdounin.ru>
>>> To: nginx@nginx.org
>>> Subject: Re: upstream hash consistent seems to depend on order of DNS
>>> entries
>>> Message-ID: <20200203130538.GR12894@mdounin.ru>
>>> Content-Type: text/plain; charset=utf-8
>>>
>>> Hello!
>>>
>>> On Sat, Feb 01, 2020 at 11:48:00AM -0800, Ian Morris Nieves wrote:
>>>
>>>> Here is the setup:
>>>> - I am running nginx in a single docker container and it has an
>>>> upstream to a docker service which is composed of 3 docker
>>>> containers (which happens to be php-fpm)
>>>> - the service is configured to _not_ expose a single virtual ip
>>>> address (vip), instead the service exposes the ip addresses of
>>>> all 3 containers through docker?s built-in DNS. When this DNS
>>>> is asked for the IP address of the service it will respond with
>>>> a list of 3 IP address but the list will rotate in round-robin
>>>> fashion each time a lookup is performed. Thus the first IP in
>>>> the list will not be the same for any 2 consecutive lookups.
>>>>
>>>> My first question is:
>>>> Is it the correct behavior that consistent hashing depends on
>>>> the order of IP addresses in the DNS query response? I can
>>>> imagine arguments either way, and it is possible that this
>>>> critical detail is outside the scope of consistent hashing. I
>>>> will also forward this question to the author of Ketama.
>>>
>>> Consistent hashing uses the _name_ as written in the
>>> configuration, not the IP addresses the name resolves to.
>>>
>>> If a name resolves to multiple IP addresses, these addresses are
>>> considered equal and requests are distributed between them using
>>> round-robin balancing.
>>>
>>> That is, to balance multiple servers (containers) using consistent
>>> hashing, you have to configure an upstream block with multiple
>>> "server" directives in it.
>>>
>>> --
>>> Maxim Dounin
>>> http://mdounin.ru/
>>>
>>
>> _______________________________________________
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.nginx.org/pipermail/nginx/attachments/20200203/ee22d974/attachment-0001.htm
>
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx