Welcome! Log In Create A New Profile

Advanced

Re: [PATCH 5 of 6] Upstream: allow any worker to resolve upstream servers

Aleksei Bavshin
February 09, 2023 11:46AM
On 2/5/2023 7:01 PM, J Carter wrote:
> Hi Aleksei,
>
> Why not permanently assign the task of resolving a given upstream server
> group (all servers/peers within it) to a single worker?
>
> It seems that this approach would resolve the SRV issues, and remove the
> need for the shared queue of tasks.
>
> The load would still be spread evenly for the most realistic scenarios -
> which is where there are many upstream server groups of few servers, as
> opposed to few upstream server groups of many servers.

The intent of the change was exactly opposite, to avoid any permanent
assignment of periodic tasks to a worker and allow another processes to
resume resolving if the original assignee exits, no matter if normally
or abnormally. I'm not even doing enough for that -- I should've kept
in-progress tasks at the end of the queue with expires = resolver
timeout + a small constant, and retry from another process when the
timeout is reached, but the idea was abandoned for a minuscule
improvement of insertion time. I expect to be asked to reconsider, as
patch 6/6 does not cover all the possible situations where we want to
recover a stale task.

A permanent assignment of a whole upstream would also require notifying
another processes that the upstream is no longer assigned if the worker
exits or consistently recovering that assignment over a restart of
single worker (e.g. after a crash - not a regular situation, but one we
should take into account nonetheless). And the benefit is not quite
obvious - I mentioned that resolving SRVs with a lot of records may take
longer to update the list of peers, but the situation with contention is
not expected to change significantly* if we pin these tasks to a single
worker as another worker may be doing the same for another upstream.
Most importantly, this isn't even a bottleneck. It only slightly
exacerbates an existing problem with certain balancers that already
suffer from the overuse of locks, in a configuration that was
specifically crafted to amplify and highlight the difference and is far
from these most realistic scenarios.

* Pending verification on a performance test stand.
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
https://mailman.nginx.org/mailman/listinfo/nginx-devel
Subject Author Views Posted

[PATCH 0 of 6] Upstream: re-resolvable servers.

Aleksei Bavshin 559 January 31, 2023 08:40PM

[PATCH 1 of 6] Upstream: re-resolvable servers

Aleksei Bavshin 192 January 31, 2023 08:40PM

[PATCH 2 of 6] Stream: re-resolvable servers

Aleksei Bavshin 151 January 31, 2023 08:40PM

[PATCH 3 of 6] Upstream: construct upstream peers from DNS SRV records

Aleksei Bavshin 170 January 31, 2023 08:40PM

[PATCH 4 of 6] Upstream: per-upstream resolver

Aleksei Bavshin 158 January 31, 2023 08:42PM

[PATCH 5 of 6] Upstream: allow any worker to resolve upstream servers

Aleksei Bavshin 146 January 31, 2023 08:42PM

Re: [PATCH 5 of 6] Upstream: allow any worker to resolve upstream servers

J Carter 188 February 05, 2023 10:02PM

Re: [PATCH 5 of 6] Upstream: allow any worker to resolve upstream servers

Aleksei Bavshin 130 February 09, 2023 11:46AM

Re: [PATCH 5 of 6] Upstream: allow any worker to resolve upstream servers

J Carter 158 February 09, 2023 07:06PM

[PATCH 6 of 6] Upstream: reschedule in-progress resolve tasks at worker exit

Aleksei Bavshin 136 January 31, 2023 08:42PM



Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 235
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready