Welcome! Log In Create A New Profile

Advanced

Re: [PATCH 1 of 9] Upstream: re-resolvable servers

July 11, 2024 05:06PM
Thank you Aleksi, I think you are right. Thats the only one of our use
cases using jdomain that I was concerned wouldn't be met by this
feature.

On Fri, 12 Jul 2024 at 06:32, Aleksei Bavshin <a.bavshin@nginx.com> wrote:
>
> On 7/11/2024 12:58 PM, Mathew Heard wrote:
> > On Fri, 12 Jul 2024 at 05:42, Aleksei Bavshin <a.bavshin@nginx.com> wrote:
> >>
> >> On 7/11/2024 12:15 PM, Mathew Heard wrote:
> >>> Do you happen to know if there remains any gap in obvious capability
> >>> provided by a module like jdomain compared to this?
> >>
> >> An obvious difference is that the jdomain directive performs resolution
> >> on demand instead of periodic polling, but that's a design choice rather
> >> than a feature gap. Other that that I don't see anything missing.
> >>
> >
> > Thinking about it thats actually a surprising feature (rather than
> > minor implementation detail). We don't want the service to ever fail
> > to start, or to take any longer than it has to start. For this reason
> > we use jdomain (and a local DNS cache). By requesting on the first
> > request jdomain allows the server to start without resolution delays
> > in all cases.
> >
> > Imagine the configuration may contain xxxx.yyyy.com which takes a
> > prolonged time to resolve (for any reason) we would rather that the
> > server is able to start and for requests to that server block fail
> > until the resolution of xxxx.yyyy.com becomes healthy.
> >
> > However if the resolution of xxxx.yyyy.com resolves quickly (healthy)
> > no interruption on start occurs. Beyond the time it takes to resolve
> > the name, which with the aid of a DNS cache is immediately available
> > on server reload.
> >
> > AFAIK is this a feature gap with re-resolution?
> >
> > I guess this is not really a feature gap as when using domains in a
> > static configuration within nginx the behaviour is already like this.
>
> `server ... resolve;` actually behaves quite similar to what you
> describe: name resolution is deferred to runtime and doesn't block or
> delay the startup.
> The main difference is that the resolution starts as soon as the worker
> processes are up instead of waiting for the first request, and we update
> the expired entries in the background according to the response TTL.
>
> >
> >>>
> >>> Perhaps it would be worth checking to ensure nothing obvious is not
> >>> implemented?
> >>>
> >>> The one that I see is the ability to control if a resolution is IPv4,
> >>> IPv6 or mixed. Is this something that would be useful for this feature?
> >>
> >> That is already implemented in the resolver directive. If the http block
> >> scope is too large, one of the patches in the series allows configuring
> >> resolver per upstream.
> >>
> >> upstream backend {
> >> zone upstream_dynamic 64k;
> >> resolver 127.0.0.1 ipv4=off ipv6=on;
> >>
> >> server example.com resolve;
> >> };
> >>
> >
> > Thank you for your eyes. I was unaware of upstream level resolver
> > configuration. That does indeed look like it would provide the same
> > capability.
> >
> >
> >>>
> >>> On Fri, 12 Jul 2024 at 02:40, Aleksei Bavshin <a.bavshin@nginx.com
> >>> <mailto:a.bavshin@nginx.com>> wrote:
> >>>
> >>> On 7/9/2024 9:22 AM, Roman Arutyunyan wrote:
> >>> > Hi,
> >>> >
> >>> > On Mon, Jul 08, 2024 at 06:20:58PM +0400, Roman Arutyunyan wrote:
> >>> >> Hi,
> >>> >>
> >>> >> On Thu, Jun 13, 2024 at 03:28:56PM -0700, Aleksei Bavshin wrote:
> >>> >>> # HG changeset patch
> >>> >>> # User Ruslan Ermilov <ru@nginx.com <mailto:ru@nginx.com>>
> >>> >>> # Date 1392462754 -14400
> >>> >>> # Sat Feb 15 15:12:34 2014 +0400
> >>> >>> # Node ID 56aeae9355df8a2ee07e21b65b6869747dd9ee45
> >>> >>> # Parent 02e9411009b987f408214ab4a8b6b6093f843bcd
> >>> >>> Upstream: re-resolvable servers.
> >>> >>>
> >>> >>> Specifying the upstream server by a hostname together with the
> >>> >>> "resolve" parameter will make the hostname to be periodically
> >>> >>> resolved, and upstream servers added/removed as necessary.
> >>> >>>
> >>> >>> This requires a "resolver" at the "http" configuration block.
> >>> >>>
> >>> >>> The "resolver_timeout" parameter also affects when the failed
> >>> >>> DNS requests will be attempted again. Responses with NXDOMAIN
> >>> >>> will be attempted again in 10 seconds.
> >>> >>>
> >>> >>> Upstream has a configuration generation number that is
> >>> incremented each
> >>> >>> time servers are added/removed to the primary/backup list.
> >>> This number
> >>> >>> is remembered by the peer.init method, and if peer.get detects
> >>> a change
> >>> >>> in configuration, it returns NGX_BUSY.
> >>> >>>
> >>> >>> Each server has a reference counter. It is incremented by
> >>> peer.get and
> >>> >>> decremented by peer.free. When a server is removed, it is
> >>> removed from
> >>> >>> the list of servers and is marked as "zombie". The memory
> >>> allocated by
> >>> >>> a zombie peer is freed only when its reference count becomes zero.
> >>> >>>
> >>> >>> Re-resolvable servers utilize timers that also hold a reference. A
> >>> >>> reference is also held while upstream keepalive caches an idle
> >>> >>> connection.
> >>> >>>
> >>> >>> Co-authored-by: Roman Arutyunyan <arut@nginx.com
> >>> <mailto:arut@nginx.com>>
> >>> >>> Co-authored-by: Sergey Kandaurov <pluknet@nginx.com
> >>> <mailto:pluknet@nginx.com>>
> >>> >>> Co-authored-by: Vladimir Homutov <vl@nginx.com
> >>> <mailto:vl@nginx.com>>
> >>> >
> >>> > [..]
> >>> >
> >>> >>> diff --git a/src/http/ngx_http_upstream_round_robin.h
> >>> b/src/http/ngx_http_upstream_round_robin.h
> >>> >>> --- a/src/http/ngx_http_upstream_round_robin.h
> >>> >>> +++ b/src/http/ngx_http_upstream_round_robin.h
> >>> >>> @@ -14,8 +14,23 @@
> >>> >>> #include <ngx_http.h>
> >>> >>>
> >>> >>>
> >>> >>> +typedef struct ngx_http_upstream_rr_peers_s
> >>> ngx_http_upstream_rr_peers_t;
> >>> >>> typedef struct ngx_http_upstream_rr_peer_s
> >>> ngx_http_upstream_rr_peer_t;
> >>> >>>
> >>> >>> +
> >>> >>> +#if (NGX_HTTP_UPSTREAM_ZONE)
> >>> >>> +
> >>> >>> +typedef struct {
> >>> >>> + ngx_event_t event; /* must be
> >>> first */
> >>> >>> + ngx_uint_t worker;
> >>> >
> >>> > Missed this last time. This field should be removed since all
> >>> resolving is in
> >>> > worker #0.
> >>>
> >>> Unfortunately, that would break the ABI compatibility between OSS and
> >>> Plus. Replacing the field with yet another NGX_COMPAT_BEGIN isn't any
> >>> better than leaving it in the opensource code.
> >>>
> >>> >
> >>> >>> + ngx_str_t name;
> >>> >>> + ngx_http_upstream_rr_peers_t *peers;
> >>> >>> + ngx_http_upstream_rr_peer_t *peer;
> >>> >>> +} ngx_http_upstream_host_t;
> >>> >>> +
> >>> >>> +#endif
> >>> >>> +
> >>> >>> +
> >>> >>> struct ngx_http_upstream_rr_peer_s {
> >>> >>> struct sockaddr *sockaddr;
> >>> >>> socklen_t socklen;
> >>> >>> @@ -46,7 +61,12 @@ struct ngx_http_upstream_rr_peer_s {
> >>> >>> #endif
> >>> >>>
> >>> >>> #if (NGX_HTTP_UPSTREAM_ZONE)
> >>> >>> + unsigned zombie:1;
> >>> >>
> >>> >> I suggest declaring this as in other similar places:
> >>> >>
> >>> >> ngx_uint_t zombie; /* unsigned
> >>> zombie:1; */
> >>> >>
> >>> >>> +
> >>> >>> ngx_atomic_t lock;
> >>> >>> + ngx_uint_t id;
> >>> >>
> >>> >> This field is not used in open source nginx and should not be
> >>> added or assigned.
> >>> >>
> >>> >>> + ngx_uint_t refs;
> >>> >>> + ngx_http_upstream_host_t *host;
> >>> >>> #endif
> >>> >>>
> >>> >>> ngx_http_upstream_rr_peer_t *next;
> >>> >
> >>> > [..]
> >>> >
> >>> > --
> >>> > Roman Arutyunyan
> >>> > _______________________________________________
> >>> > nginx-devel mailing list
> >>> > nginx-devel@nginx.org <mailto:nginx-devel@nginx.org>
> >>> > https://mailman.nginx.org/mailman/listinfo/nginx-devel
> >>> https://mailman.nginx.org/mailman/listinfo/nginx-devel
> >>> _______________________________________________
> >>> nginx-devel mailing list
> >>> nginx-devel@nginx.org <mailto:nginx-devel@nginx.org>
> >>> https://mailman.nginx.org/mailman/listinfo/nginx-devel
> >>> https://mailman.nginx.org/mailman/listinfo/nginx-devel
> >>>
> >>>
> >>> _______________________________________________
> >>> nginx-devel mailing list
> >>> nginx-devel@nginx.org
> >>> https://mailman.nginx.org/mailman/listinfo/nginx-devel
> >> _______________________________________________
> >> nginx-devel mailing list
> >> nginx-devel@nginx.org
> >> https://mailman.nginx.org/mailman/listinfo/nginx-devel
> > _______________________________________________
> > nginx-devel mailing list
> > nginx-devel@nginx.org
> > https://mailman.nginx.org/mailman/listinfo/nginx-devel
> _______________________________________________
> nginx-devel mailing list
> nginx-devel@nginx.org
> https://mailman.nginx.org/mailman/listinfo/nginx-devel
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
https://mailman.nginx.org/mailman/listinfo/nginx-devel
Subject Author Views Posted

[PATCH 0 of 9] Upstream: re-resolvable servers.

Aleksei Bavshin 312 June 13, 2024 06:30PM

[PATCH 1 of 9] Upstream: re-resolvable servers

Aleksei Bavshin 71 June 13, 2024 06:30PM

Re: [PATCH 1 of 9] Upstream: re-resolvable servers

Roman Arutyunyan 59 July 08, 2024 10:22AM

Re: [PATCH 1 of 9] Upstream: re-resolvable servers

Roman Arutyunyan 60 July 09, 2024 12:24PM

Re: [PATCH 1 of 9] Upstream: re-resolvable servers

Aleksei Bavshin 58 July 11, 2024 12:42PM

Re: [PATCH 1 of 9] Upstream: re-resolvable servers

splitice 118 July 11, 2024 03:18PM

Re: [PATCH 1 of 9] Upstream: re-resolvable servers

Aleksei Bavshin 80 July 11, 2024 03:44PM

Re: [PATCH 1 of 9] Upstream: re-resolvable servers

splitice 108 July 11, 2024 04:00PM

Re: [PATCH 1 of 9] Upstream: re-resolvable servers

Aleksei Bavshin 63 July 11, 2024 04:34PM

Re: [PATCH 1 of 9] Upstream: re-resolvable servers

splitice 138 July 11, 2024 05:06PM

Re: [PATCH 1 of 9] Upstream: re-resolvable servers

Aleksei Bavshin 61 July 09, 2024 04:22PM

Re: [PATCH 1 of 9] Upstream: re-resolvable servers

Roman Arutyunyan 66 July 10, 2024 11:44AM

[PATCH 2 of 9] Stream: re-resolvable servers

Aleksei Bavshin 66 June 13, 2024 06:30PM

[PATCH 3 of 9] Upstream: construct upstream peers from DNS SRV records

Aleksei Bavshin 67 June 13, 2024 06:30PM

[PATCH 4 of 9] Core: inheritance of non-reusable shared memory zones

Aleksei Bavshin 55 June 13, 2024 06:32PM

Re: [PATCH 4 of 9] Core: inheritance of non-reusable shared memory zones

Roman Arutyunyan 46 July 08, 2024 10:24AM

[PATCH 5 of 9] Upstream: pre-resolve servers on reload

Aleksei Bavshin 58 June 13, 2024 06:32PM

[PATCH 6 of 9] Upstream: per-upstream resolver

Aleksei Bavshin 56 June 13, 2024 06:32PM

Re: [PATCH 6 of 9] Upstream: per-upstream resolver

Roman Arutyunyan 59 July 08, 2024 11:18AM

[PATCH 7 of 9] Upstream: copy upstream zone DNS valid time during config reload

Aleksei Bavshin 57 June 13, 2024 06:32PM

Re: [PATCH 7 of 9] Upstream: copy upstream zone DNS valid time during config reload

Roman Arutyunyan 53 July 08, 2024 11:08AM

[PATCH 8 of 9] Upstream: disable re-resolve functionality on Windows

Aleksei Bavshin 89 June 13, 2024 06:34PM

Re: [PATCH 8 of 9] Upstream: disable re-resolve functionality on Windows

Roman Arutyunyan 55 July 10, 2024 09:18AM

Re: [PATCH 8 of 9] Upstream: disable re-resolve functionality on Windows

Roman Arutyunyan 54 July 11, 2024 02:52AM

Re: [PATCH 8 of 9] Upstream: disable re-resolve functionality on Windows

Aleksei Bavshin 50 July 11, 2024 12:26PM

[PATCH 9 of 9] Tests: upstream configuration tests with re-resolvable servers

Aleksei Bavshin 71 June 13, 2024 06:34PM



Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 147
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 500 on July 15, 2024
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready