Hi,

On Tue, Sep 20, 2022 at 05:46:50AM +0300, Maxim Dounin wrote:
> Hello!
>
> On Tue, Sep 13, 2022 at 07:03:04PM +0400, Roman Arutyunyan wrote:
>
> > On Tue, Sep 13, 2022 at 12:30:17AM +0300, Maxim Dounin wrote:
> > > Hello!
> > >
> > > On Fri, Sep 09, 2022 at 07:46:58PM +0400, Roman Arutyunyan wrote:
> > >
> > > > On Mon, Sep 05, 2022 at 06:58:38PM +0300, Maxim Dounin wrote:
> > > > > Hello!
> > > > >
> > > > > On Mon, Sep 05, 2022 at 05:23:18PM +0400, Roman Arutyunyan wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > On Mon, Sep 05, 2022 at 03:52:49AM +0300, Maxim Dounin wrote:
> > > > > > > Hello!
> > > > > > >
> > > > > > > On Wed, Aug 31, 2022 at 07:52:15PM +0400, Roman Arutyunyan wrote:
> > > > > > >
> > > > > > > > # HG changeset patch
> > > > > > > > # User Roman Arutyunyan <arut@nginx.com>
> > > > > > > > # Date 1661436099 -14400
> > > > > > > > # Thu Aug 25 18:01:39 2022 +0400
> > > > > > > > # Node ID 4b856f1dff939e4eb9c131e17af061cf2c38cfac
> > > > > > > > # Parent 069a4813e8d6d7ec662d282a10f5f7062ebd817f
> > > > > > > > Core: support for reading PROXY protocol v2 TLVs.
> > > > > > >
> > > > > > > First of all, could you please provide details on the use case?
> > > > > > > I've seen requests for writing proxy protocol TLVs to upstream
> > > > > > > servers (see ticket #1639), but not yet seen any meaningful
> > > > > > > reading requests.
> > > > > >
> > > > > > The known cases are these:
> > > > > >
> > > > > > - https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-target-groups.html#proxy-protocol
> > > > > > - https://docs.microsoft.com/en-us/azure/private-link/private-link-service-overview#getting-connection-information-using-tcp-proxy-v2
> > > > > > - https://cloud.google.com/vpc/docs/configure-private-service-connect-producer#proxy-protocol
> > > > > >
> > > > > > The data may need further parsing, but it can be done in njs or perl.
> > > > >
> > > > > Thanks for the details. So, basically, it's about vendor-specific
> > > > > endpoint IDs.
> > > > >
> > > > > > > > The TLV values are available in HTTP and Stream variables
> > > > > > > > $proxy_protocol_tlv_0xN, where N is a hexadecimal TLV type number with no
> > > > > > > > leading zeroes.
> > > > > > >
> > > > > > > I can't say I like the "hexadecimal TLV type number with no
> > > > > > > leading zeroes" approach, especially given that the specification
> > > > > > > uses leading zeroes in TLV types. With leading zeros might be
> > > > > > > better, to match specification.
> > > >
> > > > With on-demand approach this is no longer an issue.
> > > >
> > > > > > > Also, it might worth the effort to actually add names for known
> > > > > > > types instead or in addition to numbers.
> > > > > >
> > > > > > This is indeed a good idea and we have such plans as a further extenion of this
> > > > > > work. One of the problems is however that the abovementioned TLV variables
> > > > > > are specified in internal documents of AWS/Azure/GCP which are not standards.
> > > > > > They can be changed anytime, while we have to maintain those variables in
> > > > > > nginx. Also, raw variables give more flexibility in supporting less known TLVs.
> > > > >
> > > > > Of course I'm not suggesting to ditch raw variables, at least not
> > > > > for unknown/non-standard values. But for known/standard values it
> > > > > should be easy enough to provide alternative names for easier use,
> > > > > probably with type-specific parsing.
> > > > >
> > > > > With on-demand parsing it would be trivial to support both
> > > > > $proxy_protocol_tlv_alpn and $proxy_protocol_tlv_0x01. Further,
> > > > > it will be trivial to support $proxy_protocol_tlv_aws_vpc_id while
> > > > > still providing $proxy_protocol_tlv_0xea for raw data.
> > >
> > > [...]
> > >
> > > > # HG changeset patch
> > > > # User Roman Arutyunyan <arut@nginx.com>
> > > > # Date 1662718130 -14400
> > > > # Fri Sep 09 14:08:50 2022 +0400
> > > > # Node ID 832f6c96b26c3009640072e3f4b1f0bf43e644d0
> > > > # Parent 80714f1b0f597ce5e530e7274457450db4587fc9
> > > > Standard PROXY protocol v2 TLV variables.
> > > >
> > > > diff --git a/src/core/ngx_proxy_protocol.h b/src/core/ngx_proxy_protocol.h
> > > > --- a/src/core/ngx_proxy_protocol.h
> > > > +++ b/src/core/ngx_proxy_protocol.h
> > > > @@ -13,7 +13,18 @@
> > > > #include <ngx_core.h>
> > > >
> > > >
> > > > -#define NGX_PROXY_PROTOCOL_MAX_HEADER 107
> > > > +#define NGX_PROXY_PROTOCOL_MAX_HEADER 107
> > > > +
> > > > +#define NGX_PROXY_PROTOCOL_TLV_ALPN 0x01
> > > > +#define NGX_PROXY_PROTOCOL_TLV_AUTHORITY 0x02
> > > > +#define NGX_PROXY_PROTOCOL_TLV_UNIQUE_ID 0x05
> > > > +#define NGX_PROXY_PROTOCOL_TLV_SSL 0x20
> > > > +#define NGX_PROXY_PROTOCOL_TLV_SSL_VERSION 0x21
> > > > +#define NGX_PROXY_PROTOCOL_TLV_SSL_CN 0x22
> > > > +#define NGX_PROXY_PROTOCOL_TLV_SSL_CIPHER 0x23
> > > > +#define NGX_PROXY_PROTOCOL_TLV_SSL_SIG_ALG 0x24
> > > > +#define NGX_PROXY_PROTOCOL_TLV_SSL_KEY_ALG 0x25
> > > > +#define NGX_PROXY_PROTOCOL_TLV_NETNS 0x30
> > > >
> > > >
> > > > struct ngx_proxy_protocol_s {
> > > > @@ -25,6 +36,12 @@ struct ngx_proxy_protocol_s {
> > > > };
> > > >
> > > >
> > > > +typedef struct {
> > > > + u_char client;
> > > > + u_char verify[4];
> > >
> > > Note that these two are completely ignored.
> >
> > Yes, currently they are. I thought about adding variables for them too,
> > but then decided to postpone this. Do you think these have enough value
> > to be added to this patch?
>
> The "client" field seems to be mostly useless, since all the flags
> can be derived from the TLVs provided. In particular,
> PP2_CLIENT_SSL from PP2_SUBTYPE_SSL_VERSION, and
> PP2_CLIENT_CERT_SESS from PP2_SUBTYPE_SSL_CN (if provided). The
> PP2_CLIENT_CERT_CONN seems to be completely useless, since from
> SSL point of view there is no difference if the certificate was
> provided in this connection or previously within the same SSL
> session.
>
> I'm not sure if "verify" can be derived from the other data
> provided though. Looks like it's not, at least that's my
> impression from the specification and the code available[1]. As
> far as I understand, all certificate-related variables are
> basically useless without checking the verify field it is
> explicitly known that all clients with certificate verification
> errors were rejected by the proxy.
>
> [1] https://github.com/haproxy/haproxy/blob/master/src/connection.c#L1951
>
> >
> > > > +} ngx_proxy_protocol_tlv_ssl_t;
> > > > +
> > > > +
> > > > u_char *ngx_proxy_protocol_read(ngx_connection_t *c, u_char *buf,
> > > > u_char *last);
> > > > u_char *ngx_proxy_protocol_write(ngx_connection_t *c, u_char *buf,
> > > > diff --git a/src/http/ngx_http_variables.c b/src/http/ngx_http_variables.c
> > > > --- a/src/http/ngx_http_variables.c
> > > > +++ b/src/http/ngx_http_variables.c
> > > > @@ -63,6 +63,10 @@ static ngx_int_t ngx_http_variable_proxy
> > > > ngx_http_variable_value_t *v, uintptr_t data);
> > > > static ngx_int_t ngx_http_variable_proxy_protocol_tlv_0x(ngx_http_request_t *r,
> > > > ngx_http_variable_value_t *v, uintptr_t data);
> > > > +static ngx_int_t ngx_http_variable_proxy_protocol_tlv(ngx_http_request_t *r,
> > > > + ngx_http_variable_value_t *v, uintptr_t data);
> > > > +static ngx_int_t ngx_http_variable_proxy_protocol_tlv_ssl(
> > > > + ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data);
> > > > static ngx_int_t ngx_http_variable_server_addr(ngx_http_request_t *r,
> > > > ngx_http_variable_value_t *v, uintptr_t data);
> > > > static ngx_int_t ngx_http_variable_server_port(ngx_http_request_t *r,
> > > > @@ -220,6 +224,42 @@ static ngx_http_variable_t ngx_http_cor
> > > > ngx_http_variable_proxy_protocol_tlv_0x,
> > > > 0, NGX_HTTP_VAR_PREFIX, 0 },
> > > >
> > > > + { ngx_string("proxy_protocol_tlv_alpn"), NULL,
> > > > + ngx_http_variable_proxy_protocol_tlv,
> > > > + NGX_PROXY_PROTOCOL_TLV_ALPN, 0, 0 },
> > > > +
> > > > + { ngx_string("proxy_protocol_tlv_authority"), NULL,
> > > > + ngx_http_variable_proxy_protocol_tlv,
> > > > + NGX_PROXY_PROTOCOL_TLV_AUTHORITY, 0, 0 },
> > > > +
> > > > + { ngx_string("proxy_protocol_tlv_unique_id"), NULL,
> > > > + ngx_http_variable_proxy_protocol_tlv,
> > > > + NGX_PROXY_PROTOCOL_TLV_UNIQUE_ID, 0, 0 },
> > > > +
> > > > + { ngx_string("proxy_protocol_tlv_ssl_version"), NULL,
> > > > + ngx_http_variable_proxy_protocol_tlv_ssl,
> > > > + NGX_PROXY_PROTOCOL_TLV_SSL_VERSION, 0, 0 },
> > > > +
> > > > + { ngx_string("proxy_protocol_tlv_ssl_cn"), NULL,
> > > > + ngx_http_variable_proxy_protocol_tlv_ssl,
> > > > + NGX_PROXY_PROTOCOL_TLV_SSL_CN, 0, 0 },
> > > > +
> > > > + { ngx_string("proxy_protocol_tlv_ssl_cipher"), NULL,
> > > > + ngx_http_variable_proxy_protocol_tlv_ssl,
> > > > + NGX_PROXY_PROTOCOL_TLV_SSL_CIPHER, 0, 0 },
> > > > +
> > > > + { ngx_string("proxy_protocol_tlv_ssl_sig_alg"), NULL,
> > > > + ngx_http_variable_proxy_protocol_tlv_ssl,
> > > > + NGX_PROXY_PROTOCOL_TLV_SSL_SIG_ALG, 0, 0 },
> > > > +
> > > > + { ngx_string("proxy_protocol_tlv_ssl_key_alg"), NULL,
> > > > + ngx_http_variable_proxy_protocol_tlv_ssl,
> > > > + NGX_PROXY_PROTOCOL_TLV_SSL_KEY_ALG, 0, 0 },
> > > > +
> > > > + { ngx_string("proxy_protocol_tlv_netns"), NULL,
> > > > + ngx_http_variable_proxy_protocol_tlv,
> > > > + NGX_PROXY_PROTOCOL_TLV_NETNS, 0, 0 },
> > > > +
> > >
> > > Further, these provide no interface to support arbitrary SSL TLVs
> > > (that is, something like "$proxy_protocol_tlv_ssl_0x10").
> > >
> > > Overall, I tend to think it would be much easier / less intrusive
> > > to keep all the TLV handling logic in the
> > > src/core/ngx_proxy_protocol.c, including both arbitrary TLVs with
> > > hexadecimal type and known named TLVs, and only provide a single
> > > prefix variable in stream / http modules.
> > >
> > > That is, use "$proxy_protocol_tlv_" prefix variable and call a
> > > function to obtain TLV with the rest of the variable name.
> > >
> > > This might imply slightly more complex/less efficient name parsing
> > > logic for known TLVs, but I don't think it will be noticeable. On
> > > the other hand, it will reduce clutter in the variables hash, and
> > > therefore will save some resources in configurations where these
> > > variables are not used.
> >
> > OK. Stream/HTTP part now looks simpler.
> >
> > --
> > Roman Arutyunyan
>
> > # HG changeset patch
> > # User Roman Arutyunyan <arut@nginx.com>
> > # Date 1663080928 -14400
> > # Tue Sep 13 18:55:28 2022 +0400
> > # Node ID 0736b29bfafac445fb8cf57312c3b7a2c3247a60
> > # Parent ba5cf8f73a2d0a3615565bf9545f3d65216a0530
> > PROXY protocol v2 TLV variables.
> >
> > The variables have prefix $proxy_protocol_tlv_ and are accessible by name
> > and by type. Examples are: $proxy_protocol_tlv_0x01, $proxy_protocol_tlv_alpn.
> >
> > diff --git a/src/core/ngx_proxy_protocol.c b/src/core/ngx_proxy_protocol.c
> > --- a/src/core/ngx_proxy_protocol.c
> > +++ b/src/core/ngx_proxy_protocol.c
> > @@ -40,6 +40,24 @@ typedef struct {
> > } ngx_proxy_protocol_inet6_addrs_t;
> >
> >
> > +typedef struct {
> > + u_char type;
> > + u_char len[2];
> > +} ngx_proxy_protocol_tlv_t;
> > +
> > +
> > +typedef struct {
> > + u_char client;
> > + u_char verify[4];
> > +} ngx_proxy_protocol_tlv_ssl_t;
> > +
> > +
> > +typedef struct {
> > + ngx_str_t name;
> > + ngx_uint_t type;
> > +} ngx_proxy_protocol_tlv_entry_t;
> > +
> > +
> > static u_char *ngx_proxy_protocol_read_addr(ngx_connection_t *c, u_char *p,
> > u_char *last, ngx_str_t *addr);
> > static u_char *ngx_proxy_protocol_read_port(u_char *p, u_char *last,
> > @@ -48,6 +66,26 @@ static u_char *ngx_proxy_protocol_v2_rea
> > u_char *last);
> >
> >
> > +static ngx_proxy_protocol_tlv_entry_t ngx_proxy_protocol_tlv_entries[] = {
> > + { ngx_string("alpn"), 0x01 },
> > + { ngx_string("authority"), 0x02 },
> > + { ngx_string("unique_id"), 0x05 },
> > + { ngx_string("ssl"), 0x20 },
> > + { ngx_string("netns"), 0x30 },
> > + { ngx_null_string, 0x00 }
> > +};
> > +
> > +
> > +static ngx_proxy_protocol_tlv_entry_t ngx_proxy_protocol_tlv_ssl_entries[] = {
> > + { ngx_string("version"), 0x21 },
> > + { ngx_string("cn"), 0x22 },
> > + { ngx_string("cipher"), 0x23 },
> > + { ngx_string("sig_alg"), 0x24 },
> > + { ngx_string("key_alg"), 0x25 },
> > + { ngx_null_string, 0x00 }
> > +};
> > +
> > +
> > u_char *
> > ngx_proxy_protocol_read(ngx_connection_t *c, u_char *buf, u_char *last)
> > {
> > @@ -412,11 +450,128 @@ ngx_proxy_protocol_v2_read(ngx_connectio
> > &pp->src_addr, pp->src_port, &pp->dst_addr, pp->dst_port);
> >
> > if (buf < end) {
> > - ngx_log_debug1(NGX_LOG_DEBUG_CORE, c->log, 0,
> > - "PROXY protocol v2 %z bytes of tlv ignored", end - buf);
> > + pp->tlvs.data = ngx_pnalloc(c->pool, end - buf);
> > + if (pp->tlvs.data == NULL) {
> > + return NULL;
> > + }
> > +
> > + ngx_memcpy(pp->tlvs.data, buf, end - buf);
> > + pp->tlvs.len = end - buf;
> > }
> >
> > c->proxy_protocol = pp;
> >
> > return end;
> > }
> > +
> > +
> > +ngx_int_t
> > +ngx_proxy_protocol_lookup_tlv(ngx_connection_t *c, ngx_str_t *tlvs,
> > + ngx_uint_t type, ngx_str_t *value)
> > +{
> > + u_char *p;
> > + size_t n, len;
> > + ngx_proxy_protocol_tlv_t *tlv;
> > +
> > + ngx_log_debug1(NGX_LOG_DEBUG_CORE, c->log, 0,
> > + "PROXY protocol v2 lookup tlv:%02xi", type);
> > +
> > + p = tlvs->data;
> > + n = tlvs->len;
> > +
> > + while (n) {
> > + if (n < sizeof(ngx_proxy_protocol_tlv_t)) {
> > + ngx_log_error(NGX_LOG_ERR, c->log, 0, "broken PROXY protocol TLV");
> > + return NGX_ERROR;
> > + }
> > +
> > + tlv = (ngx_proxy_protocol_tlv_t *) p;
> > + len = ngx_proxy_protocol_parse_uint16(tlv->len);
> > +
> > + p += sizeof(ngx_proxy_protocol_tlv_t);
> > + n -= sizeof(ngx_proxy_protocol_tlv_t);
> > +
> > + if (n < len) {
> > + ngx_log_error(NGX_LOG_ERR, c->log, 0, "broken PROXY protocol TLV");
> > + return NGX_ERROR;
> > + }
> > +
> > + ngx_log_debug2(NGX_LOG_DEBUG_CORE, c->log, 0,
> > + "PROXY protocol v2 tlv:0x%02xd len:%uz", tlv->type, len);
> > +
> > + if (tlv->type == type) {
> > + value->data = p;
> > + value->len = len;
> > + return NGX_OK;
> > + }
> > +
> > + p += len;
> > + n -= len;
> > + }
> > +
> > + return NGX_DECLINED;
> > +}
> > +
> > +
> > +ngx_int_t
> > +ngx_proxy_protocol_get_tlv(ngx_connection_t *c, ngx_str_t *name,
> > + ngx_str_t *value)
> > +{
> > + u_char *p;
> > + size_t n;
> > + ngx_str_t ssl, *tlvs;
> > + ngx_int_t rc, type;
> > + ngx_proxy_protocol_tlv_entry_t *te;
> > +
> > + if (c->proxy_protocol == NULL) {
> > + return NGX_DECLINED;
> > + }
> > +
> > + ngx_log_debug1(NGX_LOG_DEBUG_CORE, c->log, 0,
> > + "PROXY protocol v2 get tlv \"%V\"", name);
> > +
> > + te = ngx_proxy_protocol_tlv_entries;
> > + tlvs = &c->proxy_protocol->tlvs;
> > +
> > + p = name->data;
> > + n = name->len;
> > +
> > + if (n >= 4 && p[0] == 's' && p[1] == 's' && p[2] == 'l' && p[3] == '_') {
> > +
> > + rc = ngx_proxy_protocol_lookup_tlv(c, tlvs, 0x20, &ssl);
> > + if (rc != NGX_OK) {
> > + return rc;
> > + }
> > +
> > + if (ssl.len < sizeof(ngx_proxy_protocol_tlv_ssl_t)) {
> > + return NGX_ERROR;
> > + }
> > +
> > + ssl.data += sizeof(ngx_proxy_protocol_tlv_ssl_t);
> > + ssl.len -= sizeof(ngx_proxy_protocol_tlv_ssl_t);
> > +
> > + te = ngx_proxy_protocol_tlv_ssl_entries;
> > + tlvs = &ssl;
> > +
> > + p += 4;
> > + n -= 4;
> > + }
> > +
> > + if (n >= 2 && p[0] == '0' && p[1] == 'x') {
> > +
> > + type = ngx_hextoi(p + 2, n - 2);
> > + if (type == NGX_ERROR) {
> > + return NGX_ERROR;
> > + }
> > +
> > + return ngx_proxy_protocol_lookup_tlv(c, tlvs, type, value);
> > + }
> > +
> > + for ( /* void */ ; te->type; te++) {
> > + if (te->name.len == n && ngx_strncmp(te->name.data, p, n) == 0) {
> > + return ngx_proxy_protocol_lookup_tlv(c, tlvs, te->type, value);
> > + }
> > + }
> > +
> > + return NGX_DECLINED;
> > +}
> > diff --git a/src/core/ngx_proxy_protocol.h b/src/core/ngx_proxy_protocol.h
> > --- a/src/core/ngx_proxy_protocol.h
> > +++ b/src/core/ngx_proxy_protocol.h
> > @@ -21,6 +21,7 @@ struct ngx_proxy_protocol_s {
> > ngx_str_t dst_addr;
> > in_port_t src_port;
> > in_port_t dst_port;
> > + ngx_str_t tlvs;
> > };
> >
> >
> > @@ -28,6 +29,10 @@ u_char *ngx_proxy_protocol_read(ngx_conn
> > u_char *last);
> > u_char *ngx_proxy_protocol_write(ngx_connection_t *c, u_char *buf,
> > u_char *last);
> > +ngx_int_t ngx_proxy_protocol_lookup_tlv(ngx_connection_t *c, ngx_str_t *tlvs,
> > + ngx_uint_t type, ngx_str_t *value);
> > +ngx_int_t ngx_proxy_protocol_get_tlv(ngx_connection_t *c, ngx_str_t *name,
> > + ngx_str_t *value);
> >
> >
> > #endif /* _NGX_PROXY_PROTOCOL_H_INCLUDED_ */
> > diff --git a/src/http/ngx_http_variables.c b/src/http/ngx_http_variables.c
> > --- a/src/http/ngx_http_variables.c
> > +++ b/src/http/ngx_http_variables.c
> > @@ -61,6 +61,8 @@ static ngx_int_t ngx_http_variable_proxy
> > ngx_http_variable_value_t *v, uintptr_t data);
> > static ngx_int_t ngx_http_variable_proxy_protocol_port(ngx_http_request_t *r,
> > ngx_http_variable_value_t *v, uintptr_t data);
> > +static ngx_int_t ngx_http_variable_proxy_protocol_tlv(ngx_http_request_t *r,
> > + ngx_http_variable_value_t *v, uintptr_t data);
> > static ngx_int_t ngx_http_variable_server_addr(ngx_http_request_t *r,
> > ngx_http_variable_value_t *v, uintptr_t data);
> > static ngx_int_t ngx_http_variable_server_port(ngx_http_request_t *r,
> > @@ -214,6 +216,10 @@ static ngx_http_variable_t ngx_http_cor
> > ngx_http_variable_proxy_protocol_port,
> > offsetof(ngx_proxy_protocol_t, dst_port), 0, 0 },
> >
> > + { ngx_string("proxy_protocol_tlv_"), NULL,
> > + ngx_http_variable_proxy_protocol_tlv,
> > + 0, NGX_HTTP_VAR_PREFIX, 0 },
> > +
> > { ngx_string("server_addr"), NULL, ngx_http_variable_server_addr, 0, 0, 0 },
> >
> > { ngx_string("server_port"), NULL, ngx_http_variable_server_port, 0, 0, 0 },
> > @@ -1387,6 +1393,39 @@ ngx_http_variable_proxy_protocol_port(ng
> >
> >
> > static ngx_int_t
> > +ngx_http_variable_proxy_protocol_tlv(ngx_http_request_t *r,
> > + ngx_http_variable_value_t *v, uintptr_t data)
> > +{
> > + ngx_str_t *name = (ngx_str_t *) data;
> > +
> > + ngx_int_t rc;
> > + ngx_str_t tlv, value;
> > +
> > + tlv.len = name->len - (sizeof("proxy_protocol_tlv_") - 1);
> > + tlv.data = name->data + sizeof("proxy_protocol_tlv_") - 1;
> > +
> > + rc = ngx_proxy_protocol_get_tlv(r->connection, &tlv, &value);
> > +
> > + if (rc == NGX_ERROR) {
> > + return NGX_ERROR;
> > + }
> > +
> > + if (rc == NGX_DECLINED) {
> > + v->not_found = 1;
> > + return NGX_OK;
> > + }
> > +
> > + v->len = value.len;
> > + v->valid = 1;
> > + v->no_cacheable = 0;
> > + v->not_found = 0;
> > + v->data = value.data;
> > +
> > + return NGX_OK;
> > +}
> > +
> > +
> > +static ngx_int_t
> > ngx_http_variable_server_addr(ngx_http_request_t *r,
> > ngx_http_variable_value_t *v, uintptr_t data)
> > {
> > diff --git a/src/stream/ngx_stream_variables.c b/src/stream/ngx_stream_variables.c
> > --- a/src/stream/ngx_stream_variables.c
> > +++ b/src/stream/ngx_stream_variables.c
> > @@ -23,6 +23,8 @@ static ngx_int_t ngx_stream_variable_pro
> > ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
> > static ngx_int_t ngx_stream_variable_proxy_protocol_port(
> > ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
> > +static ngx_int_t ngx_stream_variable_proxy_protocol_tlv(
> > + ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
> > static ngx_int_t ngx_stream_variable_server_addr(ngx_stream_session_t *s,
> > ngx_stream_variable_value_t *v, uintptr_t data);
> > static ngx_int_t ngx_stream_variable_server_port(ngx_stream_session_t *s,
> > @@ -79,6 +81,10 @@ static ngx_stream_variable_t ngx_stream
> > ngx_stream_variable_proxy_protocol_port,
> > offsetof(ngx_proxy_protocol_t, dst_port), 0, 0 },
> >
> > + { ngx_string("proxy_protocol_tlv_"), NULL,
> > + ngx_stream_variable_proxy_protocol_tlv,
> > + 0, NGX_STREAM_VAR_PREFIX, 0 },
> > +
> > { ngx_string("server_addr"), NULL,
> > ngx_stream_variable_server_addr, 0, 0, 0 },
> >
> > @@ -622,6 +628,39 @@ ngx_stream_variable_proxy_protocol_port(
> >
> >
> > static ngx_int_t
> > +ngx_stream_variable_proxy_protocol_tlv(ngx_stream_session_t *s,
> > + ngx_stream_variable_value_t *v, uintptr_t data)
> > +{
> > + ngx_str_t *name = (ngx_str_t *) data;
> > +
> > + ngx_int_t rc;
> > + ngx_str_t tlv, value;
> > +
> > + tlv.len = name->len - (sizeof("proxy_protocol_tlv_") - 1);
> > + tlv.data = name->data + sizeof("proxy_protocol_tlv_") - 1;
> > +
> > + rc = ngx_proxy_protocol_get_tlv(s->connection, &tlv, &value);
> > +
> > + if (rc == NGX_ERROR) {
> > + return NGX_ERROR;
> > + }
> > +
> > + if (rc == NGX_DECLINED) {
> > + v->not_found = 1;
> > + return NGX_OK;
> > + }
> > +
> > + v->len = value.len;
> > + v->valid = 1;
> > + v->no_cacheable = 0;
> > + v->not_found = 0;
> > + v->data = value.data;
> > +
> > + return NGX_OK;
> > +}
> > +
> > +
> > +static ngx_int_t
> > ngx_stream_variable_server_addr(ngx_stream_session_t *s,
> > ngx_stream_variable_value_t *v, uintptr_t data)
> > {
>
> Looks good, but see above: probably we should add at least the
> $proxy_protocol_tlv_ssl_verify variable (and probably with special
> handling to convert binary data into a string).

Added $proxy_protocol_tlv_ssl_verify.

Also, added patches for macro refactoring and tests.

--
Roman Arutyunyan
# HG changeset patch
# User Roman Arutyunyan <arut@nginx.com>
# Date 1664263604 -14400
# Tue Sep 27 11:26:44 2022 +0400
# Node ID 38940ff7246574aa19a19c76b072073c34f191be
# Parent ba5cf8f73a2d0a3615565bf9545f3d65216a0530
PROXY protocol v2 TLV variables.

The variables have prefix $proxy_protocol_tlv_ and are accessible by name
and by type. Examples are: $proxy_protocol_tlv_0x01, $proxy_protocol_tlv_alpn.

diff --git a/src/core/ngx_proxy_protocol.c b/src/core/ngx_proxy_protocol.c
--- a/src/core/ngx_proxy_protocol.c
+++ b/src/core/ngx_proxy_protocol.c
@@ -15,6 +15,12 @@

#define ngx_proxy_protocol_parse_uint16(p) ((p)[0] << 8 | (p)[1])

+#define ngx_proxy_protocol_parse_uint32(p) \
+ ( ((uint32_t) (p)[0] << 24) \
+ + ( (p)[1] << 16) \
+ + ( (p)[2] << 8) \
+ + ( (p)[3]) )
+

typedef struct {
u_char signature[12];
@@ -40,6 +46,24 @@ typedef struct {
} ngx_proxy_protocol_inet6_addrs_t;


+typedef struct {
+ u_char type;
+ u_char len[2];
+} ngx_proxy_protocol_tlv_t;
+
+
+typedef struct {
+ u_char client;
+ u_char verify[4];
+} ngx_proxy_protocol_tlv_ssl_t;
+
+
+typedef struct {
+ ngx_str_t name;
+ ngx_uint_t type;
+} ngx_proxy_protocol_tlv_entry_t;
+
+
static u_char *ngx_proxy_protocol_read_addr(ngx_connection_t *c, u_char *p,
u_char *last, ngx_str_t *addr);
static u_char *ngx_proxy_protocol_read_port(u_char *p, u_char *last,
@@ -48,6 +72,26 @@ static u_char *ngx_proxy_protocol_v2_rea
u_char *last);


+static ngx_proxy_protocol_tlv_entry_t ngx_proxy_protocol_tlv_entries[] = {
+ { ngx_string("alpn"), 0x01 },
+ { ngx_string("authority"), 0x02 },
+ { ngx_string("unique_id"), 0x05 },
+ { ngx_string("ssl"), 0x20 },
+ { ngx_string("netns"), 0x30 },
+ { ngx_null_string, 0x00 }
+};
+
+
+static ngx_proxy_protocol_tlv_entry_t ngx_proxy_protocol_tlv_ssl_entries[] = {
+ { ngx_string("version"), 0x21 },
+ { ngx_string("cn"), 0x22 },
+ { ngx_string("cipher"), 0x23 },
+ { ngx_string("sig_alg"), 0x24 },
+ { ngx_string("key_alg"), 0x25 },
+ { ngx_null_string, 0x00 }
+};
+
+
u_char *
ngx_proxy_protocol_read(ngx_connection_t *c, u_char *buf, u_char *last)
{
@@ -412,11 +456,145 @@ ngx_proxy_protocol_v2_read(ngx_connectio
&pp->src_addr, pp->src_port, &pp->dst_addr, pp->dst_port);

if (buf < end) {
- ngx_log_debug1(NGX_LOG_DEBUG_CORE, c->log, 0,
- "PROXY protocol v2 %z bytes of tlv ignored", end - buf);
+ pp->tlvs.data = ngx_pnalloc(c->pool, end - buf);
+ if (pp->tlvs.data == NULL) {
+ return NULL;
+ }
+
+ ngx_memcpy(pp->tlvs.data, buf, end - buf);
+ pp->tlvs.len = end - buf;
}

c->proxy_protocol = pp;

return end;
}
+
+
+ngx_int_t
+ngx_proxy_protocol_lookup_tlv(ngx_connection_t *c, ngx_str_t *tlvs,
+ ngx_uint_t type, ngx_str_t *value)
+{
+ u_char *p;
+ size_t n, len;
+ ngx_proxy_protocol_tlv_t *tlv;
+
+ ngx_log_debug1(NGX_LOG_DEBUG_CORE, c->log, 0,
+ "PROXY protocol v2 lookup tlv:%02xi", type);
+
+ p = tlvs->data;
+ n = tlvs->len;
+
+ while (n) {
+ if (n < sizeof(ngx_proxy_protocol_tlv_t)) {
+ ngx_log_error(NGX_LOG_ERR, c->log, 0, "broken PROXY protocol TLV");
+ return NGX_ERROR;
+ }
+
+ tlv = (ngx_proxy_protocol_tlv_t *) p;
+ len = ngx_proxy_protocol_parse_uint16(tlv->len);
+
+ p += sizeof(ngx_proxy_protocol_tlv_t);
+ n -= sizeof(ngx_proxy_protocol_tlv_t);
+
+ if (n < len) {
+ ngx_log_error(NGX_LOG_ERR, c->log, 0, "broken PROXY protocol TLV");
+ return NGX_ERROR;
+ }
+
+ ngx_log_debug2(NGX_LOG_DEBUG_CORE, c->log, 0,
+ "PROXY protocol v2 tlv:0x%02xd len:%uz", tlv->type, len);
+
+ if (tlv->type == type) {
+ value->data = p;
+ value->len = len;
+ return NGX_OK;
+ }
+
+ p += len;
+ n -= len;
+ }
+
+ return NGX_DECLINED;
+}
+
+
+ngx_int_t
+ngx_proxy_protocol_get_tlv(ngx_connection_t *c, ngx_str_t *name,
+ ngx_str_t *value)
+{
+ u_char *p;
+ size_t n;
+ uint32_t verify;
+ ngx_str_t ssl, *tlvs;
+ ngx_int_t rc, type;
+ ngx_proxy_protocol_tlv_ssl_t *tlv_ssl;
+ ngx_proxy_protocol_tlv_entry_t *te;
+
+ if (c->proxy_protocol == NULL) {
+ return NGX_DECLINED;
+ }
+
+ ngx_log_debug1(NGX_LOG_DEBUG_CORE, c->log, 0,
+ "PROXY protocol v2 get tlv \"%V\"", name);
+
+ te = ngx_proxy_protocol_tlv_entries;
+ tlvs = &c->proxy_protocol->tlvs;
+
+ p = name->data;
+ n = name->len;
+
+ if (n >= 4 && p[0] == 's' && p[1] == 's' && p[2] == 'l' && p[3] == '_') {
+
+ rc = ngx_proxy_protocol_lookup_tlv(c, tlvs, 0x20, &ssl);
+ if (rc != NGX_OK) {
+ return rc;
+ }
+
+ if (ssl.len < sizeof(ngx_proxy_protocol_tlv_ssl_t)) {
+ return NGX_ERROR;
+ }
+
+ p += 4;
+ n -= 4;
+
+ if (n == 6 && ngx_strncmp(p, "verify", 6) == 0) {
+
+ tlv_ssl = (ngx_proxy_protocol_tlv_ssl_t *) ssl.data;
+ verify = ngx_proxy_protocol_parse_uint32(tlv_ssl->verify);
+
+ value->data = ngx_pnalloc(c->pool, NGX_INT32_LEN);
+ if (value->data == NULL) {
+ return NGX_ERROR;
+ }
+
+ value->len = ngx_sprintf(value->data, "%uD", verify)
+ - value->data;
+ return NGX_OK;
+ }
+
+ ssl.data += sizeof(ngx_proxy_protocol_tlv_ssl_t);
+ ssl.len -= sizeof(ngx_proxy_protocol_tlv_ssl_t);
+
+ te = ngx_proxy_protocol_tlv_ssl_entries;
+ tlvs = &ssl;
+ }
+
+ if (n >= 2 && p[0] == '0' && p[1] == 'x') {
+
+ type = ngx_hextoi(p + 2, n - 2);
+ if (type == NGX_ERROR) {
+ return NGX_ERROR;
+ }
+
+ return ngx_proxy_protocol_lookup_tlv(c, tlvs, type, value);
+ }
+
+ for ( /* void */ ; te->type; te++) {
+ if (te->name.len == n && ngx_strncmp(te->name.data, p, n) == 0) {
+ return ngx_proxy_protocol_lookup_tlv(c, tlvs, te->type, value);
+ }
+ }
+
+ return NGX_DECLINED;
+}
diff --git a/src/core/ngx_proxy_protocol.h b/src/core/ngx_proxy_protocol.h
--- a/src/core/ngx_proxy_protocol.h
+++ b/src/core/ngx_proxy_protocol.h
@@ -21,6 +21,7 @@ struct ngx_proxy_protocol_s {
ngx_str_t dst_addr;
in_port_t src_port;
in_port_t dst_port;
+ ngx_str_t tlvs;
};


@@ -28,6 +29,10 @@ u_char *ngx_proxy_protocol_read(ngx_conn
u_char *last);
u_char *ngx_proxy_protocol_write(ngx_connection_t *c, u_char *buf,
u_char *last);
+ngx_int_t ngx_proxy_protocol_lookup_tlv(ngx_connection_t *c, ngx_str_t *tlvs,
+ ngx_uint_t type, ngx_str_t *value);
+ngx_int_t ngx_proxy_protocol_get_tlv(ngx_connection_t *c, ngx_str_t *name,
+ ngx_str_t *value);


#endif /* _NGX_PROXY_PROTOCOL_H_INCLUDED_ */
diff --git a/src/http/ngx_http_variables.c b/src/http/ngx_http_variables.c
--- a/src/http/ngx_http_variables.c
+++ b/src/http/ngx_http_variables.c
@@ -61,6 +61,8 @@ static ngx_int_t ngx_http_variable_proxy
ngx_http_variable_value_t *v, uintptr_t data);
static ngx_int_t ngx_http_variable_proxy_protocol_port(ngx_http_request_t *r,
ngx_http_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_http_variable_proxy_protocol_tlv(ngx_http_request_t *r,
+ ngx_http_variable_value_t *v, uintptr_t data);
static ngx_int_t ngx_http_variable_server_addr(ngx_http_request_t *r,
ngx_http_variable_value_t *v, uintptr_t data);
static ngx_int_t ngx_http_variable_server_port(ngx_http_request_t *r,
@@ -214,6 +216,10 @@ static ngx_http_variable_t ngx_http_cor
ngx_http_variable_proxy_protocol_port,
offsetof(ngx_proxy_protocol_t, dst_port), 0, 0 },

+ { ngx_string("proxy_protocol_tlv_"), NULL,
+ ngx_http_variable_proxy_protocol_tlv,
+ 0, NGX_HTTP_VAR_PREFIX, 0 },
+
{ ngx_string("server_addr"), NULL, ngx_http_variable_server_addr, 0, 0, 0 },

{ ngx_string("server_port"), NULL, ngx_http_variable_server_port, 0, 0, 0 },
@@ -1387,6 +1393,39 @@ ngx_http_variable_proxy_protocol_port(ng


static ngx_int_t
+ngx_http_variable_proxy_protocol_tlv(ngx_http_request_t *r,
+ ngx_http_variable_value_t *v, uintptr_t data)
+{
+ ngx_str_t *name = (ngx_str_t *) data;
+
+ ngx_int_t rc;
+ ngx_str_t tlv, value;
+
+ tlv.len = name->len - (sizeof("proxy_protocol_tlv_") - 1);
+ tlv.data = name->data + sizeof("proxy_protocol_tlv_") - 1;
+
+ rc = ngx_proxy_protocol_get_tlv(r->connection, &tlv, &value);
+
+ if (rc == NGX_ERROR) {
+ return NGX_ERROR;
+ }
+
+ if (rc == NGX_DECLINED) {
+ v->not_found = 1;
+ return NGX_OK;
+ }
+
+ v->len = value.len;
+ v->valid = 1;
+ v->no_cacheable = 0;
+ v->not_found = 0;
+ v->data = value.data;
+
+ return NGX_OK;
+}
+
+
+static ngx_int_t
ngx_http_variable_server_addr(ngx_http_request_t *r,
ngx_http_variable_value_t *v, uintptr_t data)
{
diff --git a/src/stream/ngx_stream_variables.c b/src/stream/ngx_stream_variables.c
--- a/src/stream/ngx_stream_variables.c
+++ b/src/stream/ngx_stream_variables.c
@@ -23,6 +23,8 @@ static ngx_int_t ngx_stream_variable_pro
ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
static ngx_int_t ngx_stream_variable_proxy_protocol_port(
ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_stream_variable_proxy_protocol_tlv(
+ ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
static ngx_int_t ngx_stream_variable_server_addr(ngx_stream_session_t *s,
ngx_stream_variable_value_t *v, uintptr_t data);
static ngx_int_t ngx_stream_variable_server_port(ngx_stream_session_t *s,
@@ -79,6 +81,10 @@ static ngx_stream_variable_t ngx_stream
ngx_stream_variable_proxy_protocol_port,
offsetof(ngx_proxy_protocol_t, dst_port), 0, 0 },

+ { ngx_string("proxy_protocol_tlv_"), NULL,
+ ngx_stream_variable_proxy_protocol_tlv,
+ 0, NGX_STREAM_VAR_PREFIX, 0 },
+
{ ngx_string("server_addr"), NULL,
ngx_stream_variable_server_addr, 0, 0, 0 },

@@ -622,6 +628,39 @@ ngx_stream_variable_proxy_protocol_port(


static ngx_int_t
+ngx_stream_variable_proxy_protocol_tlv(ngx_stream_session_t *s,
+ ngx_stream_variable_value_t *v, uintptr_t data)
+{
+ ngx_str_t *name = (ngx_str_t *) data;
+
+ ngx_int_t rc;
+ ngx_str_t tlv, value;
+
+ tlv.len = name->len - (sizeof("proxy_protocol_tlv_") - 1);
+ tlv.data = name->data + sizeof("proxy_protocol_tlv_") - 1;
+
+ rc = ngx_proxy_protocol_get_tlv(s->connection, &tlv, &value);
+
+ if (rc == NGX_ERROR) {
+ return NGX_ERROR;
+ }
+
+ if (rc == NGX_DECLINED) {
+ v->not_found = 1;
+ return NGX_OK;
+ }
+
+ v->len = value.len;
+ v->valid = 1;
+ v->no_cacheable = 0;
+ v->not_found = 0;
+ v->data = value.data;
+
+ return NGX_OK;
+}
+
+
+static ngx_int_t
ngx_stream_variable_server_addr(ngx_stream_session_t *s,
ngx_stream_variable_value_t *v, uintptr_t data)
{
# HG changeset patch
# User Roman Arutyunyan <arut@nginx.com>
# Date 1664263876 -14400
# Tue Sep 27 11:31:16 2022 +0400
# Node ID 615268a957ab930dc4be49fe5f6f88cd7e377f12
# Parent 38940ff7246574aa19a19c76b072073c34f191be
Added type cast to ngx_proxy_protocol_parse_uint16().

The cast is added to make ngx_proxy_protocol_parse_uint16() similar to
ngx_proxy_protocol_parse_uint32().

diff --git a/src/core/ngx_proxy_protocol.c b/src/core/ngx_proxy_protocol.c
--- a/src/core/ngx_proxy_protocol.c
+++ b/src/core/ngx_proxy_protocol.c
@@ -13,7 +13,9 @@
#define NGX_PROXY_PROTOCOL_AF_INET6 2


-#define ngx_proxy_protocol_parse_uint16(p) ((p)[0] << 8 | (p)[1])
+#define ngx_proxy_protocol_parse_uint16(p) \
+ ( ((uint16_t) (p)[0] << 8) \
+ + ( (p)[1]) )

#define ngx_proxy_protocol_parse_uint32(p) \
( ((uint32_t) (p)[0] << 24) \
# HG changeset patch
# User Roman Arutyunyan <arut@nginx.com>
# Date 1664200613 -14400
# Mon Sep 26 17:56:53 2022 +0400
# Node ID ff87ed4999b49433a9abecaaf2e574cbfa502961
# Parent 95ba1e704b7b29c39447135e18ed003ecd305924
Tests: client PROXY protocol v2 TLV variables.

diff --git a/proxy_protocol2.t b/proxy_protocol2.t
--- a/proxy_protocol2.t
+++ b/proxy_protocol2.t
@@ -24,7 +24,7 @@ select STDOUT; $| = 1;

my $t = Test::Nginx->new()->has(qw/http access realip/);

-$t->write_file_expand('nginx.conf', <<'EOF')->plan(23);
+$t->write_file_expand('nginx.conf', <<'EOF')->plan(26);

%%TEST_GLOBALS%%

@@ -45,6 +45,8 @@ http {
set_real_ip_from 127.0.0.1/32;
add_header X-IP $remote_addr!$remote_port;
add_header X-PP $proxy_protocol_addr!$proxy_protocol_port;
+ add_header X-TL $proxy_protocol_tlv_0x3-$proxy_protocol_tlv_0x0000ae-$proxy_protocol_tlv_0x0f;
+ add_header X-NT $proxy_protocol_tlv_unique_id-$proxy_protocol_tlv_ssl_cn-$proxy_protocol_tlv_ssl_0x22-$proxy_protocol_tlv_ssl_verify;

location /pp {
real_ip_header proxy_protocol;
@@ -76,7 +78,11 @@ my $p = pack("N3C", 0x0D0A0D0A, 0x000D0A
my $tcp4 = $p . pack("CnN2n2", 0x11, 12, 0xc0000201, 0xc0000202, 123, 5678);
my $tcp6 = $p . pack("CnNx8NNx8Nn2", 0x21, 36,
0x20010db8, 0x00000001, 0x20010db8, 0x00000002, 123, 5678);
-my $tlv = $p . pack("CnN2n2x9", 0x11, 21, 0xc0000201, 0xc0000202, 123, 5678);
+my $tlv = $p . pack("CnN2n2N3", 0x11, 24, 0xc0000201, 0xc0000202, 123, 5678,
+ 0x03000141, 0xAE000531, 0x32333435);
+my $tlv2 = $p . pack("CnN2n2N7", 0x11, 40, 0xc0000201, 0xc0000202, 123, 5678,
+ 0x05000555, 0x4E495151,
+ 0x20001100, 0xdeadbeef, 0x22000966, 0x6f6f2e62, 0x61727272);
my $unk1 = $p . pack("Cxx", 0x01);
my $unk2 = $p . pack("CnC4", 0x41, 4, 1, 2, 3, 4);
my $r;
@@ -97,6 +103,11 @@ unlike($r, qr/X-IP: (2001:DB8::1|[^!]+!1
like($r, qr/SEE-THIS/, 'tlv request');
like($r, qr/X-PP: 192.0.2.1!123\x0d/, 'tlv proxy');
unlike($r, qr/X-IP: (192.0.2.1|[^!]+!123\x0d)/, 'tlv client');
+like($r, qr/X-TL: A-12345-\x0d/, 'tlv raw variables');
+like($r, qr/X-NT: ---\x0d/, 'tlv missing variables');
+
+$r = pp_get('/t1', $tlv2);
+like($r, qr/X-NT: UNIQQ-foo.barrr-foo.barrr-3735928559\x0d/, 'tlv named variables');

$r = pp_get('/t1', $unk1);
like($r, qr/SEE-THIS/, 'unknown request 1');
_______________________________________________
nginx-devel mailing list -- nginx-devel@nginx.org
To unsubscribe send an email to nginx-devel-leave@nginx.org