Hello!

On Tue, Jan 19, 2021 at 06:34:30PM +0300, muradm wrote:

> # HG changeset patch
> # User muradm <mail@muradm.net>
> # Date 1611069863 -10800
> # Tue Jan 19 18:24:23 2021 +0300
> # Node ID 4618e767b84c5b3a7712466edb5bf37e3f0294ed
> # Parent 83c4622053b02821a12d522d08eaff3ac27e65e3

Thanks for the patch. Overall it looks much better than previous
attempts and close to what I would like to see implemented. See
below for additional comments.

> Mail: added PROXY PROTOCOL support.

The specification spells it as "PROXY protocol"
(https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt).

>
> This implements propxy protocol support for both upstream and downstream.
>
> Downstream proxy protocol support:
>
> mail {
> server {
> listen <port> [ssl] proxy_protocol;
> protocol <imap|pop3|smtp>;

Just a side notee: "[ssl]" in the examples looks unneeded. There
are multiple possible parameters of the listen directive, and
listing just one of them doesn't look right. On the other hand,
providing full current syntax of the listen directive would be a
bad idea as well.

The same applies to the "protocol <...>;" directive. This
directive isn't needed when configuring mail proxy servers on
standard ports. If it's needed, this is something up to the user
to configure.

> }
> }
>
> This will properly handle incoming connections from load balancer sending
> PROXY protocol header. Without this, it is impossible to run nginx mail
> proxy behind such balancer. Header reading is done with existing function
> "ngx_proxy_protocol_read", so it should support both v1 and v2 headers.
> This will also set "sockaddr" and "local_sockaddr" addresses from received
> header, mimicing "set_realip". While "realip_module" deals with variables
> etc., which is necessary for HTTP protocol, mail protocols are pretty
> strict, so there is no need for flexible handling of real addresses
> received.
>
> Upstream proxy protocol support:
>
> mail {
> server {
> listen <port> [ssl];
> protocol <imap|pop3|smtp>;
> proxy_protocol on;
> }
> }
>
> With this, upstream server (like Postfix, Exim, Dovecot) will have PROXY
> protocol header. Mentioned programs do support proxy protocol out of the
> box. Header is written with existing function "ngx_proxy_protocol_write"
> which supports only v1 header writing. Contents of header are written
> from "sockaddr" and "local_sockaddr".

These are two distinct features: reading PROXY protocol header
provided by the balancer, and sending PROXY protocol to backends.
As such, these should be in separate patches. In particular, this
will simplify review.

>
> Downstream and upstream proxy protocol support:
>
> mail {
> server {
> listen <port> [ssl] proxy_protocol;
> protocol <imap|pop3|smtp>;
> proxy_protocol on;
> }
> }
>
> This will combine both receiving PROXY header and sending PROXY header. With
> this, upstream server (like Postfix, Exim, Dovecot) will receive the same
> header as was sent by downstream load balancer.
>
> Above configurations work for SSL as well and should be transparent to other
> mail related configurations.
>
> Added upstream server "connect_timeout" which defaults to 1 second.

There is no need to introduce additional timeouts here: there is
already the "timeout" directive (cscf->timeout), which limits
aggregate timeout for the full login process, including connect.

If you think that a separate connect_timeout is needed for
whatever reason, consider submitting a separate patch - with
appropriate reasoning.

>
> Server configurations enabling proxy_protocol in listen directive, require
> "set_real_ip_from" configuration. Like the following:
>
> mail {
> # ...
> server {
> listen 587 proxy_protocol;
> set_real_ip_from "192.168.1.1";
> set_real_ip_from "10.10.0.0/16";
> set_real_ip_from "0.0.0.0/0";
> }
> }
>
> With enabled "proxy_protocol" and missing at least one "set_real_ip_from",
> all connections will be dropped and at startup user will see in error_log:
>
> using PROXY protocol without set_real_ip_from \
> while reading PROXY protocol header
>
> When "set_real_ip_from" is provided, but remote address on physical connection
> does not satisfy any address criteria, at "notice" level, in error_log, user
> will see:
>
> UNTRUSTED PROXY protocol provider: 127.0.0.1 \
> while reading PROXY protocol header, \
> client: 127.0.0.1, server: 127.0.0.1:8143

Such behaviour, "reject connection if not matched by
set_real_ip_from", looks wrong. It is not how things are handled
in stream/http, so such behaviour will break POLA. It also
contradicts the name of the set_real_ip_from directive: it is
expected to set addreses got from the IPs in question, but not
reject anything else.

Further, accepting PROXY protocol header and _not_ changing the
address nginx thinks the connection is from looks like a valid
use case to me. For example, the address from PROXY protocol can
be provided to the auth_http script, making it possible for the
script to do appropriate decisions.

Overall, it looks like there should be at least 3 patches here:
first one to introduce "listen ... proxy_protocol" and sending
appropriate information to auth_http script, second one for
"set_real_ip_from", and third one for "proxy_protocol on;" in the
proxy module.

>
> diff -r 83c4622053b0 -r 4618e767b84c src/mail/ngx_mail.c
> --- a/src/mail/ngx_mail.c Tue Jan 12 16:59:31 2021 +0300
> +++ b/src/mail/ngx_mail.c Tue Jan 19 18:24:23 2021 +0300
> @@ -402,6 +402,7 @@
> addrs[i].addr = sin->sin_addr.s_addr;
>
> addrs[i].conf.ctx = addr[i].opt.ctx;
> + addrs[i].conf.proxy_protocol = addr[i].opt.proxy_protocol;
> #if (NGX_MAIL_SSL)
> addrs[i].conf.ssl = addr[i].opt.ssl;
> #endif
> @@ -436,6 +437,7 @@
> addrs6[i].addr6 = sin6->sin6_addr;
>
> addrs6[i].conf.ctx = addr[i].opt.ctx;
> + addrs6[i].conf.proxy_protocol = addr[i].opt.proxy_protocol;
> #if (NGX_MAIL_SSL)
> addrs6[i].conf.ssl = addr[i].opt.ssl;
> #endif
> diff -r 83c4622053b0 -r 4618e767b84c src/mail/ngx_mail.h
> --- a/src/mail/ngx_mail.h Tue Jan 12 16:59:31 2021 +0300
> +++ b/src/mail/ngx_mail.h Tue Jan 19 18:24:23 2021 +0300
> @@ -37,6 +37,7 @@
> unsigned bind:1;
> unsigned wildcard:1;
> unsigned ssl:1;
> + unsigned proxy_protocol:1;
> #if (NGX_HAVE_INET6)
> unsigned ipv6only:1;
> #endif

It's a better idea to keep proxy_protocol similar to stream/http,
after the so_keepalive.

> @@ -56,6 +57,7 @@
> ngx_mail_conf_ctx_t *ctx;
> ngx_str_t addr_text;
> ngx_uint_t ssl; /* unsigned ssl:1; */
> + unsigned proxy_protocol:1;

The "ssl" flag needs to be converted to bitfield (as specified in
the comment) as long as another bitfield is added.

> } ngx_mail_addr_conf_t;
>
> typedef struct {
> @@ -125,6 +127,8 @@
> ngx_mail_conf_ctx_t *ctx;
>
> ngx_uint_t listen; /* unsigned listen:1; */
> +
> + ngx_array_t *realip_from; /* array of ngx_cidr_t */
> } ngx_mail_core_srv_conf_t;
>
>

It should be a good idea to put the realip_from field close to
other configuration field of the mail core module, that is,
somewhere before the pointer to other module configuration
contexts.

Alternatively, a separate module for set_real_ip_from with its own
configuration might be a good idea as well.

> @@ -190,6 +194,7 @@
> void **ctx;
> void **main_conf;
> void **srv_conf;
> + ngx_mail_addr_conf_t *addr_conf;
>
> ngx_resolver_ctx_t *resolver_ctx;
>
> @@ -197,6 +202,7 @@
>
> ngx_uint_t mail_state;
>
> + unsigned proxy_protocol:1;
> unsigned protocol:3;
> unsigned blocked:1;
> unsigned quit:1;
> diff -r 83c4622053b0 -r 4618e767b84c src/mail/ngx_mail_core_module.c
> --- a/src/mail/ngx_mail_core_module.c Tue Jan 12 16:59:31 2021 +0300
> +++ b/src/mail/ngx_mail_core_module.c Tue Jan 19 18:24:23 2021 +0300
> @@ -25,7 +25,7 @@
> void *conf);
> static char *ngx_mail_core_resolver(ngx_conf_t *cf, ngx_command_t *cmd,
> void *conf);
> -

Unrelated (and wrong from the style point of view) change, two
empty lines between functions and the following variable should be
preserved.

> +static char *ngx_mail_core_realip_from(ngx_conf_t *cf, ngx_command_t *cmd, void *conf);
>
> static ngx_command_t ngx_mail_core_commands[] = {
>
> @@ -85,6 +85,13 @@
> offsetof(ngx_mail_core_srv_conf_t, resolver_timeout),
> NULL },
>
> + { ngx_string("set_real_ip_from"),
> + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1,
> + ngx_mail_core_realip_from,
> + NGX_MAIL_SRV_CONF_OFFSET,
> + offsetof(ngx_mail_core_srv_conf_t, realip_from),
> + NULL },
> +
> ngx_null_command
> };
>
> @@ -165,6 +172,8 @@
>
> cscf->resolver = NGX_CONF_UNSET_PTR;
>
> + cscf->realip_from = NGX_CONF_UNSET_PTR;
> +
> cscf->file_name = cf->conf_file->file.name.data;
> cscf->line = cf->conf_file->line;
>
> @@ -206,6 +215,10 @@
>
> ngx_conf_merge_ptr_value(conf->resolver, prev->resolver, NULL);
>
> + ngx_conf_merge_ptr_value(conf->realip_from,
> + prev->realip_from,
> + NGX_CONF_UNSET_PTR);
> +

It is generally incorrect to preserve unset values till runtime
unless strictly necessary. A better approach would be to follow
how set_real_ip_from is handled in http/stream and use NULL if not
set. Checking for NULL to mean "no set_real_ip_from items
defined" is also more natural.

> return NGX_CONF_OK;
> }
>
> @@ -548,6 +561,11 @@
> #endif
> }
>
> + if (ngx_strcmp(value[i].data, "proxy_protocol") == 0) {
> + ls->proxy_protocol = 1;
> + continue;
> + }
> +
> ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
> "the invalid \"%V\" parameter", &value[i]);
> return NGX_CONF_ERROR;
> @@ -676,3 +694,104 @@
>
> return NGX_CONF_OK;
> }
> +
> +char *

Style: there should be two empty lines between functions.

Missing static specifier.

> +ngx_mail_core_realip_from(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
> +{
> + ngx_mail_core_srv_conf_t *cscf = conf;
> +
> + ngx_int_t rc;
> + ngx_str_t *value;
> + ngx_url_t u;
> + ngx_cidr_t c, *cidr;
> + ngx_uint_t i;
> + struct sockaddr_in *sin;
> +#if (NGX_HAVE_INET6)
> + struct sockaddr_in6 *sin6;
> +#endif
> +
> + value = cf->args->elts;
> +
> + if (cscf->realip_from == NGX_CONF_UNSET_PTR) {
> + cscf->realip_from = ngx_array_create(cf->pool, 2, sizeof(ngx_cidr_t));
> + if (cscf->realip_from == NULL) {
> + return NGX_CONF_ERROR;
> + }
> + }
> +
> +#if (NGX_HAVE_UNIX_DOMAIN)
> +
> + if (ngx_strcmp(value[1].data, "unix:") == 0) {
> + cidr = ngx_array_push(cscf->realip_from);
> + if (cidr == NULL) {
> + return NGX_CONF_ERROR;
> + }
> +
> + cidr->family = AF_UNIX;
> + return NGX_CONF_OK;
> + }
> +
> +#endif
> +
> + rc = ngx_ptocidr(&value[1], &c);
> +
> + if (rc != NGX_ERROR) {
> + if (rc == NGX_DONE) {
> + ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
> + "low address bits of %V are meaningless",
> + &value[1]);
> + }
> +
> + cidr = ngx_array_push(cscf->realip_from);
> + if (cidr == NULL) {
> + return NGX_CONF_ERROR;
> + }
> +
> + *cidr = c;
> +
> + return NGX_CONF_OK;
> + }
> +
> + ngx_memzero(&u, sizeof(ngx_url_t));
> + u.host = value[1];
> +
> + if (ngx_inet_resolve_host(cf->pool, &u) != NGX_OK) {
> + if (u.err) {
> + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
> + "%s in set_real_ip_from \"%V\"",
> + u.err, &u.host);
> + }
> +
> + return NGX_CONF_ERROR;
> + }
> +
> + cidr = ngx_array_push_n(cscf->realip_from, u.naddrs);
> + if (cidr == NULL) {
> + return NGX_CONF_ERROR;
> + }
> +
> + ngx_memzero(cidr, u.naddrs * sizeof(ngx_cidr_t));
> +
> + for (i = 0; i < u.naddrs; i++) {
> + cidr[i].family = u.addrs[i].sockaddr->sa_family;
> +
> + switch (cidr[i].family) {
> +
> +#if (NGX_HAVE_INET6)
> + case AF_INET6:
> + sin6 = (struct sockaddr_in6 *) u.addrs[i].sockaddr;
> + cidr[i].u.in6.addr = sin6->sin6_addr;
> + ngx_memset(cidr[i].u.in6.mask.s6_addr, 0xff, 16);
> + break;
> +#endif
> +
> + default: /* AF_INET */
> + sin = (struct sockaddr_in *) u.addrs[i].sockaddr;
> + cidr[i].u.in.addr = sin->sin_addr.s_addr;
> + cidr[i].u.in.mask = 0xffffffff;
> + break;
> + }
> + }
> +
> + return NGX_CONF_OK;
> +}
> diff -r 83c4622053b0 -r 4618e767b84c src/mail/ngx_mail_handler.c
> --- a/src/mail/ngx_mail_handler.c Tue Jan 12 16:59:31 2021 +0300
> +++ b/src/mail/ngx_mail_handler.c Tue Jan 19 18:24:23 2021 +0300
> @@ -12,6 +12,8 @@
>
>
> static void ngx_mail_init_session(ngx_connection_t *c);
> +static void ngx_mail_init_connection_complete(ngx_connection_t *c);
> +static void ngx_mail_proxy_protocol_handler(ngx_event_t *rev);
>
> #if (NGX_MAIL_SSL)
> static void ngx_mail_ssl_init_connection(ngx_ssl_t *ssl, ngx_connection_t *c);
> @@ -128,6 +130,7 @@
>
> s->main_conf = addr_conf->ctx->main_conf;
> s->srv_conf = addr_conf->ctx->srv_conf;
> + s->addr_conf = addr_conf;

There is no need to preserve addr_conf here, relevant flags can
(notably, ssl) can be preserved by itself, similarly to how it is
done in ngx_stream_init_connection().

>
> s->addr_text = &addr_conf->addr_text;
>
> @@ -159,13 +162,181 @@
>
> c->log_error = NGX_ERROR_INFO;
>
> + /*
> + * Before all process proxy protocol
> + */
> +

This looks too verbose.

> + if (addr_conf->proxy_protocol) {
> + s->proxy_protocol = 1;

The "s->proxy_protocol" flag looks meaningless, it is not used
anywhere.

> + c->log->action = "reading PROXY protocol header";
> + c->read->handler = ngx_mail_proxy_protocol_handler;
> +
> + ngx_add_timer(c->read, cscf->timeout);
> +
> + if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
> + ngx_mail_close_connection(c);
> + }
> +
> + return;

Here the handler is not called as long as some data is already
available for reading. While this is unlikely to cause problems
right now in the mail module due to lack of accept filters /
deferred accept support in the mail module (and iocp support being
broken), this is generally wrong. A better approach is to check
c->read->ready and act appropriately, much like stream module code
does.

> + }
> +
> + ngx_mail_init_connection_complete(c);
> +}
> +
> +
> +ngx_int_t
> +ngx_mail_proxy_protoco_set_addrs(ngx_connection_t *c)
> +{
> + ngx_addr_t addr_peer, addr_local;
> + u_char *p, text[NGX_SOCKADDR_STRLEN];
> + size_t len;

Style: wrong order of types, wrong amount of spaces between
types and variable names.

> +
> + if (ngx_parse_addr(c->pool, &addr_peer,
> + c->proxy_protocol->src_addr.data,
> + c->proxy_protocol->src_addr.len) != NGX_OK)
> + {
> + return NGX_ERROR;
> + }
> +
> + ngx_inet_set_port(addr_peer.sockaddr, c->proxy_protocol->src_port);
> +
> + if (ngx_parse_addr(c->pool, &addr_local,
> + c->proxy_protocol->dst_addr.data,
> + c->proxy_protocol->dst_addr.len) != NGX_OK)
> + {
> + return NGX_ERROR;
> + }
> +
> + ngx_inet_set_port(addr_local.sockaddr, c->proxy_protocol->dst_port);
> +
> + len = ngx_sock_ntop(addr_peer.sockaddr, addr_peer.socklen, text,
> + NGX_SOCKADDR_STRLEN, 0);
> + if (len == 0) {
> + return NGX_ERROR;
> + }
> +
> + p = ngx_pnalloc(c->pool, len);
> + if (p == NULL) {
> + return NGX_ERROR;
> + }
> +
> + ngx_memcpy(p, text, len);
> +
> + c->sockaddr = addr_peer.sockaddr;
> + c->socklen = addr_peer.socklen;
> + c->addr_text.len = len;
> + c->addr_text.data = p;
> +
> + len = ngx_sock_ntop(addr_local.sockaddr, addr_local.socklen, text,
> + NGX_SOCKADDR_STRLEN, 0);
> + if (len == 0) {
> + return NGX_ERROR;
> + }
> +
> + p = ngx_pnalloc(c->pool, len);
> + if (p == NULL) {
> + return NGX_ERROR;
> + }
> +
> + ngx_memcpy(p, text, len);
> +
> + c->local_sockaddr = addr_local.sockaddr;
> + c->local_socklen = addr_local.socklen;

Updating c->local_sockaddr looks like a bad idea even if the
address comes from trusted proxy servers. We don't do this
neither in stream nor in http, and I don't think that mail module
is a good place to start doing this.

> +
> + return NGX_OK;
> +}
> +
> +
> +void
> +ngx_mail_proxy_protocol_handler(ngx_event_t *rev)
> +{
> + ngx_mail_core_srv_conf_t *cscf;
> + ngx_mail_session_t *s;
> + ngx_connection_t *c;
> + u_char *p, buf[NGX_PROXY_PROTOCOL_MAX_HEADER];
> + size_t size;
> + ssize_t n;
> +
> + c = rev->data;
> + s = c->data;
> +
> + if (rev->timedout) {
> + ngx_log_error(NGX_LOG_INFO, c->log, NGX_ETIMEDOUT,
> + "mail PROXY protocol header timed out");

There is no need to write such error messages as there is
c->log->action to provide appropriate context. Just "client timed
out" is enough.

> + c->timedout = 1;
> + ngx_mail_close_connection(c);
> + return;
> + }
> +
> + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, c->log, 0,
> + "mail PROXY protocol handler");
> +
> + cscf = ngx_mail_get_module_srv_conf(s, ngx_mail_core_module);
> +
> + if (cscf->realip_from == NGX_CONF_UNSET_PTR) {
> + ngx_log_error(NGX_LOG_WARN, c->log, 0,
> + "using PROXY protocol without set_real_ip_from");
> + ngx_mail_close_connection(c);

With the approach taken, this is clearly a configuration error.
Such configurations should be reported (and rejected) during
configuration parsing.

Further, for configuration errors which cannot be detected during
configuration parsing yet result in things like connections being
dropped a better log level is NGX_LOG_ERR.

The approach, however, looks wrong, see above.

> + return;
> + }
> +
> + if (ngx_cidr_match(c->sockaddr, cscf->realip_from) != NGX_OK) {
> + ngx_log_error(NGX_LOG_NOTICE, c->log, 0,
> + "UNTRUSTED PROXY protocol provider: %V",
> + &c->addr_text);
> + ngx_mail_close_connection(c);

The "notice" level also looks wrong here. Client-related errors
are usually logged at the "info" level.

The approach, however, looks wrong, see above.

> + return;
> + }
> +
> + size = NGX_PROXY_PROTOCOL_MAX_HEADER;
> +
> + n = recv(c->fd, (char *) buf, size, MSG_PEEK);
> +
> + ngx_log_debug1(NGX_LOG_DEBUG, c->log, 0, "mail recv(): %z", n);
> +
> + p = ngx_proxy_protocol_read(c, buf, buf + n);

This seems to lack any error checking for the "n" value returned
by recv(), and will happily call ngx_proxy_protocol_read() with
"last" set to "buf - 1" if recv() returns an error. Most likely
this will result in a segmentation fault.

> +
> + if (p == NULL) {
> + ngx_mail_close_connection(c);
> + return;
> + }
> +
> + ngx_log_error(NGX_LOG_NOTICE, c->log, 0,
> + "PROXY protocol %V:%d => %V:%d",
> + &c->proxy_protocol->src_addr,
> + c->proxy_protocol->src_port,
> + &c->proxy_protocol->dst_addr,
> + c->proxy_protocol->dst_port);

Logging level used looks wrong. At most this should be "info",
much like connect / disconnect logging. Or shouldn't be at all,
as this information is already logged at debug level by
ngx_proxy_protocol_read().

> +
> + size = p - buf;
> +
> + if (c->recv(c, buf, size) != (ssize_t) size) {
> + ngx_mail_close_connection(c);
> + return;
> + }
> +
> + if (ngx_mail_proxy_protoco_set_addrs(c) != NGX_OK) {
> + ngx_mail_close_connection(c);
> + return;
> + }
> +
> + ngx_mail_init_connection_complete(c);
> +}
> +
> +
> +void
> +ngx_mail_init_connection_complete(ngx_connection_t *c)
> +{
> #if (NGX_MAIL_SSL)
> {
> - ngx_mail_ssl_conf_t *sslcf;
> + ngx_mail_session_t *s;
> + ngx_mail_ssl_conf_t *sslcf;
> +
> + s = c->data;

Style: since the code under #if is in a separate function now,
additional block is not needed anymore to introduce variables;
wrong number of spaces between type and variable names.

>
> sslcf = ngx_mail_get_module_srv_conf(s, ngx_mail_ssl_module);
>
> - if (sslcf->enable || addr_conf->ssl) {
> + if (sslcf->enable || s->addr_conf->ssl) {
> c->log->action = "SSL handshaking";
>
> ngx_mail_ssl_init_connection(&sslcf->ssl, c);
> @@ -348,6 +519,7 @@
> return;
> }
>
> + c->log->action = "sending client greeting line";
> c->write->handler = ngx_mail_send;
>
> cscf->protocol->init_session(s, c);

This looks like a pre-existing bug, probably should be addressed
by a separate patch.

> diff -r 83c4622053b0 -r 4618e767b84c src/mail/ngx_mail_proxy_module.c
> --- a/src/mail/ngx_mail_proxy_module.c Tue Jan 12 16:59:31 2021 +0300
> +++ b/src/mail/ngx_mail_proxy_module.c Tue Jan 19 18:24:23 2021 +0300
> @@ -19,6 +19,8 @@
> ngx_flag_t smtp_auth;
> size_t buffer_size;
> ngx_msec_t timeout;
> + ngx_msec_t connect_timeout;
> + ngx_flag_t proxy_protocol;
> } ngx_mail_proxy_conf_t;
>
>
> @@ -36,7 +38,9 @@
> static void *ngx_mail_proxy_create_conf(ngx_conf_t *cf);
> static char *ngx_mail_proxy_merge_conf(ngx_conf_t *cf, void *parent,
> void *child);
> -

Style, see above.

> +static void ngx_mail_proxy_connect_handler(ngx_event_t *ev);
> +static void ngx_mail_proxy_start(ngx_mail_session_t *s);
> +static void ngx_mail_proxy_send_proxy_protocol(ngx_mail_session_t *s);

Style: order of function prototypes does not match order of
functions.

>
> static ngx_command_t ngx_mail_proxy_commands[] = {
>
> @@ -61,6 +65,13 @@
> offsetof(ngx_mail_proxy_conf_t, timeout),
> NULL },
>
> + { ngx_string("connect_timeout"),
> + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1,
> + ngx_conf_set_msec_slot,
> + NGX_MAIL_SRV_CONF_OFFSET,
> + offsetof(ngx_mail_proxy_conf_t, connect_timeout),
> + NULL },
> +

Not needed, see above.

> { ngx_string("proxy_pass_error_message"),
> NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_FLAG,
> ngx_conf_set_flag_slot,
> @@ -82,6 +93,13 @@
> offsetof(ngx_mail_proxy_conf_t, smtp_auth),
> NULL },
>
> + { ngx_string("proxy_protocol"),
> + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_FLAG,
> + ngx_conf_set_flag_slot,
> + NGX_MAIL_SRV_CONF_OFFSET,
> + offsetof(ngx_mail_proxy_conf_t, proxy_protocol),
> + NULL },
> +
> ngx_null_command
> };
>
> @@ -156,7 +174,6 @@
> p->upstream.connection->pool = s->connection->pool;
>
> s->connection->read->handler = ngx_mail_proxy_block_read;
> - p->upstream.connection->write->handler = ngx_mail_proxy_dummy_handler;
>
> pcf = ngx_mail_get_module_srv_conf(s, ngx_mail_proxy_module);
>
> @@ -169,23 +186,139 @@
>
> s->out.len = 0;
>
> + if (rc == NGX_AGAIN) {
> + p->upstream.connection->write->handler = ngx_mail_proxy_connect_handler;
> + p->upstream.connection->read->handler = ngx_mail_proxy_connect_handler;
> +
> + ngx_add_timer(p->upstream.connection->write, pcf->connect_timeout);
> +
> + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, s->connection->log, 0, "mail proxy delay connect");

Style: too long line, should be less than 80 chars.

Further, the message looks misleading. There is no delay here,
but rather connect is in progress.

If you think that better debugging should be added to indicate the
connect() status, a better approach would be to add debugging
right after the ngx_event_connect_peer() call, like in the
ngx_http_upstream_module:

rc = ngx_event_connect_peer(&u->peer);

ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
"http upstream connect: %i", rc);

Note though that this look unrelated to the change itself, and
should be a separate patch.

> + return;
> + }
> +
> + if (pcf->proxy_protocol) {
> + ngx_mail_proxy_send_proxy_protocol(s);
> + return;
> + }
> +
> + ngx_mail_proxy_start(s);
> +}
> +
> +
> +void
> +ngx_mail_proxy_connect_handler(ngx_event_t *ev)
> +{
> + ngx_connection_t *c;
> + ngx_mail_session_t *s;
> + ngx_mail_proxy_conf_t *pcf;

Style: should be two spaces between the longest type and
variable name.

> +
> + c = ev->data;
> + s = c->data;
> +
> + if (ev->timedout) {
> + ngx_log_error(NGX_LOG_ERR, c->log, NGX_ETIMEDOUT, "upstream timed out");
> + ngx_mail_session_internal_server_error(s);
> + return;
> + }
> +
> + ngx_del_timer(c->write);
> +
> + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, c->log, 0,
> + "mail proxy connect upstream");
> +
> + pcf = ngx_mail_get_module_srv_conf(s, ngx_mail_proxy_module);
> +
> + if (pcf->proxy_protocol) {
> + ngx_mail_proxy_send_proxy_protocol(s);
> + return;
> + }
> +
> + ngx_mail_proxy_start(s);
> +}
> +
> +
> +void
> +ngx_mail_proxy_start(ngx_mail_session_t *s)
> +{
> + ngx_connection_t *pc;
> +
> + pc = s->proxy->upstream.connection;
> +
> + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, s->connection->log, 0,
> + "mail proxy starting");
> +
> + pc->write->handler = ngx_mail_proxy_dummy_handler;
> +
> switch (s->protocol) {
>
> case NGX_MAIL_POP3_PROTOCOL:
> - p->upstream.connection->read->handler = ngx_mail_proxy_pop3_handler;
> + pc->read->handler = ngx_mail_proxy_pop3_handler;
> s->mail_state = ngx_pop3_start;
> break;
>
> case NGX_MAIL_IMAP_PROTOCOL:
> - p->upstream.connection->read->handler = ngx_mail_proxy_imap_handler;
> + pc->read->handler = ngx_mail_proxy_imap_handler;
> s->mail_state = ngx_imap_start;
> break;
>
> default: /* NGX_MAIL_SMTP_PROTOCOL */
> - p->upstream.connection->read->handler = ngx_mail_proxy_smtp_handler;
> + pc->read->handler = ngx_mail_proxy_smtp_handler;
> s->mail_state = ngx_smtp_start;
> break;
> }
> +
> + if (pc->read->ready) {
> + ngx_post_event(pc->read, &ngx_posted_events);
> + }
> +}
> +
> +
> +void
> +ngx_mail_proxy_send_proxy_protocol(ngx_mail_session_t *s)
> +{
> + u_char *p;
> + ssize_t n, size;
> + ngx_connection_t *c, *pc;
> + ngx_peer_connection_t *u;
> + u_char buf[NGX_PROXY_PROTOCOL_MAX_HEADER];
> +
> + c = s->connection;
> +
> + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, c->log, 0,
> + "mail proxy send PROXY protocol header");
> +
> + p = ngx_proxy_protocol_write(c, buf, buf + NGX_PROXY_PROTOCOL_MAX_HEADER);
> + if (p == NULL) {
> + ngx_mail_proxy_internal_server_error(s);
> + return;
> + }
> +
> + u = &s->proxy->upstream;
> +
> + pc = u->connection;
> +
> + size = p - buf;
> +
> + n = pc->send(pc, buf, size);
> +
> + if (n != size) {
> +
> + /*
> + * PROXY protocol specification:
> + * The sender must always ensure that the header
> + * is sent at once, so that the transport layer
> + * maintains atomicity along the path to the receiver.
> + */
> +
> + ngx_log_error(NGX_LOG_ERR, c->log, 0,
> + "could not send PROXY protocol header at once (%z)", n);
> +
> + ngx_mail_proxy_internal_server_error(s);
> +
> + return;
> + }

The error handling seems to be incomplete here. While it is
unlikely that things like NGX_AGAIN will be returned here, it is
quite possible that c->send() will fail with an error if
connection is reset by the upstream server, and this is going to
result in duplicate and misleading error message.

> +
> + ngx_mail_proxy_start(s);
> }
>
>
> @@ -1184,6 +1317,8 @@
> pcf->smtp_auth = NGX_CONF_UNSET;
> pcf->buffer_size = NGX_CONF_UNSET_SIZE;
> pcf->timeout = NGX_CONF_UNSET_MSEC;
> + pcf->connect_timeout = NGX_CONF_UNSET_MSEC;
> + pcf->proxy_protocol = NGX_CONF_UNSET;
>
> return pcf;
> }
> @@ -1202,6 +1337,8 @@
> ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,
> (size_t) ngx_pagesize);
> ngx_conf_merge_msec_value(conf->timeout, prev->timeout, 24 * 60 * 60000);
> + ngx_conf_merge_msec_value(conf->connect_timeout, prev->connect_timeout, 1000);
> + ngx_conf_merge_value(conf->proxy_protocol, prev->proxy_protocol, 0);
>
> return NGX_CONF_OK;
> }

Below is a patch series which tries to address most of the above
comments, and also fixes several existing problems in the code
found in the process. Review is appreciated.

# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1613784928 -10800
# Sat Feb 20 04:35:28 2021 +0300
# Node ID 00bc617ccb86019850640db9c93ee6d8ea5f56d4
# Parent f77ad78046dcaa39ae4d4ddeeb52a63846f7d579
SSL: fixed build by Sun C with old OpenSSL versions.

Sun C complains about "statement not reached" if a "return" is followed
by additional statements.

diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c
--- a/src/http/modules/ngx_http_grpc_module.c
+++ b/src/http/modules/ngx_http_grpc_module.c
@@ -4841,9 +4841,9 @@ ngx_http_grpc_ssl_conf_command_check(ngx
{
#ifndef SSL_CONF_FLAG_FILE
return "is not supported on this platform";
+#else
+ return NGX_CONF_OK;
#endif
-
- return NGX_CONF_OK;
}


diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c
--- a/src/http/modules/ngx_http_proxy_module.c
+++ b/src/http/modules/ngx_http_proxy_module.c
@@ -4876,9 +4876,9 @@ ngx_http_proxy_ssl_conf_command_check(ng
{
#ifndef SSL_CONF_FLAG_FILE
return "is not supported on this platform";
+#else
+ return NGX_CONF_OK;
#endif
-
- return NGX_CONF_OK;
}


diff --git a/src/http/modules/ngx_http_ssl_module.c b/src/http/modules/ngx_http_ssl_module.c
--- a/src/http/modules/ngx_http_ssl_module.c
+++ b/src/http/modules/ngx_http_ssl_module.c
@@ -1274,9 +1274,9 @@ ngx_http_ssl_conf_command_check(ngx_conf
{
#ifndef SSL_CONF_FLAG_FILE
return "is not supported on this platform";
+#else
+ return NGX_CONF_OK;
#endif
-
- return NGX_CONF_OK;
}


diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c
--- a/src/http/modules/ngx_http_uwsgi_module.c
+++ b/src/http/modules/ngx_http_uwsgi_module.c
@@ -2398,9 +2398,9 @@ ngx_http_uwsgi_ssl_conf_command_check(ng
{
#ifndef SSL_CONF_FLAG_FILE
return "is not supported on this platform";
+#else
+ return NGX_CONF_OK;
#endif
-
- return NGX_CONF_OK;
}


diff --git a/src/mail/ngx_mail_ssl_module.c b/src/mail/ngx_mail_ssl_module.c
--- a/src/mail/ngx_mail_ssl_module.c
+++ b/src/mail/ngx_mail_ssl_module.c
@@ -682,7 +682,7 @@ ngx_mail_ssl_conf_command_check(ngx_conf
{
#ifndef SSL_CONF_FLAG_FILE
return "is not supported on this platform";
+#else
+ return NGX_CONF_OK;
#endif
-
- return NGX_CONF_OK;
}
diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c
--- a/src/stream/ngx_stream_proxy_module.c
+++ b/src/stream/ngx_stream_proxy_module.c
@@ -1026,9 +1026,9 @@ ngx_stream_proxy_ssl_conf_command_check(
{
#ifndef SSL_CONF_FLAG_FILE
return "is not supported on this platform";
+#else
+ return NGX_CONF_OK;
#endif
-
- return NGX_CONF_OK;
}


diff --git a/src/stream/ngx_stream_ssl_module.c b/src/stream/ngx_stream_ssl_module.c
--- a/src/stream/ngx_stream_ssl_module.c
+++ b/src/stream/ngx_stream_ssl_module.c
@@ -1061,9 +1061,9 @@ ngx_stream_ssl_conf_command_check(ngx_co
{
#ifndef SSL_CONF_FLAG_FILE
return "is not supported on this platform";
+#else
+ return NGX_CONF_OK;
#endif
-
- return NGX_CONF_OK;
}


# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1613784931 -10800
# Sat Feb 20 04:35:31 2021 +0300
# Node ID 63777b5cbc69027eb96ff70674d4ffa2b99aa3a4
# Parent 00bc617ccb86019850640db9c93ee6d8ea5f56d4
Events: fixed eventport handling in ngx_handle_read_event().

The "!rev->ready" test seems to be a typo, introduced in the original
commit (719:f30b1a75fd3b). The ngx_handle_write_event() code properly
tests for "rev->ready" instead.

Due to this typo, read events might be unexpectedly removed during
proxying after an event on the other part of the proxied connection.
Catched by mail proxying tests.

diff --git a/src/event/ngx_event.c b/src/event/ngx_event.c
--- a/src/event/ngx_event.c
+++ b/src/event/ngx_event.c
@@ -318,7 +318,7 @@ ngx_handle_read_event(ngx_event_t *rev,
return NGX_OK;
}

- if (rev->oneshot && !rev->ready) {
+ if (rev->oneshot && rev->ready) {
if (ngx_del_event(rev, NGX_READ_EVENT, 0) == NGX_ERROR) {
return NGX_ERROR;
}
# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1613784935 -10800
# Sat Feb 20 04:35:35 2021 +0300
# Node ID f776e728c2527301bf2cd22ef11bee1fb982f537
# Parent 63777b5cbc69027eb96ff70674d4ffa2b99aa3a4
Mail: added missing event handling after blocking events.

As long as a read event is blocked (ignored), ngx_handle_read_event()
needs to be called to make sure no further notifications will be
triggered when using level-triggered event methods, such as select() or
poll().

diff --git a/src/mail/ngx_mail_imap_handler.c b/src/mail/ngx_mail_imap_handler.c
--- a/src/mail/ngx_mail_imap_handler.c
+++ b/src/mail/ngx_mail_imap_handler.c
@@ -123,6 +123,12 @@ ngx_mail_imap_auth_state(ngx_event_t *re
if (s->out.len) {
ngx_log_debug0(NGX_LOG_DEBUG_MAIL, c->log, 0, "imap send handler busy");
s->blocked = 1;
+
+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_close_connection(c);
+ return;
+ }
+
return;
}

diff --git a/src/mail/ngx_mail_pop3_handler.c b/src/mail/ngx_mail_pop3_handler.c
--- a/src/mail/ngx_mail_pop3_handler.c
+++ b/src/mail/ngx_mail_pop3_handler.c
@@ -138,6 +138,12 @@ ngx_mail_pop3_auth_state(ngx_event_t *re
if (s->out.len) {
ngx_log_debug0(NGX_LOG_DEBUG_MAIL, c->log, 0, "pop3 send handler busy");
s->blocked = 1;
+
+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_close_connection(c);
+ return;
+ }
+
return;
}

diff --git a/src/mail/ngx_mail_smtp_handler.c b/src/mail/ngx_mail_smtp_handler.c
--- a/src/mail/ngx_mail_smtp_handler.c
+++ b/src/mail/ngx_mail_smtp_handler.c
@@ -449,6 +449,12 @@ ngx_mail_smtp_auth_state(ngx_event_t *re
if (s->out.len) {
ngx_log_debug0(NGX_LOG_DEBUG_MAIL, c->log, 0, "smtp send handler busy");
s->blocked = 1;
+
+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_close_connection(c);
+ return;
+ }
+
return;
}

# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1613787594 -10800
# Sat Feb 20 05:19:54 2021 +0300
# Node ID 0ab90ee2fd58ec5d33405333cc44049d8e5a37a7
# Parent f776e728c2527301bf2cd22ef11bee1fb982f537
Mail: added missing event handling after reading data.

If we need to be notified about further events, ngx_handle_read_event()
needs to be called after a read event is processed. Without this,
an event can be removed from the kernel and won't be reported again,
notably when using oneshot event methods, such as eventport on Solaris.

For consistency, existing ngx_handle_read_event() call removed from
ngx_mail_read_command(), as this call only covers of of the code paths
where ngx_mail_read_command() returns NGX_AGAIN. Instead, appropriate
processing added to the callers, covering all code paths where NGX_AGAIN
is returned.

diff --git a/src/mail/ngx_mail_handler.c b/src/mail/ngx_mail_handler.c
--- a/src/mail/ngx_mail_handler.c
+++ b/src/mail/ngx_mail_handler.c
@@ -722,11 +722,6 @@ ngx_mail_read_command(ngx_mail_session_t
}

if (n == NGX_AGAIN) {
- if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
- ngx_mail_session_internal_server_error(s);
- return NGX_ERROR;
- }
-
if (s->buffer->pos == s->buffer->last) {
return NGX_AGAIN;
}
diff --git a/src/mail/ngx_mail_imap_handler.c b/src/mail/ngx_mail_imap_handler.c
--- a/src/mail/ngx_mail_imap_handler.c
+++ b/src/mail/ngx_mail_imap_handler.c
@@ -136,7 +136,16 @@ ngx_mail_imap_auth_state(ngx_event_t *re

rc = ngx_mail_read_command(s, c);

- if (rc == NGX_AGAIN || rc == NGX_ERROR) {
+ if (rc == NGX_AGAIN) {
+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_session_internal_server_error(s);
+ return;
+ }
+
+ return;
+ }
+
+ if (rc == NGX_ERROR) {
return;
}

@@ -299,6 +308,11 @@ ngx_mail_imap_auth_state(ngx_event_t *re
}
}

+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_session_internal_server_error(s);
+ return;
+ }
+
ngx_mail_send(c->write);
}

diff --git a/src/mail/ngx_mail_pop3_handler.c b/src/mail/ngx_mail_pop3_handler.c
--- a/src/mail/ngx_mail_pop3_handler.c
+++ b/src/mail/ngx_mail_pop3_handler.c
@@ -151,7 +151,16 @@ ngx_mail_pop3_auth_state(ngx_event_t *re

rc = ngx_mail_read_command(s, c);

- if (rc == NGX_AGAIN || rc == NGX_ERROR) {
+ if (rc == NGX_AGAIN) {
+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_session_internal_server_error(s);
+ return;
+ }
+
+ return;
+ }
+
+ if (rc == NGX_ERROR) {
return;
}

@@ -281,6 +290,11 @@ ngx_mail_pop3_auth_state(ngx_event_t *re
s->arg_start = s->buffer->start;
}

+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_session_internal_server_error(s);
+ return;
+ }
+
ngx_mail_send(c->write);
}
}
diff --git a/src/mail/ngx_mail_proxy_module.c b/src/mail/ngx_mail_proxy_module.c
--- a/src/mail/ngx_mail_proxy_module.c
+++ b/src/mail/ngx_mail_proxy_module.c
@@ -233,6 +233,11 @@ ngx_mail_proxy_pop3_handler(ngx_event_t
rc = ngx_mail_proxy_read_response(s, 0);

if (rc == NGX_AGAIN) {
+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_proxy_internal_server_error(s);
+ return;
+ }
+
return;
}

@@ -314,6 +319,11 @@ ngx_mail_proxy_pop3_handler(ngx_event_t
return;
}

+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_proxy_internal_server_error(s);
+ return;
+ }
+
s->proxy->buffer->pos = s->proxy->buffer->start;
s->proxy->buffer->last = s->proxy->buffer->start;
}
@@ -346,6 +356,11 @@ ngx_mail_proxy_imap_handler(ngx_event_t
rc = ngx_mail_proxy_read_response(s, s->mail_state);

if (rc == NGX_AGAIN) {
+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_proxy_internal_server_error(s);
+ return;
+ }
+
return;
}

@@ -448,6 +463,11 @@ ngx_mail_proxy_imap_handler(ngx_event_t
return;
}

+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_proxy_internal_server_error(s);
+ return;
+ }
+
s->proxy->buffer->pos = s->proxy->buffer->start;
s->proxy->buffer->last = s->proxy->buffer->start;
}
@@ -482,6 +502,11 @@ ngx_mail_proxy_smtp_handler(ngx_event_t
rc = ngx_mail_proxy_read_response(s, s->mail_state);

if (rc == NGX_AGAIN) {
+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_proxy_internal_server_error(s);
+ return;
+ }
+
return;
}

@@ -763,6 +788,11 @@ ngx_mail_proxy_smtp_handler(ngx_event_t
return;
}

+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_proxy_internal_server_error(s);
+ return;
+ }
+
s->proxy->buffer->pos = s->proxy->buffer->start;
s->proxy->buffer->last = s->proxy->buffer->start;
}
diff --git a/src/mail/ngx_mail_smtp_handler.c b/src/mail/ngx_mail_smtp_handler.c
--- a/src/mail/ngx_mail_smtp_handler.c
+++ b/src/mail/ngx_mail_smtp_handler.c
@@ -462,7 +462,16 @@ ngx_mail_smtp_auth_state(ngx_event_t *re

rc = ngx_mail_read_command(s, c);

- if (rc == NGX_AGAIN || rc == NGX_ERROR) {
+ if (rc == NGX_AGAIN) {
+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_session_internal_server_error(s);
+ return;
+ }
+
+ return;
+ }
+
+ if (rc == NGX_ERROR) {
return;
}

@@ -574,6 +583,11 @@ ngx_mail_smtp_auth_state(ngx_event_t *re
s->arg_start = s->buffer->pos;
}

+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_session_internal_server_error(s);
+ return;
+ }
+
ngx_mail_send(c->write);
}
}
# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1613787635 -10800
# Sat Feb 20 05:20:35 2021 +0300
# Node ID 7361e60def463335d33d6406257b2f2f905d33dd
# Parent 0ab90ee2fd58ec5d33405333cc44049d8e5a37a7
Mail: postponed session initialization under accept mutex.

Similarly to 40e8ce405859 in the stream module, this reduces the time
accept mutex is held. This also simplifies following changes to
introduce PROXY protocol support.

diff --git a/src/mail/ngx_mail.h b/src/mail/ngx_mail.h
--- a/src/mail/ngx_mail.h
+++ b/src/mail/ngx_mail.h
@@ -197,6 +197,7 @@ typedef struct {

ngx_uint_t mail_state;

+ unsigned ssl:1;
unsigned protocol:3;
unsigned blocked:1;
unsigned quit:1;
diff --git a/src/mail/ngx_mail_handler.c b/src/mail/ngx_mail_handler.c
--- a/src/mail/ngx_mail_handler.c
+++ b/src/mail/ngx_mail_handler.c
@@ -11,6 +11,7 @@
#include <ngx_mail.h>


+static void ngx_mail_init_session_handler(ngx_event_t *rev);
static void ngx_mail_init_session(ngx_connection_t *c);

#if (NGX_MAIL_SSL)
@@ -26,6 +27,7 @@ ngx_mail_init_connection(ngx_connection_
{
size_t len;
ngx_uint_t i;
+ ngx_event_t *rev;
ngx_mail_port_t *port;
struct sockaddr *sa;
struct sockaddr_in *sin;
@@ -129,6 +131,10 @@ ngx_mail_init_connection(ngx_connection_
s->main_conf = addr_conf->ctx->main_conf;
s->srv_conf = addr_conf->ctx->srv_conf;

+#if (NGX_MAIL_SSL)
+ s->ssl = addr_conf->ssl;
+#endif
+
s->addr_text = &addr_conf->addr_text;

c->data = s;
@@ -159,13 +165,34 @@ ngx_mail_init_connection(ngx_connection_

c->log_error = NGX_ERROR_INFO;

+ rev = c->read;
+ rev->handler = ngx_mail_init_session_handler;
+
+ if (ngx_use_accept_mutex) {
+ ngx_post_event(rev, &ngx_posted_events);
+ return;
+ }
+
+ rev->handler(rev);
+}
+
+
+static void
+ngx_mail_init_session_handler(ngx_event_t *rev)
+{
+ ngx_connection_t *c;
+ ngx_mail_session_t *s;
+
+ c = rev->data;
+ s = c->data;
+
#if (NGX_MAIL_SSL)
{
ngx_mail_ssl_conf_t *sslcf;

sslcf = ngx_mail_get_module_srv_conf(s, ngx_mail_ssl_module);

- if (sslcf->enable || addr_conf->ssl) {
+ if (sslcf->enable || s->ssl) {
c->log->action = "SSL handshaking";

ngx_mail_ssl_init_connection(&sslcf->ssl, c);
# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1613787636 -10800
# Sat Feb 20 05:20:36 2021 +0300
# Node ID 698118019f618a4b746986b727189e281310d9d1
# Parent 7361e60def463335d33d6406257b2f2f905d33dd
Mail: fixed log action after SSL handshake.

diff --git a/src/mail/ngx_mail_handler.c b/src/mail/ngx_mail_handler.c
--- a/src/mail/ngx_mail_handler.c
+++ b/src/mail/ngx_mail_handler.c
@@ -365,6 +365,8 @@ ngx_mail_init_session(ngx_connection_t *

s = c->data;

+ c->log->action = "sending client greeting line";
+
cscf = ngx_mail_get_module_srv_conf(s, ngx_mail_core_module);

s->protocol = cscf->protocol->type;
# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1613787637 -10800
# Sat Feb 20 05:20:37 2021 +0300
# Node ID 23008e1a0113f5bc22028473e1c914e32d503374
# Parent 698118019f618a4b746986b727189e281310d9d1
Mail: made auth http creating request easier to extend.

diff --git a/src/mail/ngx_mail_auth_http_module.c b/src/mail/ngx_mail_auth_http_module.c
--- a/src/mail/ngx_mail_auth_http_module.c
+++ b/src/mail/ngx_mail_auth_http_module.c
@@ -1224,22 +1224,38 @@ ngx_mail_auth_http_create_request(ngx_ma
+ sizeof("Client-IP: ") - 1 + s->connection->addr_text.len
+ sizeof(CRLF) - 1
+ sizeof("Client-Host: ") - 1 + s->host.len + sizeof(CRLF) - 1
- + sizeof("Auth-SMTP-Helo: ") - 1 + s->smtp_helo.len + sizeof(CRLF) - 1
- + sizeof("Auth-SMTP-From: ") - 1 + s->smtp_from.len + sizeof(CRLF) - 1
- + sizeof("Auth-SMTP-To: ") - 1 + s->smtp_to.len + sizeof(CRLF) - 1
-#if (NGX_MAIL_SSL)
- + sizeof("Auth-SSL: on" CRLF) - 1
- + sizeof("Auth-SSL-Verify: ") - 1 + verify.len + sizeof(CRLF) - 1
- + sizeof("Auth-SSL-Subject: ") - 1 + subject.len + sizeof(CRLF) - 1
- + sizeof("Auth-SSL-Issuer: ") - 1 + issuer.len + sizeof(CRLF) - 1
- + sizeof("Auth-SSL-Serial: ") - 1 + serial.len + sizeof(CRLF) - 1
- + sizeof("Auth-SSL-Fingerprint: ") - 1 + fingerprint.len
- + sizeof(CRLF) - 1
- + sizeof("Auth-SSL-Cert: ") - 1 + cert.len + sizeof(CRLF) - 1
-#endif
+ ahcf->header.len
+ sizeof(CRLF) - 1;

+ if (s->auth_method == NGX_MAIL_AUTH_NONE) {
+ len += sizeof("Auth-SMTP-Helo: ") - 1 + s->smtp_helo.len
+ + sizeof(CRLF) - 1
+ + sizeof("Auth-SMTP-From: ") - 1 + s->smtp_from.len
+ + sizeof(CRLF) - 1
+ + sizeof("Auth-SMTP-To: ") - 1 + s->smtp_to.len
+ + sizeof(CRLF) - 1;
+ }
+
+#if (NGX_MAIL_SSL)
+
+ if (c->ssl) {
+ len += sizeof("Auth-SSL: on" CRLF) - 1
+ + sizeof("Auth-SSL-Verify: ") - 1 + verify.len
+ + sizeof(CRLF) - 1
+ + sizeof("Auth-SSL-Subject: ") - 1 + subject.len
+ + sizeof(CRLF) - 1
+ + sizeof("Auth-SSL-Issuer: ") - 1 + issuer.len
+ + sizeof(CRLF) - 1
+ + sizeof("Auth-SSL-Serial: ") - 1 + serial.len
+ + sizeof(CRLF) - 1
+ + sizeof("Auth-SSL-Fingerprint: ") - 1 + fingerprint.len
+ + sizeof(CRLF) - 1
+ + sizeof("Auth-SSL-Cert: ") - 1 + cert.len
+ + sizeof(CRLF) - 1;
+ }
+
+#endif
+
b = ngx_create_temp_buf(pool, len);
if (b == NULL) {
return NULL;
# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1613793031 -10800
# Sat Feb 20 06:50:31 2021 +0300
# Node ID e7035246851a7c50ac7083f563ba754cc3ddb2f0
# Parent 23008e1a0113f5bc22028473e1c914e32d503374
Mail: parsing of the PROXY protocol from clients.

Activated with the "proxy_protocol" parameter of the "listen" directive.
Obtained information is passed to the auth_http script in Proxy-Protocol-Addr,
Proxy-Protocol-Port, Proxy-Protocol-Server-Addr, and Proxy-Protocol-Server-Port
headers.

diff --git a/src/mail/ngx_mail.c b/src/mail/ngx_mail.c
--- a/src/mail/ngx_mail.c
+++ b/src/mail/ngx_mail.c
@@ -405,6 +405,7 @@ ngx_mail_add_addrs(ngx_conf_t *cf, ngx_m
#if (NGX_MAIL_SSL)
addrs[i].conf.ssl = addr[i].opt.ssl;
#endif
+ addrs[i].conf.proxy_protocol = addr[i].opt.proxy_protocol;
addrs[i].conf.addr_text = addr[i].opt.addr_text;
}

@@ -439,6 +440,7 @@ ngx_mail_add_addrs6(ngx_conf_t *cf, ngx_
#if (NGX_MAIL_SSL)
addrs6[i].conf.ssl = addr[i].opt.ssl;
#endif
+ addrs6[i].conf.proxy_protocol = addr[i].opt.proxy_protocol;
addrs6[i].conf.addr_text = addr[i].opt.addr_text;
}

diff --git a/src/mail/ngx_mail.h b/src/mail/ngx_mail.h
--- a/src/mail/ngx_mail.h
+++ b/src/mail/ngx_mail.h
@@ -41,6 +41,7 @@ typedef struct {
unsigned ipv6only:1;
#endif
unsigned so_keepalive:2;
+ unsigned proxy_protocol:1;
#if (NGX_HAVE_KEEPALIVE_TUNABLE)
int tcp_keepidle;
int tcp_keepintvl;
@@ -55,7 +56,8 @@ typedef struct {
typedef struct {
ngx_mail_conf_ctx_t *ctx;
ngx_str_t addr_text;
- ngx_uint_t ssl; /* unsigned ssl:1; */
+ unsigned ssl:1;
+ unsigned proxy_protocol:1;
} ngx_mail_addr_conf_t;

typedef struct {
diff --git a/src/mail/ngx_mail_auth_http_module.c b/src/mail/ngx_mail_auth_http_module.c
--- a/src/mail/ngx_mail_auth_http_module.c
+++ b/src/mail/ngx_mail_auth_http_module.c
@@ -1227,6 +1227,17 @@ ngx_mail_auth_http_create_request(ngx_ma
+ ahcf->header.len
+ sizeof(CRLF) - 1;

+ if (c->proxy_protocol) {
+ len += sizeof("Proxy-Protocol-Addr: ") - 1
+ + c->proxy_protocol->src_addr.len + sizeof(CRLF) - 1
+ + sizeof("Proxy-Protocol-Port: ") - 1
+ + sizeof("65535") - 1 + sizeof(CRLF) - 1
+ + sizeof("Proxy-Protocol-Server-Addr: ") - 1
+ + c->proxy_protocol->dst_addr.len + sizeof(CRLF) - 1
+ + sizeof("Proxy-Protocol-Server-Port: ") - 1
+ + sizeof("65535") - 1 + sizeof(CRLF) - 1;
+ }
+
if (s->auth_method == NGX_MAIL_AUTH_NONE) {
len += sizeof("Auth-SMTP-Helo: ") - 1 + s->smtp_helo.len
+ sizeof(CRLF) - 1
@@ -1314,6 +1325,26 @@ ngx_mail_auth_http_create_request(ngx_ma
*b->last++ = CR; *b->last++ = LF;
}

+ if (c->proxy_protocol) {
+ b->last = ngx_cpymem(b->last, "Proxy-Protocol-Addr: ",
+ sizeof("Proxy-Protocol-Addr: ") - 1);
+ b->last = ngx_copy(b->last, c->proxy_protocol->src_addr.data,
+ c->proxy_protocol->src_addr.len);
+ *b->last++ = CR; *b->last++ = LF;
+
+ b->last = ngx_sprintf(b->last, "Proxy-Protocol-Port: %d" CRLF,
+ c->proxy_protocol->src_port);
+
+ b->last = ngx_cpymem(b->last, "Proxy-Protocol-Server-Addr: ",
+ sizeof("Proxy-Protocol-Server-Addr: ") - 1);
+ b->last = ngx_copy(b->last, c->proxy_protocol->dst_addr.data,
+ c->proxy_protocol->dst_addr.len);
+ *b->last++ = CR; *b->last++ = LF;
+
+ b->last = ngx_sprintf(b->last, "Proxy-Protocol-Server-Port: %d" CRLF,
+ c->proxy_protocol->dst_port);
+ }
+
if (s->auth_method == NGX_MAIL_AUTH_NONE) {

/* HELO, MAIL FROM, and RCPT TO can't contain CRLF, no need to escape */
diff --git a/src/mail/ngx_mail_core_module.c b/src/mail/ngx_mail_core_module.c
--- a/src/mail/ngx_mail_core_module.c
+++ b/src/mail/ngx_mail_core_module.c
@@ -548,6 +548,11 @@ ngx_mail_core_listen(ngx_conf_t *cf, ngx
#endif
}

+ if (ngx_strcmp(value[i].data, "proxy_protocol") == 0) {
+ ls->proxy_protocol = 1;
+ continue;
+ }
+
ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
"the invalid \"%V\" parameter", &value[i]);
return NGX_CONF_ERROR;
diff --git a/src/mail/ngx_mail_handler.c b/src/mail/ngx_mail_handler.c
--- a/src/mail/ngx_mail_handler.c
+++ b/src/mail/ngx_mail_handler.c
@@ -11,6 +11,7 @@
#include <ngx_mail.h>


+static void ngx_mail_proxy_protocol_handler(ngx_event_t *rev);
static void ngx_mail_init_session_handler(ngx_event_t *rev);
static void ngx_mail_init_session(ngx_connection_t *c);

@@ -168,6 +169,22 @@ ngx_mail_init_connection(ngx_connection_
rev = c->read;
rev->handler = ngx_mail_init_session_handler;

+ if (addr_conf->proxy_protocol) {
+ c->log->action = "reading PROXY protocol";
+
+ rev->handler = ngx_mail_proxy_protocol_handler;
+
+ if (!rev->ready) {
+ ngx_add_timer(rev, cscf->timeout);
+
+ if (ngx_handle_read_event(rev, 0) != NGX_OK) {
+ ngx_mail_close_connection(c);
+ }
+
+ return;
+ }
+ }
+
if (ngx_use_accept_mutex) {
ngx_post_event(rev, &ngx_posted_events);
return;
@@ -178,6 +195,76 @@ ngx_mail_init_connection(ngx_connection_


static void
+ngx_mail_proxy_protocol_handler(ngx_event_t *rev)
+{
+ u_char *p, buf[NGX_PROXY_PROTOCOL_MAX_HEADER];
+ size_t size;
+ ssize_t n;
+ ngx_err_t err;
+ ngx_connection_t *c;
+ ngx_mail_session_t *s;
+ ngx_mail_core_srv_conf_t *cscf;
+
+ c = rev->data;
+ s = c->data;
+
+ ngx_log_debug0(NGX_LOG_DEBUG_MAIL, c->log, 0,
+ "mail PROXY protocol handler");
+
+ if (rev->timedout) {
+ ngx_log_error(NGX_LOG_INFO, c->log, NGX_ETIMEDOUT, "client timed out");
+ c->timedout = 1;
+ ngx_mail_close_connection(c);
+ return;
+ }
+
+ n = recv(c->fd, (char *) buf, sizeof(buf), MSG_PEEK);
+
+ err = ngx_socket_errno;
+
+ ngx_log_debug1(NGX_LOG_DEBUG_MAIL, c->log, 0, "recv(): %z", n);
+
+ if (n == -1) {
+ if (err == NGX_EAGAIN) {
+ rev->ready = 0;
+
+ if (!rev->timer_set) {
+ cscf = ngx_mail_get_module_srv_conf(s, ngx_mail_core_module);
+ ngx_add_timer(rev, cscf->timeout);
+ }
+
+ if (ngx_handle_read_event(rev, 0) != NGX_OK) {
+ ngx_mail_close_connection(c);
+ }
+
+ return;
+ }
+
+ ngx_connection_error(c, err, "recv() failed");
+
+ ngx_mail_close_connection(c);
+ return;
+ }
+
+ p = ngx_proxy_protocol_read(c, buf, buf + n);
+
+ if (p == NULL) {
+ ngx_mail_close_connection(c);
+ return;
+ }
+
+ size = p - buf;
+
+ if (c->recv(c, buf, size) != (ssize_t) size) {
+ ngx_mail_close_connection(c);
+ return;
+ }
+
+ ngx_mail_init_session_handler(rev);
+}
+
+
+static void
ngx_mail_init_session_handler(ngx_event_t *rev)
{
ngx_connection_t *c;
@@ -242,9 +329,10 @@ ngx_mail_ssl_init_connection(ngx_ssl_t *

s = c->data;

- cscf = ngx_mail_get_module_srv_conf(s, ngx_mail_core_module);
-
- ngx_add_timer(c->read, cscf->timeout);
+ if (!c->read->timer_set) {
+ cscf = ngx_mail_get_module_srv_conf(s, ngx_mail_core_module);
+ ngx_add_timer(c->read, cscf->timeout);
+ }

c->ssl->handler = ngx_mail_ssl_handshake_handler;

# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1613793034 -10800
# Sat Feb 20 06:50:34 2021 +0300
# Node ID 068d1bf4f2b3faaf84ec937c091274c940b5e1ca
# Parent e7035246851a7c50ac7083f563ba754cc3ddb2f0
Mail: realip module.

When configured with the "set_real_ip_from", it can set client's IP
address as visible in logs to the one obtained via the PROXY protocol.

diff --git a/auto/modules b/auto/modules
--- a/auto/modules
+++ b/auto/modules
@@ -985,6 +985,12 @@ if [ $MAIL != NO ]; then
ngx_module_srcs=src/mail/ngx_mail_proxy_module.c

. auto/module
+
+ ngx_module_name=ngx_mail_realip_module
+ ngx_module_deps=
+ ngx_module_srcs=src/mail/ngx_mail_realip_module.c
+
+ . auto/module
fi


diff --git a/src/mail/ngx_mail.h b/src/mail/ngx_mail.h
--- a/src/mail/ngx_mail.h
+++ b/src/mail/ngx_mail.h
@@ -408,6 +408,7 @@ char *ngx_mail_capabilities(ngx_conf_t *
/* STUB */
void ngx_mail_proxy_init(ngx_mail_session_t *s, ngx_addr_t *peer);
void ngx_mail_auth_http_init(ngx_mail_session_t *s);
+ngx_int_t ngx_mail_realip_handler(ngx_mail_session_t *s);
/**/


diff --git a/src/mail/ngx_mail_handler.c b/src/mail/ngx_mail_handler.c
--- a/src/mail/ngx_mail_handler.c
+++ b/src/mail/ngx_mail_handler.c
@@ -260,6 +260,11 @@ ngx_mail_proxy_protocol_handler(ngx_even
return;
}

+ if (ngx_mail_realip_handler(s) != NGX_OK) {
+ ngx_mail_close_connection(c);
+ return;
+ }
+
ngx_mail_init_session_handler(rev);
}

diff --git a/src/stream/ngx_stream_realip_module.c b/src/mail/ngx_mail_realip_module.c
copy from src/stream/ngx_stream_realip_module.c
copy to src/mail/ngx_mail_realip_module.c
--- a/src/stream/ngx_stream_realip_module.c
+++ b/src/mail/ngx_mail_realip_module.c
@@ -7,45 +7,29 @@

#include <ngx_config.h>
#include <ngx_core.h>
-#include <ngx_stream.h>
+#include <ngx_mail.h>


typedef struct {
ngx_array_t *from; /* array of ngx_cidr_t */
-} ngx_stream_realip_srv_conf_t;
-
-
-typedef struct {
- struct sockaddr *sockaddr;
- socklen_t socklen;
- ngx_str_t addr_text;
-} ngx_stream_realip_ctx_t;
+} ngx_mail_realip_srv_conf_t;


-static ngx_int_t ngx_stream_realip_handler(ngx_stream_session_t *s);
-static ngx_int_t ngx_stream_realip_set_addr(ngx_stream_session_t *s,
+static ngx_int_t ngx_mail_realip_set_addr(ngx_mail_session_t *s,
ngx_addr_t *addr);
-static char *ngx_stream_realip_from(ngx_conf_t *cf, ngx_command_t *cmd,
+static char *ngx_mail_realip_from(ngx_conf_t *cf, ngx_command_t *cmd,
void *conf);
-static void *ngx_stream_realip_create_srv_conf(ngx_conf_t *cf);
-static char *ngx_stream_realip_merge_srv_conf(ngx_conf_t *cf, void *parent,
+static void *ngx_mail_realip_create_srv_conf(ngx_conf_t *cf);
+static char *ngx_mail_realip_merge_srv_conf(ngx_conf_t *cf, void *parent,
void *child);
-static ngx_int_t ngx_stream_realip_add_variables(ngx_conf_t *cf);
-static ngx_int_t ngx_stream_realip_init(ngx_conf_t *cf);


-static ngx_int_t ngx_stream_realip_remote_addr_variable(ngx_stream_session_t *s,
- ngx_stream_variable_value_t *v, uintptr_t data);
-static ngx_int_t ngx_stream_realip_remote_port_variable(ngx_stream_session_t *s,
- ngx_stream_variable_value_t *v, uintptr_t data);
-
-
-static ngx_command_t ngx_stream_realip_commands[] = {
+static ngx_command_t ngx_mail_realip_commands[] = {

{ ngx_string("set_real_ip_from"),
- NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE1,
- ngx_stream_realip_from,
- NGX_STREAM_SRV_CONF_OFFSET,
+ NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1,
+ ngx_mail_realip_from,
+ NGX_MAIL_SRV_CONF_OFFSET,
0,
NULL },

@@ -53,23 +37,22 @@ static ngx_command_t ngx_stream_realip_
};


-static ngx_stream_module_t ngx_stream_realip_module_ctx = {
- ngx_stream_realip_add_variables, /* preconfiguration */
- ngx_stream_realip_init, /* postconfiguration */
+static ngx_mail_module_t ngx_mail_realip_module_ctx = {
+ NULL, /* protocol */

NULL, /* create main configuration */
NULL, /* init main configuration */

- ngx_stream_realip_create_srv_conf, /* create server configuration */
- ngx_stream_realip_merge_srv_conf /* merge server configuration */
+ ngx_mail_realip_create_srv_conf, /* create server configuration */
+ ngx_mail_realip_merge_srv_conf /* merge server configuration */
};


-ngx_module_t ngx_stream_realip_module = {
+ngx_module_t ngx_mail_realip_module = {
NGX_MODULE_V1,
- &ngx_stream_realip_module_ctx, /* module context */
- ngx_stream_realip_commands, /* module directives */
- NGX_STREAM_MODULE, /* module type */
+ &ngx_mail_realip_module_ctx, /* module context */
+ ngx_mail_realip_commands, /* module directives */
+ NGX_MAIL_MODULE, /* module type */
NULL, /* init master */
NULL, /* init module */
NULL, /* init process */
@@ -81,70 +64,52 @@ ngx_module_t ngx_stream_realip_module =
};


-static ngx_stream_variable_t ngx_stream_realip_vars[] = {
-
- { ngx_string("realip_remote_addr"), NULL,
- ngx_stream_realip_remote_addr_variable, 0, 0, 0 },
-
- { ngx_string("realip_remote_port"), NULL,
- ngx_stream_realip_remote_port_variable, 0, 0, 0 },
+ngx_int_t
+ngx_mail_realip_handler(ngx_mail_session_t *s)
+{
+ ngx_addr_t addr;
+ ngx_connection_t *c;
+ ngx_mail_realip_srv_conf_t *rscf;

- ngx_stream_null_variable
-};
-
-
-static ngx_int_t
-ngx_stream_realip_handler(ngx_stream_session_t *s)
-{
- ngx_addr_t addr;
- ngx_connection_t *c;
- ngx_stream_realip_srv_conf_t *rscf;
-
- rscf = ngx_stream_get_module_srv_conf(s, ngx_stream_realip_module);
+ rscf = ngx_mail_get_module_srv_conf(s, ngx_mail_realip_module);

if (rscf->from == NULL) {
- return NGX_DECLINED;
+ return NGX_OK;
}

c = s->connection;

if (c->proxy_protocol == NULL) {
- return NGX_DECLINED;
+ return NGX_OK;
}

if (ngx_cidr_match(c->sockaddr, rscf->from) != NGX_OK) {
- return NGX_DECLINED;
+ return NGX_OK;
}

if (ngx_parse_addr(c->pool, &addr, c->proxy_protocol->src_addr.data,
c->proxy_protocol->src_addr.len)
!= NGX_OK)
{
- return NGX_DECLINED;
+ return NGX_OK;
}

ngx_inet_set_port(addr.sockaddr, c->proxy_protocol->src_port);

- return ngx_stream_realip_set_addr(s, &addr);
+ return ngx_mail_realip_set_addr(s, &addr);
}


static ngx_int_t
-ngx_stream_realip_set_addr(ngx_stream_session_t *s, ngx_addr_t *addr)
+ngx_mail_realip_set_addr(ngx_mail_session_t *s, ngx_addr_t *addr)
{
- size_t len;
- u_char *p;
- u_char text[NGX_SOCKADDR_STRLEN];
- ngx_connection_t *c;
- ngx_stream_realip_ctx_t *ctx;
+ size_t len;
+ u_char *p;
+ u_char text[NGX_SOCKADDR_STRLEN];
+ ngx_connection_t *c;

c = s->connection;

- ctx = ngx_palloc(c->pool, sizeof(ngx_stream_realip_ctx_t));
- if (ctx == NULL) {
- return NGX_ERROR;
- }
-
len = ngx_sock_ntop(addr->sockaddr, addr->socklen, text,
NGX_SOCKADDR_STRLEN, 0);
if (len == 0) {
@@ -158,25 +123,19 @@ ngx_stream_realip_set_addr(ngx_stream_se

ngx_memcpy(p, text, len);

- ngx_stream_set_ctx(s, ctx, ngx_stream_realip_module);
-
- ctx->sockaddr = c->sockaddr;
- ctx->socklen = c->socklen;
- ctx->addr_text = c->addr_text;
-
c->sockaddr = addr->sockaddr;
c->socklen = addr->socklen;
c->addr_text.len = len;
c->addr_text.data = p;

- return NGX_DECLINED;
+ return NGX_OK;
}


static char *
-ngx_stream_realip_from(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
+ngx_mail_realip_from(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
{
- ngx_stream_realip_srv_conf_t *rscf = conf;
+ ngx_mail_realip_srv_conf_t *rscf = conf;

ngx_int_t rc;
ngx_str_t *value;
@@ -277,11 +236,11 @@ ngx_stream_realip_from(ngx_conf_t *cf, n


static void *
-ngx_stream_realip_create_srv_conf(ngx_conf_t *cf)
+ngx_mail_realip_create_srv_conf(ngx_conf_t *cf)
{
- ngx_stream_realip_srv_conf_t *conf;
+ ngx_mail_realip_srv_conf_t *conf;

- conf = ngx_pcalloc(cf->pool, sizeof(ngx_stream_realip_srv_conf_t));
+ conf = ngx_pcalloc(cf->pool, sizeof(ngx_mail_realip_srv_conf_t));
if (conf == NULL) {
return NULL;
}
@@ -297,10 +256,10 @@ ngx_stream_realip_create_srv_conf(ngx_co


static char *
-ngx_stream_realip_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child)
+ngx_mail_realip_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child)
{
- ngx_stream_realip_srv_conf_t *prev = parent;
- ngx_stream_realip_srv_conf_t *conf = child;
+ ngx_mail_realip_srv_conf_t *prev = parent;
+ ngx_mail_realip_srv_conf_t *conf = child;

if (conf->from == NULL) {
conf->from = prev->from;
@@ -308,94 +267,3 @@ ngx_stream_realip_merge_srv_conf(ngx_con

return NGX_CONF_OK;
}
-
-
-static ngx_int_t
-ngx_stream_realip_add_variables(ngx_conf_t *cf)
-{
- ngx_stream_variable_t *var, *v;
-
- for (v = ngx_stream_realip_vars; v->name.len; v++) {
- var = ngx_stream_add_variable(cf, &v->name, v->flags);
- if (var == NULL) {
- return NGX_ERROR;
- }
-
- var->get_handler = v->get_handler;
- var->data = v->data;
- }
-
- return NGX_OK;
-}
-
-
-static ngx_int_t
-ngx_stream_realip_init(ngx_conf_t *cf)
-{
- ngx_stream_handler_pt *h;
- ngx_stream_core_main_conf_t *cmcf;
-
- cmcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_core_module);
-
- h = ngx_array_push(&cmcf->phases[NGX_STREAM_POST_ACCEPT_PHASE].handlers);
- if (h == NULL) {
- return NGX_ERROR;
- }
-
- *h = ngx_stream_realip_handler;
-
- return NGX_OK;
-}
-
-
-static ngx_int_t
-ngx_stream_realip_remote_addr_variable(ngx_stream_session_t *s,
- ngx_stream_variable_value_t *v, uintptr_t data)
-{
- ngx_str_t *addr_text;
- ngx_stream_realip_ctx_t *ctx;
-
- ctx = ngx_stream_get_module_ctx(s, ngx_stream_realip_module);
-
- addr_text = ctx ? &ctx->addr_text : &s->connection->addr_text;
-
- v->len = addr_text->len;
- v->valid = 1;
- v->no_cacheable = 0;
- v->not_found = 0;
- v->data = addr_text->data;
-
- return NGX_OK;
-}
-
-
-static ngx_int_t
-ngx_stream_realip_remote_port_variable(ngx_stream_session_t *s,
- ngx_stream_variable_value_t *v, uintptr_t data)
-{
- ngx_uint_t port;
- struct sockaddr *sa;
- ngx_stream_realip_ctx_t *ctx;
-
- ctx = ngx_stream_get_module_ctx(s, ngx_stream_realip_module);
-
- sa = ctx ? ctx->sockaddr : s->connection->sockaddr;
-
- v->len = 0;
- v->valid = 1;
- v->no_cacheable = 0;
- v->not_found = 0;
-
- v->data = ngx_pnalloc(s->connection->pool, sizeof("65535") - 1);
- if (v->data == NULL) {
- return NGX_ERROR;
- }
-
- port = ngx_inet_get_port(sa);
-
- if (port > 0 && port < 65536) {
- v->len = ngx_sprintf(v->data, "%ui", port) - v->data;
- }
-
- return NGX_OK;
-}
# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1613827679 -10800
# Sat Feb 20 16:27:59 2021 +0300
# Node ID 30a66f88bb5d30bbfb445b57bd54c9ca35a345ec
# Parent 068d1bf4f2b3faaf84ec937c091274c940b5e1ca
Mail: sending of the PROXY protocol to backends.

Activated with the "proxy_protocol" directive. Can be combined with
"listen ... proxy_protocol;" and "set_real_ip_from ...;" to pass
client address provided to nginx in the PROXY protocol header.

diff --git a/src/mail/ngx_mail.h b/src/mail/ngx_mail.h
--- a/src/mail/ngx_mail.h
+++ b/src/mail/ngx_mail.h
@@ -178,6 +178,7 @@ typedef enum {
typedef struct {
ngx_peer_connection_t upstream;
ngx_buf_t *buffer;
+ ngx_uint_t proxy_protocol; /* unsigned proxy_protocol:1; */
} ngx_mail_proxy_ctx_t;


diff --git a/src/mail/ngx_mail_proxy_module.c b/src/mail/ngx_mail_proxy_module.c
--- a/src/mail/ngx_mail_proxy_module.c
+++ b/src/mail/ngx_mail_proxy_module.c
@@ -17,6 +17,7 @@ typedef struct {
ngx_flag_t pass_error_message;
ngx_flag_t xclient;
ngx_flag_t smtp_auth;
+ ngx_flag_t proxy_protocol;
size_t buffer_size;
ngx_msec_t timeout;
} ngx_mail_proxy_conf_t;
@@ -26,7 +27,8 @@ static void ngx_mail_proxy_block_read(ng
static void ngx_mail_proxy_pop3_handler(ngx_event_t *rev);
static void ngx_mail_proxy_imap_handler(ngx_event_t *rev);
static void ngx_mail_proxy_smtp_handler(ngx_event_t *rev);
-static void ngx_mail_proxy_dummy_handler(ngx_event_t *ev);
+static void ngx_mail_proxy_write_handler(ngx_event_t *wev);
+static ngx_int_t ngx_mail_proxy_send_proxy_protocol(ngx_mail_session_t *s);
static ngx_int_t ngx_mail_proxy_read_response(ngx_mail_session_t *s,
ngx_uint_t state);
static void ngx_mail_proxy_handler(ngx_event_t *ev);
@@ -82,6 +84,13 @@ static ngx_command_t ngx_mail_proxy_com
offsetof(ngx_mail_proxy_conf_t, smtp_auth),
NULL },

+ { ngx_string("proxy_protocol"),
+ NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_FLAG,
+ ngx_conf_set_flag_slot,
+ NGX_MAIL_SRV_CONF_OFFSET,
+ offsetof(ngx_mail_proxy_conf_t, proxy_protocol),
+ NULL },
+
ngx_null_command
};

@@ -156,7 +165,7 @@ ngx_mail_proxy_init(ngx_mail_session_t *
p->upstream.connection->pool = s->connection->pool;

s->connection->read->handler = ngx_mail_proxy_block_read;
- p->upstream.connection->write->handler = ngx_mail_proxy_dummy_handler;
+ p->upstream.connection->write->handler = ngx_mail_proxy_write_handler;

pcf = ngx_mail_get_module_srv_conf(s, ngx_mail_proxy_module);

@@ -167,6 +176,8 @@ ngx_mail_proxy_init(ngx_mail_session_t *
return;
}

+ s->proxy->proxy_protocol = pcf->proxy_protocol;
+
s->out.len = 0;

switch (s->protocol) {
@@ -186,6 +197,12 @@ ngx_mail_proxy_init(ngx_mail_session_t *
s->mail_state = ngx_smtp_start;
break;
}
+
+ if (rc == NGX_AGAIN) {
+ return;
+ }
+
+ ngx_mail_proxy_write_handler(p->upstream.connection->write);
}


@@ -230,6 +247,17 @@ ngx_mail_proxy_pop3_handler(ngx_event_t
return;
}

+ if (s->proxy->proxy_protocol) {
+ ngx_log_debug0(NGX_LOG_DEBUG_MAIL, c->log, 0, "mail proxy pop3 busy");
+
+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_proxy_internal_server_error(s);
+ return;
+ }
+
+ return;
+ }
+
rc = ngx_mail_proxy_read_response(s, 0);

if (rc == NGX_AGAIN) {
@@ -353,6 +381,17 @@ ngx_mail_proxy_imap_handler(ngx_event_t
return;
}

+ if (s->proxy->proxy_protocol) {
+ ngx_log_debug0(NGX_LOG_DEBUG_MAIL, c->log, 0, "mail proxy imap busy");
+
+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_proxy_internal_server_error(s);
+ return;
+ }
+
+ return;
+ }
+
rc = ngx_mail_proxy_read_response(s, s->mail_state);

if (rc == NGX_AGAIN) {
@@ -499,6 +538,17 @@ ngx_mail_proxy_smtp_handler(ngx_event_t
return;
}

+ if (s->proxy->proxy_protocol) {
+ ngx_log_debug0(NGX_LOG_DEBUG_MAIL, c->log, 0, "mail proxy smtp busy");
+
+ if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+ ngx_mail_proxy_internal_server_error(s);
+ return;
+ }
+
+ return;
+ }
+
rc = ngx_mail_proxy_read_response(s, s->mail_state);

if (rc == NGX_AGAIN) {
@@ -799,19 +849,92 @@ ngx_mail_proxy_smtp_handler(ngx_event_t


static void
-ngx_mail_proxy_dummy_handler(ngx_event_t *wev)
+ngx_mail_proxy_write_handler(ngx_event_t *wev)
{
ngx_connection_t *c;
ngx_mail_session_t *s;

- ngx_log_debug0(NGX_LOG_DEBUG_MAIL, wev->log, 0, "mail proxy dummy handler");
+ ngx_log_debug0(NGX_LOG_DEBUG_MAIL, wev->log, 0, "mail proxy write handler");
+
+ c = wev->data;
+ s = c->data;
+
+ if (s->proxy->proxy_protocol) {
+ if (ngx_mail_proxy_send_proxy_protocol(s) != NGX_OK) {
+ return;
+ }
+
+ s->proxy->proxy_protocol = 0;
+ }

if (ngx_handle_write_event(wev, 0) != NGX_OK) {
- c = wev->data;
- s = c->data;
+ ngx_mail_proxy_internal_server_error(s);
+ }
+
+ if (c->read->ready) {
+ ngx_post_event(c->read, &ngx_posted_events);
+ }
+}
+
+
+static ngx_int_t
+ngx_mail_proxy_send_proxy_protocol(ngx_mail_session_t *s)
+{
+ u_char *p;
+ ssize_t n, size;
+ ngx_connection_t *c;
+ u_char buf[NGX_PROXY_PROTOCOL_MAX_HEADER];
+
+ s->connection->log->action = "sending PROXY protocol header to upstream";
+
+ ngx_log_debug0(NGX_LOG_DEBUG_MAIL, s->connection->log, 0,
+ "mail proxy send PROXY protocol header");
+
+ p = ngx_proxy_protocol_write(s->connection, buf,
+ buf + NGX_PROXY_PROTOCOL_MAX_HEADER);
+ if (p == NULL) {
+ ngx_mail_proxy_internal_server_error(s);
+ return NGX_ERROR;
+ }
+
+ c = s->proxy->upstream.connection;
+
+ size = p - buf;

- ngx_mail_proxy_close_session(s);
+ n = c->send(c, buf, size);
+
+ if (n == NGX_AGAIN) {
+ if (ngx_handle_write_event(c->write, 0) != NGX_OK) {
+ ngx_mail_proxy_internal_server_error(s);
+ return NGX_ERROR;
+ }
+
+ return NGX_AGAIN;
+ }
+
+ if (n == NGX_ERROR) {
+ ngx_mail_proxy_internal_server_error(s);
+ return NGX_ERROR;
}
+
+ if (n != size) {
+
+ /*
+ * PROXY protocol specification:
+ * The sender must always ensure that the header
+ * is sent at once, so that the transport layer
+ * maintains atomicity along the path to the receiver.
+ */
+
+ ngx_log_error(NGX_LOG_ERR, s->connection->log, 0,
+ "could not send PROXY protocol header at once");
+
+ ngx_mail_proxy_internal_server_error(s);
+
+ return NGX_ERROR;
+ }
+
+ return NGX_OK;
}


@@ -1212,6 +1335,7 @@ ngx_mail_proxy_create_conf(ngx_conf_t *c
pcf->pass_error_message = NGX_CONF_UNSET;
pcf->xclient = NGX_CONF_UNSET;
pcf->smtp_auth = NGX_CONF_UNSET;
+ pcf->proxy_protocol = NGX_CONF_UNSET;
pcf->buffer_size = NGX_CONF_UNSET_SIZE;
pcf->timeout = NGX_CONF_UNSET_MSEC;

@@ -1229,6 +1353,7 @@ ngx_mail_proxy_merge_conf(ngx_conf_t *cf
ngx_conf_merge_value(conf->pass_error_message, prev->pass_error_message, 0);
ngx_conf_merge_value(conf->xclient, prev->xclient, 1);
ngx_conf_merge_value(conf->smtp_auth, prev->smtp_auth, 0);
+ ngx_conf_merge_value(conf->proxy_protocol, prev->proxy_protocol, 0);
ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size,
(size_t) ngx_pagesize);
ngx_conf_merge_msec_value(conf->timeout, prev->timeout, 24 * 60 * 60000);

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel