Welcome! Log In Create A New Profile

Advanced

Inquiry Regarding Handling of QUIC Connections During Nginx Reload

上勾拳
February 25, 2024 02:54AM
Hello,

I hope this email finds you well. I am writing to inquire about the status
of an issue I have encountered regarding the handling of QUIC connections
when Nginx is reloaded.

Recently, I delved into the Nginx eBPF implementation, specifically
examining how QUIC connection packets are handled, especially during Nginx
reloads. My focus was on ensuring that existing QUIC connection packets are
correctly routed to the appropriate worker after a reload, and the Nginx
eBPF prog have done this job perfectly.

However, I observed that during a reload, new QUIC connections might be
directed to the old worker. The underlying problem stems from the fact that
new QUIC connections fail to match the eBPF reuseport socket map. The
kernel default logic then routes UDP packets based on the hash UDP 4-tuple
in the reuseport group socket array. Since the old worker's listen FDs
persist in the reuseport group socket array (reuse->socks), there is a
possibility that the old worker may still be tasked with handling new QUIC
connections.

Given that the old worker should not process new requests, it results in
the old worker not responding to the QUIC new connection. Consequently,
clients have to endure the handshake timeout and retry the connection,
potentially encountering the old worker again, leading to an ineffective
cycle.

In my investigation, I came across a thread in the nginx-devel mailing list
[https://www.mail-archive.com/nginx-devel@nginx.org/msg10627.html], where
it was mentioned that there would be some work addressing this issue.

Considering my limited experience with eBPF, I propose a potential
solution. The eBPF program could maintain another eBPF map containing only
the listen sockets of the new worker. When the eBPF program calls
`bpf_sk_select_reuseport` and receives `-ENOENT`, it could utilize this new
eBPF map with the hash UDP 4-tuple to route the new QUIC connection to the
new worker. This approach aims to circumvent the kernel logic routing the
packet to the shutting down worker since removing the old worker's listen
socket from the reuseport group socket array seems unfeasible. Not sure
about whether this solution is a good idea and I also wonder if there are
other solutions for this. I would appreciate any insights or updates you
could provide on this matter. Thank you for your time and consideration.

Best regards,
Zhenzhong
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
https://mailman.nginx.org/mailman/listinfo/nginx-devel
Subject Author Views Posted

Inquiry Regarding Handling of QUIC Connections During Nginx Reload

上勾拳 226 February 25, 2024 02:54AM

Re: Inquiry Regarding Handling of QUIC Connections During Nginx Reload

Roman Arutyunyan 97 February 26, 2024 06:50AM



Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 189
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 500 on July 15, 2024
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready