Welcome! Log In Create A New Profile

Advanced

[PATCH] HTTP/2: fixed sendfile() aio handling

Maxim Dounin
November 10, 2021 10:12PM
# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1636599377 -10800
# Thu Nov 11 05:56:17 2021 +0300
# Node ID 76e072a6947a221868705c13973de15319c0d921
# Parent 82b750b20c5205d685e59031247fe898f011394e
HTTP/2: fixed sendfile() aio handling.

With sendfile() in threads ("aio threads; sendfile on;"), client connection
can block on writing, waiting for sendfile() to complete. In HTTP/2 this
might result in the request hang, since an attempt to continue processig
in thread event handler will call request's write event handler, which
is usually stopped by ngx_http_v2_send_chain(): it does nothing if there
are no additional data and stream->queued is set. Further, HTTP/2 resets
stream's c->write->ready to 0 if writing blocks, so just fixing
ngx_http_v2_send_chain() is not enough.

Can be reproduced with test suite on Linux with:

TEST_NGINX_GLOBALS_HTTP="aio threads; sendfile on;" prove h2*.t

The following tests currently fail: h2_keepalive.t, h2_priority.t,
h2_proxy_max_temp_file_size.t, h2.t, h2_trailers.t.

Similarly, sendfile() with AIO preloading on FreeBSD can block as well,
with similar results. This is, however, harder to reproduce, especially
on modern FreeBSD systems, since sendfile() usually do not return EBUSY.

Fix is to post a write event on HTTP/2 connection in the thread event
handler (and aio preload handler). This ensures that sendfile() will be
completed and stream processing will be resumed by HTTP/2 code.

diff --git a/src/http/ngx_http_copy_filter_module.c b/src/http/ngx_http_copy_filter_module.c
--- a/src/http/ngx_http_copy_filter_module.c
+++ b/src/http/ngx_http_copy_filter_module.c
@@ -250,6 +250,21 @@ ngx_http_copy_aio_sendfile_event_handler
r->aio = 0;
ev->complete = 0;

+#if (NGX_HTTP_V2)
+
+ if (r->stream) {
+ /*
+ * for HTTP/2, trigger a write event on the main connection
+ * to handle sendfile() preload
+ */
+
+ ngx_post_event(r->stream->connection->connection->write,
+ &ngx_posted_events);
+ return;
+ }
+
+#endif
+
r->connection->write->handler(r->connection->write);
}

@@ -323,6 +338,20 @@ ngx_http_copy_thread_event_handler(ngx_e
r->main->blocked--;
r->aio = 0;

+#if (NGX_HTTP_V2)
+
+ if (r->stream) {
+ /*
+ * for HTTP/2, trigger a write event on the main connection
+ * to handle sendfile() in threads
+ */
+
+ ngx_post_event(r->stream->connection->connection->write,
+ &ngx_posted_events);
+ }
+
+#endif
+
if (r->done) {
/*
* trigger connection event handler if the subrequest was
diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -3905,6 +3905,20 @@ ngx_http_upstream_thread_event_handler(n
r->main->blocked--;
r->aio = 0;

+#if (NGX_HTTP_V2)
+
+ if (r->stream) {
+ /*
+ * for HTTP/2, trigger a write event on the main connection
+ * to handle sendfile() in threads
+ */
+
+ ngx_post_event(r->stream->connection->connection->write,
+ &ngx_posted_events);
+ }
+
+#endif
+
if (r->done) {
/*
* trigger connection event handler if the subrequest was

_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel
Subject Author Views Posted

[PATCH] HTTP/2: fixed sendfile() aio handling

Maxim Dounin 111 November 10, 2021 10:12PM

Re: [PATCH] HTTP/2: fixed sendfile() aio handling

Sergey Kandaurov 13 November 16, 2021 07:02AM

Re: [PATCH] HTTP/2: fixed sendfile() aio handling

Maxim Dounin 12 November 16, 2021 09:42AM

Re: [PATCH] HTTP/2: fixed sendfile() aio handling

Sergey Kandaurov 23 November 18, 2021 11:48AM

Re: [PATCH] HTTP/2: fixed sendfile() aio handling

Maxim Dounin 9 November 22, 2021 10:14PM

Re: [PATCH] HTTP/2: fixed sendfile() aio handling

Sergey Kandaurov 15 November 23, 2021 02:46AM

Re: [PATCH] HTTP/2: fixed sendfile() aio handling

Maxim Dounin 7 November 23, 2021 10:52PM

Re: [PATCH] HTTP/2: fixed sendfile() aio handling

Sergey Kandaurov 8 November 24, 2021 10:52AM

Re: [PATCH] HTTP/2: fixed sendfile() aio handling

Maxim Dounin 10 November 24, 2021 02:08PM

Re: [PATCH] HTTP/2: fixed sendfile() aio handling

Sergey Kandaurov 6 November 25, 2021 08:16AM

Re: [PATCH] HTTP/2: fixed sendfile() aio handling

Maxim Dounin 4 November 25, 2021 08:56AM

Re: [PATCH] HTTP/2: fixed sendfile() aio handling

Sergey Kandaurov 5 November 25, 2021 12:44PM

Re: [PATCH] HTTP/2: fixed sendfile() aio handling

Maxim Dounin 9 November 25, 2021 02:08PM



Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 67
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready