Welcome! Log In Create A New Profile

Advanced

Re: nginx KTLS and HTTP/2 performance degradation

Vadim Fedorenko
December 02, 2021 07:34PM
Hello!
I would say that current implementation of Kernel TLS in OpenSSL will give
huge overhead because of additional syscall for every frame and it's header,
it doesn't matter if it's sendfile or not. Without sendfile it's actually
5% worse
in my tests. That's why it's better to disable Kernel TLS for HTTP/2
requests
in Nginx + OpenSSL.
The only solution for this would be implementation of sendmsg()/sendmmsg()
in OpenSSL and support for such implementation in Nginx together with mmap()
for files. This solution would have the same performance as sendfile() from
kernel perspective.

чт, 2 дек. 2021 г. в 13:08, Maxim Dounin <mdounin@mdounin.ru>:

> Hello!
>
> On Thu, Dec 02, 2021 at 02:05:52PM +0200, Lyuben Stoev wrote:
>
> > Hello,
> > I have tested the nginx with the patch
> > https://hg.nginx.org/nginx/rev/65946a191197 (SSL: SSL_sendfile()
> support
> > with kernel TLS.) following the nginx blog article
> > https://www.nginx.com/blog/improving-nginx-performance-with-kernel-tls/
> > And it sort of works, but I have bad performance when making HTTP/2
> > requests. If I made a HTTP/1.1 request there is 30-35% increase in
> > performance as the Nginx blog article stated, but when I changed the
> > request to use HTTP/2 the request was 40% slower than an ordinary nginx
> > without KTLS enabled. Does anyone have such perfomance degradation with
> > nginx KTLS and HTTP/2? I am using generic setup - Ubuntu 20.04.3 LTS and
> > kernels 5.8.0-63-generic (the same results are with 5.4.0-91-generic).
> > The nginx vritual host is the same as in the Nginx blog article with
> > exception of adding http2 to the listen! OpenSSL 3.0.0 and nginx 1.21.4
> > are used.
> > The KTLS seems to work, because the strace and debug logs show it. Just
> > the sstrange thing is when using HTTP2, the sendfile syscalls look:
> > write(39, "\0 \0\0\0\0\0\0\1", 9) = 9
> > sendfile(39, 131, [1418218] => [1426410], 8192) = 8192
> > write(39, "\0 \0\0\0\0\0\0\1", 9) = 9
> > sendfile(39, 131, [1426410] => [1434602], 8192) = 8192
> > write(39, "\0 \0\0\0\0\0\0\1", 9) = 9
> > sendfile(39, 131, [1434602] => [1442794], 8192) = 8192
> > write(39, "\0 \0\0\0\0\0\0\1", 9) = 9
> > sendfile(39, 131, [1442794] => [1450986], 8192) = 8192
> > write(39, "\0 \0\0\0\0\0\0\1", 9) = 9
> > sendfile(39, 131, [1450986] => [1459178], 8192) = 8192
> > write(39, "\0 \0\0\0\0\0\0\1", 9) = 9
> > sendfile(39, 131, [1459178] => [1467370], 8192) = 8192
> >
> > It is always 8K and there are thousands of sendfile syscalls....
>
> That's expected, because of HTTP/2 framing. Unfortunately, HTTP/2
> isn't designed to work with sendfile(), and sending large files
> over HTTP/2 require a lot of sendfile() syscalls. In general, for
> HTTP/2 it is better to keep sendfile() disabled.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx-devel mailing list
> nginx-devel@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
_______________________________________________
nginx-devel mailing list
nginx-devel@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel
Subject Author Views Posted

nginx KTLS and HTTP/2 performance degradation

Lyuben Stoev 212 December 02, 2021 07:06AM

Re: nginx KTLS and HTTP/2 performance degradation

Maxim Dounin 104 December 02, 2021 08:08AM

Re: nginx KTLS and HTTP/2 performance degradation

Vadim Fedorenko 103 December 02, 2021 07:34PM

Re: nginx KTLS and HTTP/2 performance degradation

Maxim Dounin 91 December 02, 2021 11:12PM



Sorry, you do not have permission to post/reply in this forum.

Online Users

Guests: 51
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready