Welcome! Log In Create A New Profile

Advanced

Tuning client request buffering in ngx_http_proxy_module

January 31, 2022 07:53PM
I had/have an issue where I am proxying from NGINX to a backend WebDAV server.
My upload speeds were slow and had long pauses in them including very long pauses at the end (5 minutes or more on uploads around 1GB).

Via packet captures I found that the NGINX server was not transmitting data synchronously to the backend WebDAV server, but was clearly doing buffering despite the fact I had set "proxy_buffering off".
Looking at the documentation led me to "proxy_request_buffering off" which seems to have solved my problem and seems to immediately send the entire WebDAV client request directly to the backend server in a synchronous manner.

I did however want to experiment with proxy_request_buffering not disabled and see if setting smaller buffers to the backend would perhaps not result in such log asynchronous delays.

With "proxy_request_buffering on" (default) I have also set the "client_body_temp_path /tmp/cache" and set "client_body_in_file_only on".
In this configuration I can see the client request get placed into the /tmp/cache directory, so I know my directive and paths are working.

However, if I set "client_body_in_file_only off" (default) no file gets created at all despite the fact that my client PUT request over WebDAV is undoubtedly larger than any buffer settings.

By doing a "df" on my NGINX box I can see that the space on my drive it being eaten up by the same amount equivalent to my upload (i.e. if I upload a 500 MB file I can see my free space decrease by 500 MB). I cannot however see anything in /tmp/cache and have no idea where these files are being placed.

I also don't know how/why NGINX is caching/buffering the entire file and what is controlling how it is sending this to the backend server. I have tried playing with the "client_body_buffer_size" but it does not appear to have any effect in how the data gets buffered or the size of the file.

Please help me understand how NGINX is working here in the background and how I can tune these settings.
Subject Author Posted

Tuning client request buffering in ngx_http_proxy_module

bengalih January 31, 2022 07:53PM

Re: Tuning client request buffering in ngx_http_proxy_module

Maxim Dounin February 01, 2022 09:46AM

Re: Tuning client request buffering in ngx_http_proxy_module

bengalih February 01, 2022 03:19PM

Re: Tuning client request buffering in ngx_http_proxy_module

Maxim Dounin February 01, 2022 06:34PM

Re: Tuning client request buffering in ngx_http_proxy_module

bengalih February 02, 2022 12:42PM

Re: Tuning client request buffering in ngx_http_proxy_module

Francis Daly February 03, 2022 07:58PM

Re: Tuning client request buffering in ngx_http_proxy_module

noloader February 03, 2022 11:06PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 287
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready