Welcome! Log In Create A New Profile

Advanced

Proxy buffering and slow clients related issues

October 02, 2017 11:51PM
Hi,

We are trying to use NGINX for caching service in low bandwidth, high latency mobile networks. The service is to stream 10-sec video segments of different types ranging from 2MB to 50MB.

NGINX proxy_buffering configuration is as follows:

proxy_buffering on;
proxy_buffer_size 4k;
proxy_buffers 64 4k;
proxy_busy_buffers_size 128k;
proxy_temp_file_write_size 64k;

The slow clients results in NGINX buffering of the response and writing data to disk as part of temporary buffering. The disk IO is causing higher TTFB and higher load time for video download.

We have tried to configure the proxy_max_temp_file_size to 0 to disable the buffering. This change results in interrupts not being balanced as shown below in the top command output - core0 and core15 is using 100%.

top - 13:53:04 up 6 days, 2:31, 5 users, load average: 6.42, 4.35, 3.84
Tasks: 370 total, 6 running, 364 sleeping, 0 stopped, 0 zombie
%Cpu0 : 0.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi,100.0 si, 0.0 st
%Cpu1 : 7.9 us, 10.3 sy, 0.0 ni, 79.7 id, 0.0 wa, 0.0 hi, 2.1 si, 0.0 st
%Cpu2 : 18.7 us, 14.8 sy, 0.0 ni, 49.0 id, 0.0 wa, 0.0 hi, 17.5 si, 0.0 st
%Cpu3 : 18.7 us, 11.5 sy, 0.0 ni, 52.7 id, 0.0 wa, 0.0 hi, 17.2 si, 0.0 st
%Cpu4 : 20.1 us, 12.4 sy, 0.0 ni, 47.1 id, 0.0 wa, 0.0 hi, 20.5 si, 0.0 st
%Cpu5 : 22.2 us, 12.3 sy, 0.0 ni, 45.6 id, 0.0 wa, 0.0 hi, 19.9 si, 0.0 st
%Cpu6 : 15.1 us, 13.0 sy, 0.0 ni, 68.5 id, 0.0 wa, 0.0 hi, 3.4 si, 0.0 st
%Cpu7 : 9.6 us, 10.7 sy, 0.0 ni, 77.7 id, 0.3 wa, 0.0 hi, 1.7 si, 0.0 st
%Cpu8 : 9.9 us, 12.0 sy, 0.0 ni, 75.3 id, 0.0 wa, 0.0 hi, 2.7 si, 0.0 st
%Cpu9 : 11.7 us, 11.7 sy, 0.0 ni, 73.5 id, 0.0 wa, 0.0 hi, 3.1 si, 0.0 st
%Cpu10 : 10.0 us, 11.0 sy, 0.0 ni, 74.9 id, 0.0 wa, 0.0 hi, 4.1 si, 0.0 st
%Cpu11 : 5.5 us, 12.3 sy, 0.0 ni, 80.9 id, 0.0 wa, 0.0 hi, 1.4 si, 0.0 st
%Cpu12 : 1.7 us, 3.7 sy, 0.0 ni, 93.6 id, 0.0 wa, 0.0 hi, 1.0 si, 0.0 st
%Cpu13 : 2.0 us, 3.7 sy, 0.0 ni, 93.6 id, 0.0 wa, 0.0 hi, 0.7 si, 0.0 st
%Cpu14 : 1.4 us, 2.4 sy, 0.0 ni, 95.3 id, 0.7 wa, 0.0 hi, 0.3 si, 0.0 st
%Cpu15 : 0.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi,100.0 si, 0.0 st
%Cpu16 : 18.0 us, 10.9 sy, 0.0 ni, 55.5 id, 0.0 wa, 0.0 hi, 15.6 si, 0.0 st
%Cpu17 : 18.8 us, 11.9 sy, 0.0 ni, 48.8 id, 0.0 wa, 0.0 hi, 20.4 si, 0.0 st
%Cpu18 : 6.4 us, 6.8 sy, 0.0 ni, 85.1 id, 0.0 wa, 0.0 hi, 1.7 si, 0.0 st
%Cpu19 : 3.8 us, 6.5 sy, 0.0 ni, 86.3 id, 0.0 wa, 0.0 hi, 3.4 si, 0.0 st
%Cpu20 : 1.7 us, 5.1 sy, 0.0 ni, 92.2 id, 0.3 wa, 0.0 hi, 0.7 si, 0.0 st
%Cpu21 : 2.4 us, 3.0 sy, 0.0 ni, 93.9 id, 0.0 wa, 0.0 hi, 0.7 si, 0.0 st
%Cpu22 : 1.0 us, 4.0 sy, 0.0 ni, 93.9 id, 0.0 wa, 0.0 hi, 1.0 si, 0.0 st
%Cpu23 : 1.3 us, 3.7 sy, 0.0 ni, 94.6 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st
KiB Mem : 11523494+total, 27725092 free, 10556008 used, 76953848 buff/cache
KiB Swap: 16777212 total, 16675408 free, 101804 used. 83445760 avail Mem

Couple of queries:
a) Why do we get unbalanced interrupts when buffering is disabled?
b) How to configure NGINX to throttle the upstream read and avoid temp buffering?

Thanks
Rajesh
Subject Author Posted

Proxy buffering and slow clients related issues

rnmx18 October 02, 2017 11:51PM

Re: Proxy buffering and slow clients related issues

Maxim Dounin October 03, 2017 09:12AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 92
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready