Hello!

On Mon, Oct 02, 2017 at 11:51:33PM -0400, rnmx18 wrote:

> Hi,
>
> We are trying to use NGINX for caching service in low bandwidth, high
> latency mobile networks. The service is to stream 10-sec video segments of
> different types ranging from 2MB to 50MB.
>
> NGINX proxy_buffering configuration is as follows:
>
> proxy_buffering on;
> proxy_buffer_size 4k;
> proxy_buffers 64 4k;
> proxy_busy_buffers_size 128k;
> proxy_temp_file_write_size 64k;
>
> The slow clients results in NGINX buffering of the response and writing data
> to disk as part of temporary buffering. The disk IO is causing higher TTFB
> and higher load time for video download.
>
> We have tried to configure the proxy_max_temp_file_size to 0 to disable the
> buffering. This change results in interrupts not being balanced as shown
> below in the top command output - core0 and core15 is using 100%.
>
> top - 13:53:04 up 6 days, 2:31, 5 users, load average: 6.42, 4.35, 3.84
> Tasks: 370 total, 6 running, 364 sleeping, 0 stopped, 0 zombie
> %Cpu0 : 0.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi,100.0 si, 0.0
> st

[...]

> Couple of queries:
> a) Why do we get unbalanced interrupts when buffering is disabled?

Try looking on what runs on these CPUs.

Given that it's "100.0 si", I would suggest that it is your NIC
interrupt threads trying to cope with load.

> b) How to configure NGINX to throttle the upstream read and avoid temp
> buffering?

With proxy_max_temp_file_size set to 0 nginx won't buffer anything
to disk, and will read from upstream up to available
proxy_buffers. As long as configured buffers are full, nginx will
stop reading from the upstream server till at least one buffer is
free.

That is, nginx will read from the upstream at a rate
controlled by bandwidth of connected clients. You can use normal
client limiting mechanisms such as limit_rate and limit_conn if
the rate observed is too high.

Additionly, the proxy_limit_rate directive can be used to control
rate limiting of connections to upstream servers, see
http://nginx.org/r/proxy_limit_rate. Though this is primary
useful when you don't disable disk buffering but rather have to
keep it enabled, for example, when using cache.

--
Maxim Dounin
http://nginx.org/
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx