Welcome! Log In Create A New Profile

Advanced

Buffering issues with nginx

July 17, 2017 02:06AM
No matter what configs I try, nginx still keeps buffering my requests.

These are the configs that I apply in my test:
```
proxy_pass http://localhost:80;
proxy_request_buffering off;
proxy_buffer_size 4k;
proxy_buffers 8 4k;
proxy_no_cache 1;
proxy_set_header Host $host;
proxy_max_temp_file_size 0;```

in my case I run nodejs app directly on port 80 and I run nginx on 8080, then I run my tests against proxied and direct version.

My nodejs app needs to know exact amount of data that was sent to remote (to calculate speed of transfer and for billing purposes). Ideally, I'd like to know number of ACKed bytes of a TCP connection. Node itself cannot even provide that (as this wasn't even possible until recently on windows).

In my first test I serve 25MB test binary data. Without `proxy_max_temp_file_size 0` nginx would read all 25MB in 5ms and then would continue sending that data to the originator on its own without me (e.g. my node app) ever knowing if all 25MB were delivered or transfer was aborted after 1MB.
After I applied all these configs above this started to work better, and since I proxy in http1.0 mode I can use connection.close event on my node app as a rough approximation when connection was completed.

However, when I send for example 500KB of binary data, nginx still reads in all the data in couple of milliseconds and closes connection and there is no way for me to know on node side if that data was actually properly delivered.
For the test I download that data using wget and I use --limit-rate=10000 to limit download on receiver side to 10KB/s. After 5 seconds I ctrl+C and abort transfer after loading just 50KB, while my nodejs side actually thinks that everything went well and all 500KB were loaded.


1) how can I make nginx not to buffer more than 64KB of data?
2) can my node app know how much data nginx ended up delivering?
Subject Author Posted

Buffering issues with nginx

Dan34 July 17, 2017 02:06AM

Re: Buffering issues with nginx

Francis Daly July 17, 2017 05:16AM

Re: Buffering issues with nginx

Dan34 July 17, 2017 09:47PM

Re: Buffering issues with nginx

Francis Daly July 19, 2017 04:18PM

Re: Buffering issues with nginx

Dan34 July 21, 2017 07:02AM

Re: Buffering issues with nginx

Valentin V. Bartenev July 21, 2017 09:20AM

Re: Buffering issues with nginx

Dan34 July 21, 2017 01:45PM

Re: Buffering issues with nginx

Valentin V. Bartenev July 21, 2017 05:06PM

Re: Buffering issues with nginx

Dan34 July 22, 2017 02:00AM

Re: Buffering issues with nginx

Dan34 July 24, 2017 10:28AM

Re: Buffering issues with nginx

Dan34 July 24, 2017 12:24PM

Re: Buffering issues with nginx

Francis Daly July 24, 2017 03:06PM

Re: Buffering issues with nginx

Dan34 July 29, 2017 10:16PM

Re: Buffering issues with nginx

Dan34 July 29, 2017 10:41PM

Re: Buffering issues with nginx

Valentin V. Bartenev July 30, 2017 10:16AM

Re: Buffering issues with nginx

Dan34 July 30, 2017 02:03PM

Re: Buffering issues with nginx

Dan34 July 31, 2017 02:37AM

Re: Buffering issues with nginx

Igal @ Lucee.org July 19, 2017 04:22PM

Re: Buffering issues with nginx

Dan34 July 21, 2017 05:12AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 133
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready