Welcome! Log In Create A New Profile

Advanced

Nginx proxy truncates large file downloads to 1.1G

Duane Mulder
December 05, 2010 08:26PM
Hello All:

So nginx so far has been running great. Today however I discovered that
when downloading a file larger than 1G (ie 1.5G) it will always fail
leaving the client with a file of 1.1G and an incomplete download. If I
point directly to the webserver the download completes properly.
Downloading over a slow or local or fast link does not change the
problem. Its the file size that appears to be causing the issue. Any
size of file download which is under 1G works and downloads with no issues.

I suspect its some issue with proxy buffering however I have not been
able to come up with the secret sauce to make it work. Below is the
last iteration of my config which still truncates large file downloads.
The boxes that nginx is running on are a dedicated dual core dual Xeon
3Ghz with 8G of ram.

Also there does not seem to be anything in the error logs when this occurs.

Regards,
Duane

The files
user nginx;
worker_processes 4;
events {
worker_connections 1024;
use epoll;
multi_accept on;
}

http { #http
include mime.types;
default_type application/octet-stream;
gzip on;
gzip_http_version 1.0;
gzip_comp_level 2;
gzip_proxied any;
gzip_types text/plain text/css application/x-javascript
text/xml application/xml application/xml+rss text/javascript;
gzip_buffers 16 8k;
#disable for IE 6
gzip_disable "MSIE [1-6]\.(?!.*SV1)";
# POST size limit
# Set File upload limit at 2G
client_max_body_size 2000M;
sendfile on;
keepalive_timeout 65;


server { #server80
listen 192.168.1.231:80 ;
server_name x.x.com;

##Blocked IP address message
error_page 403 = http://x.y.com/monitor/x_403.html;

### Add SSL specific settings here ###
keepalive_timeout 240;
access_log /nginx/cust-SSL/logs/access.log;
error_log /nginx/cust-SSL/logs/error.log;

location /nginx_status {
stub_status on;
access_log off;
allow 1.1.1.0.0/24;

deny all;
}


##FILE DOWNLOAD OPTIONS --this is the bit that is not working.
location /path_for_testing {
##hosts to allow
proxy_buffering on;
#proxy_buffer_size 16k;
proxy_buffer_size 64k;
proxy_buffers 64 64k;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://x.y.com:80/path_for_testing;
proxy_redirect off;
}

##FIX CONTENT

location /different_path {
subs_filter_types text/html text/css text/xml;
#subs_filter http://x.x.com/sf/ http://x.x.com/sf/ i;
subs_filter http://x.x.com/svn/ http://x.x.com/svn/ i;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://x.x.com:80/different_path/;
proxy_redirect off;
}

## Proxy the rest of the site, temporarily commented out as I am
debugging the in the path for testing section.

location / {
#proxy_buffering on;
#proxy_buffer_size 16k;
#proxy_buffers 32 16k;
#client_body_buffer_size 128k;
#proxy_busy_buffers_size 64k;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://x.x.com:80/;
proxy_redirect off;
}





_______________________________________________
nginx mailing list
nginx@nginx.org
http://nginx.org/mailman/listinfo/nginx
Subject Author Posted

Nginx proxy truncates large file downloads to 1.1G

Duane Mulder December 05, 2010 08:26PM

Re: Nginx proxy truncates large file downloads to 1.1G

mike December 05, 2010 08:36PM

Re: Nginx proxy truncates large file downloads to 1.1G

JAZZ F. September 14, 2011 02:58AM

Re: Nginx proxy truncates large file downloads to 1.1G

Maxim Dounin September 14, 2011 03:54AM

Re: Nginx proxy truncates large file downloads to 1.1G

Nicholas Sherlock December 07, 2010 05:34PM

Re: Nginx proxy truncates large file downloads to 1.1G

Cliff Wells September 14, 2011 03:42AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 313
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready