Welcome! Log In Create A New Profile

Advanced

truncated output with gzip, large files and slow rate

All files from this thread

File Name File Size   Posted by Date  
debug.log.bz2 10.7 KB open | download James Lee 04/16/2012 Read message
James Lee
April 16, 2012 04:38AM
Hello,

nginx is truncating output with the following:
+ gzip on
+ compressed size greater than 100k (approx)
+ client downloads at a slow rate


*** Steps to reproduce ***

nginx.conf:

events {
}
http {
types {
text/plain txt;
}
gzip on;
gzip_types text/plain;
server {
listen 80 default_server;
server_name _;
root /path/to/root;
}
}


Create a test file that is larger than 100k compressed:

$ head -1000 /dev/random > /path/to/root/test.txt


Request exhibiting failure:

$ curl -H "Accept-Encoding: gzip" --limit-rate 50k localhost/test.txt |
gzcat | cksum
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 180k 0 180k 0 0 2966 0 --:--:-- 0:01:02
--:--:-- 0
curl: (18) transfer closed with outstanding read data remaining

gzcat: stdin: unexpected end of file
649550918 184260


and with wget:

$ wget --header "Accept-Encoding: gzip" --limit-rate=50k -O /dev/stdout
localhost/test.txt | gzcat | cksum

--2012-04-15 11:27:04-- http://localhost/test.txt
Resolving localhost (localhost)... 127.0.0.1, ::1
Connecting to localhost (localhost)|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/plain]
Saving to: `/dev/stdout'

[ <=> ] 184,330 --.-K/s in 61s

2012-04-15 11:28:06 (2.96 KB/s) - `/dev/stdout' saved [184330]


gzcat: stdin: unexpected end of file
649550918 184260




Some variants showing success (correct size and cksum) by avoiding one of
compression or limit-rate:

$ curl -H "Accept-Encoding: gzip" localhost/test.txt | gzcat | cksum
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 235k 0 235k 0 0 5970k 0 --:--:-- --:--:-- --:--:--
6370k
2871747365 241269


$ curl --limit-rate 50k localhost/test.txt | cksum
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 235k 100 235k 0 0 47700 0 0:00:05 0:00:05 --:--:--
55650
2871747365 241269



There are also failures with major web browsers including Firefox 11.0,
MSIE 9.0 and Safari.

No error is reported by nginx. The http return code and logged code is
200. The only clue in the server is the sent size reported is smaller
than the expected compressed size.



*** Other information ***

nginx 1.1.19
Solaris 10 u10




James.

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

truncated output with gzip, large files and slow rate

James Lee April 16, 2012 04:38AM

Re: truncated output with gzip, large files and slow rate

Maxim Dounin April 16, 2012 06:18AM

Re: truncated output with gzip, large files and slow rate Attachments

James Lee April 16, 2012 09:28AM

Re: truncated output with gzip, large files and slow rate

Maxim Dounin April 16, 2012 01:58PM

Re: truncated output with gzip, large files and slow rate

James Lee April 16, 2012 02:24PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 316
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready