Hi, again

FreeBSD 9.1-STABLE #0: Sat May 18 00:32:18 EEST 2013 amd64

nginx version: nginx/1.4.4
TLS SNI support enabled
configure arguments: --prefix=/usr/local/etc/nginx --with-cc-opt='-I
/usr/local/include' --with-ld-opt='-L /usr/local/lib'
--conf-path=/usr/local/etc/nginx/nginx.conf
--sbin-path=/usr/local/sbin/nginx --pid-path=/var/run/nginx.pid
--error-log-path=/var/log/nginx-error.log --user=www --group=www
--http-client-body-temp-path=/var/tmp/nginx/client_body_temp
--http-fastcgi-temp-path=/var/tmp/nginx/fastcgi_temp
--http-proxy-temp-path=/var/tmp/nginx/proxy_temp
--http-scgi-temp-path=/var/tmp/nginx/scgi_temp
--http-uwsgi-temp-path=/var/tmp/nginx/uwsgi_temp
--http-log-path=/var/log/nginx-access.log
--with-http_image_filter_module --with-http_stub_status_module
--add-module=/usr/ports/www/nginx/work/nginx_upload_module-2.2.0
--add-module=/usr/ports/www/nginx/work/masterzen-nginx-upload-progress-module-a788dea
--add-module=/usr/ports/www/nginx/work/naxsi-core-0.50/naxsi_src
--with-pcre --with-http_ssl_module

I've meet strange behaviour when connection is established via software
download manager.

My configuration is as follows:

location some_extension {

limit_rate_after 15m;
limit_rate 250k;

}

When I open or save as URL with matched extension via browser (desktop,
mobile, etc), limit_rate works perfectly, after I hit 15MB of the file,
my speed goes down up to 1Mb/s regarding my configuration.

When I place the same URL with matched extension in some download
manager like IDM (internet download manager), FDM (free download
manager) or others, limit_rate is not matched, download speed proceeded
at max possible. [the actual problem]

As differences from browser and software download manager I met only:

- when download is processed via browser - if download is requested on
chunks I get 206 partial content (multiple http sessions sent, multiple
entries for this connection in my access log), download if processed as
expected, after hitting 15MB of the file, download speed goes down to 1 Mb/s

- when download is processed via browser - if download is requested on
one piece I get 200 ok (1 http session, 1 entry in my access log
regarding this connection), download if processed as expected, after
hitting 15MB of the file, download speed goes down to 1 Mb/s

- when download is processed via software download manager FDM (free
download manager) I receive 1 http connection with status 200 OK (1
entry in my access log, multiple established connections from remote
addr X.X.X.X and remote ports A,B,C,D,...) , but after inspecting
headers, I see multiple requests regarding this single http session, in
multiple requests is changed only start byte in Range header without
sending end byte in range header; netstat confirms multiple connections
via this single session and limit_rate is not matched in that case.

tcpdump - download performed via browser Firefox - 200 OK single
connection (1 entry in my access_log), no Range header:
GET [requested_file] HTTP/1.1
Host: [host]
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:27.0) Gecko/20100101
Firefox/27.0 AlexaToolbar/alxf-2.19
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: [referer]
Cookie: [cookie]
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache

tcpdump - download performed via browser Firefox 206 partial (multiple
connections, multiple entries in my access log) - range header sent with
end byte specified:
GET [requested_file] HTTP/1.1
Host: [host]
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:27.0) Gecko/20100101
Firefox/27.0 AlexaToolbar/alxf-2.19
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: [referer]
Cookie: [cookie]
Connection: keep-alive
Range: bytes=2621440-2686975

tcpdump - download performed via FDM (single session, 1 entry in my
access log, multiple established connections) without end byte specified
in Range header:
GET [requested_file] HTTP/1.1
Referer: [referer]
Accept: */*
Range: bytes=2541702-
User-Agent: FDM 3.x
Host: [host]
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: [cookie]

My question is what may be the reason when downloading the example file
with download manager not to match limit_rate directive and is there any
way to debug somehow limit_rate ?

I've used to inspect in the past the behaviour when end byte in Range
header is not specified (unfortunately I don't remember if is was with
some browser, but I can reproduce it if needed), then Nginx returns 416
(and when I see 416 in my access log, I block certain client for some
time, let say 1h hour ).

Actually is not a problem for me, to match these download managers and
block them, or to set some inspection regarding Range header, or to
limit the connection with limit_conn instead limit_rate, but I don't
want to limit speed by IP address or to block download managers, but I
would like to use limit_rate regarding file type extension.

Thank you

On 10/9/2013 5:41 PM, Bozhidara Marinchovska wrote:
> I'm sorry, misread documentation.
>
> Placed set $limit_rate 0 in my case1 instead limit_rate 0 and now
> works as expected.
>
>
> On 09.10.2013 17:21 ч., Bozhidara Marinchovska wrote:
>> Hi,
>>
>> nginx -V
>> nginx version: nginx/1.4.2
>> configure arguments: --prefix=/usr/local/etc/nginx --with-cc-opt='-I
>> /usr/local/include' --with-ld-opt='-L /usr/local/lib'
>> --conf-path=/usr/local/etc/nginx/nginx.conf
>> --sbin-path=/usr/local/sbin/nginx --pid-path=/var/run/nginx.pid
>> --error-log-path=/var/log/nginx-error.log --user=www --group=www
>> --http-client-body-temp-path=/var/tmp/nginx/client_body_temp
>> --http-fastcgi-temp-path=/var/tmp/nginx/fastcgi_temp
>> --http-proxy-temp-path=/var/tmp/nginx/proxy_temp
>> --http-scgi-temp-path=/var/tmp/nginx/scgi_temp
>> --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi_temp
>> --http-log-path=/var/log/nginx-access.log
>> --with-http_image_filter_module --with-http_stub_status_module
>> --add-module=/usr/ports/www/nginx/work/naxsi-core-0.50/naxsi_src
>> --with-pcre
>>
>> FreeBSD 9.1-STABLE #0: Sat May 18 00:32:18 EEST 2013 amd64
>>
>> I'm using limit_rate case if in location. Regarding documentation "if
>> in location" context is avaiable.
>>
>> My configuration is as follows:
>>
>> location some_extension {
>>
>> # case 1
>> if match_something {
>> root ...
>> break;
>> }
>>
>> # case 2
>> if match_another {
>> root ...
>> break;
>> }
>>
>> # else (case3)
>> root ...
>> something other ...
>> here it is placed also limit_rate / limit_after directives
>> }
>>
>> There is a root inside location with a (strong) reason :) (nginx
>> pitfails case "root inside location block - BAD").
>>
>> When I open in my browser http://my.website.com/myfile.ext it matches
>> case 3 from the cofiguration. Limit_rate/limit_after works correct as
>> expected.
>> I want case1 not to have limit_rate / limit_after.
>>
>> Test one:
>> In case1 I place limit_rate 0, case3 is the same limit_rate_after
>> XXm; limit_rate some_rate. When I open in my browser URL matching
>> case1 - limit_rate 0 is ignored. After hitting XXm from the file I
>> get limit_rate from case 3.
>>
>> Test 2:
>> In case 1 I place limit_rate_after 0; limit_rate 0, case3 is the same
>> limit_rate_after XXm; limit_rate some rate. When I open in my browser
>> URL matching case 1 - limit_rate_after 0 and limit_rate 0 are
>> ignored. Worst is that when I try to download the file, I even didn't
>> match case3 - my download starts from the first MB with limit_rate
>> bandwidth from case3.
>>
>> Both tests are made in interval from 20 minutes, 1 connection from my
>> IP, etc.
>>
>> I don't post my whole configuration, because may be it is
>> unnessesary. If cases are with http_referer.
>>
>> Case 1 - if I see referer some_referer, I do something. (here I don't
>> want to place any limits)
>> Case 2 - If I see another referer , I do something else.
>> Case 3 - Else ... something other (here I have some limits)
>>
>> I'm sure I match case1 when I test (nginx-error.log with debug say
>> it), I mean my configuration with cases is working as expected, the
>> new thing is limit_rate and limit_rate_after which is not working as
>> expected.
>>
>> Any thoughts ? Meanwhile I will test on another version.
>>
>> Thanks
>>
>>
>>
>> _______________________________________________
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx