Welcome! Log In Create A New Profile

Advanced

Re: limit-req and greedy UAs

September 12, 2016 05:28AM
‎I picked 444 based on the following, though I see your point in that it is a non-standard code. I guess from a multiplier standpoint, returning nothing is as minimal as it gets, but the hacker often sends the message twice due to lack of response. A 304 return to an attempt to log into WordPress would seem a bit weird. All I really need is a unique code to find in the log file. 

444 CONNECTION CLOSED WITHOUT RESPONSE
A non-standard status code used to instruct nginx to close the connection without sending a response to the client, most commonly used to deny malicious or malformed requests.

This status code is not seen by the client, it only appears in nginx log files.‎
  Original Message  
From: B.R.
Sent: Monday, September 12, 2016 1:08 AM
To: nginx ML
Reply To: nginx@nginx.org
Subject: Re: limit-req and greedy UAs

You could also generate 304 responses for content you won't provide (cf. return).
nginx is good at dealing with loads of requests, no problem on that side. And since return generates an in-memory answer by default, you won't be hammering your resources. If yo uare CPU or RAM-limited because of those requests, then I would suggest you evaluate the sizing of your server(s).
You might wish to seperate logging for these requests from the standard flow to improve their readability, or deactivate them altogether if you consider they add little-to-no value.

My 2¢,
---
B. R.

On Sun, Sep 11, 2016 at 9:16 PM, <lists@lazygranch.com> wrote:
‎https://www.nginx.com/blog/tuning-nginx/

‎I have far more faith in this write up regarding tuning than the anti-ddos, though both have similarities. 

My interpretation is the user bandwidth is connections times rate. But you can't limit the connection to one because (again my interpretation) there can be multiple users behind one IP. Think of a university reading your website. Thus I am more comfortable limiting bandwidth than I am limiting the number of connections. ‎The 512k rate limit is fine. I wouldn't go any higher. 

I don't believe their is one answer here because it depends on how the user interacts with the website. I only serve static content. In fact, I only allow the verbs "head" and "get" to limit the attack surface. A page of text and photos itself can be many things. Think of a photo gallery versus a forum page. The forum has mostly text sprinkled with avatar photos, while the gallery can be mostly images with just a line of text each. 

Basically you need to experiment. Even then, your setup may be better or worse than the typical user. That said, if you limited the rate to 512k bytes per second, most users could achieve that rate‎. 

I just don't see evidence of download managers. I see plenty of wget, curl, and python. Those people get my 444 treatment. I use the map module as indicated in my other post to do this. 

What I haven't mentioned is filtering out machines. If you are really concerned about your system being overloaded, think about the search engines you want to support. Assuming you want Google, you need to set up your website in a manner so that Google knows you own it, then you can throttle it back. Google is maybe 20% of my referrals.

If you have a lot of photos, you can set up nginx to block hit linking. This is significant because Google images will hot link everything you have. What you want is for Google itself to see your images, which it will present in reduced resolution, but block the Google hot link. If someone really wants to see your image, Google supplies the referal page. 

http://nginx.org/en/docs/http/ngx_http_referer_module.html

I make my own domain a valid, but maybe that is assumed. If you want to place a link to an image on your website in a forum, you need to make that forum valid. 

Facebook will steal your images.
http://badbots.vps.tips/info/facebookexternalhit-bot

I would use the nginx map module since you will probably be blocking many bots. 

Finally, you may want to block "the cloud"‎ using your firewall. Only block the browser ports since mail servers will be on the cloud. I block all of AWS for example. My nginx.conf also flags certain requests such as logging into WordPress since I'm not using WordPress! Clearly that IP is a hacker. I have plenty more signatures in the map. I have a script that pulls the IP addresses out of the access.log. I get maybe 20 addresses a day. I feed them to ip2location. Any address that goes to a cloud, VPS, colo, hosting company gets added to the firewall blocking list. I don't just block the IP, but I use the Hurricane Electric BGP tool to get the entire IP space to block. As a rule, I don't block schools, libraries, or ISPs. The idea here is to allow eyeballs but not machines. 

You can also use commercial blocking services if you trust them. (I don't. )


  Original Message  
From: Grant
Sent: Sunday, September 11, 2016 10:28 AM
To: nginx@nginx.org
Reply To: nginx@nginx.org
Subject: Re: limit-req and greedy UAs

> ‎This page has all the secret sauce, including how to limit the number of connections.
>
> https://www.nginx.com/blog/mitigating-ddos-attacks-with-nginx-and-nginx-plus/
>
> I set up the firewall with a higher number as a "just in case."


Should I basically duplicate my limit_req and limit_req_zone
directives into limit_conn and limit_conn_zone? In what sort of
situation would someone not do that?

- Grant

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx


_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

limit-req and greedy UAs

Grant September 08, 2016 09:26PM

Re: limit-req and greedy UAs

gariac September 08, 2016 09:40PM

Re: limit-req and greedy UAs

Richard Stanway September 09, 2016 09:02AM

Re: limit-req and greedy UAs

gariac September 09, 2016 12:42PM

Re: limit-req and greedy UAs

Grant September 11, 2016 08:32AM

Re: limit-req and greedy UAs

gariac September 11, 2016 11:32AM

Re: limit-req and greedy UAs

Grant September 11, 2016 01:30PM

Re: limit-req and greedy UAs

gariac September 11, 2016 03:18PM

Re: limit-req and greedy UAs

B.R. September 12, 2016 04:10AM

Re: limit-req and greedy UAs

gariac September 12, 2016 05:28AM

Re: limit-req and greedy UAs

Grant September 12, 2016 04:24PM

Re: limit-req and greedy UAs

Richard Stanway September 12, 2016 05:40PM

Re: limit-req and greedy UAs

gariac September 12, 2016 06:32PM

Re: limit-req and greedy UAs

gariac September 13, 2016 02:56AM

Re: limit-req and greedy UAs

gariac September 13, 2016 04:30AM

Re: limit-req and greedy UAs

Grant September 13, 2016 12:04PM

Re: limit-req and greedy UAs

Grant September 13, 2016 12:10PM

Re: limit-req and greedy UAs

Grant September 24, 2016 08:58PM

Re: limit-req and greedy UAs

c0nw0nk September 12, 2016 08:51AM

Re: limit-req and greedy UAs

Grant September 11, 2016 08:38AM

Re: limit-req and greedy UAs

gariac September 11, 2016 10:42AM

Re: limit-req and greedy UAs

Grant September 11, 2016 01:24PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 107
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready