Welcome! Log In Create A New Profile

Advanced

Re: limit_req per subnet?

December 14, 2016 02:18PM
‎They claim to obey robots.txt. They also claim to to use consecutive IP addresses.
https://www.semrush.com/bot/

Some dated posts (2011) indicate semrush uses AWS. I block all of AWS IP space and can say I've never seen a semrush bot. So that might be a solution. I got the AWS IP space from some Amazon Web page. 

I get a bit of kick back about blocking things that are not eyeballs like colos and VPS, but it works for me.  I only block after seeing a hacking attempt(s) in my logs.



  Original Message  
From: Grant
Sent: Wednesday, December 14, 2016 10:31 AM
To: nginx@nginx.org
Reply To: nginx@nginx.org
Subject: Re: limit_req per subnet?

> I am curious what is the request uri they was hitting. Was it a dynamic page
> or file or a static one.


It was semrush and it was all manner of dynamic pages.

- Grant
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

limit_req per subnet?

Grant December 13, 2016 05:02PM

Re: limit_req per subnet?

gariac December 13, 2016 07:44PM

Re: limit_req per subnet?

c0nw0nk December 14, 2016 02:36AM

Re: limit_req per subnet?

Grant December 14, 2016 01:32PM

Re: limit_req per subnet?

gariac December 14, 2016 02:50PM

Re: limit_req per subnet?

Grant December 14, 2016 05:00PM

Re: limit_req per subnet?

Grant December 14, 2016 01:32PM

Re: limit_req per subnet?

gariac December 14, 2016 02:18PM

Re: limit_req per subnet?

shiz December 14, 2016 03:22PM

Re: limit_req per subnet?

Grant December 14, 2016 05:02PM

Re: limit_req per subnet?

Grant December 14, 2016 05:16PM

Re: limit_req per subnet?

gariac December 14, 2016 07:08PM

Re: limit_req per subnet?

shiz December 14, 2016 08:24PM

Re: limit_req per subnet?

gariac December 15, 2016 03:14AM

Re: limit_req per subnet?

c0nw0nk December 15, 2016 05:23AM

Re: limit_req per subnet?

gariac December 15, 2016 06:04PM

Re: limit_req per subnet?

c0nw0nk December 14, 2016 11:04PM

Re: limit_req per subnet?

Grant December 15, 2016 06:52PM

Re: limit_req per subnet?

c0nw0nk December 16, 2016 12:03AM

Re: limit_req per subnet?

Francis Daly December 29, 2016 06:20AM

Re: limit_req per subnet?

Grant December 30, 2016 07:32AM

Re: limit_req per subnet?

Francis Daly December 31, 2016 05:38AM

Re: limit_req per subnet?

Grant January 02, 2017 10:44AM

Re: limit_req per subnet?

Francis Daly January 04, 2017 01:34PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 110
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready