Welcome! Log In Create A New Profile

Advanced

Denying illegal host headers blocks bots sometimes

October 28, 2014 09:12AM
Hi,

I've set my conf to deny illegal host headers, as per below option.
However, I've noticed that bots from Google, Bing, Baidu, etc, when
trying to fetch /robots.txt, often get blocked by nginx as it just
terminates the connection with 444 response. Sometimes they succeed
but more often they get blocked. Yes, I've checked to see if these
bots are not impostors but they look legit to me. My nginx version is
1.7.6 and the code I use is below

if ($host !~ ^(mydomain.net|www.mydomain.net)$ ) {
return 444;
}

I've also tried with !~* but I get the same outcome. Am I doing something wrong?

--
Yours truly

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Denying illegal host headers blocks bots sometimes

microchip October 28, 2014 09:12AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 91
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready