Welcome! Log In Create A New Profile


Re: fake googlebots

Rainer Duffner
September 25, 2016 06:08PM
> Am 25.09.2016 um 23:58 schrieb lists@lazygranch.com:
> I got a spoofed googlebot hit. It was easy to detect since there were
> probably a hundred requests that triggered my hacker detection map
> scheme. Only two requests received a 200 return and both were harmless.
> 200 - - [25/Sep/2016:17:45:23 +0000] "GET / HTTP/1.1" 847 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" "-"
> For the fake googlebot:
> # host
> Host not found: 3(NXDOMAIN)
> For a real googlebot:
> # host
> domain name pointer crawl-66-249-69-184.googlebot.com.
> IP2location shows it is a Chinese ISP:
> 3(NXDOMAIN)http://www.ip2location.com/
> Nginx has a reverse DNS module:
> https://github.com/flant/nginx-http-rdns
> I see it has a 10.1 issue:
> https://github.com/flant/nginx-http-rdns/issues/8
> Presuming this bug gets fixed, does anyone have code to verify
> googlebots? Or some other method?

Sorry to be so blunt - but what’s the point?

You can also password-protect your site and give the credentials only to your friends.
Problem solved.

Most of the traffic of the web these days is created by bots (unless you’re a popular shop or offer original, often updated content for popular topics, then you’ll actually get visitors).

If it’s not the Big G, it might be bing or baidu or yandex or some other bot.

nginx mailing list
Subject Author Posted

fake googlebots

gariac September 25, 2016 06:00PM

Re: fake googlebots

wandenberg September 25, 2016 06:06PM

Re: fake googlebots

gariac September 25, 2016 06:28PM

Re: fake googlebots

Robert Paprocki September 25, 2016 06:30PM

Re: fake googlebots

Rainer Duffner September 25, 2016 06:08PM

Re: fake googlebots

dommyet September 25, 2016 11:02PM

Re: fake googlebots

gariac September 28, 2016 07:02PM

Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 73
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready