Welcome! Log In Create A New Profile

Advanced

Re: Geo blocking, but allowing google index robot to pass-thru

All files from this thread

File Name File Size   Posted by Date  
smime.p7s 6 KB open | download GreenGecko 02/18/2013 Read message
February 18, 2013 04:12PM
On Mon, 2013-02-18 at 14:00 +0200, Pekka.Panula@sofor.fi wrote:
> Hi
>
> I have a site where i want to geo block all but one country, but
> perhaps allow Google to index site, perhaps some other index bot too.
>
> So what sort of configuration is needed so i can detect Google bot and
> let it pass-thru? Would be nice if there is example configuration. Is
> only good way to check user-agent?
>
> ______________________________________________________________________
>
> Pekka Panula | Jatkuvat Palvelut | Direct +358 10 235 9232 |
> Pekka.Panula@sofor.fi
> Sofor Oy | www.sofor.fi| Takakaarre 3 | PL 51 | FIN-62200 Kauhava
> tel. +358 10 235 90 | fax +358 10 235 9100
>
Here's some code I use for a similar setup...

map $geoip_country_code $external_redirects {
default 'Block';

US http://www.example.com;
}

#Whitelist crawlers
map $http_user_agent $crawler {
default 0;

~*(AdsBot-Google|Googlebot-Mobile|Googlebot-Image|Mediapartners-Google|
bingbot|Feedfetcher-Google|Googlebot|Yahoo\ !Slurp|msnbot|msnbot-media|
YahooCacheSystem) 1;
}
# You'll also probably need to override for specific IP addresses too...
geo $whitelisted {
default $crawler;

# localhost
127.0.0.0/8 1;

# CloudFlare

204.93.240.0/24 1;
204.93.177.0/24 1;
199.27.128.0/21 1;
173.245.48.0/20 1;
103.21.244.0/22 1;
103.22.200.0/22 1;
103.31.4.0/22 1;
141.101.64.0/18 1;
108.162.192.0/18 1;
190.93.240.0/20 1;
188.114.96.0/20 1;
197.234.240.0/22 1;
198.41.128.0/17 1;
}


You can then use your own logic with the values of $external_redirects
and $whitelisted to control redirections.

hth,


Steve

--
Steve Holdoway BSc(Hons) MIITP
http://www.greengecko.co.nz
Skype: sholdowa
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Attachments:
open | download - smime.p7s (6 KB)
Subject Author Posted

Geo blocking, but allowing google index robot to pass-thru

Anonymous User February 18, 2013 07:02AM

Re: Geo blocking, but allowing google index robot to pass-thru Attachments

GreenGecko February 18, 2013 04:12PM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 176
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 500 on July 15, 2024
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready