Welcome! Log In Create A New Profile

Advanced

Re: block robots for some urls

July 18, 2010 05:44PM
On Sun, Jul 18, 2010 at 22:34, toto2008 <nginx-forum@nginx.us> wrote:
> Hello,
>
> Why not using a robots.txt file ? http://www.robotstxt.org/

It won't block bots that don't honor it (not would anything block bots
that present themselves as mozilla).

--
()  ascii-rubanda kampajno - kontraŭ html-a retpoŝto
/\  ascii ribbon campaign - against html e-mail

_______________________________________________
nginx mailing list
nginx@nginx.org
http://nginx.org/mailman/listinfo/nginx
Subject Author Posted

block robots for some urls

bvidinli July 14, 2010 06:56PM

Re: block robots for some urls

toto2008 July 18, 2010 05:34PM

Re: block robots for some urls

vesperto July 18, 2010 05:44PM

Re: block robots for some urls

bvidinli July 18, 2010 07:44PM

Re: block robots for some urls

Igor Sysoev July 19, 2010 07:56AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 186
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready