Welcome! Log In Create A New Profile

Advanced

Re: limit_req for spiders only

October 14, 2013 09:25AM
Hello.

Doesnt robots.txt "Crawl-Delay" directive satisfy your needs?
Normal spiders should obey robots.txt, if they dont - they can be banned.
Subject Author Posted

limit_req for spiders only

Toni Mueller October 14, 2013 08:00AM

Re: limit_req for spiders only

Sylvia October 14, 2013 09:25AM

Re: limit_req for spiders only

Toni Mueller October 14, 2013 10:04AM

Re: limit_req for spiders only

Francis Daly October 14, 2013 10:24AM

Re: limit_req for spiders only

Toni Mueller October 14, 2013 10:54AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 61
Record Number of Users: 6 on August 08, 2016
Record Number of Guests: 386 on August 02, 2016
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready