Welcome! Log In Create A New Profile

Advanced

nginx blocking GoogleBot

Posted by Loggy 
nginx blocking GoogleBot
October 22, 2012 04:07PM
I have nginx 1.2.4 as a reverse proxy to apache2 and WordPress and this runs very sweetly. However today's upgrade in Google Page Ranks, far from promoting the various sites on my server, have set them to P/R=n/a.

When I look at Webmaster Tools, I see that all recent scans have resulted in a 403 Access Denied result. But I cannot see why this should be.

It is the usual setup with the purge module ngx_cache_purge-1.6 compiled in and WordPress running with nginx-proxy-cache-purge as a must-use plugin. This works very sweetly, refreshing where expected and caching where expected. The problem is the Google doesn't see this at all and for the past week, has obviously returned a 403. When I try to index the site directly in WMT, it appears to work OK but I haven't seen the result yet.

The only place where I am testing the user agent is for mobiles. I have tried commenting this out.

I attach the full config file. Any clues?
Attachments:
open | download - tttt.nginx (3.6 KB)
Re: nginx blocking GoogleBot
October 22, 2012 05:33PM
Not every GoogleBot enquiry is blocked which is a puzzle. But enough to render the page rank n/a.

I have tried to add a trailing "/" after the docroot as would be required by the alias command.

I am also trying to remove the "/" from the redirect as well as the "/" in the substitution so it is:

server {
listen 80;
server_name mydomain.com;
rewrite ^(.*) http://www.mydomain.com$1 permanent;
}

in case a page is requested with a ? without the "/".

I am sure it is not in the apache (on port 8081) config which is very simple:

<VirtualHost *:8081>
ServerAdmin webmaster@mydomain.com
DocumentRoot /home/user/www/mydomain.com/docroot
ServerName www.mydomain.com

CustomLog /home/user/logs/mydomain.com/access_log combined
ErrorLog /home/user/logs/mydomain.com/error_log

<DIRECTORY /home/user/www/mydomain.com/docroot >
OPTIONS FollowSymLinks
AllowOverride All
</DIRECTORY>
IndexOptions FancyIndexing
</VirtualHost>
Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 165
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready