Could if be possible to change the 301 redirect HTML body of nginx?
I know it sounds really presomptuous. so, let's dive into the deep.
So, when you perform the following:
jojo@DESKTOP-IRG1A6S:~$ curl http://forum.nginx.org
<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.17.5</center>
</body>
</html>
In June 2014, RFC7231 from IETF mentionned that:
The server SHOULD generate a Location header field in the response
containing a preferred URI reference for the new permanent URI. The
user agent MAY use the Location field value for automatic
redirection. The server's response payload usually contains a short
hypertext note with a hyperlink to the new URI(s).
Reference:
https://tools.ietf.org/html/rfc7231#section-6.4.2
As you saw before the HTML body of Nginx does not mention the destination URL in the 301 redirect which should be something like <a href="https://forum.nginx">here</a> (see example of apache at the bottom)
This RFC follows another older RFC (deprecated):
https://tools.ietf.org/html/rfc2616#section-10.3.2
The new permanent URI SHOULD be given by the Location field in the
response. Unless the request method was HEAD, the entity of the
response SHOULD contain a short hypertext note with a hyperlink to
the new URI(s).
I did an experiment and I found that Googlebot has a problem with the Nginx redirections because the Googlebot code expects the destination URL in the HTML body of the 301 redirect. On the other hand, I think that if the HTML body is empty, googlebot likes it and it relies on the location field for redirection.
There are two options, either to delete the whole HTML and rely on the location field only or add the destination URL in the HTML body.
I have noticed that twitter.com and facebook.com chose to delete the HTML body of their 301 redirect.
if you perform a "curl http://twitter.com" or a "curl http://facebook.com" it does not return anything. It means that the HTML body is empty for a 301 redirect from those websites.
FOr reference, the HTML body of Apache 301 redirect is like this:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://xx/">here</a>.</p>
<hr>
<address>Apache Server at xx.com Port 80</address>
</body></html>
