Welcome! Log In Create A New Profile

Advanced

Re: Question regarding $invalid_referer

J Carter
March 07, 2024 05:14AM
Hello,

On Tue, 5 Mar 2024 13:07:53 -0800
"lists@lazygranch.com" <lists@lazygranch.com> wrote:

> I am presently using a scheme like this to prevent scraping documents.
> ************************************
> location /images/ {
> valid_referers none blocked www.example.com example.com forums.othersite.com ;
> # you can tell the browser that it can only download content from the domains you explicitly allow
> # if ($invalid_referer) {
> # return 403;
> if ($invalid_referer) {
> return 302 $scheme://www.example.com;
> ***************************************
> I commented out some old code which just sends an error message. I
> pulled that from the nginx website. I later added the code which sends
> the user to the top level of the website.
>
> It works but the results really aren't user friendly. What I rather do
> is if I find an invalid_referer to some document, I would like to
> redirect the request to the html page that has my link to the document.
>
> I am relatively sure I will need to hand code the redirection for every
> file, but plan on only doing this for pdfs. Probably 20 files.
>
> Here is a google referral I pulled from the log file
>
> *********************************************
> 302 172.0.0.0 - - [05/Mar/2024:20:18:52 +0000] "GET /images/ttr/0701crash.pdf HTTP/2.0" 145 "https://www.google.com/" "Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Mobile Safari/537.36" "-"
> **********************************************
> So I would need to map /images/ttr/0701crash.pdf to the referring page
> on the website.
> _______________________________________________

There is really a question in your email :) however, you could use the
SSI module[1] to auto generate the referring page with the link
dynamically if don't already have that.

[1] https://nginx.org/en/docs/http/ngx_http_ssi_module.html

In terms of doing the mapping to some static set of referring pages if
you already have those, that will depend upon what path scheme you plan
for those in relation to original files.

A sensible way would be to make
the referring pages's path related to pdf name, (something like
/referring/0701crash).

In nginx when you do redirect, you can do those mappings
dynamically using regex captures. Something like this using nested
locations:

location /images {
...
location ~/(.+).pdf {
if ($invalid_referer) {
return 302 $scheme://www.example.com/referring/${1};
}
}
}
_______________________________________________
nginx mailing list
nginx@nginx.org
https://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Question regarding $invalid_referer

gariac March 05, 2024 04:10PM

Re: Question regarding $invalid_referer

J Carter March 07, 2024 05:14AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 216
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready