Yes, I should have explained the problem up front. I made the wrong assumption I was asking a simple question and quickly realized I was getting good answers but my approach was likely flawed from the start.
We are using Nginx just as a cache mechanism across many custom DNS names. As a result we have many server {} blocks handling a variety of different DNS names, vanity URLs and in a few cases SSL redirects. Where possible a single server {} block handles a wildcard DNS name but there are reasons we separate some of them out. Given the number of DNS entries, I have also created a system for creating *.conf files, so I do not use a single conf file for all the servers, instead I divide them out into many conf files. For special configuration items that are only included sometimes, I use *.inc files with the idea being that nginx.conf includes *.conf but does not include *.inc so that way I can make my server{} blocks cleaner and put common configurations into inc file. The main benefit being that almost all the config files fit on the screen without scrolling that much in VIM.
Within each of these DNS instances we have a mix of login required content. We use three different login methods, IP whitelists, local user/password set in our CMS and then we use NetID via CAS/Shibboleth. We don’t want any of the login pages cached.
The way we differentiate this relates to the mechanics on the backend of the web application. So we created a custom header called X-Secured-Page which is either true or false. In the event the value is true, we do not want to cache.
To go a step further, I have made use the add_header to set different headers for when we cache a page versus when we do not cache a page. The idea being that Tier1 support could view these headers to deflect support calls when things are working as intended but also to create Jira tickets with the custom header content when things do not seem to be working correctly. In addition to this, when the secure page is true, I use this setting proxy_pass_header Set-Cookie; but when the secured page is false I use this setting: proxy_hide_header Set-Cookie; As for why, it is actually the CAS/Shibboleth authentication that fails to work without this setting change. I could easily just always pass the cookie but I was trying to eliminate any noise I could. We have a fairly complicated stack given the Nginx passes most traffic to a server cluster that handles URL rewrites and Vanity URLs at a level we did not want to approach in Nginx. We may in the future but right now that is off the table given the complexity and work that went into the management interfaces used to handle the url rewrites the content owners use.
So my thought process started with a test for $http_x_secured_page which for obvious reasons would not work because I was failing to distinguish request versus response headers and am very new to Nginx.
Using the responses yesterday, I was 99% sure I had a working model nailed down. I had a map block that looked like this
Map $upstream_http_x_secured_page $nocache {
default “0”;
“true” “1”;
“false” “2”;
}
Then in my server block I had logic that looks like this
Server{
Location / {
If ($nocache = “1”) { return 418; }
If ($nocache = “2”) { return 419; };
}
}
Then I had a custom location for each, again this was all just to test and it worked. I used a combination of cURL and just using the inspector in firefox to see the headers and I was consistently getting the correct code from either the 418 or 419 block. So I moved the same identical code into production but used a new server block. In all this testing I was working with a DNS name not yet put into Nginx, so I was using all HOSTS file manipulation. But the content of the Server block was accurate so the responses were correct. This is the same method I used to test every DNS name moved in and seemed like the simplest way to test in production, not to mention we run 4 nginx servers as part of a globally redundant solution, using the HOSTS file allows me to easily test against a single production server before rolling my configs to all servers.
In trying to figure out the problem, I read about how the map{} block worked and was confused by it saying that it stores the last request made which caused me to wonder if the design I had with many server{} blocks all sharing the same map{} directive was somehow flawed so my testing in production was doomed from the start. I also really do not understand the order of operation taking place. If the map{} took place inside a server block, then I would understand it better and see it encapsulated from other server{} requests. Or if I was to use upstream{} blocks and it went inside that, again I would understand it more.
I did try implementing upstream{} sections but it did not seem to make sense, I only had one server defined inside of it and since I did not see a direct relationship with map{} and in testing it made no difference whether I had this defined or not, so I removed it.
The bottom line is that I need a solution that will cache almost all pages and then in some cases when the X-Secured-Page is true, I don’t want to cache it. I am totally flexible, we could easily set it up so that the X-Secured-Page header only exists when the value would be true and does not exist at all when it would be false if that would make things easier or more reliable. I’m also open to doing just about anything else to make this happen.
On the code side, this value is determined in a security class that was already setting some headings we used for debugging at the application level and it would not be hard to setup a service that could be called (even cached behind nginx) where a URL is cURL’d to see if the page should be secure or not. I’m not sure if having that sort of logic inside a server{} block is possible, good practice or the stupidest idea in the world.
But like I said, I’m willing to approach this problem however necessary with one exception. As it stands now, I have a server block that we moved in which is when we realized there was going to be a problem with login pages. So inside the server{} block I setup location{} blocks for the folders that were secure and handled them with the alternative add_header and cookie settings. Security on folders can be a little more dynamic than I’d like and also not predictable. We have hundreds of web content editors that can make secure content easily in our CMS and they make significant use of this option. So creating a system that would use location{} blocks has to be off the table. It would be a tremendous amount of work to maintain that setup.
___________________________________________
Michael Friscia
Office of Communications
Yale School of Medicine
(203) 737-7932 - office
(203) 931-5381 - mobile
http://web.yale.edu http://web.yale.edu/
On 2/14/18, 5:13 PM, "nginx on behalf of Peter Booth" <nginx-bounces@nginx.org on behalf of peter_booth@me.com> wrote:
I think that part of the power and challenge of using nginx’s caching is that there are many different ways
of achieving the same or similar results, but some of the approaches will be more awkward than others.
I think that it might help if you could express what the issue is that you are trying to solve,
as opposed to the mechanism that you are wanting to use to solve your problem, for example,
“Don’t cache personalized pages” or “don’t cache pages where the user is outside the US” or
“don’t cache pages if the user has logged in” or ...
> On Feb 14, 2018, at 4:44 PM, Friscia, Michael <michael.friscia@yale.edu> wrote:
>
> Maybe that’s the problem, I want to disable cache if the response header is true but not do anything if it is false. I can change my logic in creating this header to only have it on pages where cache should be disabled if it is not possible to use an IF statement around it. I will post my config here once I get rid of the sensitive things, but the confusing thing is that it worked, then it stopped working.
>
>
>
> ___________________________________________
>
> Michael Friscia
>
> Office of Communications
> Yale School of Medicine
>
> (203) 737-7932 – office
> (203) 931-5381 – mobile
>
> http://web.yale.edu
>
>
>
>
> On 2/14/18, 4:39 PM, "nginx on behalf of webopsx" <nginx-bounces@nginx.org on behalf of nginx-forum@forum.nginx.org> wrote:
>
> Hi,
>
> The map is processed on each request and should be very consistent.
>
> I thought you wanted to disable cache on the existence of a response header,
> not a request header.
>
> Otherwise I think we need more information to understand, such as how are
> you testing? Perhaps paste your full configuration here after removing any
> sensitive text.
>
> Posted at Nginx Forum: https://urldefense.proofpoint.com/v2/url?u=https-3A__forum.nginx.org_read.php-3F2-2C278558-2C278576-23msg-2D278576&d=DwICAg&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=x0V3h5hp-I18hCxDQuN0U7f_JYUQYISVWlygl1QK-FU&s=sUMR29LFjCI97P-LSb-RwuojlvhtDhvfRiX_YAio19E&e=
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.nginx.org_mailman_listinfo_nginx&d=DwICAg&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=x0V3h5hp-I18hCxDQuN0U7f_JYUQYISVWlygl1QK-FU&s=833NZxJsLHFd1YlQmOtXp22oR4K0OXPS-xz0mYs6r04&e=
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.nginx.org_mailman_listinfo_nginx&d=DwIGaQ&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=JuKVbxbm0g2sxo5YSYeiKyCXurdeZujyxenVsST6Ce8&s=m6rC0_mW4r0GP-Yl6edLAFDbcVLuRViJIOg-r3yQ9vc&e=
_______________________________________________
nginx mailing list
nginx@nginx.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__mailman.nginx.org_mailman_listinfo_nginx&d=DwIGaQ&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=JuKVbxbm0g2sxo5YSYeiKyCXurdeZujyxenVsST6Ce8&s=m6rC0_mW4r0GP-Yl6edLAFDbcVLuRViJIOg-r3yQ9vc&e=
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
