Welcome! Log In Create A New Profile

Advanced

Re: Avoiding Nginx restart when rsyncing cache across machines

Quintin Par
September 12, 2018 07:24PM
Hi Lucas,



The cache is pretty big and I want to limit unnecessary requests if I can.
Cloudflare is in front of my machines and I pay for load balancing,
firewall, Argo among others. So there is a cost per request.



Admittedly I have a not so complex cache architecture. i.e. all cache
machines in front of the origin and it has worked so far. This is also
because I am not that great a programmer/admin :-)



My optimization is not primarily around hits to the origin, but rather
bandwidth and number of requests.



- Quintin


On Wed, Sep 12, 2018 at 1:06 PM Lucas Rolff <lucas@lucasrolff.com> wrote:

> Can I ask, why do you need to start with a warm cache directly? Sure it
> will lower the requests to the origin, but you could implement a secondary
> caching layer if you wanted to (using nginx), so you’d have your primary
> cache in let’s say 10 locations, let's say spread across 3 continents (US,
> EU, Asia), then you could have a second layer that consist of a smaller
> amount of locations (1 instance in each continent) - this way you'll warm
> up faster when you add new servers, and it won't really affect your origin
> server.
>
> It's a lot more clean also because you're able to use proxy_cache which is
> really what (in my opinion) you should use when you're building caching
> proxies.
>
> Generally I'd just slowly warm up new servers prior to putting them into
> production, get a list of top X files accessed, and loop over them to pull
> them in as a normal http request.
>
> There's plenty of decent solutions (some more complex than others), but
> there should really never be a reason to having to sync your cache across
> machines - even for new servers.
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Avoiding Nginx restart when rsyncing cache across machines

Quintin Par September 11, 2018 07:46PM

Re: Avoiding Nginx restart when rsyncing cache across machines

Maxim Dounin September 12, 2018 10:46AM

Re: Avoiding Nginx restart when rsyncing cache across machines

Quintin Par September 12, 2018 03:42PM

Re: Avoiding Nginx restart when rsyncing cache across machines

Lucas Rolff September 12, 2018 04:08PM

Re: Avoiding Nginx restart when rsyncing cache across machines

Quintin Par September 12, 2018 07:24PM

Re: Avoiding Nginx restart when rsyncing cache across machines

Peter Booth via nginx September 12, 2018 07:32PM

Re: Avoiding Nginx restart when rsyncing cache across machines

Quintin Par September 13, 2018 02:16AM

Re: Avoiding Nginx restart when rsyncing cache across machines

Lucas Rolff September 13, 2018 02:40AM

Re: Avoiding Nginx restart when rsyncing cache across machines

Quintin Par September 13, 2018 02:46PM

Re: Avoiding Nginx restart when rsyncing cache across machines

Lucas Rolff September 13, 2018 04:04PM

Re: Avoiding Nginx restart when rsyncing cache across machines

Peter Booth via nginx September 13, 2018 09:24PM

Re: Avoiding Nginx restart when rsyncing cache across machines

itpp2012 September 14, 2018 02:20AM

Question

Saint Michael September 14, 2018 08:08AM

Re: Avoiding Nginx restart when rsyncing cache across machines

Maxim Dounin September 13, 2018 06:48AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 134
Record Number of Users: 6 on February 13, 2018
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready