Welcome! Log In Create A New Profile

Advanced

nginx as reverse caching proxy for CentOS repositories

Wouter Schoot
March 14, 2010 09:08AM
Dear list,

In an attempt to create an offline copy for my homenetwork to quickly
install CentOS I wanted to use nginx as a reverse caching proxy so that
nginx stores all the files the that the CentOS installer requests after
the first time.

However, after most of the startup files (product.img, stage2.img etc)
are downloaded and cached succesfully, it bails on the chkconfig rpm
(which happens to be the first file it actually downloads for the
install, from the CentOS/ folder.

I'm running nginx/0.8.34, compiled with a standard ./configure (only
using the prefix, so the rest is left as default).

The following relevant changes to the default configuration were made:

proxy_cache_path /storage/cache/proxy_temp/ levels=1:2 keys_zone=one:10m
inactive=100d max_size=1000M;
proxy_temp_path /storage/cache/;

location / {
proxy_pass http://mirror.oxilion.nl;
proxy_cache one;
proxy_cache_key "$scheme$host$request_uri";
proxy_cache_valid 200 100d;
}

As far as I could google, this seemed only fair. Basic caching works as
a charm, but it somehow chokes on the chkconfig rpm. However, when I use
my proxy_pass mirror (and doing so, bypass nginx), it works like a
charm. So, the mirror is up2date and up&running.

I've tried emptying my cache several times, and it doesn't matter.

This will be unable to troubleshoot without some debug. I've enabled
caching debug and traced the (requesting) HTTP headers

T 192.168.1.113:33652 -> 192.168.1.104:80 [AP]
GET /centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm
HTTP/1.1..Accept-Encoding: identity..Range: bytes=440-25087..Connection:
close..Host: 192.168.1.104..User-agent:
urlgrabber/3.1.0 yum/3.2.22....

Accesslog:

192.168.1.113 - - [13/Mar/2010:19:16:05 +0100] "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200
162614 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200
143352 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200
133216 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200
162614 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200
162614 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200
162614 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200
123238 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200
162614 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200
152040 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200
128872 "-" "urlgrabber/3.1.0 yum/3.2.22"


It tries to redownload the file several times and seems to give up after
a few failed attempts.

Cache log:

***14/Mar/2010:13:56:12 +0100 MISS Cache-Control: - Expires: - "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200)
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:12 +0100 MISS Cache-Control: - Expires: - "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200)
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:12 +0100 HIT Cache-Control: - Expires: - "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200)
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200)
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200)
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200)
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200)
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 MISS Cache-Control: - Expires: - "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200)
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200)
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - "GET
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200)
"urlgrabber/3.1.0 yum/3.2.22"

The file does exists, and downloads succesfully when I simply feed this
link to firefox or any other tool.

Anaconda (the CentOS installer) gives me a
"chkconfig-1.3.30.1-2.i386.rpm cannot be opened. This is due to a
missing file, a corrupt package, or corrupt media." in the installer
screen, with no other options than retry and reboot.

For what it's worth; I've also tried nginx-0.7.65 with a comparable
config, but to no avail.

Any hints or pointers I might have missed? I've tried several
configoptions, but none of them seem to help. If you have a comparable
setup, I'd love to see the relevant config parts.

Thanks,

Wouter

_______________________________________________
nginx mailing list
nginx@nginx.org
http://nginx.org/mailman/listinfo/nginx
Subject Author Posted

nginx as reverse caching proxy for CentOS repositories

Wouter Schoot March 14, 2010 09:08AM

Re: nginx as reverse caching proxy for CentOS repositories

Wouter Schoot March 14, 2010 04:40PM

Re: nginx as reverse caching proxy for CentOS repositories

Maxim Dounin March 14, 2010 04:56PM

Re: nginx as reverse caching proxy for CentOS repositories

Wouter Schoot March 14, 2010 05:04PM

Re: nginx as reverse caching proxy for CentOS repositories

Wouter Schoot March 18, 2010 05:10AM

Re: nginx as reverse caching proxy for CentOS repositories

Wouter Schoot March 18, 2010 02:34PM

Re: nginx as reverse caching proxy for CentOS repositories

Maxim Dounin March 18, 2010 04:56PM

Re: nginx as reverse caching proxy for CentOS repositories

Wouter Schoot March 18, 2010 05:02PM

Re: nginx as reverse caching proxy for CentOS repositories

Wouter Schoot March 19, 2010 05:10AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 226
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready