Igor Sysoev wrote:
> On Sat, Aug 29, 2009 at 01:58:11PM -0400, Jim Ohlstein wrote:
>
>
>> Igor Sysoev wrote:
>>
>>> On Sat, Aug 29, 2009 at 11:41:16AM -0400, Jim Ohlstein wrote:
>>>
>>>
>>>
>>>> We're dealing with a high degree of fraud from certain countries and
>>>> would like to simply ban all IP's from those countries.
>>>>
>>>> I seem to recall reading here that using the Geo module is more
>>>> efficient for this purpose than the GeoIP module.
>>>>
>>>> Currently I have the following in nginx.conf:
>>>>
>>>> geo $country {
>>>> include geo.conf;
>>>> }
>>>>
>>>> where geo.conf is generated from MaxMind country lite csv database using
>>>> geo2nginx.pl supplied with nginx.
>>>>
>>>> In the site config I have multiple if statements like:
>>>>
>>>> server {
>>>> ...
>>>>
>>>> if ($country = XX) {
>>>> return 403;
>>>> }
>>>>
>>>> if ($country = YY) {
>>>> return 403;
>>>> }
>>>>
>>>> if ($country = ZZ) {
>>>> return 403;
>>>> }
>>>>
>>>> ...
>>>>
>>>> }
>>>>
>>>> Is this more efficient than using GeoIP module? Is there a more
>>>> efficient way of doing this?
>>>>
>>>>
>>> Yes:
>>>
>>> # convert MaxMind's blocks to nginx format:
>>> ./nginx.pl GeoLiteCity_20090701/GeoLiteCity-Blocks.csv > maxmind.conf
>>>
>>> # convert MaxMand location to country names:
>>> ./country.pl GeoLiteCity_20090701/GeoLiteCity-Location.csv maxmind.conf >
>>> countries.conf
>>>
>>> # set forbidden countries to 1, ignore others:
>>> perl -ne 'print "$1 1\n" if /^(\S+) (US|RU|CN|...);$/' < countries.conf >
>>> networks.conf
>>>
>>> # aggregate networks:
>>> ./compress.pl networks.conf > forbidden.conf
>>>
>>> Then use it:
>>>
>>> geo $forbidden {
>>> default 0;
>>> include forbidden.conf;
>>> }
>>>
>>> server {
>>> if ($forbidden) {
>>> return 403;
>>> }
>>>
>>>
>>> nginx.pl:
>>> -----------
>>> #!/usr/bin/perl -w
>>>
>>> use Net::CIDR::Lite;
>>> use strict;
>>> use warnings;
>>>
>>> while (<>) {
>>> if (/^"([^"]+)","([^"]+)","([^"]+)"/){
>>> my($start, $end, $region) = ($1, $2, $3);
>>> my $cidr = Net::CIDR::Lite->new(ip($start) . "-" . ip($end));
>>> print((join " $region;\n", $cidr->list), " $region;\n");
>>> }
>>> }
>>>
>>> sub ip {
>>> my $n = shift;
>>> return (($n >> 24) & 0xff) . "." .
>>> (($n >> 16) & 0xff) . "." .
>>> (($n >> 8) & 0xff) . "." .
>>> ($n & 0xff);
>>> }
>>> -----------
>>>
>>> country.pl:
>>> -----------
>>> #!/usr/bin/perl -w
>>>
>>> use warnings;
>>> use strict;
>>>
>>> my %country;
>>>
>>> while (<>) {
>>> if (/^(\d+),"([^"]+)","([^"]*)"/) {
>>> $country{$1} = $2;
>>> next;
>>> }
>>>
>>> if (/^\S+ \d+;$/) {
>>> last;
>>> }
>>> }
>>>
>>> do {
>>> if (/^(\S+) (\d+);$/) {
>>> print "$1 $country{$2};\n";
>>> } else {
>>> print STDERR;
>>> }
>>>
>>> } while (<>);
>>> -----------
>>>
>>> compress.pl:
>>> -----------
>>> #!/usr/bin/perl -w
>>>
>>> use Net::CIDR::Lite;
>>> use strict;
>>> use warnings;
>>>
>>> my %cidr;
>>>
>>> while (<>) {
>>> if (/^(\S+) (\S+);/) {
>>> my($net, $region) = ($1, $2);
>>> if (!defined $cidr{$region}) {
>>> $cidr{$region} = Net::CIDR::Lite->new;
>>> }
>>> $cidr{$region}->add($net);
>>> }
>>> }
>>>
>>> for my $region (sort { $a <=> $b } keys %cidr) {
>>> print((join " $region;\n", $cidr{$region}->list), " $region;\n");
>>> }
>>> -----------
>>>
>>>
>>>
>> Thank you Igor. Unfortunately forbidden.conf comes up as an empty file.
>> I have confirmed the three perl files are as above and I have run it twice.
>>
>> [jim@saturn geo]# ls -lh
>> total 195M
>> -rwx------ 1 jim jim 422 Aug 29 13:15 compress.pl
>> -rw-r--r-- 1 jim jim 91M Aug 29 13:49 countries.conf
>> -rwx------ 1 jim jim 334 Aug 29 13:15 country.pl
>> -rw-r--r-- 1 jim jim 0 Aug 29 13:50 forbidden.conf
>> drwxr-xr-x 2 jim jim 4.0K Aug 29 13:25 GeoLiteCity_20090801
>> -rw-r--r-- 1 jim jim 104M Aug 29 13:48 maxmind.conf
>> -rw-r--r-- 1 jim jim 156K Aug 29 13:50 networks.conf
>> -rwx------ 1 jim jim 497 Aug 29 13:14 nginx.pl
>>
>
> Sorry, I mistaked:
>
> -perl -ne 'print "$1 1\n" ...
> +perl -ne 'print "$1 1;\n" ...
>
>
>

Thank you again Igor. It's working like a charm and loads are down a bit
as well.

I'm assuming that "forbidden.conf" is loaded in memory and so if I
replace it then I need to reload nginx. Correct?


Jim