Hi Max,

In my opinion you dont want to rely on nginx to do the analytics simply to
log suspicious activity but rather need to look at a better
log analyzing solution. Have you checked out splunk? It is a very powerful
log analyzer that will allow you to more intelligently parse the logs and
has a free licence.

Rami

On Tue, Aug 9, 2011 at 5:17 PM, Maxime Ducharme <max@techboom.com> wrote:

> Hi guys
>
> We are looking for a way to detect suspicious activity on high-traffic
> websites. Parsing log files is not good option here, our current nginx
> config generates around 90G of logs for around 412K http requests each
> days.
>
> We are looking to use nginx to detect suspicious activity and generate
> precise log when it happens for post-processing.
>
> Some tools we are looking for would be something like
>
> - Detect IPs which accessed /uri1/ X times without accessing other URI
> in a period of time Y.
>
> - Detect IPs that are indexing our site by accessing sequential uris
> like /uri123, /uri124, /uri125, ...
>
> We are using load balancing services (haproxy), we enabled realip module
> in nginx, we need something that can work with it.
>
> If you have any pointers / ideas / module names that could help us,
> please let me know.
>
> Have a good day
>
> Max
>
>
>
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
>


--
Cheers,
Rami
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx