Hello,

On Fri, 29 Dec 2023 09:54:30 -0300
Rejaine Monteiro <rejaine@bhz.jamef.com.br> wrote:

> Hi all,
>
> I´m running Nginx community edition and need to implement rate limiting
>
> There's plenty of guides out there on how to do this, but no guides on how
> to get real values/stats from the access logs
>
>
> What I need to get from the NGINX access logs is:
>
> - Requests per minute, per IP address
>
> - Average requests per minute, derived from all IP addresses
>
> - Max requests per minute, per IP address
>
> We have a few endpoints with different functionalities and we simply cannot
> have a common rule that works for everyone.
>
> Any tips on a tool or script that could help generate this type of
> information (if possible in real time or collect this data for future
> analysis)?
>
>
> I appreciate any tips.
>

There isn't an existing bespoke tool for this (at least publicly
available).

Normally such metrics are generated by:
A) Feeding access logs into a log aggregator platform (Splunk, Loki)
B) Performing / creating custom queries on that platform, to generate
such reports.

Of note, Loki (which is AGPL/free) has a nice user experience for
this.

https://grafana.com/docs/loki/latest/query/metric_queries/

Other than log aggregators, writing a script (python, perl, bash) is
likely the fastest approach. Consider sharing it if you do, I'm sure
others will find it useful.

If you're looking for existing scripts as a starting point, there may
be similar tools for Apache that you could adapt for nginx. Both use
the 'Common Log Format' for access logs by default.

https://github.com/ajohnstone/apache-log-stats

Something like this.
_______________________________________________
nginx mailing list
nginx@nginx.org
https://mailman.nginx.org/mailman/listinfo/nginx