You could use nginx-sflow-module (binary logging over UDP). The "sflowtool" program can convert this back to a continuous feed of common log file format at the collector.

This way you can receive from multiple servers all sending to the same UDP port. You can also apply random 1-in-N sampling at source as an efficient data-reduction measure if required.

Neil Mckee



On Aug 1, 2011, at 6:53 AM, John Macleod <jcdmacleod@gmail.com> wrote:

> I'm looking for a near real-time script to parse log files and insert interesting data into a db.
>
> Does anyone know of an existing script to do this?
>
> John
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx