Welcome! Log In Create A New Profile

Advanced

Re: Log Parsing - Near Real Time

David Yu
August 01, 2011 10:24PM
On Tue, Aug 2, 2011 at 3:08 AM, Harold Sinclair <haroldsinclair@gmail.com>wrote:

> I cobbled something like this together with open source tools and have been
> using it on hundreds of servers.. pls contact me offline if you'd like a
> copy :)
>
Or you can share it on github to make it open to anyone.

>
> -Harold
>
>
> On Mon, Aug 1, 2011 at 2:57 PM, Dennis Jacobfeuerborn <
> dennisml@conversis.de> wrote:
>
>> An alternative is to tail -F (aka. "--follow=name --retry") the log file
>> and pipe the output into a script. This allows you to parse the entries as
>> they come in and rotate the log file as often as you want independently of
>> the parsing script.
>>
>> Regards,
>> Dennis
>>
>> On 08/01/2011 04:57 PM, Randy Parker wrote:
>>
>>> My app has a request that opens the log file, fseeks to the end, backs up
>>> as many bytes as it takes to get to the size the log file was on the last
>>> similar request by that user, and runs a regex over the novel part to get
>>> interesting metrics before closing the file. Since this happens less
>>> than
>>> once per minute, I have not done anything fancy to optimize.
>>>
>>> - Randy
>>>
>>> On Mon, Aug 1, 2011 at 10:39 AM, Reinis Rozitis <r@roze.lv
>>> <mailto:r@roze.lv>> wrote:
>>>
>>> I'm looking for a near real-time script to parse log files and
>>> insert interesting data into a db.
>>> Does anyone know of an existing script to do this?
>>>
>>>
>>> You can check/try http://www.splunk.com
>>>
>>> rr
>>>
>>>
>>> ______________________________**___________________
>>> nginx mailing list
>>> nginx@nginx.org <mailto:nginx@nginx.org>
>>> http://mailman.nginx.org/__**mailman/listinfo/nginxhttp://mailman.nginx.org/__mailman/listinfo/nginx
>>> <http://mailman.nginx.org/**mailman/listinfo/nginxhttp://mailman.nginx.org/mailman/listinfo/nginx
>>> >
>>>
>>>
>>>
>>>
>>> --
>>> http://mobiledyne.com
>>>
>>>
>>> ______________________________**_________________
>>> nginx mailing list
>>> nginx@nginx.org
>>> http://mailman.nginx.org/**mailman/listinfo/nginxhttp://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>
>> ______________________________**_________________
>> nginx mailing list
>> nginx@nginx.org
>> http://mailman.nginx.org/**mailman/listinfo/nginxhttp://mailman.nginx.org/mailman/listinfo/nginx
>>
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
>


--
When the cat is away, the mouse is alone.
- David Yu
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Log Parsing - Near Real Time

John Macleod August 01, 2011 09:56AM

Re: Log Parsing - Near Real Time

splitice August 01, 2011 10:26AM

Re: Log Parsing - Near Real Time

Reinis Rozitis August 01, 2011 10:40AM

Re: Log Parsing - Near Real Time

Randy Parker August 01, 2011 10:58AM

Re: Log Parsing - Near Real Time

Dennis Jacobfeuerborn August 01, 2011 02:58PM

Re: Log Parsing - Near Real Time

Harold Sinclair August 01, 2011 03:14PM

Re: Log Parsing - Near Real Time

splitice August 01, 2011 09:10PM

Re: Log Parsing - Near Real Time

Neil Mckee August 04, 2011 01:00PM

Re: Log Parsing - Near Real Time

Dennis Jacobfeuerborn August 04, 2011 03:18PM

Re: Log Parsing - Near Real Time

Neil Mckee August 04, 2011 08:32PM

Re: Log Parsing - Near Real Time

David Yu August 01, 2011 10:24PM

Re: Log Parsing - Near Real Time

Akins, Brian August 01, 2011 11:50AM

Re: Log Parsing - Near Real Time

Oscar Fernandez August 02, 2011 07:14AM

Re: Log Parsing - Near Real Time

Neil McKee August 01, 2011 12:26PM

Re: Log Parsing - Near Real Time

Ed W August 05, 2011 02:32AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 279
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 421 on December 02, 2018
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready