Welcome! Log In Create A New Profile

Advanced

Re: Monitoring http returns

April 11, 2018 09:06PM
Just to be clear, I’m not contrasting active synthetic testing with monitoring resource consumption. I think that the highest value variable is $, or those variables that have highest correlation to profit. The real customer experience is probably #2 after sales. Monitoring things like active connections, cache hit ratios etc is important to understand “what is normal?” It’s easy for our mental model of how a site works to differ markedly from reality.

Sent from my iPhone

> On Apr 11, 2018, at 2:04 AM, Jeff Abrahamson <jeff@p27.eu> wrote:
>
>> On Wed, Apr 11, 2018 at 01:17:14AM -0400, Peter Booth wrote:
>> There are some very good reasons for doing things in what sounds
>> like a heavy inefficient manner.
>
> I suspected, thanks for the explanations.
>
>
>> The first point is that there are some big differences between
>> application code /business logic and monitoring code:
>>
>> [...]
>
> good summary, I agree with you.
>
>
>> tailing a log file doesnt sound sexy, but its also pretty hard to
>> mess it up. I monitored a high traffic email site with a very short
>> Ruby script that would tail an nginx log, pushing messages ten at a
>> time as UDP datagrams to an influxdb. The script would do its thing
>> for 15 mins then die. cron ensured a new instance started every 15
>> minutes. It was more efficient than a shell script because it didn't
>> start new processes in a pipeline.
>
> It's hard to mess up as long as you're not interested in
> exactly-once. ;-)
>
> The tail solution has the particularity that (1) it could miss things
> if the short gap between process death and process start sees more
> events than tail catches at startup or if the log file rotates a few
> seconds into that 15 minute period, and (2) it could duplicate things
> in case of very few events in that period. Now, with telegraf/influx,
> duplicates aren't a concern, because influx keys on time, and our site
> is probably not getting so much traffic that a tail restart is a big
> deal, although log rotation could lead to gaps we don't like.
>
> Of course, this is why Logwatch was written...
>
>
>> I like the scalar guide but I disagree with their advice on active
>> monitoring I think its smarter to use real user requests to test if
>> servers are up. i have seen many high profile sites that end up
>> serving more synthetic requests than real customer initiated
>> requests.
>
> I'm not sure I understood what you mean by "active monitoring". I've
> understood "sending http queries to see if they are handled properly".
>
> In that context: I think both submitting queries (from outside one's
> own network) and passively watching stats on the service itself are
> essential. Passively watching stats gives me information on internal
> state, useful in itself but also when debugging problems. Active
> monitoring from a different network can alert me to problems that may
> not be specific to any one service, maybe even are at the network
> level.
>
> Of course, yes, active monitoring shouldn't be trying to DoS my
> service. ;-)
>
> Jeff Abrahamson
> https://www.p27.eu/jeff/
>
>
>> On 11 Apr 2018, at 12:19 AM, Jeff Abrahamson <jeff@p27.eu> wrote:
>>
>> I want to monitor nginx better: http returns (e.g., how many
>> 500's, how many 404's, how many 200's, etc.), as well as request
>> rates, response times, etc. All the solutions I've found start
>> with "set up something to watch and parse your logs, then ..."
>>
>> Here's one of the better examples of that:
>>
>> https://www.scalyr.com/community/guides/how-to-monitor-nginx-the-essential-guide
>>
>> Perhaps I'm wrong to find this curious. It seems somewhat heavy
>> and inefficient to put this functionality into log watching,
>> which means another service and being sensitive to an eventual
>> change in log format.
>>
>> Is this, indeed, the recommended solution?
>>
>> And, for my better understanding, can anyone explain why this
>> makes more sense than native nginx support of sending UDP
>> packets to a monitor collector (in our case, telegraf)?
>>
>> --
>>
>> Jeff Abrahamson
>> +33 6 24 40 01 57
>> +44 7920 594 255
>>
>> http://p27.eu/jeff/
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
Subject Author Posted

Monitoring http returns

Jeff Abrahamson April 11, 2018 12:22AM

Re: Monitoring http returns

Frank Liu April 11, 2018 12:52AM

Re: Monitoring http returns

pbooth April 11, 2018 01:18AM

Re: Monitoring http returns

Jeff Abrahamson April 11, 2018 02:06AM

Re: Monitoring http returns

pbooth April 11, 2018 09:06PM

Re: Monitoring http returns

pbooth April 11, 2018 09:12PM

Re: Monitoring http returns

oscaretu April 13, 2018 12:14PM

Re: Monitoring http returns

Liu Lantao April 26, 2018 08:30AM



Sorry, only registered users may post in this forum.

Click here to login

Online Users

Guests: 129
Record Number of Users: 8 on April 13, 2023
Record Number of Guests: 500 on July 15, 2024
Powered by nginx      Powered by FreeBSD      PHP Powered      Powered by MariaDB      ipv6 ready