splunk?

[Steve Conover]

I hope this is an appropriate forum for this kind of question, if not
please rebuke away.

Does anyone here use splunk? If so, I'm particularly interested in
how you make use of it (as a fancy log grepper, as a visualization
tool, etc).

Regards,
Steve

[Ilan Rabinovitch]

We've been using it for both searching/grepping through logs, as well as
visualizing data in them.
Its also been pretty helpful to use it for alerting based on specific
patterns in the logs, and generating nightly or weekly reports with
statistics on our logged events.

[Brian Dunbar]

We're still demoing it. So far it's been awesome for troubleshooting
by grepping logs and event-based alerts.

No visualization or statistics, yet.

My biggest problem is remembering to use it: you spend a few years
grepping a log file from terminal it's hard to remember in a crunch to
switch gears .

--
Brian Dunbar
Geidus

"Display some adaptability"

[Allen Bettilyon]

We've found it to be quite helpful in exposing production log data to other groups in the organization that wouldn't otherwise have production level access.
 

--

You received this message because you are subscribed to the Google Groups "Agile System Administration" group.
To post to this group, send email to agile-system-...@googlegroups.com.
To unsubscribe from this group, send email to agile-system-admini...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/agile-system-administration?hl=en.

--
/ab

[Patrick Debois]

I've used it in the past to correlate log files together within an
identity and access mgt project:
we had syslog files, but also application logs, and network logs,
firewall logs, database logs.

If we would have put everything within the database , the schema would
have to be adapted constantly because of the
different file formats. Splunk allowed us to store this in one central
repository, but still define different field by specifying
formatters/parsers of each format.
in this way we could easily slurp in all different/custom log formats
every legacy application produced instead of writing custom agents.

biggest bummer for us, was that the license depends on the daily volume
you can process with it, which could grow a lot if you need to do
archiving of it.

[Gildas Le Nadan]

> Steve Conover wrote:
[snip]

>> Does anyone here use splunk? If so, I'm particularly interested in
>> how you make use of it (as a fancy log grepper, as a visualization
>> tool, etc).

Patrick Debois wrote:
[snip]

> biggest bummer for us, was that the license depends on the daily volume
> you can process with it, which could grow a lot if you need to do
> archiving of it.

Hi,

There was a discussion about splunk! during the "tools" session of
DevOpsDAys'09.

I think Patrick is right to mention the price as the biggest
showstopper, as there was a consensus on the fact an OSS replacement for
Splunk! was one of the dearly missed tool.

Lindsay Holmwood made mention of a prototype/QnD tool based on mysql he
wrote, but I don't know if he has posted/published it yet (and
functionaly it is probably quite far from splunk! anyway).

Cheers,
Gildas

[Paul Nasrat]

2009/11/7 Gildas Le Nadan <3ntr...@gmail.com>:

> There was a discussion about splunk! during the "tools" session of
> DevOpsDAys'09.
>
> I think Patrick is right to mention the price as the biggest
> showstopper, as there was a consensus on the fact an OSS replacement for
> Splunk! was one of the dearly missed tool.
>
> Lindsay Holmwood made mention of a prototype/QnD tool based on mysql he
> wrote, but I don't know if he has posted/published it yet (and
> functionaly it is probably quite far from splunk! anyway).

So what is the core value of splunk that we'd want in an OSS tool (or
set of tools)?

Thinking about the problem I think it falls into several components:

*) Host based Log Collection (live tailing, etc)
*) Log Aggregation (getting it efficiently across the network)
*) Log analysis
*) Visualisation
*) Search/Query

I really like what Data Wrangling have done with wikipedia's squid
logs plus hadoop:

http://www.trendingtopics.org/

It is open source and uses a combination of Hive/Hadoop Streaming
(python) to do the analysis with a rails app and google visualisations
and charts for the front end.

Paul

[Steve Conover]

Thanks for everyone's responses, this was very helpful.

-Steve

[Kris Buytaert]

On Sat, 2009-11-07 at 13:40 +0100, Gildas Le Nadan wrote:
> > Steve Conover wrote:
> [snip]
> >> Does anyone here use splunk? If so, I'm particularly interested in
> >> how you make use of it (as a fancy log grepper, as a visualization
> >> tool, etc).
> Patrick Debois wrote:
> [snip]
> > biggest bummer for us, was that the license depends on the daily volume
> > you can process with it, which could grow a lot if you need to do
> > archiving of it.
>
> Hi,
>
> There was a discussion about splunk! during the "tools" session of
> DevOpsDAys'09.
>

So I missed that session .. and it has been ages since I looked at
Splunk, to me it looked like a log parser..

Would Rivermuse as an eventhandling platform be a potential
replacement ?

greetings

Kris

> I think Patrick is right to mention the price as the biggest
> showstopper, as there was a consensus on the fact an OSS replacement for
> Splunk! was one of the dearly missed tool.
>
> Lindsay Holmwood made mention of a prototype/QnD tool based on mysql he
> wrote, but I don't know if he has posted/published it yet (and
> functionaly it is probably quite far from splunk! anyway).
>
> Cheers,
> Gildas
>

[Gildas Le Nadan]

Hi Paul,

I've been willing to answer this mail for a long time now...

Paul Nasrat wrote:
> 2009/11/7 Gildas Le Nadan <3ntr...@gmail.com>:
>
>> There was a discussion about splunk! during the "tools" session of
>> DevOpsDAys'09.
>>
>> I think Patrick is right to mention the price as the biggest
>> showstopper, as there was a consensus on the fact an OSS replacement for
>> Splunk! was one of the dearly missed tool.
>>
>> Lindsay Holmwood made mention of a prototype/QnD tool based on mysql he
>> wrote, but I don't know if he has posted/published it yet (and
>> functionaly it is probably quite far from splunk! anyway).
>
> So what is the core value of splunk that we'd want in an OSS tool (or
> set of tools)?
>
> Thinking about the problem I think it falls into several components:
>
> *) Host based Log Collection (live tailing, etc)
> *) Log Aggregation (getting it efficiently across the network)

I think the actual status quo on this is remote logging via syslog-ng
and writing logs on nfs volumes. Would you say this is ok?

> *) Log analysis
> *) Visualisation
> *) Search/Query

I would add trending. I find it really valuable to know what is the
"normal rate of errors" in application logs for instance. It allows you
to avoid red herrings when you have a real problem, hence a smaller Time
To Diagnose.

It must be said that the system probably need to allow access to non
technical/non admin people to the "live" dataset. And you want to avoid
DoS (think of stupid queries on live systems).

> I really like what Data Wrangling have done with wikipedia's squid
> logs plus hadoop:
>
> http://www.trendingtopics.org/
>
> It is open source and uses a combination of Hive/Hadoop Streaming
> (python) to do the analysis with a rails app and google visualisations
> and charts for the front end.
>
> Paul

Indeed the Data Wrangling solution is neat. It uses "off line" datas though.

It seems there is a new challenger:
http://www.roadtofailure.com/2010/01/25/logging-unsexy-important-and-now-usable/
(thanks @fs111 for the link).

The plus sides: it's shared nothing so it should scale linearly.

Gildas