> I hope this is an appropriate forum for this kind of question, if not
> please rebuke away.
> Does anyone here use splunk? If so, I'm particularly interested in
> how you make use of it (as a fancy log grepper, as a visualization
> tool, etc).
We've been using it for both searching/grepping through logs, as well as visualizing data in them.
Its also been pretty helpful to use it for alerting based on specific patterns in the logs, and generating nightly or weekly reports with statistics on our logged events.
On Fri, Nov 6, 2009 at 9:41 AM, Steve Conover <scono...@gmail.com> wrote:
> I hope this is an appropriate forum for this kind of question, if not
> please rebuke away.
> Does anyone here use splunk? If so, I'm particularly interested in
> how you make use of it (as a fancy log grepper, as a visualization
> tool, etc).
> Regards,
> Steve
We're still demoing it. So far it's been awesome for troubleshooting
by grepping logs and event-based alerts.
No visualization or statistics, yet.
My biggest problem is remembering to use it: you spend a few years
grepping a log file from terminal it's hard to remember in a crunch to
switch gears .
We've found it to be quite helpful in exposing production log data to other
groups in the organization that wouldn't otherwise have production level
access.
On Fri, Nov 6, 2009 at 2:28 PM, Brian Dunbar <brian.dun...@gmail.com> wrote:
> On Fri, Nov 6, 2009 at 9:41 AM, Steve Conover <scono...@gmail.com> wrote:
> > I hope this is an appropriate forum for this kind of question, if not
> > please rebuke away.
> > Does anyone here use splunk? If so, I'm particularly interested in
> > how you make use of it (as a fancy log grepper, as a visualization
> > tool, etc).
> > Regards,
> > Steve
> We're still demoing it. So far it's been awesome for troubleshooting
> by grepping logs and event-based alerts.
> No visualization or statistics, yet.
> My biggest problem is remembering to use it: you spend a few years
> grepping a log file from terminal it's hard to remember in a crunch to
> switch gears .
> --
> Brian Dunbar
> Geidus
> "Display some adaptability"
> --
> You received this message because you are subscribed to the Google Groups
> "Agile System Administration" group.
> To post to this group, send email to
> agile-system-administration@googlegroups.com.
> To unsubscribe from this group, send email to
> agile-system-administration+unsubscribe@googlegroups.com<agile-system-admin istration%2Bunsubscribe@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/agile-system-administration?hl=en.
I've used it in the past to correlate log files together within an identity and access mgt project:
we had syslog files, but also application logs, and network logs, firewall logs, database logs.
If we would have put everything within the database , the schema would have to be adapted constantly because of the
different file formats. Splunk allowed us to store this in one central repository, but still define different field by specifying formatters/parsers of each format.
in this way we could easily slurp in all different/custom log formats every legacy application produced instead of writing custom agents.
biggest bummer for us, was that the license depends on the daily volume you can process with it, which could grow a lot if you need to do archiving of it.
Steve Conover wrote:
> I hope this is an appropriate forum for this kind of question, if not
> please rebuke away.
> Does anyone here use splunk? If so, I'm particularly interested in
> how you make use of it (as a fancy log grepper, as a visualization
> tool, etc).
> Regards,
> Steve
> --
> You received this message because you are subscribed to the Google Groups "Agile System Administration" group.
> To post to this group, send email to agile-system-administration@googlegroups.com.
> To unsubscribe from this group, send email to agile-system-administration+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/agile-system-administration?hl=en.
> Steve Conover wrote:
[snip]
>> Does anyone here use splunk? If so, I'm particularly interested in
>> how you make use of it (as a fancy log grepper, as a visualization
>> tool, etc).
Patrick Debois wrote:
[snip]
> biggest bummer for us, was that the license depends on the daily volume > you can process with it, which could grow a lot if you need to do > archiving of it.
Hi,
There was a discussion about splunk! during the "tools" session of DevOpsDAys'09.
I think Patrick is right to mention the price as the biggest showstopper, as there was a consensus on the fact an OSS replacement for Splunk! was one of the dearly missed tool.
Lindsay Holmwood made mention of a prototype/QnD tool based on mysql he wrote, but I don't know if he has posted/published it yet (and functionaly it is probably quite far from splunk! anyway).
> There was a discussion about splunk! during the "tools" session of
> DevOpsDAys'09.
> I think Patrick is right to mention the price as the biggest
> showstopper, as there was a consensus on the fact an OSS replacement for
> Splunk! was one of the dearly missed tool.
> Lindsay Holmwood made mention of a prototype/QnD tool based on mysql he
> wrote, but I don't know if he has posted/published it yet (and
> functionaly it is probably quite far from splunk! anyway).
So what is the core value of splunk that we'd want in an OSS tool (or
set of tools)?
Thinking about the problem I think it falls into several components:
*) Host based Log Collection (live tailing, etc)
*) Log Aggregation (getting it efficiently across the network)
*) Log analysis
*) Visualisation
*) Search/Query
I really like what Data Wrangling have done with wikipedia's squid
logs plus hadoop:
It is open source and uses a combination of Hive/Hadoop Streaming
(python) to do the analysis with a rails app and google visualisations
and charts for the front end.
On Sat, Nov 7, 2009 at 5:17 AM, Paul Nasrat <pnas...@googlemail.com> wrote:
> 2009/11/7 Gildas Le Nadan <3ntr0...@gmail.com>:
>> There was a discussion about splunk! during the "tools" session of
>> DevOpsDAys'09.
>> I think Patrick is right to mention the price as the biggest
>> showstopper, as there was a consensus on the fact an OSS replacement for
>> Splunk! was one of the dearly missed tool.
>> Lindsay Holmwood made mention of a prototype/QnD tool based on mysql he
>> wrote, but I don't know if he has posted/published it yet (and
>> functionaly it is probably quite far from splunk! anyway).
> So what is the core value of splunk that we'd want in an OSS tool (or
> set of tools)?
> Thinking about the problem I think it falls into several components:
> *) Host based Log Collection (live tailing, etc)
> *) Log Aggregation (getting it efficiently across the network)
> *) Log analysis
> *) Visualisation
> *) Search/Query
> I really like what Data Wrangling have done with wikipedia's squid
> logs plus hadoop:
> It is open source and uses a combination of Hive/Hadoop Streaming
> (python) to do the analysis with a rails app and google visualisations
> and charts for the front end.
> Paul
> --
> You received this message because you are subscribed to the Google Groups "Agile System Administration" group.
> To post to this group, send email to agile-system-administration@googlegroups.com.
> To unsubscribe from this group, send email to agile-system-administration+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/agile-system-administration?hl=en.
On Sat, 2009-11-07 at 13:40 +0100, Gildas Le Nadan wrote:
> > Steve Conover wrote:
> [snip]
> >> Does anyone here use splunk? If so, I'm particularly interested in
> >> how you make use of it (as a fancy log grepper, as a visualization
> >> tool, etc).
> Patrick Debois wrote:
> [snip]
> > biggest bummer for us, was that the license depends on the daily volume > > you can process with it, which could grow a lot if you need to do > > archiving of it.
> Hi,
> There was a discussion about splunk! during the "tools" session of > DevOpsDAys'09.
So I missed that session .. and it has been ages since I looked at
Splunk, to me it looked like a log parser..
Would Rivermuse as an eventhandling platform be a potential
replacement ?
> I think Patrick is right to mention the price as the biggest > showstopper, as there was a consensus on the fact an OSS replacement for > Splunk! was one of the dearly missed tool.
> Lindsay Holmwood made mention of a prototype/QnD tool based on mysql he > wrote, but I don't know if he has posted/published it yet (and > functionaly it is probably quite far from splunk! anyway).
> Cheers,
> Gildas
> --
> You received this message because you are subscribed to the Google Groups "Agile System Administration" group.
> To post to this group, send email to agile-system-administration@googlegroups.com.
> To unsubscribe from this group, send email to agile-system-administration+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/agile-system-administration?hl=en.
|
