Elastix Search/Logstach intgration

[tbaror]

Hi Group,

Is there any future plan to integrate Elastix search + Logstach
http://www.elasticsearch.org/ http://logstash.net/

I worked with it recently on little project we made those tools are very powerful , really would like to see it integrated with Security Onion
Thanks

[Doug Burks]

Hi tbaror,

Unfortunately, we don't have the manpower to maintain and support an ElasticSearch integration right now. Please consider using ELSA instead since it's already integrated. 

--
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.
Visit this group at http://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

--
Doug Burks
Need Security Onion Training or Commercial Support?
http://securityonionsolutions.com

[Scott Ellis]

Tharor, I see this is an old message but it predates my intro to SO. We've been talking about how to shove all logs into Elastic Search as well. is this an area where you would want to contribute the tools you've developed? I feel there is value in searching through the content of all pcaps, and so far I am not seeing away to do that (GREP?) inside of ELSA or any of the other tools packaged with SO. I have some experience with managing git projects in the past, maybe (if you have some code you want to share) we could ask Doug to give us a branch or SO to start developing something? Let me know. Doug, what do you think (I'm assuming you will see this...let me know if you don't. You can best to do this by not saying anything ;)

......s

[Doug Burks]

Hi Scott,

Can you provide more information about what exactly you want to search
and how you want to search it? Perhaps we can come up with a way to
do that without resorting to Elastic Search.

[Scott Ellis]

Hi Doug,

I'd like to be able to conduct string searches against the content of pcap files. For example, maybe I want go search all ftp traffic for a certain file name, or perhaps I want to search against all html for particular keywords.

Thanks, I appreciate your thoughts. I began putting together a shell script today that would leverage tShark to do this as an interim solution, but hadn't gotten much further than a basic script.

Sent from my iPhone

> You received this message because you are subscribed to a topic in the Google Groups "security-onion" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/security-onion/fAn5eoJYfFY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to security-onio...@googlegroups.com.

[Doug Burks]

Replies inline.

On Tue, Nov 24, 2015 at 10:10 PM, Scott Ellis <scor...@gmail.com> wrote:
> Hi Doug,
>
> I'd like to be able to conduct string searches against the content of pcap files.

If you were to "shove all logs into Elastic Search", you wouldn't be
able to do this any easier than you can today in ELSA.

> For example, maybe I want go search all ftp traffic for a certain file name,

Have you tried the ELSA query "FTP - Top Arg"?

> or perhaps I want to search against all html for particular keywords.

You can easily search the HTTP URI in ELSA today. If you're looking
for particular keywords in the raw HTML content, you may want to
consider putting those into Snort rules so that alerts are generated
whenever they are seen. For retrospective analysis of raw HTML
content, you may want to consider running ngrep across the raw pcaps.

[Scott Ellis]

Thanks for the great insight Doug.

Well, perhaps "shove" wasn't the best choice of words. Maybe a better word would have been finesse.

You can either "shove" or "finesse" when it comes to pcap files and ES. I haven't worked with it myself, but we have a team that has - they pointed me to this link:

https://discuss.elastic.co/t/reading-packets-from-pcap-file/26278/4

I envision retroactive needs to do forensic searches, so rules in Snort won't help me capture things for searches I don't yet know I need to do.

Thanks, and have a great Thaksgiving!

[Doug Burks]

I'm looking at the PacketBeat online demo right now and it appears to
provide similar data to what we already have with the Bro logs going
into ELSA. I can see high level details like HTTP status code, URI,
etc., but it doesn't look like PacketBeat allows you to search through
the full payload of the HTTP transactions.

> You received this message because you are subscribed to the Google Groups "security-onion" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.

> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.

[Scott Ellis]

Elastic can consume entire pcaps as well, it's just slow (so I'm told, I don't know how slow).

[Bamm Visscher]

It sounds like you are wanting Moloch (https://github.com/aol/moloch).

-Bamm

[Doug Burks]

Hi Bamm,

Does Moloch index the entire payload? It looks like it only allows
you to search the first 8 bytes of the payload.

[Bamm Visscher]

Hi Doug,

It looks like you are right. It does appear to have some type of protocol decoding/indexing, but not pure packet to ES. 

Bamm

sguil - The Analyst Console for NSM
http://www.sguil.net