Netflow monitoring

[JB]

I have just had a trial of Scrutinizer and it is amazing how much network insight you get and the alarms. But naturally for that wonderful piece of software they want their pound of flesh. Which is way outside of budget. What are you guys using to monitor your netflow data and troubleshoot problems?

[Larry Sampas]

On Monday, November 10, 2014 11:05:13 AM UTC-5, JB wrote:
> I have just had a trial of Scrutinizer and it is amazing how much network insight you get and the alarms. But naturally for that wonderful piece of software they want their pound of flesh. Which is way outside of budget. What are you guys using to monitor your netflow data and troubleshoot problems?

SiLK and Yaf, following Chris Sanders and Jason Smith's methods. (http://www.appliednsm.com/silk-on-security-onion/ -- I have their book, too.)

The nice thing about Yaf is that I can give something back to the network team after inserting our devices into their critical links. I a Yaf process to collect flows into Silk locally and another to send IPFIX to our network team's tool.

With the recent release of Flowbat (http://www.flowbat.com/), you can do charts and the queries get easier in SiLK. However, learning the SiLK queries (rwcut and rwfilter) forces you to think about exactly what you're looking for instead of relying on the pretty charts that Plixer sells.

I also use Zabbix for measuring bandwidth utilization on particular sensor interfaces. Zabbix can alert when thresholds are exceeded, and then you can drill in to the details with SiLK.

-- Larry

[Giles Coochey]

On 10/11/2014 16:05, JB wrote:
> I have just had a trial of Scrutinizer and it is amazing how much network insight you get and the alarms. But naturally for that wonderful piece of software they want their pound of flesh. Which is way outside of budget. What are you guys using to monitor your netflow data and troubleshoot problems?
>

Not part of security onion, or nsm, but I use nfdump via the nfsen web
frontend.

--
Regards,

Giles Coochey, CCNP, CCNA, CCNAS
NetSecSpec Ltd
+44 (0) 8444 780677
+44 (0) 7584 634135
http://www.coochey.net
http://www.netsecspec.co.uk
gi...@coochey.net

[Doug Burks]

Hi JB,

One option would be to collect your netflow data with either argus or
logstash and then store it in ELSA so it can be searched alongside
your other logs.

On Mon, Nov 10, 2014 at 11:05 AM, JB <jonbrown...@gmail.com> wrote:
> I have just had a trial of Scrutinizer and it is amazing how much network insight you get and the alarms. But naturally for that wonderful piece of software they want their pound of flesh. Which is way outside of budget. What are you guys using to monitor your netflow data and troubleshoot problems?
>

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks
Need Security Onion Training or Commercial Support?
http://securityonionsolutions.com

[JB]

Ok This is a dumb question and I'm sure I will show my ignorance. I plan on running PFsense to push netflow along with my switches to my collectors. Can I have netflow go to Elsa/Argus/Bro and go to either Silk with Flowbat or Nfsen?

Bro will work as an IDS based on the flow data will Argus do this as well? Basically I would like to use netflow for troubleshooting and seeing things that are off but have SO use either argus or bro as an IDS on anything I maybe missing.

[jswan]

On Monday, November 10, 2014 11:58:04 AM UTC-7, JB wrote:
> Ok This is a dumb question and I'm sure I will show my ignorance. I plan on running PFsense to push netflow along with my switches to my collectors. Can I have netflow go to Elsa/Argus/Bro and go to either Silk with Flowbat or Nfsen?
>

Bro already produces "flow-like" data natively with the conn log. If you want to export NetFlow from your routers and switches and have that searchable in ELSA, you'll need to come up with a way to ingest NetFlow data into ELSA. There is no built-in method for doing this in Security Onion.

The most easily accessible option would probably be to send your NetFlow data to Argus or Logstash, then send that to ELSA via syslog. This will require some plumbing on your part; it's not built into SO. Either one is going to increase your resource utilization, so be prepared.

If you can live with having your data outside ELSA, SiLK with FlowBAT is the most versatile turnkey solution that I've seen in the free/open source world. There are many other options, but FlowBAT is really nice.

> Bro will work as an IDS based on the flow data will Argus do this as well? Basically I would like to use netflow for troubleshooting and seeing things that are off but have SO use either argus or bro as an IDS on anything I maybe missing.

I'm not entirely sure what you mean by "work as an IDS", but if you want to create notices or get alerts from ELSA, the easiest way to do that would be to work with the existing tools built into SO.

[Jonathan Brown]

You are right I wanted to get my NetFlow into Elsa and hopefully Bro/Argus could detect potential problems like scrutinizer does with alarms for botnets. Has can be seen here under FA section http://www.plixer.com/manual/!SSL!/WebHelp/flow_analytics/flowanalytics.html

Goal is ELSA could show me when these type of issues arise and or Squil. Hoping for additional insight into the network.

--
You received this message because you are subscribed to a topic in the Google Groups "security-onion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/security-onion/NfKMYqcMgYs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to security-onio...@googlegroups.com.

[Jay Swan]

We run Plixer Scrutinizer, so I'm familiar with how their stuff works. It's a great product for pure NetFlow/IPFIX analytics. As an aside, their professional services team can actually integrate Bro logs into the product for you, but this isn't something we've pursued.

For "TopN" style reporting (top N IPs, top N domains, top N websites, etc): most of that stuff is already built into the canned ELSA reports that you see when you log into ELSA.

For IP reputation tracking: the best way to do this is to get your IP reputation feed into the Bro Intel framework, and set up Bro notices for alerting if you need to. Search the mailing list archive for info on this, or check out the Bro channel on YouTube. There are also a bunch of IP reputation feeds and IP-based botnet signatures included in the ET and VRT Snort signature packages. These really increase the compute resource overhead for Snort, though, so putting them into the Bro intel framework is better if you can do it.

In general, I find that between Bro logs, Snort alerts, and full packet capture, Security Onion gives me much better situational awareness and a much better ability to distinguish false and true positives than I do with Plixer flow analytics. The problem with pure flow-based solutions is that you usually have insufficient context to distinguish false and true positives.

The places that pure NetFlow solutions shine (and I think Scrutinizer is among the best) are:

* giving visibility into parts of the network where you don't have SO sensors
* capacity planning
* visualization
* traffic analysis (because of NetFlow's awareness of interfaces, subnets, and routing metadata such as nexthop and BGP ASN)
* dealing with very large volumes of data