Security onion on a VM

[Chris Teodorski]

Doug and everyone else,

I'm looking at deploying SO on a VM in a fairly high throughput network.  I know the specs on the SO site say:

If you're deploying Security Onion in production to a large network (500Mbps - 1000Mbps), you should plan on 128GB - 256GB RAM or more.

Has anyone here deployed a virtual security onion on a production network of that size?  Am I foolish to even consider it?  

Chris

[Lee Sharp]

I have an install running on a network with a sustained traffic of about
75Mbps. It is on a Core i7 with 32 gig of ram and a single hard drive,
capturing through 3 different Intel based nics. CPU is never above 50%
overall, and memory is not fully utilized. The hard drive is a bit
busy, but not bad. I suspect hard drive will be the first bottleneck I
will see on this system.

My problem is that the default rules are so tight that turning it on
slams the system with alerts faster than I can clear them. :) You may
end up installing a few times to get a good disable set before it is usable.

Now, what is your actual sustained traffic? Is it only one interface or
more than one?

Lee

[Doug Burks]

Hi Chris,

If you are averaging 500Mbps - 1000Mbps traffic and you want to run
our recommended stack of Snort/Suricata, Bro, and netsniff-ng, then
I'd highly recommend using bare metal instead of virtual.

> --
> You received this message because you are subscribed to the Google Groups
> "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks
Need Security Onion Training or Commercial Support?
http://securityonionsolutions.com

[Chris Teodorski]

Doug,

That is what I was thinking/expecting.  Any idea where you would start to have problem with virtuals?   Have any concept of a theoretical limit?

Thanks,

Chris

[Doug Burks]

I don't have any data to give you, as I always use physical machines
for deployments. Perhaps others will chime in with their experiences.

On Wed, Oct 8, 2014 at 10:22 AM, Chris Teodorski

[Chris Teodorski]

Thanks again Doug.  As always your level of support of this project amazes me.  

I would love to hear about others experience using security onion on virtual machines in a production environment.  In the particular environment I'm looking at physicals are not a possibility so this question is make or break for us regarding whether we use security onion.

Chris

[C. L. Martinez]

On Wed, Oct 8, 2014 at 2:37 PM, Chris Teodorski
<chris.t...@gmail.com> wrote:
> Thanks again Doug. As always your level of support of this project amazes
> me.
>
> I would love to hear about others experience using security onion on virtual
> machines in a production environment. In the particular environment I'm
> looking at physicals are not a possibility so this question is make or break
> for us regarding whether we use security onion.
>
> Chris

I have tried to deploy some suricata, snort and Bro-IDS sensors under
VMware vSphere and KVM (CentOS 6.5) platforms ... and results are not
good ... I loose packets ever ... But, if you can isolate what type of
traffic you would monitor, maybe you can reach good results. In my
case, I could get good results just by monitoring outgoing traffic of
a proxy server (0,6% of dropped packets).

Bye.

[Chris Teodorski]

Thanks for the response.  Any idea how much traffic the security onion was seeing?  Also when it was dropping packets like crazy are we talking 25% packet loss or 75% packet loss?

Chris

[C. L. Martinez]

There aren't SO vms ... I used FreeBSD (with netmap) and CentOS 6.x
(with pf_ring) vms ... And I try to monitor 1gb traffic only. Without
using bpf filter, I loss more than 80% packets in most cases.

On Wed, Oct 8, 2014 at 2:47 PM, Chris Teodorski

[Lee Sharp]

On 10/08/2014 09:22 AM, Chris Teodorski wrote:
> Doug,
>
> That is what I was thinking/expecting. Any idea where you would start
> to have problem with virtuals? Have any concept of a theoretical limit?

"A VM" is a poorly defined concept. With some server boards, and a good
NAS you can make a VM more powerful than most stand alone systems.
However...

The big problem is the nic. Sniffing a busy wire is not trivial, and on
a VM server you have several layers of abstraction. The VM server nic.
The hypervisor nic driver. The hypervisor nic passthrough as a
virtual device. The guest nic driver... And nic themselves have issues
as well. Capture 100 Mbps on a Realtek and on an Intel nic and compare
CPU loads... Now, you can do direct hardware pass-through so that
Security Onion has direct access to the nic hardware. And you would
need a STRONG VM server to allow you to give it a stupid amount of
resources. Better have an amazing disk array with a ton of iops as
well. And since you are doing hardware passthrough, you can forget
fault tolerance and vmotion on this one. :)

It may be cheaper to just buy a decent server. :) The system I
described in my prior post was just $750.

Lee

[Greg Williams]

We used to monitor our infrastructure on ESXi. Traffic was roughly 500-800 Mbps. The bottleneck really wasn't the RAM (52GB), it was the number of cores dedicated to it. We only had Bro and Snort running and packet loss was always <1%. I had 16 cores dedicated to it, but it they would hover around 90%. I would recommend that you make sure you have enough cores. Additionally we used direct passthrough NICs to make sure the virtual NICs didn't create a bottleneck. We switched to baremetal because of a network redesign. As long as you have the cores you need, and passthrough NICs, I don't see a problem with a virtual infrastructure.