Re: [security-onion] How are people provisioning storage for SO?

[Doug Burks]

Hi jswan,

I like local disk. How much disk space are you projecting?

Doug

On Thu, Oct 11, 2012 at 6:37 PM, jswan <sanju...@gmail.com> wrote:
> I've recently deployed a standalone SO server with local disk in my extremely Windows-centric environment. That's been a big success, so when I rebuild with 12.04 I want to provision a bunch more storage on a SAN. We are mostly an iSCSI shop and my extremely Windows-centric sysadmins stare blankly when I mention NFS.
>
> iSCSI on Linux looks a bit daunting and is well outside my experience. What are other people doing? Should I press onward with iSCSI, push for NFS, or provision the whole thing on vSphere to abstract the storage problem away completely? Something else?
>
> --
>
>



--
Doug Burks
http://securityonion.blogspot.com

[KenFury]

We went with a Dell 2950 8 core box with 8 gig ram and used all 6 bays for 2 Tb sata drives in a raid 10 giving us 6Tb useable mounted to /nsm and then slapped a 110GB ssd on the pcie for /.  Server is 300 on eBay and drives were in the 700 range. It watches a 200 mbit line just fine.  YMMV.

--

[Martin Holste]

ELSA will use about 1 TB per 500 million logs indexed. So, it really
depends on how many logs/sec you'll be recording to know if you need
to use the box with the big disk. Since ELSA can work in a
distributed way, if you can load-balance your Bro logs across all
three servers and get a big performance boost at the cost of having to
deal with boxes with smaller log space. How many Bro logs/sec are you
currently recording?

On Thu, Oct 18, 2012 at 3:48 PM, jswan <sanju...@gmail.com> wrote:
> Hi Doug,
>
> Right now I have 1.3 TB of usable space on my SO 10.04 installation and I have room for 2-3 days of full pcaps and a few more days of Bro logs. If ELSA works out well for us in 12.04 (and I'm sure it will), I'd like to store Bro logs for months if possible.
>
> My main issue right now is that I just got 3 servers with decent RAM/CPU specs handed off to me, but they only have two drive bays each. What configuration would you recommend here?
>
> Thanks,
> Jay

[Martin Holste]

I'd recommending running a simple mirror of RAID 0 if you only have
two drives, because I've lost a lot of drives in the last year, and
I'd prefer not to rebuild a server unless I have to. That said, I
wouldn't fault anyone for going RAID 1 and just hoping for the best.
If you run RAID 0, it's kind of like one drive dies every week from a
data retention standpoint, because you're always running at half
capacity. If RAID 0 won't even get you 24 hours of data, then it's
probably going to be necessary to go RAID 1 to make the build even
worth it.

One other thing: Have you considered mapping to a Windows share via
Samba instead of NFS?

On Fri, Oct 19, 2012 at 4:06 PM, jswan <sanju...@gmail.com> wrote:
> Right now I'm averaging around 50/sec over the course of a day. So round up to 5M per day, that gives me 100 days per TB, which is great.
>
> What disk configuration would the more experienced users recommend with 3 x 2-bay servers available? Right now the links I'm monitoring average only around 50-75 Mbps combined, but I'd like to add to that until I start pushing the capabilities of the hardware.
>
> Jay

>
>
> On Thursday, October 18, 2012 4:11:48 PM UTC-6, Martin wrote:
>> ELSA will use about 1 TB per 500 million logs indexed. So, it really
>>
>> depends on how many logs/sec you'll be recording to know if you need
>>
>> to use the box with the big disk. Since ELSA can work in a
>>
>> distributed way, if you can load-balance your Bro logs across all
>>
>> three servers and get a big performance boost at the cost of having to
>>
>> deal with boxes with smaller log space. How many Bro logs/sec are you
>>
>> currently recording?
>>
>>
>

> --
>
>

[Doug Burks]

Mirroring is RAID 1, right?
Doug

[Martin Holste]

Yes, sorry, flip 1 and 0 for the RAID talk.

> --
>
>