Backup/Restore Strategy for Security Onion

[Kevin Branch]

As I am near to putting my first SO system into production, I'm trying to work out a good backup and restore strategy that will accommodate a fairly quick rebuild an an SO system which preserves all configuration/customizations, and as much NSM data as I can afford to back up.  I am working with an Ubuntu 12.04 Server PPA install of SO on a physical server.

If my SO system dies and I have to completely rebuild it, what I'd like to be able to do is:

• Reinstall and update Ubuntu 12.04.

• Install packages needed by SO via apt-get from native Ubuntu sources and from the SO PPA.

• Restore /etc/network/interfaces from backup instead of running sosetup phase 1.
  

• Run sosetup phase 2 only, skipping the network config phase and configuring things the same way as was originally done.

• Shut down absolutely all SO stuff that is running.

• Restore all the files and MySQL databases/tables needed to get me back to my pre-crashed SO system state.

• Reboot and enjoy a working system again, with all my customizations intact, but minus the large data items I couldn't afford to back up.

If I back up the right files and mysqldump the right databases, would this be a feasible recovery strategy?

I'm thinking of dumping the following databases

• mysql (to get all the MySQL db users and permissions backed up)

• elsa_web

• securityonion_db (excluding the SANCP tables)

• snorby

• syslog

• NOT syslog_data which contains the huge ELSA log data

with something like this:

• mysqldump --databases mysql elsa_web securityonion_db snorby syslog --ignore-table=securityonion_db.sancp | gzip > so-backup.sql.gz

and then also backing up the following files/directories

• /nsm/   - except for /nsm/sensor_data/*/dailylogs/ and /nsm/elsa/data/

• /etc/cron.d/

• /etc/nsm/

• /etc/sphinxsearch/

• /var/ossec/

• /opt/bro/

• /opt/elsa/   

• /opt/snorby/

• /opt/xplico/

• /var/www/

• /etc/networking/interfaces

• /etc/udev/rules.d/70-persistent-net.rules

• /lib/ufw/user.rules

• /etc/syslog-ng/syslog-ng.conf

• /etc/elsa_node.conf

• /etc/elsa_web.conf

• /etc/modprobe.d/pf_ring.conf

• /etc/mysql/my.cnf

• /etc/apparmor.d/

I think this would back up enough to catch all my SO customizations, whether done at the file or web interface level, such that by restoring all these databases and files on top of a generic install of SO, I'd have my old system back, minus ELSA logs and daily pcaps.

Can you think of anything I'm missing here?  Is there an authoritative list somewhere of all the files and directories when configuration data lives on an SO system?  Would such a list be useful to the wider SO community?

Thanks in advance for any advice or commentary on this topic. 

Kevin

[Doug Burks]

Hi Kevin,

That appears to be a fairly comprehensive list! However, as with all
backups, please test and verify that this does exactly what you expect
before putting your strategy into production. If it does work as
expected, let us know and we'll add it to the Wiki.

Thanks!

> --
> You received this message because you are subscribed to the Google Groups
> "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/groups/opt_out.



--
Doug Burks

[Robert Pirner]

Hi Kevin,

any (backup) scripts you can / wanna share with the community?

Thx Robert

[Richard Barrow]

Afternoon Gents,

I am going through the same scenario for my current jobs sensors and servers, PCI requirement is for us to be able to do a full restore, and be able to pull detailed information spanning a period back into the SO deployment.

pulling up the ILO now.. backups perform will let you know how it goes.

regards Richard

[Richard Barrow]

Hey Kevin ..

We where half way through testing backups when the central server fell over
:-( The words Sod's and law come to mind

I am working through getting the central server back in place then we are
going to run the backup tests on the DR platform.

Business has now found time for the team to explore a backup strategy, I am
though thinking along the lines of "HowTo" do a N+1 production deployment.

Would appreciate the sounding board if your interested.

Regards Richard

[Doug Burks]

One option to consider for DR specifically for ELSA is that our new
ELSA packages should be able to do store-and-forward, so you could
have one ELSA box that just receives copies of logs from all the other
ELSA nodes:
https://code.google.com/p/enterprise-log-search-and-archive/wiki/Documentation#elsa_node.conf:

[Frank Oduro]

I know this is an old post but I'm new to SO and have a task of backing up our existing one and recreating it on a RAID1.
Was this method successful? Any ups and downs that can be shared?

Regards,
Frank

[Wes]

Frank,

You could try taking a look at the following scripts located in /usr/sbin:

nsm_server_backup-data
nsm_server_backup-config
nsm_sensor_backup-data
nsm_sensor_backup-config

I don't believe the scripts are tested regularly, but it may be worth it to look into them to see if they meet your need(s).

Thanks,
Wes

[Clint Conner]

My apologies for replying to such an old post, but I felt it was better than creating a new one. We have a client who is going for their PCI "stamp-'o-approval". We need to keep logs for a full year, but that seems like it may be quite costly due to the size needed. We were using ShadowProtect SPX to do a full-disk image, but that has proved to be ineffective since it breaks whenever there is a kernel update and StorageCraft is slow to update SPX to work with the new kernels.

I have moved to using Back In Time to copy everything in the /nsm folder aside from the DailyLogs in the sensors; as well as various config files in other folders. I am also doing a mysqldump on the database that includes everything.

I have noticed that the syslog database is quite large as is the /nsm/elsa/data/sphinx folder. Is any of this data duplicated? Can I cut out backing up one or the other and still have the logs required for PCI compliancy?

If there is anyone out there who has a Security Onion running in a PCI environment, I would love to hear about your setup and how you've maintained compliancy.