How to completely wipe and reinitialize Snorby without touching the rest of SO

[Kevin Branch]

I recently had a problem with a broken Snorby database that was jamming barnyard2 and thus breaking pretty much anything Suricata-alert related in my SO system.  My final solution was to completely wipe out the Snorby application along with it's data and start fresh.  I did not want to rebuild SO.  This is the method that worked well for me, and since I seem to remember someone asking a while back about resetting Snorby while leaving SO intact, I offer it here to the wider group.  Refinements are welcome.

The following procedure will completely wipe out all of your Snorby data and any custom configurations you have made to Snorby, restoring Snorby to like-new condition while leaving the rest of SO alone.  I have only tested this on a standalone install of SO.

Get to the CLI of your SO box.

sudo su -

# shut down NSM services

service nsm stop

# adapt these two lines to reflect your existing local snorby credentials and then run them

SNORBY_EMAIL="***EMAIL ADDR HERE***"

SGUIL_CLIENT_PASSWORD_1="***PASSWORD HERE***"

# these are needed by the script below

LOG=log.txt

ELSA=YES

# run this code section which was lifted directly from sosetup

# Kill any existing Snorby processes.

pkill delayed_job

# Delete any existing Snorby data.

if [ -d /var/lib/mysql/snorby ]; then

mysql -e "drop database snorby" >> $LOG 2>&1

fi

# Set email and password

cp /opt/snorby/db/seeds.rb.securityonion /opt/snorby/db/seeds.rb

sed -i "s|ReplaceWithDesiredEmail|$SNORBY_EMAIL|g" /opt/snorby/db/seeds.rb

sed -i "s|ReplaceWithDesiredPassword|$SGUIL_CLIENT_PASSWORD_1|g" /opt/snorby/db/seeds.rb

# Set FPC options

IP=`ifconfig |grep "inet addr" | awk '{print $2}' |cut -d\: -f2 |grep -v "127.0.0.1" |head -1`

sed -i "s|packet_capture_url, nil|packet_capture_url, 'https://$IP/capme/'|g" /opt/snorby/db/seeds.rb

sed -i "s|packet_capture, nil|packet_capture, 1|g" /opt/snorby/db/seeds.rb

sed -i "s|packet_capture_auto_auth, 1|packet_capture_auto_auth, nil|g" /opt/snorby/db/seeds.rb

# Initialize Snorby DB

su www-data -c "cd /opt/snorby; bundle exec rake snorby:setup RAILS_ENV=production" >> $LOG 2>&1

# Shred the Snorby password

shred -u /opt/snorby/db/seeds.rb >> $LOG 2>&1

# Pivot from Snorby to ELSA

[ "$ELSA" = "YES" ] && mysql -uroot -Dsnorby -e "INSERT INTO lookups (title,value) VALUES('ELSA Search By IP Address','https://$IP:3154/?query_string=\"\${ip}\"%20groupby:program')"

# review the log if you wish

less log.txt

# get a fresh start

shutdown -r now

[Michał Purzyński]

This is so cool! Thanks a lot, should go on a wiki.

[Doug Burks]

Added to Wiki and created a link under the "Tricks and Tips" section:
https://code.google.com/p/security-onion/wiki/WipingSnorby

Thanks, Kevin!

On Thu, May 15, 2014 at 5:50 PM, Michał Purzyński
<michalpu...@gmail.com> wrote:
> This is so cool! Thanks a lot, should go on a wiki.
>

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks

[Ric Woodard]

I keep receiving "-bash: $LOG: ambiguous redirect" messages and since my password contains a !, I also receive "-bash: !: event not found"

Please advise

[Doug Burks]

Replies inline.

On Thu, Sep 18, 2014 at 4:48 PM, Ric Woodard <ricwo...@gmail.com> wrote:
> I keep receiving "-bash: $LOG: ambiguous redirect" messages

Did you do this part of the procedure?
LOG=log.txt

> and since my password contains a !, I also receive "-bash: !: event not found"

Try a password without a !. You can always change your password later
through the Snorby interface.


--
Doug Burks
Need Security Onion Training or Commercial Support?
http://securityonionsolutions.com

[Kevin Branch]

I've been doing a fair bit of tuning on a couple of new SO installs this month and in the process have refined the steps in that Wiki page into a script which wipes Snorby data while retaining Snorby configuration, including username/password.  I call it wipe-snorby:

#!/bin/bash

# shut down NSM services and Snorby

service nsm stop

# purge all unified2 files since otherwise they tend to get reloaded into Snorby by barnyard2

rm -f /nsm/sensor_data/*-*/snort.unified2.*

# backup snorby's config-related db tables

mysqldump snorby settings users lookups sensor > /tmp/snorby-config-backup.sql

# Kill any existing Snorby processes.

pkill delayed_job

# Delete any existing Snorby data.

if [ -d /var/lib/mysql/snorby ]; then

   mysql -e "drop database snorby"

fi

# Set a dummy email and password to facilitate initialization (will be replaced by db table restore) 

cp /opt/snorby/db/seeds.rb.securityonion /opt/snorby/db/seeds.rb

sed -i "s|ReplaceWithDesiredEmail|du...@dummy.com|g" /opt/snorby/db/seeds.rb

sed -i "s|ReplaceWithDesiredPassword|dummy_password|g" /opt/snorby/db/seeds.rb

# Set FPC options

IP=`ifconfig |grep "inet addr" | awk '{print $2}' |cut -d\: -f2 |grep -v "127.0.0.1" |head -1`

sed -i "s|packet_capture_url, nil|packet_capture_url, 'https://$IP/capme/'|g" /opt/snorby/db/seeds.rb

sed -i "s|packet_capture, nil|packet_capture, 1|g" /opt/snorby/db/seeds.rb

sed -i "s|packet_capture_auto_auth, 1|packet_capture_auto_auth, nil|g" /opt/snorby/db/seeds.rb

# Initialize Snorby DB - will take while

su www-data -c "cd /opt/snorby; bundle exec rake snorby:setup RAILS_ENV=production"

# Shred the Snorby password

shred -u /opt/snorby/db/seeds.rb

# restore config tables

mysql snorby < /tmp/snorby-config-backup.sql

# update Snorby's reference tables

rule-update

# restart all SO services

service nsm stop

service nsm start

I've done a number of Snorby wipes with it and it seems to be performing consistently for me.  I'm thinking of putting it up in place of my original Wiki page on this subject.  When you all look at the script, do you see any issues to address before I publish it?

Kevin

[Doug Burks]

Great job, Kevin!

I've created Issue 598 to add this as a new script called so-snorby-wipe:
https://code.google.com/p/security-onion/issues/detail?id=598

[6st...@gmail.com]

I ran the script and receive the following errors at various places:

Writing v1 /etc/nsm/rules/sid-msg.map....
Done

WARNING: Ignoring bad line in SID file: 'v1'

______ -*> Barnyard2 <*-
/ ,,_ \ Version 2.1.13 (Build 333) TCL
|o" )~| By Ian Firns (SecurixLive): http://www.securixlive.com/
+ '''' + (C) Copyright 2008-2013 Ian Firns <fir...@securixlive.com>

Before Stopping and Start Section:
ERROR: Unable to open directory '' (No such file or directory)
ERROR: Unable to find the next spool file!

Bottom line, I was able to login to Snorby ; but I did not see some test snort alerts I created, but which showed up in Sguil.

Thank you!
Steve

[Doug Burks]

Replies inline.

On Wed, Oct 15, 2014 at 6:15 PM, <6st...@gmail.com> wrote:
> I ran the script and receive the following errors at various places:
>
> Writing v1 /etc/nsm/rules/sid-msg.map....
> Done
>
> WARNING: Ignoring bad line in SID file: 'v1'
>
> ______ -*> Barnyard2 <*-
> / ,,_ \ Version 2.1.13 (Build 333) TCL
> |o" )~| By Ian Firns (SecurixLive): http://www.securixlive.com/
> + '''' + (C) Copyright 2008-2013 Ian Firns <fir...@securixlive.com>
>
> Before Stopping and Start Section:
> ERROR: Unable to open directory '' (No such file or directory)
> ERROR: Unable to find the next spool file!

These are all normal messages from rule-update.

> Bottom line, I was able to login to Snorby ; but I did not see some test snort alerts I created, but which showed up in Sguil.

There may be an initial delay while Snorby and barnyard2 are still
initializing, but at some point, alerts should appear in Sguil and
Snorby at the same time.

[6st...@gmail.com]

I will test and monitor for a few day and let you know how it goes. Thank you to all those that help on these forums, but Doug, your consistency and patience really impress me, thank you for Security Onion and all the work you do on this project!

[Tim Desrochers]

On Thursday, October 16, 2014 3:10:43 PM UTC, 6st...@gmail.com wrote:
> I will test and monitor for a few day and let you know how it goes. Thank you to all those that help on these forums, but Doug, your consistency and patience really impress me, thank you for Security Onion and all the work you do on this project!

I ran this on a test environment and got the following when I click on the snorby icon on my desktop:

NoMethodError in Page#dashboard

Showing /opt/snorby/app/views/page/dashboard.html.erb where line #142 raised:

undefined method `name' for nil:NilClass
Extracted source (around line #142):

139: <ul class='box-list'>
140: <% @recent_events.each do |event| %>
141: <li>
142: <%= link_to truncate(event.signature.name, :length => 28), results_path(:title => "#{truncate(event.signature.name, :length => 40)}",
143: "match_all"=>"true", "search"=>{"sensor"=>{"column"=>"signature", "operator"=>"is", "value"=> event.signature.sig_id } }), :title => event.signature.name %>
144: <span>
145: <%= number_with_delimiter (event.signature.events_count.zero? ? Event.count(:sig_id => event.signature.sig_id) : event.signature.events_count) %>
Rails.root: /opt/snorby

Application Trace | Framework Trace | Full Trace
app/views/page/dashboard.html.erb:142:in `block in _app_views_page_dashboard_html_erb__2440591551938018138_41197140'
app/views/page/dashboard.html.erb:140:in `each'
app/views/page/dashboard.html.erb:140:in `_app_views_page_dashboard_html_erb__2440591551938018138_41197140'
app/controllers/page_controller.rb:56:in `dashboard'
Request

Parameters:

None
Show session dump

session_id: "97e6935bede47407ebb5efdeaffe7c86"
warden.user.user.key: ["User", [1], "$2a$10$GzBPmwEBT7zArxA8oUkIK."]
Show env dump

Response

Headers:

None

Any thoughts as to why.

[Tim Desrochers]

Fixed the issue. I had to rerun the script as root, not sudo. Not sure why it mattered but it did.

[Tim Desrochers]

Sorry to blow this thread up but now when I try to force cache update I get the same error as above. So back to the original question....Any thoughts as to why this is happening??

[Doug Burks]

Is it possible no alerts have come in yet?

Have you tried forcing an alert using "curl testmyids.com"?

> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.

--
Doug Burks
Need Security Onion Training or Commercial Support?
http://securityonionsolutions.com

Last day to register for 3-Day Training Class in Augusta GA is 12/11!

[Doug Burks]

I've updated the script a little, added it to the NSM package, and submitted it for testing:
https://groups.google.com/d/topic/security-onion-testing/5C7j_gBWxbc/discussion