Local rules

398 views
Skip to first unread message

lasy gsd

unread,
Nov 12, 2015, 10:35:03 PM11/12/15
to security-onion
Hello

I am new to securityonion and I just deployed my production environment as per the deployment guide.

I have updated my trusted (LAN) under the HOME_NET on the sensors. I do not see much alerts in squert so I am not sure it is because of less rules in the local rules. In fact I have not created any rules.

Do we really need to create any rules and there are default rules to alert for all traffic from the external network as I do not see any rules under local.rules

Wes

unread,
Nov 12, 2015, 10:58:04 PM11/12/15
to security-onion
lasy gsd,

PulledPork downloads new signatures every night, providing you with a large set of rules to draw from. Based on your network, you could disable or re-write signatures to best fit your needs. You can see more about this here:

https://github.com/Security-Onion-Solutions/security-onion/wiki/ManagingAlerts

Local rules are mostly used when you would like to rewrite an existing signature to better fit your requirements. You can see more about this in the aforementioned section, and here:

https://github.com/Security-Onion-Solutions/security-onion/wiki/AddingLocalRules

Also, in order to provide further recommendations as to why you aren't seeing many alerts (could be normal, maybe not), could you please provide the output of "sudo sostat-redacted"?

Hope this helps.

Thanks,
Wes

lasy gsd

unread,
Nov 13, 2015, 7:24:07 AM11/13/15
to security-onion
Below you will find the output of Sensor and server

So please provide your recommendations. Couple of clarifications I needed

1. I should always have the internet connection on the server to download the rules
2. local rules need to be added only if I need tuning



Server sostat
---------------------
=========================================================================
Service Status
=========================================================================
Status: securityonion
* sguil server[ OK ]
Status: HIDS
* ossec_agent (sguil)[ OK ]

Link Statistics
=========================================================================
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
RX: bytes packets errors dropped overrun mcast
230987332 596610 0 0 0 0
RX errors: length crc frame fifo missed
0 0 0 0 0
TX: bytes packets errors dropped carrier collsns
230987332 596610 0 0 0 0
TX errors: aborted fifo window heartbeat
0 0 0 0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:50:56:b0:06:84 brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped overrun mcast
86392783 198919 0 0 0 0
RX errors: length crc frame fifo missed
0 0 0 0 0
TX: bytes packets errors dropped carrier collsns
304620906 174261 0 0 0 0
TX errors: aborted fifo window heartbeat
0 0 0 0
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:50:56:b0:05:55 brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped overrun mcast
1462316 16762 0 0 0 0
RX errors: length crc frame fifo missed
0 0 0 0 0
TX: bytes packets errors dropped carrier collsns
13129 119 0 0 0 0
TX errors: aborted fifo window heartbeat
0 0 0 0
IDS Rules Update
=========================================================================
Fri Nov 13 07:01:01 UTC 2015
Backing up current local_rules.xml file.
Cleaning up local_rules.xml backup files older than 30 days.
Backing up current downloaded.rules file before it gets overwritten.
Cleaning up downloaded.rules backup files older than 30 days.
Backing up current local.rules file before it gets overwritten.
Cleaning up local.rules backup files older than 30 days.
Sleeping for 27 minutes to avoid overwhelming rule sites.
LOCAL_NIDS_RULE_TUNING is enabled.
This will cause PulledPork to use the existing rules in /opt/emergingthreats/
instead of downloading new rules from the Internet.
If you want PulledPork to download new rules from the Internet,
set the following in /etc/nsm/securityonion.conf:
LOCAL_NIDS_RULE_TUNING=no
Running PulledPork.
http://code.google.com/p/pulledpork/
_____ ____
`----,\ )
`--==\\ / PulledPork v0.7.0 - Swine Flu!
`--==\\/
.-~~~~-.Y|\\_ Copyright (C) 2009-2013 JJ Cummings
@_/ / 66\_ cumm...@gmail.com
| \ \ _(")
\ /-| ||'--' Rules give me wings!
\_\ \_\\
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Prepping rules from emerging.rules.tar.gz for work....
Done!
Reading rules...
Generating Stub Rules....
Done
Reading rules...
Reading rules...
Modifying Sids....
Done!
Processing /etc/nsm/pulledpork/enablesid.conf....
Modified 0 rules
Done
Processing /etc/nsm/pulledpork/dropsid.conf....
Modified 0 rules
Done
Processing /etc/nsm/pulledpork/disablesid.conf....
Modified 0 rules
Done
Setting Flowbit State....
Enabled 38 flowbits
Done
Writing /etc/nsm/rules/downloaded.rules....
Done
Generating sid-msg.map....
Done
Writing v1 /etc/nsm/rules/sid-msg.map....
Done
Writing /var/log/nsm/sid_changes.log....
Done
Rule Stats...
New:-------0
Deleted:---0
Enabled Rules:----17229
Dropped Rules:----0
Disabled Rules:---3891
Total Rules:------21120
No IP Blacklist Changes
=========================================================================
Sguil Uncategorized Events
=========================================================================
COUNT(*)
7545

=========================================================================
Sguil events summary for yesterday
=========================================================================
Totals GenID:SigID Signature
4 1:2100366 GPL ICMP_INFO PING *NIX
3 1:2101603 GPL WEB_SERVER DELETE attempt
2 1:2003068 ET SCAN Potential SSH Scan OUTBOUND
2 1:2001219 ET SCAN Potential SSH Scan
============================================================================



Sensor -sostat
-------------------------------
Link Statistics
=========================================================================
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
RX: bytes packets errors dropped overrun mcast
10799460 34430 0 0 0 0
RX errors: length crc frame fifo missed
0 0 0 0 0
TX: bytes packets errors dropped carrier collsns
10799460 34430 0 0 0 0
TX errors: aborted fifo window heartbeat
0 0 0 0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:50:56:b0:23:3d brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped overrun mcast
173805335 213276 0 0 0 0
RX errors: length crc frame fifo missed
0 0 0 0 0
TX: bytes packets errors dropped carrier collsns
40916626 173613 0 0 0 0
TX errors: aborted fifo window heartbeat
0 0 0 0
3: eth1: <BROADCAST,MULTICAST,NOARP,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:50:56:b0:12:03 brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped overrun mcast
283513209 583661 0 0 0 0
RX errors: length crc frame fifo missed
0 0 0 0 0
TX: bytes packets errors dropped carrier collsns
90 1 0 0 0 0
TX errors: aborted fifo window heartbeat
Packets received during last monitoring interval (600 seconds)
=========================================================================
eth1: 4249

=========================================================================
Log Archive
=========================================================================
/nsm/sensor_data/vm-nidsqtss1-prod-eth0/dailylogs/ - 0 days
4.0K .

/nsm/sensor_data/vm-nidsqtss1-prod-eth1/dailylogs/ - 3 days
465M .
30M ./2015-11-11
165M ./2015-11-12
270M ./2015-11-13

/nsm/bro/logs/ - 3 days
4.5M .
392K ./2015-11-11
1.9M ./2015-11-12
868K ./2015-11-13
1.4M ./stats

=========================================================================
Bro netstats
=========================================================================
Average packet loss as percent across all Bro workers: 0.000000

vm-nidsqtss1-prod-eth1-1: 1447402577.043732 recvd=57990 dropped=0 link=57990
vm-nidsqtss1-prod-eth1-2: 1447402577.243877 recvd=74623 dropped=0 link=74623

=========================================================================
IDS Engine (snort) packet drops
=========================================================================
/nsm/sensor_data/vm-nidsqtss1-prod-eth1/snort-1.stats last reported pkt_drop_percent as 0.000
/nsm/sensor_data/vm-nidsqtss1-prod-eth1/snort-2.stats last reported pkt_drop_percent as 0.000

=========================================================================
pf_ring stats
=========================================================================
PF_RING Version : 6.0.2 ($Revision: $)
Total rings : 4

Standard (non DNA) Options
Ring slots : 4096
Slot version : 16
Capture TX : Yes [RX+TX]
IP Defragment : No
Socket Mode : Standard
Transparent mode : Yes [mode 0]
Total plugins : 0
Cluster Fragment Queue : 0
Cluster Fragment Discard : 0

/proc/net/pf_ring/24145-eth1.86
Appl. Name : bro-eth1
Tot Packets : 74625
Tot Pkt Lost : 0
TX: Send Errors : 0
Reflect: Fwd Errors: 0
Min Num Slots : 4096
Num Free Slots : 4096

/proc/net/pf_ring/24146-eth1.87
Appl. Name : bro-eth1
Tot Packets : 57992
Tot Pkt Lost : 0
TX: Send Errors : 0
Reflect: Fwd Errors: 0
Min Num Slots : 4096
Packets received during last monitoring interval (600 seconds)
=========================================================================
eth1: 4249

=========================================================================
Log Archive
=========================================================================
/nsm/sensor_data/vm-nidsqtss1-prod-eth0/dailylogs/ - 0 days
4.0K .

/nsm/sensor_data/vm-nidsqtss1-prod-eth1/dailylogs/ - 3 days
465M .
30M ./2015-11-11
165M ./2015-11-12
270M ./2015-11-13

/nsm/bro/logs/ - 3 days
4.5M .
392K ./2015-11-11
1.9M ./2015-11-12
868K ./2015-11-13
1.4M ./stats

=========================================================================
Bro netstats
=========================================================================
Average packet loss as percent across all Bro workers: 0.000000

vm-nidsqtss1-prod-eth1-1: 1447402577.043732 recvd=57990 dropped=0 link=57990
vm-nidsqtss1-prod-eth1-2: 1447402577.243877 recvd=74623 dropped=0 link=74623

=========================================================================
IDS Engine (snort) packet drops
=========================================================================
/nsm/sensor_data/vm-nidsqtss1-prod-eth1/snort-1.stats last reported pkt_drop_percent as 0.000
/nsm/sensor_data/vm-nidsqtss1-prod-eth1/snort-2.stats last reported pkt_drop_percent as 0.000

=========================================================================
pf_ring stats
=========================================================================
PF_RING Version : 6.0.2 ($Revision: $)
Total rings : 4

Standard (non DNA) Options
Ring slots : 4096
Slot version : 16
Capture TX : Yes [RX+TX]
IP Defragment : No
Socket Mode : Standard
Transparent mode : Yes [mode 0]
Total plugins : 0
Cluster Fragment Queue : 0
Cluster Fragment Discard : 0

/proc/net/pf_ring/24145-eth1.86
Appl. Name : bro-eth1
Tot Packets : 74625
Tot Pkt Lost : 0
TX: Send Errors : 0
Reflect: Fwd Errors: 0
Min Num Slots : 4096
Num Free Slots : 4096

/proc/net/pf_ring/24146-eth1.87
Appl. Name : bro-eth1
Tot Packets : 57992
Tot Pkt Lost : 0
TX: Send Errors : 0
Reflect: Fwd Errors: 0
Min Num Slots : 4096
Num Free Slots : 4096

/proc/net/pf_ring/24399-eth1.88
Appl. Name : snort-cluster-52-socket-0
Tot Packets : 57764
Tot Pkt Lost : 0
TX: Send Errors : 0
Reflect: Fwd Errors: 0
Min Num Slots : 4098
Num Free Slots : 4098

/proc/net/pf_ring/24443-eth1.89
Appl. Name : snort-cluster-52-socket-0
Tot Packets : 74482
Tot Pkt Lost : 0
TX: Send Errors : 0
Reflect: Fwd Errors: 0
Min Num Slots : 4098
Num Free Slots : 4096

=========================================================================
Netsniff-NG - Reported Packet Loss (per interval)
=========================================================================
0 Loss

=========================================================================
Last update
=========================================================================
Syslog-ng
Checking for process:
18515 supervising syslog-ng
18516 /usr/sbin/syslog-ng -p /var/run/syslog-ng.pid
Checking for connection:
Connection to localhost 514 port [tcp/shell] succeeded!

MySQL
Checking for process:
2212 /usr/sbin/mysqld
Checking for connection:
Connection to localhost 50000 port [tcp/*] succeeded!

Sphinx
Checking for process:
2059 su -s /bin/sh -c exec "$0" "$@" sphinxsearch -- /usr/bin/searchd --nodetach
Checking for connection:
Connection to localhost 9306 port [tcp/*] succeeded!

ELSA Buffers in Queue:
2
If this number is consistently higher than 20, please see:
https://github.com/Security-Onion-Solutions/security-onion/wiki/FAQ#why-does-sostat-show-a-high-number-of-elsa-buffers-in-queue

ELSA Directory Sizes:
106M /nsm/elsa/data
2.8M /var/lib/mysql/syslog
2.3M /var/lib/mysql/syslog_data

ELSA Index Date Range:
MIN(start) MAX(end)
2015-11-11 20:16:59 2015-11-13 08:15:48

autossh
Checking for process:
4198 /usr/lib/autossh/autossh -M 0 -q -N -o ServerAliveInterval 60 -o ServerAliveCountMax 3 -i /root/.ssh/securityonion -L 3306:127.0.0.1:3306 -R 50000:localhost:3154 sen...@10.210.12.10

namobud...@gmail.com

unread,
Nov 13, 2015, 9:28:41 AM11/13/15
to security-onion
Hello Lasy gsd,

A properly placed sensor in a production environment (I.E. taping any pre-vpn choke points on a switch or tap) should initial generate A LOT of traffic and alerts which will require tuning in the /etc/nsm/rules/threshold.conf file.

I tune on the server and the threshold.conf gets pushed out I believe every 24 hours. My tuning strategy is to initially let the noise be there for a couple of days or week as these can be very valuable in understanding your environment, and then I slowly tune out obvious false positives.

I think working with your operations guys to determine what is a false positive and help them understand valuable configuration info which SO can generate is a good. False positives can be invaluable in understudying the environment and operations.

Wes

unread,
Nov 13, 2015, 9:51:33 AM11/13/15
to security-onion
lasy gsd,

Namobud is correct, in that you can modify certain rules within threshold.conf.

https://github.com/Security-Onion-Solutions/security-onion/wiki/ManagingAlerts

There are multiple ways to manage rules with Security Onion. How you tune depends mostly on your environment and what is and isn't normal activity for you.

PulledPork simply downloads new signatures from the community nightly.

With threshold.conf, you can use suppression so that you can make granular decisions without having to re-write the rule/sig.

With disablesid.conf, you can simply add a rule (or category) to be disabled.

If you are using Salt, rules you modify will get pushed out from the master server to the sensors every 15 minutes--otherwise, when are finished modifying the rule(s) you could run "sudo /usr/bin/rule-update" on both the server and the sensor to update the configuration on each, or wait until the nightly rule update.

A whole of really helpful information in regard to rules/signatures can be found here:
https://github.com/Security-Onion-Solutions/security-onion/wiki/AddingLocalRules
https://github.com/Security-Onion-Solutions/security-onion/wiki/ManagingAlerts
https://github.com/Security-Onion-Solutions/security-onion/wiki/Salt

Also, take a look at the SO best practices, so that your machine(s) will be running at peak performance:

https://github.com/Security-Onion-Solutions/security-onion/wiki/Best-Practices

Thanks,
Wes

lasy gsd

unread,
Nov 13, 2015, 12:41:55 PM11/13/15
to security-onion
Thank you for the reply

I got your points but my issues are I do not see enough alerts in my environment and it supposed to be other way and I am not sure what is wrong with the setup

If I understand correctly, I do not need to play with rules, threshold and signature until I lot of alerts

I have made my HOME_NET to very limited network so I can see lot of traffic but even that did not help me

Doug Burks

unread,
Nov 13, 2015, 1:00:28 PM11/13/15
to securit...@googlegroups.com
The sostat output from your sensor appears to be incomplete. Can you
attach the entire output as a .txt file, please?

Is your sensor connected to a tap or span port?

If span port, are you sure the span port is configured correctly?
> --
> You received this message because you are subscribed to the Google Groups "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at http://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks
Need Security Onion Training or Commercial Support?
http://securityonionsolutions.com
Message has been deleted

Doug Burks

unread,
Nov 15, 2015, 3:53:14 PM11/15/15
to securit...@googlegroups.com
It looks like you included an extra file not related to your sostat
output. You might want to go back and delete that post.

Also, whenever you submit sostat output, make sure that you run
"sostat-redacted" and manually redact any remaining sensitive info.

Looking at sensor1, it's only seeing a small amount of traffic:

eth1 Link encap:Ethernet HWaddr 00:0c:29:3f:09:8c
UP BROADCAST RUNNING NOARP PROMISC MULTICAST MTU:1500 Metric:1
RX packets:65657 errors:0 dropped:8512 overruns:0 frame:0
TX packets:1 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8270156 (8.2 MB) TX bytes:90 (90.0 B)

I'll ask my previous question again: are you sure your span port is
configured correctly?

On Sat, Nov 14, 2015 at 3:53 PM, lasy gsd <las...@gmail.com> wrote:
> Please find attached sostat output
>
> Yes sensor is connected to SPAN port
>
> And in the few alters I get, I do not source and destination IP everything is 0.0.0.0 and 0.0.0.0
Reply all
Reply to author
Forward
0 new messages