Sagan 1.2.2 released!
411 views
Skip to first unread message
Da Beave
Jul 3, 2019, 2:34:37 PM7/3/19
to sagan-users
Quadrant Information Security is proud to release Sagan version 1.2.2 along with a new "stable" Sagan rule set (20190703). For more information about Sagan, go to https://quadrantsec.com/sagan_log_analysis_engine/. This release has a lot of new features & bug fixes. Below is the ChangeLog:
2019/07/03 - Sagan 1.2.2 released.
* Sagan has new and improved documentation! Go to:
The new documentation system makes it a part of the development cycle. We are
continuing to add and improve our documentation.
* New performance enhancements - This version was "stress tested" in our lab to
identify bottlenecks. The first major issue we noticed was pthread_mutex_lock()/
pthread_mutex_unlock() calls and how expensive they are. For simple lock
scenarios, we now use __atomic_add_fetch() and __atomic_sub_fetch(). For bigger
lock issues, we introduced "batching" in Sagan. This allows Sagan to "lock"
memory for more than one object (log line) at a time. To control how many logs
lines get sent to a thread, adjust the "batch-size" in the sagan.yaml. By
default, the "batch-size" is set to 1. This allows Sagan to perform like you
would normally expect. In situations of 5-10k EPS or higher, consider
adjusting the "batch-size" to a higher value. For more information see:
* Suricata style 'xbits' and Sagan 'flexbits' - Sagan has aligned the
'xbits' keyword usage with Suricata. If you understand how Suricata xbits
function, then you understand how the new Sagan 'xbits' function. What had
previously been known as 'xbits' (prior to this version) are now known as
'flexbits'. In certain situation, 'flexbits' may be more flexible in that
they can be set on TCP/IP port usage and direction.
i * New 'xbits' in Redis support - Normally 'xbits' are stored to a mmap() file.
This is done to not only store xbit values but allows multiple Sagan processes
to "share" (intra-process communications) xbit data. This also allows Sagan
persistent xbit storage (xbits can survive processes restarts). The 'xbit' in
in Redis concept allows Sagan to share 'xbits' between multiple Sagan systems
over a network. That is, a Sagan system physically in one location can share
xbit data with another system in another physical location via the network.
In the sagan.yaml, you can set your Redis server location (TCP/IP address,
password, etc). You can also set the 'xbit-storage' type to 'redis' (defaults
to mmap). Once done, 'xbits' will be stored to Redis rather than mmap() files.
While this sort of network level communication of threats is great, it comes at
a cost of network speed and latency. Any xbits that fire in rapid succession
might perform poorly. In those cases, you may want to consider using 'xbits'
for high level communications and detection and 'flexbits', which is mmap()
only, for low level communications and detection.
Sagan 'xbits' are stored using the following key format (via SET):
sagan:{cluster name}:{xbit name}:{triggering IP}
For example:
sagan:my_company_name:brute_force:192.168.2.1
Data about the xbit it stored in JSON can can be retrieved using "GET {keyname}".
* "cluster-name" is added to Sagan - You can define a 'cluster-name' within
the sagan.yaml. This is currently only used by Redis xbits. This allows
you to control what xbits can be shared in Redis between Sagan instances.
* The "external" output module has been removed from the sagan.yaml. The "external"
rule keyword is still valid. This calls an external program to processes data
from an event.
* When using "external" within a rule, the data is now passed in a sane JSON
format via stdin to the program you are calling. This makes processing the data
much easier.
* Fixed various Bluedot JSON parsing issues. Added Bluedot lookup returns to EVE
output.
* Added "program" field to EVE output.
* You can now enabled/disabled base64 payload encoding in the sagan.yaml file by
using the 'alerts-base64' configuration option. By default, to remain compatible
with Suricata style EVE output, it is enabled. If set to 'false', Sagan will
dump the data as plan text.
* Fixed an issue where using the "ignore" option can consume all worker threads.
* Removed support for the "selector" in Sagan. This was remove to keep the Sagan
code clean. It was also our opinion that this would function better at the
syslog daemon level rather than within Sagan.
* Sagan now shuts down more cleanly. Sagan will attempt to "wait" for any worker
threads to complete there tasks. The default "wait" time is 15 seconds. Upon
reaching 15 seconds, Sagan will for worker threads to abort.
* In certain situations, the "after" keyword would misfire. This was due to a
timestamp update issue that was out of place.
* This version makes the "saganpeek" (src/tools/saganpeek.c) more useful. This
tool allows you to "see" what is currently in Sagan memory. For example, what
has a 'threshold' on it, what 'xbits' are set, what 'after' has tripped. This
utility give you statistics along with the "offending" log line.
* Multiple bug fixes that addressed stability. New JSON bug fixes.
Reply all
Reply to author
Forward
0 new messages