Barnyard2 Kafka

[Jaime Nebrera]

Dear all,

The redBorder team is pleased to announce the availability of the Beta
release of Barnyard2-Kafka plugin in our Github repository
https://github.com/redBorder/ under GPL license.

This is an extension of Barnyard2 2-1.13 official release to add the
following capabilities:

* Ability to send Snort events using an Apache Kafka messaging system
(http://kafka.apache.org/)
* Preprocessing of certain Unified2 fields in order to provide enhanced
meta data information
- Geolocation of IPs based on Maxmind libraries
- IP translation based on /etc/hosts & /etc/barnyard_networks information

In future releases we hope to extend the meta data fields provided (fe
services information extracted from /etc/services) but for now we
believe this is ok. This patch is usable, but beta quality, use at your
own risk. Of course, we would really appreciate any help to extend the
number of Unified2 fields supported as well as testing in real
scenarios. We have based our contribution in CSV and SQL Barnyard2 plugins.

Apache Kafka is a new messaging system several orders of magnitude
faster than AMPQ or similar. By using this framework, we will be able to
more easily plug Snort events into a BigData environment. Just two ideas
in this regard, it would enable to save Snort events in a Hadoop
(http://hadoop.apache.org/) cluster as well as preprocess them using
Twitter's Storm (http://storm-project.net/)

As for redBorder project, we are working on the real time management of
the events for the GUI as well as a scale out capable correlation
engine, that will not only process events generated by Snort but also
from other elements in our framework. More information here
(http://redborder.net/redborder-roadmap/)

Of course, we would like to thank our sponsors and clients for
supporting us into making this public. Also, the Barnyard2 and Snort
developers for their great software. We just hope this patch helps the
community.

Regards

PS.- I work for the company developing redBorder

***********************************************************************************

1 Using alert_json barnyard2 plugin
*****************************
If you want to use alert_json barnyard2 plugin, you have to put it in
barnyard2.conf file.
The format of the argument passed to the plugin is:

output alert_json: kafka://<host>:<port>@<topic>

Where host, port and topic are the kafka host, port and topic (not
zookeepers one).

2 Host and network in readable format:
*******************************
Alert_json can can print a human readable string plus the default host
string. For example, if you
have the hostname �foo PC� associated with the �192.168.100.3� ip in
/etc/hosts file, alert_json will
print �foo PC� plus �192.168.100.3� and the number representation of the ip.

In the same way, alert_json can print the destination or source network
of the packet. You have to make
an entry in �/etc/barnyard_networks� indicating this. For example, the
entry �192.168.100.0/24 foo
network� will make alert_json print the network name plus the network id.

In case alert_json does not locate the network in the file, it will
print �0.0.0.0/0� instead.

3 GeoIP
*******
Alert_json can locate the region of the IP too. You just have to have
libGeoIP installed and compile
the sources with GEO_IP macro defined (it's defined by default). Also,
you have to put the database
in �/usr/local/share/GeoIP/GeoIP.dat�

If you want �alert_json� print geo-localization information too, you
have to compile barnyard with
geo-ip support:

./configure --enable-geo-ip

--
Jaime Nebrera -jneb...@eneotecnologia.com
Consultor TI - ENEO Tecnologia SL
C/ Manufactura 2, Edificio Euro, Oficina 3N
Mairena del Aljarafe - 41927 - Sevilla
Telf.- 955 60 11 60 / 619 04 55 18