X-Forwarded-For Patch for Barnyard2-1.9

[neha]

Has anyone else had success with this patch? 

http://code.google.com/p/svmids/downloads/detail?name=barnyard2-1.9_xff-patch.patch

I am able to apply the patch without any trouble, and barnyard2 compiles without any errors. 

But Barnyard2 never seems to actually update the ip_source header field with the Original Client IP. 

I wonder if the Barnyard2 devs can give any insight. I have sent this same email to the author of the patch as well.

 - platform is debian squeeze

 - using mysql backend 5.1.61-0+squeeze1

 - snort 2.9.2.2

 - barnyard2-1.9

 

I am starting barnyard2 like so (please note the "-j" option, which is added by the patch):

/usr/local/bin/barnyard2 -j -c /usr/local/etc/barnyard2.conf -d /var/log/snort/VLAN15 -f snortunified2.log -w /tmp/waldo -G /etc/snort/gen-msg.map -S /etc/snort/sid-msg.map -C /etc/snort/classification.config

From what I can tell in unified2 logs, the extrahdr data is logged and recorded appropriately. Record type=110 is being set and logged correctly by snort. Which I can clearly see in u2spewfoo, along with the Original Client IP field.

Barnyard2 w/ Patch appears to process the record (spi_unified2.c -> spooler.c), but the code path in spo_database.c that inserts xff_srcip into database queries never seems to get triggered.

Here is the relevant debug output from Barnyard2 w/ BARNYARD2_DEBUG=

spi_unified2.c:133: Header: Reading at byte position 596

spi_unified2.c:157: Header: Type=110 (36 bytes)

spi_unified2.c:175: Reading record type=110 (36 bytes)

spi_unified2.c:187: Record: Reading at byte position 604

spi_unified2.c:223: No debug available for record type: 110

spooler.c:744: XXF: Extra Data Header found

spooler.c:750: XXF: Data Header is type 4 (ok)

spooler.c:761: Event ID 1 1 1 (blog_lenght: 12

spooler.c:765: XXF: Extra Data is info and data type 1, and data size is 12 (ok)

spooler.c:769: XXF IP: 213.181.x.x [redacted for public consumption]

spooler.c:1107: Waldo write

        dir:  /var/log/snort/VLAN15

        base: snortunified2.log

        time: 1335174311

        idx:  3

--

spi_unified2.c:133: Header: Reading at byte position 1601

spi_unified2.c:157: Header: Type=110 (36 bytes)

spi_unified2.c:175: Reading record type=110 (36 bytes)

spi_unified2.c:187: Record: Reading at byte position 1609

spi_unified2.c:223: No debug available for record type: 110

spooler.c:744: XXF: Extra Data Header found

spooler.c:750: XXF: Data Header is type 4 (ok)

spooler.c:761: Event ID 2 1 1 (blog_lenght: 12

spooler.c:765: XXF: Extra Data is info and data type 1, and data size is 12 (ok)

spooler.c:769: XXF IP: 188.70.x.x [ redacted for public consumption ]

spooler.c:1107: Waldo write

        dir:  /var/log/snort/VLAN15

        base: snortunified2.log

        time: 1335174311

        idx:  7

Database debug looks like this. I've verified the integer converteted ip_src is the IP of the proxy server and *not* the IP from XFF:

spo_database.c:2901: database(debug): (INSERT INTO iphdr (sid, cid, ip_src, ip_dst, ip_ver, ip_hlen,        ip_tos, ip_len, ip_id, ip_flags, ip_off,       ip_ttl, ip_proto, ip_csum) VALUES (1,497,167841290,167838224,4,5,0,1500,56164,0,0,64,6,13724)) executed

Here's the spo_database.c code for XFF:

/* XFF patch - It changes IP packet src ip by alert src ip! */

            if (BcUseXFF()) xff_srcip=ntohl(((Unified2IDSEvent *)event)->ip_source);

            else xff_srcip=(u_long)ntohl(p->iph->ip_src.s_addr);

            if(data->detail)

            {

                ret = SnortSnprintf(query->val, MAX_QUERY_LENGTH,

                                    "INSERT INTO "

                                    "iphdr (sid, cid, ip_src, ip_dst, ip_ver, ip_hlen, "

                                    "       ip_tos, ip_len, ip_id, ip_flags, ip_off,"

                                    "       ip_ttl, ip_proto, ip_csum) "

                                    "VALUES (%u,%u,%lu,%lu,%u,%u,%u,%u,%u,%u,%u,%u,%u,%u)",

                                    data->shared->sid,

                                    data->shared->cid,

                                    xff_srcip,

                                    (u_long)ntohl(p->iph->ip_dst.s_addr),

                                    IP_VER(p->iph),

                                    IP_HLEN(p->iph),

                                    p->iph->ip_tos,

                                    ntohs(p->iph->ip_len),

                                    ntohs(p->iph->ip_id),

                                    p->frag_flag,

                                    ntohs(p->frag_offset),

                                    p->iph->ip_ttl,

                                    p->iph->ip_proto,

                                    ntohs(p->iph->ip_csum));

                if (ret != SNORT_SNPRINTF_SUCCESS)

                    goto bad_query;

            }

My grasp of this code is pretty weak, and I'm not quite sure where or how this is failing. I'm also not sure if there's some other way to do this - do the git branches for the latest barnyard2-1.10 beta support storing the XFF header as ip_src? My IDS resides behind a proxy, so we need a consistent and reliable way to include X-Forwarded-For headers as ip_src for all packets that trigger alerts.

Thanks,

Neha 

[Lysemose]

Hi

I have the same question and is in the same situation regarding IDS behind proxy.

I doing a little research to help Doug Burks and his SO, http://securityonion.blogspot.com.

Is the patch, http://code.google.com/p/svmids/downloads/detail?name=barnyard2-1.9_xff-patch.patch, valid for Barnyard2-1.10? Or is there any plans for building this directly into future Barnyard?

/Lysemose

[beenph]

Greeting Lysemose,

The patch would need some adaptation to actually work since some code
of the 2.1-10 release
changed, especially in the database output code. Full compatibility
with EXTRA_DATA type will come
in a next release. Especially since the spooler need its new lifting
and all output plugins.

Cheers,
-elz

[Heine Lysemose]

Hi

Thanks for your quick reply!

When you write full compatibility with EXTRA_DATA does that imply that Barnyard by native will replace the srcIP/destIP with the one X-Forwarded-For IP if available/by adding a command lien option?

Any ETA on this?

/Lysemose

[beenph]

On Fri, Jun 1, 2012 at 5:27 AM, Heine Lysemose <lyse...@gmail.com> wrote:
> Hi
>
> Thanks for your quick reply!
> When you write full compatibility with EXTRA_DATA does that imply that
> Barnyard by native will replace the srcIP/destIP with the one
> X-Forwarded-For IP if available/by adding a command lien option?
>

No, this mean that each output plugin will correctly process each type of event
depending on their output restriction/features.
Recontruction/alteration/substitution should only be done
if its evident. In this context i think its not absolute maping (i
could be wrong) and this should be addressed
in a correlation context.

Why output plugin do you use?


> Any ETA on this?
>
Unfortunatly not. stay tuned.

-elz

[beenph]

> Why output plugin do you use?
>
>

Oups press sent before i could correct this "WHY" i meant to write Which.

sorry.

-elz

[Nelo Belda]

Hi all,

Here is a good post about Extra Data and Barnyard2. It's written in spanish so you can use Google Translate.
http://www.securityartwork.es/2012/03/06/snort-extraccion-avanzada-de-informacion-en-preprocesadores-http-y-smtp-i/
http://www.securityartwork.es/2012/03/14/snort-extraccion-avanzada-de-informacion-en-preprocesadores-http-y-smtp-ii/

Try it and let us know if it's what you want.

Regards.

2012/6/1 beenph <bee...@gmail.com>

[Heine Lysemose]

@beenph

I'm using the SecurityOnion distribution and it's using database and sguil output plugin.

@Nelo

I will read those post, thanks!

/Lysemose

[beenph]

On Fri, Jun 1, 2012 at 7:23 AM, Nelo Belda <nbe...@gmail.com> wrote:
> Hi all,
>
> Here is a good post about Extra Data and Barnyard2. It's written in spanish
> so you can use Google Translate.
> http://www.securityartwork.es/2012/03/06/snort-extraccion-avanzada-de-informacion-en-preprocesadores-http-y-smtp-i/
> http://www.securityartwork.es/2012/03/14/snort-extraccion-avanzada-de-informacion-en-preprocesadores-http-y-smtp-ii/
>
> Try it and let us know if it's what you want.
>
> Regards.
>

I didin't find a link to the patch but if you can send it to the ML so
we can see what it touches.

Have you tried to adapt your patch to 2-1.10 pre-stable?

-elz

[Neha Chriss]

Hello All - 

I had an opportunity to work with the original author of the patch - he was very helpful in diagnosing the issue. He informed me of the same changes that beeph describes. This was a critical piece of the nids deployment, and my final solution was to downgrade to 2.9.0.5, which worked without issue. Although, the downside is that my lab environment doesn't have the rather useful javascript obfuscation decoding features.. (comparing custom snort with a commercial nids).

cheers,

Neha

[beenph]

>
> Hello All -
>
> I had an opportunity to work with the original author of the patch - he was
> very helpful in diagnosing the issue. He informed me of the same changes
> that beeph describes. This was a critical piece of the nids deployment, and
> my final solution was to downgrade to 2.9.0.5, which worked without issue.
> Although, the downside is that my lab environment doesn't have the rather
> useful javascript obfuscation decoding features.. (comparing custom snort
> with a commercial nids).
>

Why did you have to downgrade snort version if the by2 patch worked?

-elz

[Neha Chriss]

On Sun, Jun 3, 2012 at 10:12 PM, beenph <bee...@gmail.com> wrote:

>

Why did you have to downgrade snort version if the by2 patch worked?

-elz

I was not clear: the XFF patch did *not* work on my initial deployment of 2.9.2.2.and barnyard2-1.9. Debugging output revealed that snort's output extra data header format had changed, and it was at that time that the author became aware of the need to refactor the barnyard spooler. He recommended I downgrade instead of waiting for a patch, since the necessary rewrite/refactor would take some time.

cheers,

n

[Neha Chriss]

On Sun, Jun 3, 2012 at 10:29 PM, Neha Chriss <ne...@youzee.com> wrote:

On Sun, Jun 3, 2012 at 10:12 PM, beenph <bee...@gmail.com> wrote:

>

Why did you have to downgrade snort version if the by2 patch worked?

-elz

Also - I'm just now reviewing the spanish sites and I have not yet tried this method with a current version of snort. I will update when I get a chance to test a fresh 2.9.10 installation..  

[beenph]

FYI the 2.1-10 version does not have a new spooler. Also note that
every output plugins will need to be rewritten to actually handle new
type of event.

Thus the main priority will be the folowing:

1-db
2-syslog
3-alert/sguil

-elz