Matching netfilter

[DigiAngel]

So ok...I think I'm close, but no taco yet. Here's my rule..the
New,invalid is my log prefix:

alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"New, invalid TCP
Connection"; program: kernel; content: "New,invalid"; content: "TCP";
parse_ip; parse_port; classtype: network-scan; sid: 6000000; rev:1;)

From a debug of Sagan:
[*] 127.0.0.1|kern|warning|warning|kernel:|2011-10-19|14:14:16|kernel|
[94407.546076] New,invalid IN=ppp0 OUT= MAC= SRC=60.51.92.131
DST=My_Ext_IP LEN=60 TOS=0x00 PREC=0x00 TTL=54 ID=1061 DF PROTO=TCP
SPT=2865 DPT=23 WINDOW=5808 RES=0x00 SYN URGP=0

What I see in BASE:
#0-(9-114) [rule] [EmThreats] New, invalid TCP Connection
2011-10-19 14:14:16 60.51.92.131:514 192.168.1.1:514 TCP

So...my issues are ports aren't being seen, and the DST IP is being
read as the internal IP of the server, not the external one. My
question is, do I need to use a normalize with this? And if so, not
sure I how I tell normalize that "hey, DST= is the destination IP,
SPT= is the source port, and DPT= is the destination port" in lognorm
terms ;) Thanks for any help you can provide.

James

[DigiAngel]

So here's the new rule:

alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"New, invalid TCP
Connection"; program: kernel; content: "New,invalid"; content: "TCP";

parse_ip; parse_port; normalize: kernel; classtype: tcp-connection;

sid: 6000000; rev:1;)

and the matching kernel-normalize.rulebase:
prefix=
rule=: SRC=%src-ip:ipv4% DST=%dst-ip:ipv4% SPT=%src-port:number% DPT=
%dst-port:number%

this crashes Sagan (see other post) so I'm at an impass until I can
figure out what the issue it.

James

[Champ Clark III [Quadrant]]

You do have the "kernel-normalize.rulebase" (which you created) in the sagan.conf file,  right?

Let me catch up on your other emails and see what I find..  Worst case,  it looks easy to reproduce. 

Champ Clark III
(office) 904.253.7856
(mobile) 850.443.2440
ccl...@quadrantsec.com
www.quadrantsec.com

[Champ Clark III [Quadrant]]

Ok.. not sure if it's related.. but. 

On Oct 19, 2011, at 4:49 PM, DigiAngel wrote:

So here's the new rule:

alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"New, invalid TCP
Connection"; program: kernel; content: "New,invalid"; content: "TCP";
parse_ip; parse_port; normalize: kernel; classtype: tcp-connection;
sid: 6000000; rev:1;)

You're specifying "parse_ip" and "parse_port" on a rule you're trying to "normalize: kernel". 

"parse_ip" and "parse_port" should only be used if you're _not_ using normalization (and visa-versa).

Basically, you're saying,  dynamically find the IP address (parse_ip) and dynamically find the 

port (parse_port) and also normalize it (find all the same information) via liblognorm. 

It's still a bug (IMHO) because it shouldn't let you do both.   Remove the parse_ip/port stuff and

let me know how it works...  

  

and the matching kernel-normalize.rulebase:
prefix=
rule=:  SRC=%src-ip:ipv4% DST=%dst-ip:ipv4% SPT=%src-port:number% DPT=
%dst-port:number%

 

    The above looks find to me...   Anyways,  let me know how it works out without the parse_ip/port 

stuff.

[Digital X]

Hey Clark!

So here's what I got in sagan.conf:

normalize: kernel, $RULE_PATH/kernel-normalize.rulebase

include $RULE_PATH/kernel.rules

And I've tried:

alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"New, invalid TCP Connection"; program: kernel; content: "New,invalid"; content: "TCP"; parse_ip; parse_port;  classtype: tcp-connection; sid: 6000000; rev:1;)

and

alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"New, invalid TCP Connection"; program: kernel; content: "New,invalid"; content: "TCP"; normalize: kernel; classtype: tcp-connection; sid: 6000000; rev:1;)

With parse_ip; parse_port I get:

New, invalid TCP Connection 2011-10-20 05:45:38 62.135.97.126:514 192.168.1.1:514

With the second one I get the segfault.

So parse_ip is working for the first IP, but not the ports, and possible not the last IP.  This machine is a router…2 nics, so I'm thinking maybe I need to specify the nic with the external IP address, since that's actually where I'm seeing these firewall hits.  Thanks Clark.

James

[Champ Clark III [Quadrant]]

On Oct 20, 2011, at 7:54 AM, Digital X wrote:

Hey Clark!

So here's what I got in sagan.conf:

normalize: kernel, $RULE_PATH/kernel-normalize.rulebase

include $RULE_PATH/kernel.rules

And I've tried:

alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"New, invalid TCP Connection"; program: kernel; content: "New,invalid"; content: "TCP"; parse_ip; parse_port;  classtype: tcp-connection; sid: 6000000; rev:1;)

and

alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"New, invalid TCP Connection"; program: kernel; content: "New,invalid"; content: "TCP"; normalize: kernel; classtype: tcp-connection; sid: 6000000; rev:1;)

With parse_ip; parse_port I get:

New, invalid TCP Connection 2011-10-20 05:45:38 62.135.97.126:514 192.168.1.1:514

With the second one I get the segfault.

So parse_ip is working for the first IP, but not the ports, and possible not the last IP.  This machine is a router…2 nics, so I'm thinking maybe I need to specify the nic with the external IP address, since that's actually where I'm seeing these firewall hits.  Thanks Clark.

That's the nature how parse_ip and parse_port work.  They are very simple log scanning algorithms and could probably be improved a little bit (source is at : https://github.com/beave/sagan/tree/master/src/parsers).  Those parses simply grab what Sagan believes is the first IP address and first port.   I'll probably modify that later,  but it was always meant for simple parsing. However,  for complete normalization,  liblognorm is where it's at.   I'll have to take your rules and run them in my test environment.   Me and other people have been using liblognorm successfully for some time now,  however,  it's very possible you've found a bug.  Until I can replicate the issue,  I wont know what the fix is.  If it's a liblognorm issue,  I'll fix a bug report for Rainer to look into.  

Thanks and I'll let you know the outcome.

[DigiAngel]

Ok...here's what I got...from kernel.rules:

alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"New, invalid TCP
Connection"; program: kernel; content: "New,invalid"; content: "TCP";
normalize: kernel; classtype: tcp-connection; sid: 6000000; rev:1;)

and from kernel-normalize.rules
prefix=[%garbage:number%.%garbage:number%] New,invalid
rule=: IN=ppp0 OUT= MAC= SRC=%src-ip:ipv4% DST=%dst-ip:ipv4% LEN=60
TOS=0x10 PREC=0x00 TTL=60 ID=%garbage:number% DF PROTO=TCP SPT=%src-
port:number% DPT=%dst-port:number% WINDOW=%garbage:number% RES=0x00
SYN URGP=0

And what's unparsed:
unparsed-data="IN=ppp0 OUT= MAC= SRC=70.56.158.130 DST=my_ext_ip
LEN=60 TOS=0x10 PREC=0x00 TTL=60 ID=58738 DF PROTO=TCP SPT=41258
DPT=23 WINDOW=14600 RES=0x00 SYN URGP=0 "]

Is there something I'm missing here? I'm matching the src/dst ip, src/
dst port, and the rest of the variables are garbage that I don't need
to see. Thanks for looking at this.

James

On Oct 20, 11:35 am, "Champ Clark III [Quadrant]"

[DigiAngel]

And even more tries here...I've tried making almost everything that
can have a possible variable reflect it. as well as moving stuff into
prefix= just to see what happens...it's just not matching.

raw syslog entry:
Oct 28 11:13:48 gateway kernel: [110475.092235] New,invalid IN=ppp0

OUT= MAC= SRC=70.56.158.130 DST=my_ext_ip LEN=60 TOS=0x10 PREC=0x00

TTL=60 ID=55693 DF PROTO=TCP SPT=59786 DPT=23 WINDOW=14600 RES=0x00
SYN URGP=0


normalize rule:
prefix=[%garbage:number%.%garbage:number%] New,invalid IN=%int:word%
OUT= MAC=
rule=: SRC=%src-ip:ipv4% DST=%dst-ip:ipv4% LEN=%len:number% TOS=
%tos:number% PREC=%prec:word% TTL=%ttl:number% ID=%garbage:number% DF
PROTO=%proto:word% SPT=%src-port:number% DPT=%dst-port:number% WINDOW=
%garbage:number% RES=%res:word% SYN URGP=%ugrp:number%

normalize debug:
[*] Normalize output: [cee@115 originalmsg="[110475.092235\] New
\,invalid IN=ppp0 OUT= MAC= SRC=70.56.158.130 DST=my_ext_ip LEN=60
TOS=0x10 PREC=0x00 TTL=60 ID=55693 DF PROTO=TCP SPT=59786 DPT=23
WINDOW=14600 RES=0x00 SYN URGP=0 " unparsed-data="SRC=70.56.158.130
DST=my_ext_ip LEN=60 TOS=0x10 PREC=0x00 TTL=60 ID=55693 DF PROTO=TCP
SPT=59786 DPT=23 WINDOW=14600 RES=0x00 SYN URGP=0 "]

[cclark]

I'll look at this ASAP, but I'm a bit slammed with "real work" right
now :)

Also, this is actually more of a liblognorm issue that Sagan. Also,
Rainer
(the author) might also have some ideas as well.

That mailing list is at:

http://lists.adiscon.net/mailman/listinfo/lognorm

Again, I'll look at it ASAP.

[DigiAngel]

Awesome..I'll get on that list as well and start to annoy them as
well :D Thank you!

James

[Da Beave]

DigiAngel,

I had a chance to produce some rules/normalization very similar to
what your doing. These are generic netfilter/iptables Linux kernel
message. That is, these are without any iptables --prefix LOG
customization, just what comes straight out of the box. With slight
modification they should work do you. I've pushed the up to github.
You can check them out at:

The rules:

https://github.com/beave/sagan-rules/blob/master/linux-kernel.rules

rulebase/normalization files:

https://github.com/beave/sagan-rules/blob/master/linux-kernel-normalize.rulebase

The interesting thing is, these type of log message are pretty rich
with correlate'able information. Sagan grabs most of it, but it
could even grab stuff like TOS/Window Size/etc.

Let me know how it works for you.

[DigiAngel]

Thank you. I had a chance to look at these...looks like you're
running bridged, I am not so I made the below modifications:

rule=: %unusedtime:word% IN=%in:word% OUT=%out:word% MAC=%mac:word%
SRC=%src-ip:ipv4% DST=%dst-ip:ipv4% LEN=%len:number% TOS=%tos:word%
PREC=%prec:word% TTL=%ttl:number% ID=%id:number% %DF:word% PROTO=

%proto:word% SPT=%src-port:number% DPT=%dst-port:number% WINDOW=

%window:number% RES=%res:word% %pkt-type:word% URGP=%urgp:number%

This currently does not match:

originalmsg="[364235.641704\] IN=ppp0 OUT= MAC= SRC=95.156.14.20
DST=my_ext_ip LEN=52 TOS=0x00 PREC=0x00 TTL=116 ID=65402 DF PROTO=TCP
SPT=1861 DPT=445 WINDOW=65535 RES=0x00 SYN URGP=0 " unparsed-
data="[364235.641704\] IN=ppp0 OUT= MAC= SRC=95.156.14.20
DST=my_ext_ip LEN=52 TOS=0x00 PREC=0x00 TTL=116 ID=65402 DF PROTO=TCP
SPT=1861 DPT=445 WINDOW=65535 RES=0x00 SYN URGP=0 "]

Thank you.

James

On Oct 30, 5:33 pm, Da Beave <ccl...@quadrantsec.com> wrote:
> DigiAngel,
>
> I had a chance to produce some rules/normalization very similar to
> what your doing.  These are generic netfilter/iptables Linux kernel
> message.  That is,  these are without any iptables --prefix LOG
> customization,  just what comes straight out of the box.  With slight
> modification they should work do you.  I've pushed the up to github.
> You can check them out at:
>
> The rules:
>
> https://github.com/beave/sagan-rules/blob/master/linux-kernel.rules
>
> rulebase/normalization files:
>

> https://github.com/beave/sagan-rules/blob/master/linux-kernel-normali...

[Champ Clark III [Quadrant]]

Yep.  Bridge mode,  but the concept should be about the same.  I didnt have a non-bridged firewall I had access to to 

make rules/rulebase signatures for.  However,  when you get it working,  send it to me :)

Otherwise,  I'll add them in my self later when.  

Champ Clark III
(office) 904.253.7856

(mobile) 850.443.2440 

(SOC) 800.538.9357 ext 101

ccl...@quadrantsec.com
www.quadrantsec.com

[DigiAngel]

Champ,

Could it be possible that netfilter is inserting a space for Ubuntu
after the 0? The above unparsed data shows:
<snip> RES=0x00 SYN URGP=0 "]
Did Sagan insert that space? Or no? I've tried changing:

URGP=%urgp:number%
to
URGP=%urgp:word%

but so far no go.

James

On Oct 31, 9:49 am, "Champ Clark III [Quadrant]"

[Champ Clark III [Quadrant]]

On Oct 31, 2011, at 12:13 PM, DigiAngel wrote:

Champ,

Could it be possible that netfilter is inserting a space for Ubuntu
after the 0?  The above unparsed data shows:
<snip> RES=0x00 SYN URGP=0 "]
Did Sagan insert that space?  Or no?  I've tried changing:

URGP=%urgp:number%
to
URGP=%urgp:word%

but so far no go.

The space is being inserted by netfilter.  So you'll need to add the space at the end.  For example:

"URGP=%urgp:number% "

Note the space _after_ the %urgp:number%! Yeah - it can be that picky.  You'll noticed the rules I pushed to git last night have that same space at the end. 

[DigiAngel]

So...I think I got this identified...it's the leading space thing.
From debugging syslog:

[*] 127.0.0.1|kern|warning|warning|kernel:|2011-10-31|14:10:05|kernel|
[380252.719678] IN=ppp0

Everything else is like the below:

[*] 127.0.0.1|mail|info|info|postfix/qmgr[16128]:|2011-10-31|14:24:47|
postfix| 175A311614E: removed

I tested this with a simple rule called bleh. Rule:
alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"bleh"; content:
"bleh"; classtype: bad-unknown; normalize: bleh; program: bleh; sid:
6000000; rev:1;)

normalize:
prefix=
rule=: %bleh:word%

Command:
echo "127.0.0.1|auth|info|info|bleh:|2011-10-31|13:50:58|bleh| bleh" >
sagan.fifo
[*] Normalize output: [cee@115 originalmsg=" bleh" unparsed-data=""]

Note the space after |bleh|

Now, this command fails:
echo "127.0.0.1|kern|warning|warning|kernel:|2011-10-31|13:50:58|
kernel|bleh" > sagan.fifo
[*] Normalize output: [cee@115 originalmsg="bleh" unparsed-
data="bleh"]

Changing the normalize rule to the below had no effect:
rule=:%bleh:word%

Seems like the Ubuntu, rsyslog, and kernel messages combo neglect the
leading space.