[Snort-users] Reliability of signatures
Fraser, Hugh
rule itself, or in the classification. Is there anywhere that the
reliability (ie. the chance of it not reporting a false positive) of the
signature is recorded?
-----Original Message-----
From: Joe Gedeon [mailto:joe.g...@gmail.com]
Sent: Wednesday, February 02, 2011 12:53 PM
To: snort...@lists.sourceforge.net
Subject: [Snort-users] Increase in ASN.1 alerts
Has anyone else noticed an increase in the number of alerts for
SPECIFIC-THREATS ASN.1 constructed bit string? The payload seems
different than the kill-bill script.
An example can be found here.
http://pastebin.com/y9jAigbb
Sincerely,
Joe
--
Registered Linux User # 379282
------------------------------------------------------------------------
------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better
price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires February
28th, so secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________
Snort-users mailing list
Snort...@lists.sourceforge.net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users
------------------------------------------------------------------------------
The modern datacenter depends on network connectivity to access resources
and provide services. The best practices for maximizing a physical server's
connectivity to a physical network are well understood - see how these
rules translate into the virtual world?
http://p.sf.net/sfu/oracle-sfdevnlfb
_______________________________________________
Snort-users mailing list
Snort...@lists.sourceforge.net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users
Martin Holste
> rule itself, or in the classification. Is there anywhere that the
> reliability (ie. the chance of it not reporting a false positive) of the
> signature is recorded?
>
No. There has been a lot of discussion regarding whether or not
something like that would be helpful. I think the short answer is
that environments and preferences vary too widely to be able to
effectively communicate a signature's fidelity. I would also argue
for those same reasons priority should not be suggested either and it
should be deprecated.
I ignore both priority and classification for signatures as they are
terribly broken right now. For instance, the signature "CHAT MSN
messenger http link transmission attempt" is classified as Trojan
activity. Sure, links in an MSN message can point to malware, but I
hardly think that every MSN message with a link in it should be
classified as "Trojan activity." This is not good intel.
An effort is underway to redo the classification system, which is very
welcome. However, I believe the new classification system will be
almost as unhelpful because though more specific, it only allows for a
signature to be placed in one category. I favor a tagging system in
which a signature can have many tags applied to it for a comprehensive
representation of the signature author's intent.
Jim Hranicky
Martin Holste <mcho...@gmail.com> wrote:
> > The snort signatures have a priority associated with them, either in the
> > rule itself, or in the classification. Is there anywhere that the
> > reliability (ie. the chance of it not reporting a false positive) of the
> > signature is recorded?
> >
>
> No. There has been a lot of discussion regarding whether or not
> something like that would be helpful. I think the short answer is
> that environments and preferences vary too widely to be able to
> effectively communicate a signature's fidelity. I would also argue
> for those same reasons priority should not be suggested either and it
> should be deprecated.
Seems like there'd almost need to be a central place that various
entities could report their findings. I know we've got rules that we
rely on heavily and work very well for us, but other than mailing lists
there's no place to report our findings.
Anyone want to volunteer ? Sounds trivial :-p
--
Jim Hranicky
IT Security Engineer
Office of Information Security and Compliance
University of Florida
Martin Holste
> entities could report their findings. I know we've got rules that we
> rely on heavily and work very well for us, but other than mailing lists
> there's no place to report our findings.
>
Hm, you mean like a vote up/down system like StackOverflow.com? That
could be really interesting. It would be very valuable to see what
others are finding to be helpful.
I also think that a basic wiki would be good enough for a SID -> tags
relationship.
Jim Hranicky
Martin Holste <mcho...@gmail.com> wrote:
> > Seems like there'd almost need to be a central place that various
> > entities could report their findings. I know we've got rules that we
> > rely on heavily and work very well for us, but other than mailing lists
> > there's no place to report our findings.
> >
>
> Hm, you mean like a vote up/down system like StackOverflow.com? That
> could be really interesting. It would be very valuable to see what
> others are finding to be helpful.
Sure, something like that - that would actually be very cool.
--
Jim Hranicky
IT Security Engineer
Office of Information Security and Compliance
University of Florida
------------------------------------------------------------------------------
Martin Roesch
> On Fri, 4 Feb 2011 09:13:12 -0600
> Martin Holste <mcho...@gmail.com> wrote:
>
>> > Seems like there'd almost need to be a central place that various
>> > entities could report their findings. I know we've got rules that we
>> > rely on heavily and work very well for us, but other than mailing lists
>> > there's no place to report our findings.
>> >
>>
>> Hm, you mean like a vote up/down system like StackOverflow.com? That
>> could be really interesting. It would be very valuable to see what
>> others are finding to be helpful.
>
> Sure, something like that - that would actually be very cool.
I like that idea too. It'd make a lot of sense to integrate it into
snort.org - in fact there's probably a lot of data about Snort
detection performance, config options and rule quality we could put up
there. Communication favors the defender...
Marty
--
Martin Roesch - Founder/CTO, Sourcefire Inc. - +1-410-290-1616
Sourcefire - Security for the Real World - http://www.sourcefire.com
Snort: Open Source IDP - http://www.snort.org
Martin Holste
>> snort.org - in fact there's probably a lot of data about Snort
>> detection performance, config options and rule quality we could put up
>> there. Communication favors the defender...
>>
Thanks, Marty. I'm all for free resources, but that would make this
project vendor-sponsored, which makes my spider senses tingle... I'd
feel better if a non-profit hosted, or at least a company that doesn't
sell signatures. Otherwise, it'd be like Starbucks sponsoring a
coffee rating site. Up-vote for Trenta!
>
> I would think it would need to have some kind of automatic reporting method,
> perhaps with manual commenting?
> J
What do you mean by automatic? I'd think we'd want this to remain
manual, but as integrated into the analysis process as possible via
whatever GUI you're using. For SF products, a button built into the
GUI, and maybe something to click on in Snorby, et al.? And, of
course, there would need to be the manual vote page on the site. A
basic JSON API to receive submissions would do fine on the web side.
Actually, I could probably code this up this weekend if someone
volunteers a neutral hosting space. Will Jeff Atwood sue if we use
snortoverflow.com?
Martin Holste
>
If all sigs start neutral, then each sig can be categorized as people
get around to it. It seems like a daunting task, but there is a
linear benefit to each signature categorized/rated, so every little
bit helps.
>I was thinking that further refinement effort could be driven by the signatures that are most active at any time, like the way SANS directs their efforts using dshield to identify what's most important. Over time, the most active signatures receive the most attention.
>
That could work, but I wonder if enough people use the default
configuration that it would overpower folks who are tuning. Might be
worth a shot, though.
Nigel Houghton
> On 2/4/11 11:17 AM, Joel Esler wrote:
>> What I was thinking was having a reputation (hit) count score from
>> gid:sid and maybe from the IP involved, then allow people to comment
>> on said results manually.
> good idea. would give you ip reputation also.
We've been working on something, stay tuned :D
--
Nigel Houghton
Head Mentalist
SF VRT Department of Intelligence Excellence
http://vrt-blog.snort.org/ && http://labs.snort.org/
Jason Wallace
see how this could produce any form of reliable data.
1) Isn't accuracy of rules in part reliant on how well the sensor is tuned?
2) Isn't the determination of a legit hit vs. FP partially dependent
on the analysis skill?
3) GID:SID wouldn't be enough. You have to use GID:SID:REV since rev
bumps are often done to fix FP issues.
4) Wouldn't an open submission process/tool be vulnerable to malicious
bad data submissions?
Thx,
Wally
On Fri, Feb 4, 2011 at 11:17 AM, Joel Esler <jes...@sourcefire.com> wrote:
> On Fri, Feb 4, 2011 at 10:51 AM, Martin Holste <mcho...@gmail.com> wrote:
>>
>> >> I like that idea too. It'd make a lot of sense to integrate it into
>> >> snort.org - in fact there's probably a lot of data about Snort
>> >> detection performance, config options and rule quality we could put up
>> >> there. Communication favors the defender...
>> >>
>>
>> Thanks, Marty. I'm all for free resources, but that would make this
>> project vendor-sponsored, which makes my spider senses tingle... I'd
>> feel better if a non-profit hosted, or at least a company that doesn't
>> sell signatures. Otherwise, it'd be like Starbucks sponsoring a
>> coffee rating site. Up-vote for Trenta!
>>
> Vendor sponsored projects are okay I think, especially since we have the
> resources to donate to a project that is going to make everyone's detection
> better.
>
>>
>> > I would think it would need to have some kind of automatic reporting
>> > method,
>> > perhaps with manual commenting?
>> > J
>>
>> What do you mean by automatic? I'd think we'd want this to remain
>> manual, but as integrated into the analysis process as possible via
>> whatever GUI you're using. For SF products, a button built into the
>> GUI, and maybe something to click on in Snorby, et al.? And, of
>> course, there would need to be the manual vote page on the site. A
>> basic JSON API to receive submissions would do fine on the web side.
>>
>> Actually, I could probably code this up this weekend if someone
>> volunteers a neutral hosting space. Will Jeff Atwood sue if we use
>> snortoverflow.com?
>
>
> What I was thinking was having a reputation (hit) count score from gid:sid
> and maybe from the IP involved, then allow people to comment on said results
> manually.
> Using that information could build a high or low reputation score based upon
> actual results, allowing the ruleset to be better tuned and formed, allowing
> reduction of false positives or false negatives.
> Just thinking outloud (which is usually a bad habit)
> Joel
Martin Holste
> resources to donate to a project that is going to make everyone's detection
> better.
>
LOL says the vendor... Seriously though, I guess I'd be ok with SF
hosting as long as they agreed to be source-agnostic and were very
clear about how the whole thing works.
> What I was thinking was having a reputation (hit) count score from gid:sid
> and maybe from the IP involved, then allow people to comment on said results
> manually.
Right, but what is the mechanism for the auto reporting, that is,
what's the criteria?
Martin Holste
>
> 2) Isn't the determination of a legit hit vs. FP partially dependent
> on the analysis skill?
>
Yep, see above.
> 3) GID:SID wouldn't be enough. You have to use GID:SID:REV since rev
> bumps are often done to fix FP issues.
>
Yep, I would actually go with G:S:R along with the SHA1 of the signature.
> 4) Wouldn't an open submission process/tool be vulnerable to malicious
> bad data submissions?
>
Yep. You would have to put in a threshold for submissions of some
sort and see how it goes. Worst-case, a captcha.
In my mind, this only works if each up/down vote is a manual action
done during the course of an investigation. Basically, I want to know
what signatures were helpful to other IR teams during their
investigations. I want to be sure those rules are included in my
ruleset. Obviously, all submissions would have to be anonymous. IP's
would be nice, but then there's a chance someone could mess up src/dst
IP and accidentally de-anonymize themselves.
Martin Holste
>
to aid this discussion?
Nigel Houghton
>> We've been working on something, stay tuned :D
>>
> Um, that intelligence isn't very excellent. Can you be more specific
> to aid this discussion?
Not at the moment, we're still working on it. When we have more data to
share, we will.
--
Nigel Houghton
Head Mentalist
SF VRT Department of Intelligence Excellence
http://vrt-blog.snort.org/ && http://labs.snort.org/
------------------------------------------------------------------------------
Martin Holste
> share, we will.
>
Seriously, are you saying that this entire discussion should be
abandoned right now, or what? What do you want me to do with your
information? Could you at least tell Joel or Marty what you're
planning?
Nigel Houghton
>> Not at the moment, we're still working on it. When we have more data to
>> share, we will.
>>
>
> Seriously, are you saying that this entire discussion should be
> abandoned right now, or what? What do you want me to do with your
> information? Could you at least tell Joel or Marty what you're
> planning?
Nope, keep going with the discussion, we are reading it. It is actually
helping us. I was just letting you all know that we have been thinking
about this problem for a while now and we have been working on
something that we think might help out.
--
Nigel Houghton
Head Mentalist
SF VRT Department of Intelligence Excellence
http://vrt-blog.snort.org/ && http://labs.snort.org/
------------------------------------------------------------------------------
Martin Holste
> interested in.
>
Ok, cool.
So, here's my feedback to SF/ET regarding what will help, and I'll try
to summarize the above comments to be sure I have understood them:
1. Up/down vote per gid:sid:rev my analysts can click on at the tail
end of an investigation to indicate that something's been helpful with
a way to make a note of how it was helpful.
2. Dshield/sidreporter-style automated submissions so that you guys
can see the sigs that are flagging on all kinds of FP's right off the
bat and also to get a cross-section of what IP's are flagging alerts.
3. Up/down vote for category confidence on a given gid:sid:rev.
And, I'd personally add a fourth that I feel is very important:
4. Tag suggestion for a gid:sid:rev with corresponding up/down vote
for confidence.
I personally want to see 1 and 4 implemented ASAP, and they can be
started without retrofitting to all existing signatures. Each datum
contributed is value added.
beenph
ressource or paid subscription you should
give your self a process to filter the detection input you receive and be able
to adapt it for your setup.
If your analyst can't do it there is probably an in your process somewhere.
Martin Holste
> the most important (as measured, perhaps, as the most hits)
Ok, hang on--I'd actually say that you can get a pretty good idea of
the most important signatures by sorting them in ascending order by
hits. The higher the number of hits, the greater probability that
each hit is an FP and the signature isn't helpful. Important caveats
would be for the sigs that aren't alerting on "bad" traffic, but
traffic that is usually good unless it's from a certain IP address
(JAR files, exe files, etc.) or SCAN signatures. That nuance actually
makes this kind of hard to do in a helpful way.
It's for this reason that I want the manual submissions, not based on logs.
Martin Holste
> ressource or paid subscription you should
> give your self a process to filter the detection input you receive and be able
> to adapt it for your setup.
>
We're trying to create a catalyst for that process.
> If your analyst can't do it there is probably an in your process somewhere.
>
I think there's an in YOUR somewhere.
Jason Wallace
On Fri, Feb 4, 2011 at 1:45 PM, Martin Holste <mcho...@gmail.com> wrote:
>> Personally, I'd like to know what
>> the most important (as measured, perhaps, as the most hits)
>
> Ok, hang on--I'd actually say that you can get a pretty good idea of
> the most important signatures by sorting them in ascending order by
> hits. The higher the number of hits, the greater probability that
> each hit is an FP and the signature isn't helpful. Important caveats
> would be for the sigs that aren't alerting on "bad" traffic, but
> traffic that is usually good unless it's from a certain IP address
> (JAR files, exe files, etc.) or SCAN signatures. That nuance actually
> makes this kind of hard to do in a helpful way.
>
> It's for this reason that I want the manual submissions, not based on logs.
>
------------------------------------------------------------------------------
Matthew Jonkman
I don't see real good ways to make that distinction en mass, I certainly wouldn't want to have to mark events that way in addition to the usual handling of events.
I think there is definitely value in just tracking raw hits. Few things off the top of my head:
1. A new sig is out and we get massive numbers of hits, more than should be expected (ie hits per site ratio). That should be a red alert for a bad sig.
2. Once a sig is established for a few weeks and is stable, then any fluctuation is significant. Especially malware sigs and bot stuff. New strain, new outbreak, old strain using a new o-day, etc.
3. Established malware/bot sig, suddenly drops to zero after a period of hits. We're being evaded and it needs attention asap (unless the botnet was infiltrated and killed)
4. Geo location of sources would also be extremely interesting at scale.
These were some of the things I wanted to do with sidreporter, but never had the resources to pursue. I'm sure there are many more things we could infer just from raw hit patterns.
Matt
On Feb 4, 2011, at 12:56 PM, Martin Holste wrote:
> Ok, cool.
>
> So, here's my feedback to SF/ET regarding what will help, and I'll try
> to summarize the above comments to be sure I have understood them:
>
> 1. Up/down vote per gid:sid:rev my analysts can click on at the tail
> end of an investigation to indicate that something's been helpful with
> a way to make a note of how it was helpful.
> 2. Dshield/sidreporter-style automated submissions so that you guys
> can see the sigs that are flagging on all kinds of FP's right off the
> bat and also to get a cross-section of what IP's are flagging alerts.
> 3. Up/down vote for category confidence on a given gid:sid:rev.
> And, I'd personally add a fourth that I feel is very important:
> 4. Tag suggestion for a gid:sid:rev with corresponding up/down vote
> for confidence.
>
> I personally want to see 1 and 4 implemented ASAP, and they can be
> started without retrofitting to all existing signatures. Each datum
> contributed is value added.
>
> ------------------------------------------------------------------------------
> The modern datacenter depends on network connectivity to access resources
> and provide services. The best practices for maximizing a physical server's
> connectivity to a physical network are well understood - see how these
> rules translate into the virtual world?
> http://p.sf.net/sfu/oracle-sfdevnlfb
> _______________________________________________
> Snort-users mailing list
> Snort...@lists.sourceforge.net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
----------------------------------------------------
Matthew Jonkman
Emergingthreats.net
Emerging Threats Pro
Open Information Security Foundation (OISF)
Phone 765-807-8630
Fax 312-264-0205
http://www.emergingthreatspro.com
http://www.openinfosecfoundation.org
----------------------------------------------------
PGP: http://www.jonkmans.com/mattjonkman.asc
beenph
> threshold/event_filter statements will fudge that up too...
>
> On Fri, Feb 4, 2011 at 1:45 PM, Martin Holste <mcho...@gmail.com> wrote:
>>> Personally, I'd like to know what
>>> the most important (as measured, perhaps, as the most hits)
>>
>> Ok, hang on--I'd actually say that you can get a pretty good idea of
>> the most important signatures by sorting them in ascending order by
>> hits. The higher the number of hits, the greater probability that
>> each hit is an FP and the signature isn't helpful. Important caveats
>> would be for the sigs that aren't alerting on "bad" traffic, but
>> traffic that is usually good unless it's from a certain IP address
>> (JAR files, exe files, etc.) or SCAN signatures. That nuance actually
>> makes this kind of hard to do in a helpful way.
>>
>> It's for this reason that I want the manual submissions, not based on logs.
>>
>
>
> I think there's an in YOUR somewhere.
>
A typo, shit happens, english is not my first language.
This aside if you can't create multiple rule set instances for the
same traffic and make clear distinction on
rule importances for each instance, you wont be able to acheive
somethig valuable at the end.
If you are still managing your clients rule with something like
pullthepork or by hand you might also have an issue.
Opensource UI have been lacking for years on "sensor" management, rule
context like thresholding and suppression
(for individual instance or multiple instances or even system wide).
Some of em even ignore revision.
All of this point out to the usage you make of the public available
data and or even subscribed data.
All the changes to a submission system will still be context relevant
and might not apply to you and you will
end up having people who will not try to understand what they actually
see but will basicly rely on that
to apply automatic rules tunning and the problem will still be there
for the "unknown" proportion of people
who do no take time to tune and manage their rule set.
-elz
waldo kitty
> For instance, the signature "CHAT MSN
> messenger http link transmission attempt" is classified as Trojan
> activity. Sure, links in an MSN message can point to malware, but I
> hardly think that every MSN message with a link in it should be
> classified as "Trojan activity." This is not good intel.
agreed which is why i questioned, on another list, the verbiage used in the
snort MSG and classification portion of the rules... in the case that i
questioned, the priority was the same for the classification that fit better to
the rules... the "relaxed" MSG and classification text would not raise hackles
as much as they currently do... the case in question was much the same as you
depict... some traffic that fit a certain rule was classed as "trojan activity"
when it was not and only matched the rule in question...
while i agree that the tags can help in these cases, i'd much rather see the
classifications of rules better conform to what they are truly detecting... an
example is RBN rules... not all traffic from RBN related addresses is trojan or
bad traffic... reclassifying those rules to indicate /possible/ bad traffic is
better than what is currently in place...
the next question is if this is going to be done... i don't specifically recall
any responses to my post in that other thread on this topic, though :?
waldo kitty
>> Personally, I'd like to know what
>> the most important (as measured, perhaps, as the most hits)
>
> Ok, hang on--I'd actually say that you can get a pretty good idea of
> the most important signatures by sorting them in ascending order by
> hits. The higher the number of hits, the greater probability that
> each hit is an FP and the signature isn't helpful.
on the surface, i can't agree with this... in my environment, which has been
carefully tuned for my network(s), i see almost no false positives... almost
every rule alerted on is properly alerted on the contents of the network
packet(s) analyzed... the problem that i've found is that while a packet might
match the rule, the rule MSG is on the "scare" side of the fence such that all
traffic that matches the rule is classified incorrectly... while some traffic
might be classified correctly, the "FP" traffic is not even though it /does/
match the rule in question...
> Important caveats
> would be for the sigs that aren't alerting on "bad" traffic, but
> traffic that is usually good unless it's from a certain IP address
> (JAR files, exe files, etc.) or SCAN signatures. That nuance actually
> makes this kind of hard to do in a helpful way.
i think i see what you are saying and that i can agree with it ;)
> It's for this reason that I want the manual submissions, not based on logs.
+1.5 with a caveat that this means more manual labor for those who are already
stuffed to the gills if they want to contribute... i'm not sure, off the top of
my head, how this might be handled... especially in an environment where there
is no reporting participation capabilities in place :?
waldo kitty
> All the changes to a submission system will still be context relevant
> and might not apply to you and you will
> end up having people who will not try to understand what they actually
> see but will basicly rely on that
> to apply automatic rules tunning and the problem will still be there
> for the "unknown" proportion of people
> who do no take time to tune and manage their rule set.
i can agree almost 100% on this paragraph... my tool and environment require
tuning for the network being protected... even if one doesn't use my tool, they
must tune snort's rules for their network or else they will be quite overrun by
alerts that are not problematic for their network...
i had the chance to work (from remote) on a system over in norway the other
week... their snort was quite overloaded with "too many small tcp packets"
alerts... it didn't take me long to discover that they have several cisco
products in their setup... is also didn't take me long to discover that they
have a lot of snmp traffic even though they are not using much/any of it... they
may not even know that it is available or they may simply not have the tools to
utilize the information in that snmp traffic...
anyway, once i thresholded several snort rules and completely disabled other
extremely talkative ones, it was much easier to see things on their network that
were of interest and indicating possible problems... the sad part of this tale
is that i've been working with them for over a year and describing the necessity
and method of tuning but this was the first time that i had a chance to actively
enter their system and do it myself... it was quite satisfying to get things
cleaned up enough for them to actually start assisting in the protection of
their network rather than them spending so much time wading thru cr4p alerts and
basically giving up because of being overwhelmed...
Fraser, Hugh
-----Original Message-----
From: Martin Holste [mailto:mcho...@gmail.com]
Sent: Friday, February 04, 2011 10:52 AM
To: Joel Esler
Cc: Martin Roesch; snort...@lists.sourceforge.net; Fraser, Hugh
Subject: Re: [Snort-users] Reliability of signatures
>> I like that idea too. It'd make a lot of sense to integrate it into
>> snort.org - in fact there's probably a lot of data about Snort
>> detection performance, config options and rule quality we could put
>> up there. Communication favors the defender...
>>
Thanks, Marty. I'm all for free resources, but that would make this project vendor-sponsored, which makes my spider senses tingle... I'd feel better if a non-profit hosted, or at least a company that doesn't sell signatures. Otherwise, it'd be like Starbucks sponsoring a coffee rating site. Up-vote for Trenta!
>
> I would think it would need to have some kind of automatic reporting
> method, perhaps with manual commenting?
> J
What do you mean by automatic? I'd think we'd want this to remain manual, but as integrated into the analysis process as possible via whatever GUI you're using. For SF products, a button built into the GUI, and maybe something to click on in Snorby, et al.? And, of course, there would need to be the manual vote page on the site. A basic JSON API to receive submissions would do fine on the web side.
Actually, I could probably code this up this weekend if someone volunteers a neutral hosting space. Will Jeff Atwood sue if we use snortoverflow.com?
------------------------------------------------------------------------------
Fraser, Hugh
make a decision, but I use OSSIM as the decision making tool and
incorporate a lot of other sources of information (including a
commercial IDS/IPS) along with associated correlation rules to try to
make an intelligent decision about what's happening. What I'm looking
for is less the characterization, but more the "how sure are we this is
really what we think it is" to help with the calculation of risk.
-----Original Message-----
From: Martin Holste [mailto:mcho...@gmail.com]
Sent: Friday, February 04, 2011 9:51 AM
To: Fraser, Hugh
Cc: snort...@lists.sourceforge.net
Subject: Re: [Snort-users] Reliability of signatures
> The snort signatures have a priority associated with them, either in
> the rule itself, or in the classification. Is there anywhere that the
> reliability (ie. the chance of it not reporting a false positive) of
> the signature is recorded?
>
No. There has been a lot of discussion regarding whether or not
something like that would be helpful. I think the short answer is that
environments and preferences vary too widely to be able to effectively
communicate a signature's fidelity. I would also argue for those same
reasons priority should not be suggested either and it should be
deprecated.
I ignore both priority and classification for signatures as they are
terribly broken right now. For instance, the signature "CHAT MSN
messenger http link transmission attempt" is classified as Trojan
activity. Sure, links in an MSN message can point to malware, but I
hardly think that every MSN message with a link in it should be
classified as "Trojan activity." This is not good intel.
An effort is underway to redo the classification system, which is very
welcome. However, I believe the new classification system will be
almost as unhelpful because though more specific, it only allows for a
signature to be placed in one category. I favor a tagging system in
which a signature can have many tags applied to it for a comprehensive
representation of the signature author's intent.
Fraser, Hugh
asked for represents two views of the signature world; my local view,
and the community's view as a whole. Personally, I'd like to know what
the most important (as measured, perhaps, as the most hits) 20 or so
signatures in the past day, week, and month. I'd like to know trends,
since they're often good predictors of what the bad guys are up to,
leading to an idea of what I should be watching for. And I'd like to be
able to rate each of those top 20 things based upon their impact on my
environment. As Martin says, it's a manual thing. To me, this is a
simple application that takes the snort log files (or database records)
and does some statistical analysis, and a form to present the results
and allow me to rate the signatures.
I'd then like to be able to submit this information somewhere, and
receive daily the same stats aggregated for the entire community,
perhaps embedded within my snort signature feed. In the form I use for
rating my private part of the world, I could also see results for the
community as well, again to give me a heads up about what's happening
from a higher altitude. This is what dshield does with network traffic,
and I'm sure they have some analysis going on in the background that
throttles multiple submissions and tosses statistically insignificant
results to ensure that the stats aren't easilly distorted by malicious
reports.
I'd certainly make reporting this part of our incident response process
here, and would likely check the information daily to stay current with
emerging trends. Add a link to a web site for a discussion area for the
current top hitters, and I'm in heaven.
-----Original Message-----
From: Martin Holste [mailto:mcho...@gmail.com]
Sent: Friday, February 04, 2011 11:52 AM
To: Jason Wallace
Cc: snort...@lists.sourceforge.net; Martin Roesch
Subject: Re: [Snort-users] Reliability of signatures
------------------------------------------------------------------------
------