Drop rate
Jaime Nebrera
I have noticed snort drops packets even at moderate packet per second rates
May I ask the published results what kind of limit to drop rate were
considered?
Enviado desde mi iPhone
livio Ricciulli
Jaime Nebrera
If packet drop was 0 why stop at that level?
Of course we see 0 drop rate at low speeds, I'm asking when should be
considered a test has failed, 0,5%, 1%?
Enviado desde mi iPhone
livio Ricciulli
threshold.
The other thing is that you should measure sustained throughput so that
you fill up all the buffers.
We measure total packets transmitted at a constant rate V.S. total
packets processed/forwarded over traces long enough to reduce
the measurement errors due to buffering.
Also, in our experience, the type and composition of traffic is a huge
factor; so absolute numbers are kind of indicative.
That is why in all our tests, we emphasize the trend as a function of
sustained bandwidth; this way you can compare relative
performance by fixing the traffic type and varying the setup (drivers,
number of processes, number of rules, etc..).
I wish somebody proposed a set of relevant public traces and an
affordable, reproducible traffic generation methodology so that we
could all compare performance numbers based on a scientific process
rather than anecdotal results. Lincoln Labs at
one point published some traces for a DARPA IDS evaluation project but
those traces are way too old for today's requirements..
Livio.
On 03/06/2012 12:56 PM, Jaime Nebrera wrote:
> Hi Livio,
>
> If packet drop was 0 why stop at that level?
>
> Of course we see 0 drop rate at low speeds, I'm asking when should be
> considered a test has failed, 0,5%, 1%?
>
> Enviado desde mi iPhone
>
> El 06/03/2012, a las 21:41, livio Ricciulli<li...@metaflows.com> escribi�:
Jaime Nebrera
> It is arbitrary; but as you can see from our graphs we use <5% as the
> threshold.
Mmm, I didnt notice. We were using 0,5% and thus were getting much
lower results !!
> The other thing is that you should measure sustained throughput so
> that you fill up all the buffers.
> We measure total packets transmitted at a constant rate V.S. total
> packets processed/forwarded over traces long enough to reduce
> the measurement errors due to buffering.
Ok
> Also, in our experience, the type and composition of traffic is a huge
> factor; so absolute numbers are kind of indicative.
> That is why in all our tests, we emphasize the trend as a function of
> sustained bandwidth; this way you can compare relative
> performance by fixing the traffic type and varying the setup (drivers,
> number of processes, number of rules, etc..).
I fully agree here
> I wish somebody proposed a set of relevant public traces and an
> affordable, reproducible traffic generation methodology so that we
> could all compare performance numbers based on a scientific process
> rather than anecdotal results. Lincoln Labs at
> one point published some traces for a DARPA IDS evaluation project but
> those traces are way too old for today's requirements..
As related to public traces, the best listing we were able to find was:
http://sourceforge.net/apps/mediawiki/networkminer/index.php?title=Publicly_available_PCAP_files
And from here, our particular favorite is:
http://www.itoc.usma.edu/research/dataset/index.html
As includes recent traffic (2009) and both internal network (lightly
risk) and external traffic (heavy attacked)
But our main lack is how to take one of those nice traces and
reproduce it in a stateful way those pcap files at high speed and be
able to get packets dropped as well as latency.
I fought quite a bit with tcpreplay and companions that allowed me to
make this bibirectional and stateful, but the speed was not that great
and was extremelly complex to get packet drop information and was
completelly unable to get latency value.
Of course, Ixia and friends do this, but well, you know the price tag :D
Right now we are just generating synthetic UDP traffic.