PCAP Indexing?

[TOoSmOotH]

Is it possible to index PCAP as it writes it to disk? I really like netsniff-ng as it scales well with high traffic but the downside to that is a lot of pcap. This means searching through the pcap takes a long time especially when there are lots of writes going on. 

Thanks!

[Daniel Borkmann]

Thanks for using netsniff-ng!

Currently, there is no such a feature built-in. I agree that on huge
pcap files, searching through it might be a bit of a pain, even in
case of an efficient BPF filter for an offline analysis. There is such
as thing as pcapIndex [1], but from what I know seems to be patented.

[1] http://www.sigcomm.org/node/3230

[Daniel Borkmann]

Let me think about it and maybe in short till mid-term future we will
come up with a solution.

[Daniel Borkmann]

For now, maybe https://github.com/taterhead/PCAP-Index can be of help for you.

[Markus Amend]

I think a pcap-indexer in netsniff-ng is missing. Netsniff-ng is a great
tool with a lot of performance in handling (especially in recording) network
traffic, but it's not performant to get the informations out of it. An
indexer could help to get informations faster especially when the same
network traffic is examined again and again. Also an intelligent search
instead of bpf is possible :-)

We should think about hacking something like a pcap-indexer in the future.

-----Ursprüngliche Nachricht-----
Von: netsn...@googlegroups.com [mailto:netsn...@googlegroups.com] Im
Auftrag von Daniel Borkmann
Gesendet: Donnerstag, 4. Oktober 2012 14:57
An: netsn...@googlegroups.com
Betreff: Re: [netsniff-ng] PCAP Indexing?

--

[Daniel Borkmann]

On Thu, Oct 4, 2012 at 3:42 PM, Markus Amend <mar...@netsniff-ng.org> wrote:
> I think a pcap-indexer in netsniff-ng is missing. Netsniff-ng is a great
> tool with a lot of performance in handling (especially in recording) network
> traffic, but it's not performant to get the informations out of it. An
> indexer could help to get informations faster especially when the same
> network traffic is examined again and again. Also an intelligent search
> instead of bpf is possible :-)
>
> We should think about hacking something like a pcap-indexer in the future.

Agreed.

> --
>
>

[TOoSmOotH]

Right now cxtracker supports it but I can't get the same performance out of cxtracker as I do with netsniff when it comes to full packet capture on high speed links. I can use another process to do it but then I am putting some hurt on the IO which makes the most sense to do it as its written. :) Let me know if you all move forward with something as I would be glad to test it for you. I have several multi-gig sensors running netsniff today for FPC.

Thanks!! 

[Daniel Borkmann]

On Thu, Oct 4, 2012 at 7:20 PM, TOoSmOotH <reev...@gmail.com> wrote:
> Right now cxtracker supports it but I can't get the same performance out of
> cxtracker as I do with netsniff when it comes to full packet capture on high
> speed links. I can use another process to do it but then I am putting some
> hurt on the IO which makes the most sense to do it as its written. :) Let me
> know if you all move forward with something as I would be glad to test it
> for you. I have several multi-gig sensors running netsniff today for FPC.

Thanks, good to know.

Just out of curiosity, what capturing speed do you achieve with netsniff-ng?

Thanks,
Daniel

> --
>
>

[leero...@gmail.com]

Hey, can some one please explain me how to implement the taterhead project please ?