Absurd memory footprint!

[Leonardo da Silva]

Hi folks,

I'm trying to parse a large pcap file, the main loop looks something like this:

/?

std::vector<Packet*> packets;

 while(packet = pcap_next(pcap_handle, &header)) {

        Packet* pkt;        

        pkt = new Packet(packet, header.len, Ethernet::PROTO);

        packets.push_back(pkt);

}

On a 1GB pcap file it drains out all the 8GB RAM plus 8GB swap memory and then crashes. Am i doing something wrong or is it the expected memory consumption?

Thanks in advance,

Leo

[Esteban Pellegrino]

I'm not sure about the rate of the real memory of a packet / libcrafter memory of a packet but is expected to have a much more memory usage. the main issue is that current implementation of libcrafter saves inside each layer the name of the protocol ("Ethernet", "IP", etc) plus the information of each one of the fields of the protocol, including the name,

--
You received this message because you are subscribed to the Google Groups "libcrafter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to libcrafter+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Esteban Pellegrino | Software Developer
Twitter | Zimperium.com

[Leonardo da Silva]

Hi Esteban,

Yeah, it's impossible to know the real memory taken by a packet because the human readable and metadata will actually define it. The consumption is really high as for the benchmarks i'm doing at the moment although slim memory consumption is not the main deal of the library which by the way kicks ass (Congratulations)  :) ! I will solve my problem by reading some feasible amount of data and then dealing with the closed tcp streams. Btw i will do a pull request soon. I'm a bit rusty on c++. Long time i don't use it.

Cheers,

Leo

[Esteban Pellegrino]

thanks :-) looking forward for the contribution.

cheers,

Esteban