tcpflow freezes for a long time during a pcap processing

[tcpflow user]

Hi,

When I run tcpflow ( 1.4.0b1,64bit,windows) on relatively big pcap file, the program is freezes for a long time (some hours) with full cpu and big ram usage (4 GB in the case) and after a long time the ram is being freed very slowly, in a rate of about 1 MB per 5 minutes (I didn't wait for the end of the process). I think that the problem is happens after the program is finished output files processing (There are many files in the output folder, including HTTPBODY files and there is no harddrive activity) though I haven't evidence for that. 

Is there a way to know what the reason for the problem? Is there a way to disable some unnecessary processes (for example the pdf report producing)?

Unfortunately I can't  upload the pcap file because it's contains private data...

Thank you very much.

[Simson Garfinkel]

Can you send me the command line you are using?

Are you making the PDF output?

How large is the input.

--
You received this message because you are subscribed to the Google Groups "tcpflow-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tcpflow-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[tcpflow user]

The command:

tcpflow64.exe -a -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r pcapFile.pcap -o .\output

Just finished successfully the same command on the same file with tcpflow32.exe. Comparing the results- the 64bit version was stuck approximately in the 1\5 of the process (and not as I wrote in the previous massage), according to the number of the files.

I'm not making the pdf output (I thought it's made automatically).

The file is 2GB pcap.

BTW, Is there a way to avoid report.xml producing? It's take a lot of storage space and in some cases I would like to disable its creation....

[Simson Garfinkel]

Specifying the "-a" option generates the PDF output file. If you don't need it, I suggest you turn off the netviz feature:

tcpflow64.exe -a -x netviz -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r pcapFile.pcap -o .\output

[tcpflow user]

Thanks. I would try to enable only what I need and will write the results.

[tcpflow user]

I have tried the following commands:

tcpflow64.exe -a -x netviz -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

the process was stuck in the same place as the previous.

tcpflow64.exe -E http -x netviz -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

the process was completed successfully but not produced HTTPBODY files

Thanks

[Simson Garfinkel]

-E http turns off all scanners, then enables http.

You probably also want to run the tcpdemux scanner.

I suggest you run with :

tcpflow64.exe -e tcpdemux -e http -e md5 -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

[tcpflow user]

Did you mean 

tcpflow64.exe -E tcpdemux -E http -E md5 -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

?

[Simson Garfinkel]

I do. I’m sorry, bulk_extractor and tcpflow use different command line arguments because ‘-e’ was already being used in tcpflow for something else.

Ignore what I wrote before. Use this:

tcpflow64.exe -x netviz -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

That is, just turn off the netviz.

[tcpflow user]

Again, the process was completed successfully but not produced HTTPBODY files

[Simson Garfinkel]

Try this:

tcpflow64.exe -e http -x netviz -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

If you are using tcpflow 1.4, that should work (the usage seems to be switched back to -e)

You need to ENABLE HTTP and DISABLE NETVIZ.

This should also work:

tcpflow64.exe -e all -x netviz -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

[tcpflow user]

tcpflow64.exe -e http -x netviz -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

I already tried this, but with -E and the program didn't produced HTTPBODY file. with -e it doesn't work.

tcpflow64.exe -e all -x netviz -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

Isn't it the same as:

tcpflow64.exe -a -x netviz -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

? 

[Simson Garfinkel]

On Oct 21, 2013, at 11:04 AM, tcpflow user <tcpflo...@gmail.com> wrote:

tcpflow64.exe -e http -x netviz -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

I already tried this, but with -E and the program didn't produced HTTPBODY file. with -e it doesn't work.

tcpflow64.exe -e all -x netviz -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

Isn't it the same as:

tcpflow64.exe -a -x netviz -T %A/%a/%B/%b/%A.%a-%B.%b%C%c -r .\pcapFile.pcap -o .\output

? 

Yes, that is. What version are you using?

[tcpflow user]

1.4.0b1,64bit,windows

[Simson Garfinkel]

Try -X /dev/null to send report.xml to /dev/null

You are the first person who has asked to suppress it.

On Oct 20, 2013, at 11:30 PM, tcpflow user <tcpflo...@gmail.com> wrote: