Parse a Wireshark pcap file

[Kevin Holleran]

Is there a way to parse out a wireshark pcap file and extract key value pairs from the data?  I am illustrated a sniff of some traffic and why it needs utilize HTTPS instead of HTTP but I was hoping to run the pcap through a python script and just output some interesting key value pairs.... 

Thanks for your help.

Kevin

[Kevin Holleran]

Thanks, I have been trying to get it to work but I am on Mac OS 10.8.2.  I tried to get it from Macports and download/install it myself.  Both seem to get me to here:

ImportError: No module named dnet

I tried to download libdnet but no matter what I do this is what I get.  Granted I am doing;

from scapy.all import *

But I have no idea what I need.  I am not trying to craft packets but filter packets based on tcp.dstport 80 & frame matches signin.aspx.  Then my goal is to parse the data looking for post vars txtUserId & txtPwd and extract them, dumping them to the screen as userid_value => password.

Thanks for your help.

--
Kevin Holleran
Master of Science, Computer Information Systems
Grand Valley State University
Master of Business Administration
Western Michigan University
SANS GCFA, SANS GCFE, CCNA, ISA, MCSA, MCDST, MCP

"Do today what others won't, do tomorrow what others can't" - SEALFit

"We are what we repeatedly do. Excellence, then, is not an act, but a habit." - Aristotle

On Tue, Jan 22, 2013 at 10:03 PM, Dave Angel <d...@davea.name> wrote:

Sure.  scapy can create and/or parse pcap files.

http://pypi.python.org/pypi/Scapy


--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

[Dave Angel]

[Kevin Holleran]

I also found this:

http://code.google.com/p/py-greppcap/

Which I can leverage to do what I want but I also get that dnet error!

--
Kevin Holleran
Master of Science, Computer Information Systems
Grand Valley State University
Master of Business Administration
Western Michigan University
SANS GCFA, SANS GCFE, CCNA, ISA, MCSA, MCDST, MCP

"Do today what others won't, do tomorrow what others can't" - SEALFit

"We are what we repeatedly do. Excellence, then, is not an act, but a habit." - Aristotle

On Tue, Jan 22, 2013 at 10:15 PM, Kevin Holleran <kda...@gmail.com> wrote:

Thanks, I have been trying to get it to work but I am on Mac OS 10.8.2.  I tried to get it from Macports and download/install it myself.  Both seem to get me to here:

ImportError: No module named dnet

I tried to download libdnet but no matter what I do this is what I get.  Granted I am doing;

from scapy.all import *

But I have no idea what I need.  I am not trying to craft packets but filter packets based on tcp.dstport 80 & frame matches signin.aspx.  Then my goal is to parse the data looking for post vars txtUserId & txtPwd and extract them, dumping them to the screen as userid_value => password.

Thanks for your help.

--
Kevin Holleran
Master of Science, Computer Information Systems
Grand Valley State University
Master of Business Administration
Western Michigan University
SANS GCFA, SANS GCFE, CCNA, ISA, MCSA, MCDST, MCP

"Do today what others won't, do tomorrow what others can't" - SEALFit

"We are what we repeatedly do. Excellence, then, is not an act, but a habit." - Aristotle

On Tue, Jan 22, 2013 at 10:03 PM, Dave Angel <d...@davea.name> wrote:

--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

[Dave Angel]

On 01/22/2013 10:15 PM, Kevin Holleran wrote:
> Thanks, I have been trying to get it to work but I am on Mac OS 10.8.2. I
> tried to get it from Macports and download/install it myself. Both seem to
> get me to here:
>
> ImportError: No module named dnet
>
> I tried to download libdnet but no matter what I do this is what I get.
> Granted I am doing;
>
> from scapy.all import *
>
>
> But I have no idea what I need. I am not trying to craft packets but
> filter packets based on tcp.dstport 80 & frame matches signin.aspx. Then
> my goal is to parse the data looking for post vars txtUserId & txtPwd and
> extract them, dumping them to the screen as userid_value => password.
>

I've never worked on Mac OSx And the only times I had and used scapy
were on a work machine that's long gone. I still run Linux, but
versions of everything have changed since then.

I don't know if there's anyone here that's more current with scapy
and/or with Mac, but in case there is, you could be lots clearer about
what you're doing and how it fails.

Version of OS. You said that well.
Version of Python,.
Exact location you got scapy from, what version it was
How you installed it (I don't know Macports, clearly)
What the full traceback was when it died.



--
DaveA

[Kevin Holleran]

--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Noted, I will try to be more verbose.

Mac OS 10.8.2

Python v.2.7

I downloaded from the sourceforge site, then tried to install with MacPorts when some dependencies were failing.  I then downloaded & installed pcapy-0.10.6 when that dependency still failed.  That solved that but I received the dnet error:

    from scapy.all import conf

  File "/Library/Python/2.7/site-packages/scapy/all.py", line 16, in <module>

    from arch import *

  File "/Library/Python/2.7/site-packages/scapy/arch/__init__.py", line 75, in <module>

    from bsd import *

  File "/Library/Python/2.7/site-packages/scapy/arch/bsd.py", line 12, in <module>

    from unix import *

  File "/Library/Python/2.7/site-packages/scapy/arch/unix.py", line 20, in <module>

    from pcapdnet import *

  File "/Library/Python/2.7/site-packages/scapy/arch/pcapdnet.py", line 160, in <module>

    import dnet

ImportError: No module named dnet

So I downloaded and compiled libdnet-1.11 with a:

 $ sudo ./configure && make

I see it compile fine & the libraries have been installed to:

/usr/local/sbin/dnet

However, python can't find it... I am not clear on how to point Python there...

Thanks again.

Kevin

[John Evans]

The import "from scapy.all import *" does work for me with macports and 10.6.8  When I installed the scapy port, I did see that macports installed the py27-libdnet package as well.  

On Wed, Jan 23, 2013 at 1:24 AM, Dennis Lee Bieber <wlf...@ix.netcom.com> wrote:

On Tue, 22 Jan 2013 22:43:24 -0500, Kevin Holleran <kda...@gmail.com>

declaimed the following in gmane.comp.python.general:

>

        "libdnet" is likely a shared object binary... What I /think/ you are
missing is the Python library that interfaces with that binary...

        Could http://pypi.python.org/pypi/dnet answer the question?

--
        Wulfraed                 Dennis Lee Bieber         AF6VN
        wlf...@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

--
http://mail.python.org/mailman/listinfo/python-list

--
John Evans

[Kevin Holleran]

--
http://mail.python.org/mailman/listinfo/python-list

I downloaded scapy manually since for some reason, after using macports, it wouldn't find the package at all. 

I am also trying to install libdnet manually as mentioned above, so after ./configure && make I go into the python directory & do a  python setup.py install, which generates a bunch of warnings & the following two errors:

/dnet.c:2729:4: error: assignment to cast is illegal, lvalue casts are not supported

  ((PyObject*)__pyx_v_next) = Py_None; Py_INCREF(((PyObject*)__pyx_v_next));

  ~^~~~~~~~~~~~~~~~~~~~~~~~ ~

./dnet.c:2741:6: error: assignment to cast is illegal, lvalue casts are not supported

    ((PyObject *)__pyx_v_next) = __pyx_3;

    ~^~~~~~~~~~~~~~~~~~~~~~~~~ ~

Thanks again for any help.  Need to get all this working for this mini-project and also because I am starting a SANS class that leverages scapy quite a bit...

Kevin

[John Evans]

It looks like there was some very recent reorganization of the scapy packaging on macports, see http://lists.macosforge.org/pipermail/macports-dev/2013-January/021620.html

Did you have an updated port tree when you installed?  If not, I'd suggest uninstalling whatever port you installed, resync with "port selfupdate", then install the scapy port again.  You should see not only the libdnet port installed, but also py27-libdnet, which is the missing glue you need for interfacing with libdnet.

--
John Evans

[Grant Edwards]

On 2013-01-23, Kevin Holleran <kda...@gmail.com> wrote:

> Is there a way to parse out a wireshark pcap file and extract key value
> pairs from the data?

You can use pylibpcap to read pcap files (or to capture live data).
I'm afraid I don't know what "parse out" or "extract key value pairs"
means. pylibpcap doesn't have access to any of wireshark's packet
disecter plugins, if that's what you're after.

> I am illustrated a sniff of some traffic and why it needs utilize
> HTTPS instead of HTTP but I was hoping to run the pcap through a
> python script and just output some interesting key value pairs....

To what does "key value pairs" refer?

--
Grant Edwards grant.b.edwards Yow! I am a traffic light,
at and Alan Ginzberg kidnapped
gmail.com my laundry in 1927!

[1991mani...@gmail.com]

I have a pcap file, I want to parse that file & fetch some information like Timestamp, Packet Size, Source/Dest IP Address, Source/Dest Port, Source/ Dest MAC address.

I am trying this in Django.

other that Source/ Dest Port details, I am able to fetch everything.
Please tell me how I can get port details from pcap file.

This is my python code: https://github.com/manishkk/pcap-parser/blob/master/webapp/views.py

Thanking you in advance

[Michiel Overtoom]

> On 2016-12-27, at 20:46, 1991mani...@gmail.com wrote:
>
> I have a pcap file, I want to parse that file & fetch some information like Timestamp, Packet Size, Source/Dest IP Address, Source/Dest Port, Source/ Dest MAC address.

pcapy can do this.

import pcapy
pcap = pcapy.open_offline("httpsession.pcap")

def callback(hdr, data):
... do something with hdr and data, which is the captured packet

pcap.loop(0, callback)

Greetings,

[Grant Edwards]

On 2016-12-27, 1991mani...@gmail.com <1991mani...@gmail.com> wrote:
>
> I have a pcap file, I want to parse that file & fetch some
> information like Timestamp, Packet Size, Source/Dest IP Address,
> Source/Dest Port, Source/ Dest MAC address.

I've been using pylibpcap for ages. It's a bit old, but still works
fine for me:

https://sourceforge.net/projects/pylibpcap/

There's also pypcap:

https://github.com/pynetwork/pypcap

--
Grant Edwards grant.b.edwards Yow! Now KEN and BARBIE
at are PERMANENTLY ADDICTED to
gmail.com MIND-ALTERING DRUGS ...