adding a new layer to libcrafter

[kalden]

How would I go about adding a new layer to libcrafter?

[Esteban Pellegrino]

Lets take as an example the TCP layer.

    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Source Port          |       Destination Port        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Sequence Number                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Acknowledgment Number                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Data |           |U|A|P|R|S|F|                               |
   | Offset| Reserved  |R|C|S|S|Y|I|            Window             |
   |       |           |G|K|H|T|N|N|                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           Checksum            |         Urgent Pointer        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

I code a little program to create the *.cpp and .h files of a new layer from a text file to automate the layer creation on libcrafter. What you should do is:

1) Clone the development version of libcrafter and compile this utility (protogen):

git clone https://code.google.com/p/libcrafter/
cd libcrafter/libcrafter/crafter/ProtoSource/
make

2) On that directory you will see a lot of *.src files. Those files are *txt* files used to generate the *.cpp and .h files with the protogen program you just compile. For example, for the TCP layer the text file looks like:

name TCP
protoid 0x06
ShortField  SrcPort     0 0   0
ShortField  DstPort     0 2   80
WordField   SeqNumber   1 0   0
WordField   AckNumber   2 0   0
BitsField   DataOffset  3 0 4 0
BitsField   Reserved    3 4 3 0
TCPFlags    Flags       3 7   0
ShortField  WindowsSize 3 2   5840
XShortField CheckSum    4 0   0
ShortField  UrgPointer  4 2   0

The name of this layer is TCP and the protoid is 0x06. The protoid is the number that should be on the Protocol field on the IP header when this layer is on top of it. With that number, libcrafter knows that a TCP layer should be created after a IP layer with the Protocol field equal to 0x06.
The first column is the type of field, the second is the name of the field, and the last column is the default value of the field. The columns on the middle (sometimes 2 columns, sometimes 3) are parameters that defines the position of the field inside the header and the values depends on the type of the field.

For example,
ShortField  DstPort     0 2   80
means a short field (2 bytes field) on word "zero" (the first word) on byte "two" inside that word.

This one,
BitsField   Reserved    3 4 3 0
is a bit field inside the word "three" starting at bit 4, 3 bits long (in the figure is 6 bits long, but according to RFC3540 is 3 bits long).

You can take a look at the *.src file of different protocols and you should be able to infer the meaning and parameters of each field type.

3) Generate the .cpp and .h for this layer:

$ ./protogen TCP.src
[@] Protocol name = TCP
[@] Protocol ID = 0x06
[@] Protocol size (bits) = 160
[@] Protocol size (bytes) = 20

This will generate 3 files: TCPCraft.cpp, TCPConstructor.cpp and TCP.h. The only file you should edit is TCPCraft.cpp. Inside this file there are four virtual methods that could be implemented on the child class (you can inherit the base class behavior if you want). But you should implement them or delete them from the base class (do not leave the methods in blank).

Craft method
Is the most important method. This method is executed before a packet is sent and is where you set all the fields tedious for the library user. For example is where the checksum should be calculated (you can see examples on the TCPCraft.cpp and UDPCraft.cpp files).

ParseLayerData
Basically, this method is the one that gives information to libcrafter of which is going to be the next layer to be created. If the layer you are creating doesn't contains that information, you just need to inherit the default behavior. The definition of the ParseInfo structure is on Layer.h.

MatchFilter
Returns an expression for the SendRecv method to match an answer from the net using libpcap filters.

ReDefineActiveFields
This method is used only on the ICMP layer. Is for handling situations where the field names or sizes depends on values inside the header.

4) Then copy the three files into the Protocol directory:

cp TCP.h TCPCraft.cpp TCPConstructor.cpp ../Protocols/

5) Then, you should register that protocol into the factory. Add the next lines into the "InitCrafter" function:

gedit ../InitCrafter.cpp

/* +++++ Lines to be added */

        TCP tcp_dummy;
        /* Register the protocol, this is executed only once */
        Protocol::AccessFactory()->Register(&tcp_dummy);

/* +++++ End of lines */

And also, open the Crafter.h header (upper-case)

gedit ../Crafter.h

and add this line at the end of the file

#include "Protocols/TCP.h"

6) Finally, open the Makefile.am file and add the TCP*.cpp and TCP.h into the list.

7) ./configure && make && sudo make install

Now you should be able to create TCP layers on libcrafter programs:

Ethernet ether_layer;

TCP tcp_layer;

Packet pck = ether_layer / tcp_layer;

You should try all this with a DummyProtocol and post here you results or any problems you had.

[kalden]

This will be a great help.  Thanks.  In your example you did:

Packet pck = ether_layer / tcp_layer; 

Was the IP layer just forgotten, or does TCP auto include an IP layer somehow?

Thanks

[Esteban Pellegrino]

Yes, sorry, I forgot the IP layer :-p. Should be

Packet pck = ether_layer / ip_layer / tcp_layer;

[kalden]

Another followup.  What if my layer has a variable length header.  Does the library have facilities for this, or should I have different static layer objects for the different layer lengths?

Thanks

[Esteban Pellegrino]

No support for that yet... Probably on the future. The header should have a fixed size and everything "variable" should go on the payload of the Layer. The payload is at the end of the Layer an can contain an arbitrary numbers of bytes.

[Esteban Pellegrino]

Take a look at DNS  and DHCP layers which have variable header size.

[Jim Lloyd]

Hi, I have a .pcap file with "Null" headers made from sniffing the loopback interface on a BSD unix system (MacOS X). Here is the relevant Wireshark wiki page: http://wiki.wireshark.org/NullLoopback

This header is fixed length of 4 bytes, and meant to be a 32-bit integer containing the protocol family ID, i.e AF_INET for IPv4 or AF_INET6 for IPv6. But unfortunately, the value is specified to be in the machine's native byte order, so it won't be possible to simply declare a constant for the protoid.

I can think of some possible hacks, such as declaring two different protocols for Null IPv4: NullBigEndianIpv4 and NullLittleEndianIpv4. But I wonder if there is a more elegant method for libcrafter to support Null layer headers?

And in the meantime, I know that an IP layer header begins at offset 4 in the raw packets. What is the recipe for constructing valid packets from the packets read in from the .pcap file?

Thanks,

Jim Lloyd

[Esteban Pellegrino]

Hello! The 4 bytes value  on the NULL header is the value of the *next* layer after the null header, right? I think the protoid of the null header should be an arbitrary value because the null header will never be atop of other layer (is like an ethernet layer which have a protoid equal to 0xfff2). And the 4-byte value should be parsed as a WordField and used to create the layer on the top of the null layer.

Seems pretty trivial to do, if you send me a pcap file with captures containing null headers I can do it and commit the changes on the git tree.

Anyway, a workaround in the meantime could be:

#include <iostream>
#include <crafter.h>
#include <boost/shared_ptr.hpp>

using namespace std;
using namespace Crafter;

/* Offset for the first layer (4 for null header) */
const size_t offset = 4;

int main() {
  typedef boost::shared_ptr<Packet> packet_ptr;

  vector<packet_ptr> pck_cont;

  ReadPcap(&pck_cont,"data.pcap");

  vector<packet_ptr>::iterator it_pck;
  for(it_pck = pck_cont.begin() ; it_pck != pck_cont.end() ; it_pck++) {
    /* Get the raw data buffer */
    const byte* read_data = (*it_pck)->GetRawPtr();
    size_t read_length = (*it_pck)->GetSize();
    /*
     * Create a packet from the 4 bytes offset (probably need to find out if is IPv4 or IPv6)
     * I'm going to suppose is IPv4
     */
    Packet ip_pck(read_data + offset, read_length - offset, IP::PROTO);

    /* You got a packet from the IP layer to the top :-) */
    ip_pck.Print();
  }

  return 0;
}

This code will work with the last commit on the git tree (not working on version 0.2).

[Jim Lloyd]

On Wed, Sep 5, 2012 at 6:48 AM, Esteban Pellegrino <pellegre...@gmail.com> wrote:

Hello! The 4 bytes value  on the NULL header is the value of the *next* layer after the null header, right? I think the protoid of the null header should be an arbitrary value because the null header will never be atop of other layer (is like an ethernet layer which have a protoid equal to 0xfff2). And the 4-byte value should be parsed as a WordField and used to create the layer on the top of the null layer.

Ah yes, that seems right.

 

Seems pretty trivial to do, if you send me a pcap file with captures containing null headers I can do it and commit the changes on the git tree.

Ok, will do.

 

Anyway, a workaround in the meantime could be:

Cool. Thanks! 

[Jim Lloyd]

On Wed, Sep 5, 2012 at 6:48 AM, Esteban Pellegrino <pellegre...@gmail.com> wrote:

Hello! The 4 bytes value  on the NULL header is the value of the *next* layer after the null header, right? I think the protoid of the null header should be an arbitrary value because the null header will never be atop of other layer (is like an ethernet layer which have a protoid equal to 0xfff2). And the 4-byte value should be parsed as a WordField and used to create the layer on the top of the null layer.

Seems pretty trivial to do, if you send me a pcap file with captures containing null headers I can do it and commit the changes on the git tree.

See the attached tar ball, which has two short .pcap files, one for IPv4 and one for IPv6. The captures were made by sniffing lo0 with filter "tcp port 80" while executing "wget http://127.0.0.1" and "wget http://localhost" respectively on my Mac running Mac OS 10.7.4.

Thanks,

Jim

 

[Jim Lloyd]

I should note that since the 4 bytes of the null header are in machine byte order, these files are good examples only for little-endian machines. But since the vast majority of computation is now done on x86, this may be moot. 

The other note is that while 2 is the universal value for AF_INET, different values are used for AF_INET6 on different flavors of BSD. From the wireshark wiki page: "AF_INET is 2 on all BSD-based operating systems, as it was introduced at the same time the BSD versions with networking were released; however, AF_INET6, unfortunately, has different values in {NetBSD,OpenBSD,BSD/OS}, {FreeBSD,DragonFlyBSD}, and {Darwin/Mac OS X}, so an IPv6 packet might have a link-layer header with 24, 28, or 30 as the AF_ value."

[Esteban Pellegrino]

Ok, thanks! I guess I should take a look to wireshark code and see how to deal with this stuff.

[Esteban Pellegrino]

The tree is updated with the NullLoopback layer.

[Jim Lloyd]

Very cool, thanks!

[Rana Umar]

Dear Pellegrino

I intend to decode SCTP using libcrafter . i have implemented SCTP protocol like you mentioned for TCP . but i am facing difficulty in while implementing SCTP's chunk layer field . how to implement it in libcrafter . i have seen tcpoptionlayer and tcpoptionpad but i could understand it . can you please guide me how to implement it ?

regards

Umar Majeed

[Esteban Pellegrino]

Hello Umar :)

The main problem implementing that type of layers is that libcrafter don't support fields with variable length. More precisely, don't support fields with a length defined in other field of the header. I'm planning to do that for future releases, but is not an immediate change because I need to do a lot of code refactoring to achieve that.

However, is possible to model that behavior with the current layer layout in a portable way (i.e. future releases won't break your code). And you are in the correct path looking at TCP options, because that is how should be implemented. Layers has a variable portion which is the Layer's payload. The main drawback of that implementation is that decoding a layer can be tricky, but still doable.

First of all, my recommendation would be to implement a layer for each type of chunk (DataChunk, InitChunk, and so on), and the you can just stack them to create a full SCTP Packet :

packet = SCTP / InitChunk / WhateverChunk / etc....

Decoding a packet from the network will be tricky but should be done is a similar manner than IP / TCP options. If you open a new thread about this issue we can follow the topic there.

Best,

Esteban

--
You received this message because you are subscribed to the Google Groups "libcrafter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to libcrafter+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Rana Umar]

Hello Estben :)

Thanks for your kind reply . i was able to both  encode and decode SCTP layer . but yet i have only implemented data chunk not init or other chunk . next i would be deocding m3ua .

libcrafter is awesome :)

Regards

Umar

[Esteban Pellegrino]

Glad to hear that! Thanks :-)

Best,

Esteban