feature request: easy packet mangling

[phil]

Hi Matias,

I've done some stuff with your library now and it's really pretty easy to use. Thank you very much for sharing your great work!

I think a very useful feature could be to integrate any kind of packet mangling, enabling the programmer to modify packets before they are delivered to their final destination. With this feature your library would be great for the development of open source network security software (not only monitoring but also active prevention like active intrusion prevention systems).

One of the possible ways to do that is the nfqueue module in iptables. Sadly I didn't find an easy way to use the nfqueue features with your library but it would simplify packet mangling a lot.

There is a good example for the use in python with a similar library like libtins called "scapy" for manipulating the TTL (copied from http://blog.yancomm.net/2011/05/nfqueue-packet-mangling-with-python.html):

in the terminal:
iptables -A OUTPUT -p tcp -j NFQUEUE


python script with use of scapy:

    import nfqueue
    from scapy.all import IP

    q = None

    def cb(dummy, payload):
            pkt = IP(payload.get_data())

            # set the TTL
            pkt.ttl = 10

            # clear the IP checksum so that Scapy recalculates it, since we modified the IP header
            del pkt.chksum

            # reinject the packet!
            payload.set_verdict_modified(nfqueue.NF_ACCEPT, str(pkt), len(pkt))

    q = nfqueue.queue()
    q.open()
    q.bind(socket.AF_INET)
    q.set_callback(cb)
    q.create_queue(0)

    try:
            q.try_run()
    except KeyboardInterrupt:
            print "Exiting..."

    q.unbind(socket.AF_INET)
    q.close()

Maybe there's also an easy way to do it with libtins but for now I didn't find a good solution.

I would be glad to get some feedback for this feature request.

Thank you,

Philipp

[Matias Fontanini]

Seems like this is pretty easy to do (there's this tutorial here https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/). I don't know about adding this to the library, since it's Linux specific. I would however add an example on how to use libnetfilter_queue + libtins to do this.

[phil]

Yes a tutorial for the use of libtins together with libnetfilter_queue would be very great and I'm sure it would help many people.

But that's not the full essence of my feature suggestion. What I tried to explain with libnetfilter_queue was just an example. The point I tried to reach is to develop an option to use libtins directly for packet mangling without any other libraries.
It would be great if there's a way to get the packets out of the network queue (/out of the kernel), modify them and then give it back to the network queue (/the kernel) for further handling. I know that's exactly what libnetfilter_queue allows us to do but a full integration of this feature for all plattforms directly included in libtins would be really great - it would be a unique feature against the other packet monitoring and crafting libraries. Sadly I've got no idea how to realize that but it offers new possiblities - more than just monitoring and crafting - it would take "nearly real time" manipulations to reality without duplicated packets (which happens sometimes while crafting DNS spoofing packets etc.)..

With this option it would be easy to use libtins as a filter system while running it on a network gateway. It would be possible to gain full control over all packet headers without the need to use other additional software.

Thank you so much for your great work, maybe there's a way how to do this in the future without too much effort,

Regards,

Philipp

[phil]

I've tried to use libtins for packet manipulation with libnetfilter_queue for a while now.
Sadly I wasn't able to get it working.

The IP-Data could be retrieved with this example code (usable within the callback function in the sample file nfqnl_test.c):

nfq_get_payload(tb, &data)
struct iphdr * ip_info = (struct iphdr *)data;

After that we've got a pointer for an iphdr structure (like the standard iphdr structure in linux (from ip.h)).
What I didn't get working: How do I create a libtins-PDU with this iphdr structure pointer? I can't find a constructor for this task (I've tried it with RawPDU and also with the IP-constructors).
It wold be great if it's possible to keep it as an iphdr pointer so that we could modify the header directly without the need to give it back to the original data format seperately.

Am Mittwoch, 7. Oktober 2015 21:34:56 UTC+2 schrieb Matias Fontanini:

[Matias Fontanini]

You have to use the IP constructor that takes a pointer and a size:

IP pkt(data, data_size);

I think the data size is the return value of nfq_get_payload.

And keeping a pointer to the original IP header just won't work with the way the library is designed. Basically each packet is constructed from the original buffer but it's completely decoupled from it. If you want to re-inject the packet, you need to serialize it (using pkt.serialize()) and then copy the contents of the buffer back to the original buffer. 

This might be overkill, depending on what you're trying to do though. If you just want to do something super simple like changing the TTL, you might as well just do something like:

struct iphdr * ip_info = (struct iphdr *)data;

ip_info->ttl = 42;

[phil]

Thanks for your kind support and your hints. You're right it's an overkill for such simple things like ttl modifications but it's very pretty if you want to get control over higher OSI-levels (TCP for example).

Here are the steps as a little guide for the use with libnetfilter_queue on Debian systems:

• Install libtins as usual (guide in the download section)
  
• Install libnetfilter_queue with the following commands:

sudo apt-get install libnfnetlink-dev

sudo apt-get install libnetfilter-queue-dev

 

• Compile the sample code with the following command:
  

g++ -o test test.cpp -lnfnetlink -lnetfilter_queue -ltins

And here's the sample file including a TTL modifaction with the libtins library - based on the sample file from the libnetfilter_queue library:

The code for the libtins-part including the copy-back mechanism to the original buffer is highlighted in bold. The rest is still the original sample file from libnetfilter_queue.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <netinet/in.h>
#include <linux/types.h>
#include <linux/netfilter.h>        /* for NF_ACCEPT */
#include <errno.h>

#include <libnetfilter_queue/libnetfilter_queue.h>

#include <iostream>
#include <tins/tins.h>

using namespace Tins;

/* returns packet id */
static u_int32_t print_pkt (struct nfq_data *tb)
{
    int id = 0;
    struct nfqnl_msg_packet_hdr *ph;
    struct nfqnl_msg_packet_hw *hwph;
    u_int32_t mark,ifi;
    int ret;
    unsigned char *data;
    ret = nfq_get_payload(tb, &data);

    struct iphdr * ip_info = (struct iphdr *)data;

    IP mypack(data, ret);
    std::cout << "TTL original:" << unsigned(mypack.ttl());
    mypack.ttl(64);
    std::cout << ", new libtins TTL:" << unsigned(mypack.ttl());
    std::vector<unsigned char> v(mypack.size());
    v = mypack.serialize();
    std::copy(v.begin(), v.end(), data);
    ret = nfq_get_payload(tb, &data);
    struct iphdr * ip_infon = (struct iphdr *)data;
    IP mypack2(data, ret);
    std::cout << ", manipulated IP-packet TTL " << unsigned(mypack2.ttl());
    ph = nfq_get_msg_packet_hdr(tb);
    if (ph) {
        id = ntohl(ph->packet_id);
        printf("hw_protocol=0x%04x hook=%u id=%u ",
            ntohs(ph->hw_protocol), ph->hook, id);
    }

    hwph = nfq_get_packet_hw(tb);
    if (hwph) {
        int i, hlen = ntohs(hwph->hw_addrlen);

        printf("hw_src_addr=");
        for (i = 0; i < hlen-1; i++)
            printf("%02x:", hwph->hw_addr[i]);
        printf("%02x ", hwph->hw_addr[hlen-1]);
    }

    mark = nfq_get_nfmark(tb);
    if (mark)
        printf("mark=%u ", mark);

    ifi = nfq_get_indev(tb);
    if (ifi)
        printf("indev=%u ", ifi);

    ifi = nfq_get_outdev(tb);
    if (ifi)
        printf("outdev=%u ", ifi);
    ifi = nfq_get_physindev(tb);
    if (ifi)
        printf("physindev=%u ", ifi);

    ifi = nfq_get_physoutdev(tb);
    if (ifi)
        printf("physoutdev=%u ", ifi);

    ret = nfq_get_payload(tb, &data);
    if (ret >= 0)
        printf("payload_len=%d ", ret);

    fputc('\n', stdout);

    return id;
}
   

static int cb(struct nfq_q_handle *qh, struct nfgenmsg *nfmsg,
          struct nfq_data *nfa, void *data)
{
    u_int32_t id = print_pkt(nfa);
    printf("entering callback\n");
    return nfq_set_verdict(qh, id, NF_ACCEPT, 0, NULL);
}

int main(int argc, char **argv)
{
    struct nfq_handle *h;
    struct nfq_q_handle *qh;
    struct nfnl_handle *nh;
    int fd;
    int rv;
    char buf[4096] __attribute__ ((aligned));

    printf("opening library handle\n");
    h = nfq_open();
    if (!h) {
        fprintf(stderr, "error during nfq_open()\n");
        exit(1);
    }

    printf("unbinding existing nf_queue handler for AF_INET (if any)\n");
    if (nfq_unbind_pf(h, AF_INET) < 0) {
        fprintf(stderr, "error during nfq_unbind_pf()\n");
        exit(1);
    }

    printf("binding nfnetlink_queue as nf_queue handler for AF_INET\n");
    if (nfq_bind_pf(h, AF_INET) < 0) {
        fprintf(stderr, "error during nfq_bind_pf()\n");
        exit(1);
    }

    printf("binding this socket to queue '0'\n");
    qh = nfq_create_queue(h,  0, &cb, NULL);
    if (!qh) {
        fprintf(stderr, "error during nfq_create_queue()\n");
        exit(1);
    }

    printf("setting copy_packet mode\n");
    if (nfq_set_mode(qh, NFQNL_COPY_PACKET, 0xffff) < 0) {
        fprintf(stderr, "can't set packet_copy mode\n");
        exit(1);
    }

    fd = nfq_fd(h);

    for (;;) {
        if ((rv = recv(fd, buf, sizeof(buf), 0)) >= 0) {
            printf("pkt received\n");
            nfq_handle_packet(h, buf, rv);
            continue;
        }
        /* if your application is too slow to digest the packets that
         * are sent from kernel-space, the socket buffer that we use
         * to enqueue packets may fill up returning ENOBUFS. Depending
         * on your application, this error may be ignored. Please, see
         * the doxygen documentation of this library on how to improve
         * this situation.
         */
        if (rv < 0 && errno == ENOBUFS) {
            printf("losing packets!\n");
            continue;
        }
        perror("recv failed");
        break;
    }

    printf("unbinding from queue 0\n");
    nfq_destroy_queue(qh);

#ifdef INSANE
    /* normally, applications SHOULD NOT issue this command, since
     * it detaches other programs/sockets from AF_INET, too ! */
    printf("unbinding from AF_INET\n");
    nfq_unbind_pf(h, AF_INET);
#endif

    printf("closing library handle\n");
    nfq_close(h);

    exit(0);
}

 

[phil]

One last thing I haven't checked: I don't know if libtins calculates correct ip checksums.
I'll try that later but maybe Matias can answer this question directly.

...

[phil]

Sorry, you don't need the

struct iphdr *

lines in my example above, it's superfluous and an relict from old code - my fault.

Am Donnerstag, 15. Oktober 2015 22:49:41 UTC+2 schrieb phil:

...

[Matias Fontanini]

Thanks for the example, I'll set up an example on the repo.

Regarding the IP checksum, it should be calculated correctly, except for one case (that I just though about :P): if the packet you are serializing is not actually an outbound one (which would be the case here if you set up the rule for the INPUT chain), then I don't think that will work. Basically when the lowest layer is IP, you need a source address to get the checksum correctly. So far, all packets were assumed to be outbound, so the source address was set to the address of the interface in which the packet would be sent (based on destination address). In this case, you don't need to change the source address, since the packet is inbound, so the checksum is likely to be invalid.

I'll create a ticket for this and work on it when I have some spare time (likely later today :P)

Thanks!

[Einar Jon Gunnarsson]

Hi

I'd just like to note that we have been using Libtins for major "packet mangling" where we strip the EthernetII / IPv6 / TCP headers and replace them with shorter versions to send over a low-bandwidth radio link.
The EthernetII / IPv6 / TCP headers are then reconstructed on the other side and the checksums are correct (accepted by wireshark and the kernel) when you call ethII.serialize()

For the IP checksums, we had issues with split IP packets, as the checksum is calculated from the payload from all packets (or something).
Those are very rare in the wild, but can be created with "ping -s 1600" or "ping6 -s 1600"  (assuming an MTU <= 1500)

We switched to IPv6-only before Tins::IPv4Reassembler was added, but maybe that fixes the issue.

Cheers
Einar Jón

On Thursday, 15 October 2015 23:37:10 UTC+2, Matias Fontanini wrote:

Thanks for the example, I'll set up an example on the repo.

Regarding the IP checksum, it should be calculated correctly, except for one case (that I just though about :P): if the packet you are serializing is not actually an outbound one (which would be the case here if you set up the rule for the INPUT chain), then I don't think that will work. Basically when the lowest layer is IP, you need a source address to get the checksum correctly. So far, all packets were assumed to be outbound, so the source address was set to the address of the interface in which the packet would be sent (based on destination address). In this case, you don't need to change the source address, since the packet is inbound, so the checksum is likely to be invalid.

I'll create a ticket for this and work on it when I have some spare time (likely later today :P)

Thanks!

...

[phil]

Are there any news on ticket  #105?

Thanks!

[Matias Fontanini]

Okay so IP checksums are always calculated now. I added a few tests for this and it works fine.