[erlang-questions] IP packet manipulation within Erlang
Martti Kuparinen
How suitable is Erlang for a small project where I'd like to
- grab all incoming IP packets
- perform some IP header manipulation
- send out the modified packets
I'm aware I need some kind of bpf/pcap driver to get the packets into Erlang (or
does Erlang now have something in this front).
Do you guys have any idea how fast this whole thing would be compared to doing
same in normal userland C program? Or even compared to doing the manipulation
within Linux kernel.
Martti
________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org
Leon de Rooij
I didn't try it myself yet, but I as I understand it, you can use
tuntap to route ip traffic into erlang:
http://cean.process-one.net/packages/index.yaws?action=detail&name=tuntap
regards,
Leon
Chandru
> Hi,
>
> How suitable is Erlang for a small project where I'd like to
>
> - grab all incoming IP packets
> - perform some IP header manipulation
> - send out the modified packets
>
> I'm aware I need some kind of bpf/pcap driver to get the packets into
> Erlang (or does Erlang now have something in this front).
>
> Do you guys have any idea how fast this whole thing would be compared to
> doing same in normal userland C program? Or even compared to doing the
> manipulation within Linux kernel.
>
I attempted this a few years ago. I tried to write a linked in driver which
got packets from libipq [1]. That was painful. So I dumped the idea of using
erlang at all, and did it all in C which was pretty straightforward and
worked quite well. I didn't put any large load on it, but it was enough for
what I was doing at the time.
cheers
Chandru
Hans Nilsson R
I think Erlang binary syntax makes it trivialy straightforward for
handling e.g. IP packets. The following example shows the technique.
The argument to decode/1 is an Ethernet frame in a binary. I used this
for post-processing of a pcap-file so I have no idea of the speed in a
"wireline" application, but it was surprisingly fast in my application.
I guess that the new NIFs will make it faster than with the built-in driver.
decode(<<ED1:8, ED2:8, ED3:8, ED4:8, ED5:8, ED6:8,
ES1:8, ES2:8, ES3:8, ES4:8, ES5:8, ES6:8,
EtherType:16,
Pkt/binary>> ) ->
#ethernet{dst = {ED1, ED2, ED3, ED4, ED5, ED6},
src = {ES1, ES2, ES3, ES4, ES5, ES6},
type = EtherType,
data = dec_pkt(EtherType,Pkt)}.
dec_pkt(2048, %% IPv4
<<Version:4,HdrLen:4, TOS:8, TotLen:16,
ID:16, Flg:3, FragmentOffset:13,
TTL:8, Protocol:8, HdrChkSum:16,
IPsrc1:8, IPsrc2:8, IPsrc3:8, IPsrc4:8,
IPdst1:8, IPdst2:8, IPdst3:8, IPdst4:8,
Data/binary>>) ->
OptLen = HdrLen*4 - 20,
<<Options:OptLen/binary, PayLoad/binary>> = Data,
#ipv4{version = Version,
hdr_length = HdrLen,
tos = TOS,
length = TotLen,
id = ID,
flags = Flg,
fragment_offset = FragmentOffset,
ttl = TTL,
protocol = Protocol,
checksum = HdrChkSum,
src_ip = {IPsrc1, IPsrc2, IPsrc3, IPsrc4},
dst_ip = {IPdst1, IPdst2, IPdst3, IPdst4},
options = Options,
data = case FragmentOffset of
0 -> dec_ipv4(Protocol,PayLoad);
_ -> PayLoad
end
};
dec_pkt(EtherType,X) -> {'??',EtherType,X}.
... and so on for #udp, #tcp, #sctp .....
/Hans
Chandru skrev:
Ulf Wiger
>
> Do you guys have any idea how fast this whole thing would be compared to
> doing same in normal userland C program? Or even compared to doing the
> manipulation within Linux kernel.
For just header manipulation, you shouldn't pay that
much overhead, but of course there will be extra costs
associated with going into Erlang in the first place.
There is a fairly ambitious reference, in the form of a
complete TCP/IP stack implemented in Erlang.
A high performance Erlang Tcp/Ip stack
(Paris, Gulias, Valderruten),
ACM SIGPLAN Erlang Workshop, Tallinn, 2005
http://portal.acm.org/citation.cfm?id=1088372&dl=GUIDE&coll=GUIDE&CFID=70015131&CFTOKEN=10074876
Here are some quotes on performance from the paper:
"To measure throughput a 1GB file was transmitted between
the two machines in both directions, and measured
several times (Fig.5), taking the mean of the measures. As
expected, the larger the Maximum Tranfer Unit (MTU), the
higher the throughput. The total throughput in the best case
is around 150Mbps. As a comparison, Linux TCP/IP stack
in the same situation gave a total throughput of 615 Mbps,
which is around 4 times higher. The costs in copying from
userspace and of the virtual machine explain the difference
in performance. Linux also takes advantage of advanced features
present in the network cards, such as segmentation
offload, checksum computing in the card, or scatter-gather
copies from memory." (p 58)
[...]
"Up to 3000 concurrent connections were opened
with a remote machine, but this reduced the throughput.
A similar test was done with Linux, but opening only 500
connections. Linux showed no reduction in throughput with
those connections, but a few connections used all the available
bandwidth while the others did nothing. The Erlang
stack loses throughput but each of the connections gets the
same share of the bandwidth." (p 59)
BR,
Ulf W
--
Ulf Wiger
CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd
http://www.erlang-solutions.com
---------------------------------------------------
---------------------------------------------------
WE'VE CHANGED NAMES!
Since January 1st 2010 Erlang Training and Consulting Ltd. has become ERLANG SOLUTIONS LTD.
Trevor Woollacott
Hi Chandru
Do you mind elaborating on the issues that you experienced when writing the
linked in driver using libipq?
Regards,
Trevor
Martti Kuparinen
> - grab all incoming IP packets
> - perform some IP header manipulation
> - send out the modified packets
Let's assume I have N hosts (on the same subnet) which perform this. These hosts
have a shared database which contains some information how to manipulate the IP
header. The database entry is inserted when the first packet arrives to any of
the hosts.
How fast is Mnesia to make the new information be available on the other hosts?
Are we talking about M*100 milliseconds or M seconds or M*10 seconds here?
The reason I'm asking this is that, if/when the second packet for the same
session arrives on any other host, it can't do anything unless it first finds
the information within the database.
Chandru
2010/1/7 Trevor Woollacott <tre...@pharos-avantgard.com>
> 2010/1/7 Martti Kuparinen <martti.k...@iki.fi>
>>
>> Hi,
>>>
>>> How suitable is Erlang for a small project where I'd like to
>>>
>>> - grab all incoming IP packets
>>> - perform some IP header manipulation
>>> - send out the modified packets
>>>
>>> I attempted this a few years ago. I tried to write a linked in driver
>> which
>> got packets from libipq [1]. That was painful. So I dumped the idea of
>> using
>> erlang at all, and did it all in C which was pretty straightforward and
>> worked quite well. I didn't put any large load on it, but it was enough
>> for
>> what I was doing at the time.
>>
>> cheers
>> Chandru
>>
>> [1] http://en.wikipedia.org/wiki/Libipq
>>
>>
> Hi Chandru
>
> Do you mind elaborating on the issues that you experienced when writing the
> linked in driver using libipq?
>
Sorry Trevor, I tried hard to recollect, but I can't remember. I do remember
giving up using Erlang in frustration, and writing it in C, much to my
annoyance because I could've done some really complex processing with the
packet stream if I could get them into Erlang.
Chandru
Geoff Cant
> Hi,
>
> How suitable is Erlang for a small project where I'd like to
>
> - grab all incoming IP packets
> - perform some IP header manipulation
> - send out the modified packets
>
> I'm aware I need some kind of bpf/pcap driver to get the packets into
> Erlang (or does Erlang now have something in this front).
>
> Do you guys have any idea how fast this whole thing would be compared
> to doing same in normal userland C program? Or even compared to doing
> the manipulation within Linux kernel.
>
> Martti
Hi there, I've been working on a project which sounds like it could be
suitable for the packet manipulation you want to do. Enet
(http://github.com/archaelus/enet) is a very early stage network stack
written in Erlang. The project also includes a port program that
connects to a TAP device to send and receive raw ethernet frames to and
from the host operating system.
The tap program I have has only been tested on OS X, but I suspect it
could work on Linux with small modifications.
Enet can currently handle arp/icmp/ip and some aspects of udp for en/decoding.
Cheers,
--
Geoff Cant
Zoltan Lajos Kis
Do you have any figures on how the TAP device solution compares to others
like PF_RING sockets and MMAP-ing in terms of required CPU power for a
given traffic?
BTW, how "efficient" is Erlang for IP packet defragmentation (RFC815)? I
can't imagine reassembling a packet without some hipe wizardry at least.
Toby Thain
Are you saying the complex processing would have been simpler in Erlang?
What specifically was impossible in C?
--Toby
>
> Chandru
Valentin Micic
Well... when was the last time you've tried to amend the C/C++ code while it
was running?
V.
Chandru
>
> On 7-Jan-10, at 6:11 PM, Chandru wrote:
>
>> Sorry Trevor, I tried hard to recollect, but I can't remember. I do
>> remember
>> giving up using Erlang in frustration, and writing it in C, much to my
>> annoyance because I could've done some really complex processing with the
>> packet stream if I could get them into Erlang.
>>
>
> Are you saying the complex processing would have been simpler in Erlang?
>
> What specifically was impossible in C?
>
>
I didn't say anything was impossible, it is just that I write code much
faster in erlang than in C. It feels like a chore having to write code in C
these days. Sure there are good reasons for it, but I can't be bothered.
Chandru
Toby Thain
Right, understood. Just didn't quite parse that from your comment.
--T