trafgen usage to test throughput

[b...@coco.fr]

Hi Daniel,

I wish to stress test a packet sniffer to see what throughput it can have.
is it possible with trafgen to send over a million TCP packet per second to an IP ?
what command should I use?

Regards,
Zack

[Daniel Borkmann]

On 03/05/2013 05:15 PM, b...@coco.fr wrote:

> I wish to stress test a packet sniffer to see what throughput it can have.
> is it possible with trafgen to send over a million TCP packet per second to an IP ?

Depending on your hardware, yes.

> what command should I use?

See `trafgen -e` for an example. Otherwise `trafgen -h` will further assist you.

There are also community wikis, e.g. for Ubuntu https://help.ubuntu.com/community/Netsniff-NG .

[Jon Schipp]

Zack, the _quickest_ way would be to generate the packet you want with
mausezhan or another similar packet crafting tool e.g. hping3, nmap scapy etc.
Write the single packet to disk and then convert it to a trafgen configuration
and send it out.

sth like this ( off top of head ).
# Send a single TCP segment with SYN bit set
$ hping3 -c 1 -I eth0 -p 443 -S 192.168.1.1
# Capture single packet, write to disk
$ netsniff-ng --in eth0 --out tcpsyn.pcap
# Convert pcap to trafgen config
$ netsniff-ng --in tcpsyn.pcap --out tcpsyn.cfg
# Send packet x 1000
$ trafgen --in tcpsyn.cfg --out eth0 --num 1000

[Daniel Borkmann]

Why so complicated ?! Seriously ...

trafgen -e, the built-in example already has a TCP SYN packet with dynamic
elemets in it. It's really not that difficult.

Alternatively, mausezahn would do the job. If you need a full stream,
then use iperf.

[Jon Schipp]

He didn't say what kind of TCP packet he wanted. If it wasn't a TCP SYN packet
he would need to configure his own with the trafgen language. I suppose he could
modify the -e config instead. He also mentioned speed though. Okay,
well, maybe mine
was a little more complicated.

[Daniel Borkmann]

Yep, what exactly is needed was not really specified, but as one can see there
are quite a lot of possibilities available. ;-)

[b...@coco.fr]

thanks John,

I succeeded but outgoing Syn packets are 2048 bytes long :

trafgen --in tcpsyn.cfg --out eth0 --num 1000

trafgen 0.5.7
1 packets to schedule
54 bytes in total
TX: 238.41 MiB, 122064 Frames, each 2048 Byte allocated
MD: TX fastpath RR 10us

Running! Hang up with ^C!

1000 frames outgoing
54000 bytes outgoing
0 sec, 2723 usec in total

how can I make then 54 bytes long ?

Thanks

[Daniel Borkmann]

On 03/05/2013 08:54 PM, b...@coco.fr wrote:
> thanks John,
>
> I succeeded but outgoing Syn packets are 2048 bytes long :
>
> trafgen --in tcpsyn.cfg --out eth0 --num 1000
> trafgen 0.5.7
> 1 packets to schedule
> 54 bytes in total
> TX: 238.41 MiB, 122064 Frames, each 2048 Byte allocated

I assume you are referring to this line?

It's just a description of allocated ring buffer slots for the TX ring
in the Linux kernel. So just a status message that is not really related
to the content you're sending.

> MD: TX fastpath RR 10us
>
> Running! Hang up with ^C!
>
>
> 1000 frames outgoing
> 54000 bytes outgoing
> 0 sec, 2723 usec in total

As you can see here in the TX stats, it already is 54 bytes long. Please
read those things more carefully ...

[b...@coco.fr]

Hi Daniel and John,

I succesfully ran the test from another server but I am limited in outgoing traffic.
that is why I wish to send the maximum number of TCP packet on the same machine to test throughput of my program.
is that possible with trafgen ?
what command should I execute locally ?

Zack

[Daniel Borkmann]

You can use a traffic shaper on your trafgen outgoing device, so tc(7) would
so the job to rate limit.

[b...@coco.fr]

my purpose is to test my Tcp read packet program locally.
I would like to know the command to execute to send 50 M packets locally to see at what speed my program can read and decode packets.

I am just needing trafgen to bench my program on port 84 but I don't know how to tell trafgen to send all thoses packets as if it was real traffic
what command to use?

thanks

[Jon Schipp]

You can send 50 million packets to TCP destination port 84 with
trafgen but you will need
to modify the destination port in the configuration file and set --num 50000000
/* TCP Dest Port */
c16(84),

Though, if you need a _real_ TCP stream (two-way) you should use iperf
as Daniel mentioned previously.
Trafgen does not handle TCP streams (higher-level), it's a low-level
packet generator.

> --
> You received this message because you are subscribed to the Google Groups "netsniff-ng" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to netsniff-ng...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[b...@coco.fr]

thanks John,

I only neeed to send raw tcp packets so trafgen is ok.
but my question is what command should I enter to send incoming packet to my application listening packet on the same server.
I would need the command line to enter
Sorry for my low knowledge of linux

[Daniel Borkmann]

If your server is listening on the lo device, you can do sth like:

trafgen -i your_conf -o lo -n1000 [plus other options you need]

Also, you might want to consider using a newer version of trafgen
from upstream Git, since I can see your's is quite old.

[b...@coco.fr]

Thanks daniel,

I have trafgen 0.5.7 isn't it the latest stable version ?

other point. when sending 1 million packet I get 1.33s duration.
how can I make it quicker so I can send 5 Mpps with trafgen?

trafgen --in tcpsyn.cfg -o lo --num 1000000

trafgen 0.5.7
1 packets to schedule
54 bytes in total

TX: 64.00 MiB, 32768 Frames, each 2048 Byte allocated

MD: TX fastpath RR 10us

Running! Hang up with ^C!

1000000 frames outgoing
54000000 bytes outgoing
1 sec, 341868 usec in total

Thanks

[Daniel Borkmann]

On 03/06/2013 04:40 PM, b...@coco.fr wrote:
> Thanks daniel,
>
> I have trafgen 0.5.7 isn't it the latest stable version ?

Yes, but you want it faster, those features are upstream in Git.

> other point. when sending 1 million packet I get 1.33s duration.
> how can I make it quicker so I can send 5 Mpps with trafgen?

Do:

git clone git://github.com/borkmann/netsniff-ng.git
cd netsniff-ng
make trafgen
make trafgen_install
trafgen --in tcpsyn.cfg -o lo -k100

[b...@coco.fr]

Hi,

I have this when I make trafgen :

make trafgen
NACL_LIB_DIR/NACL_INC_DIR is undefined, build libnacl first for curvetun!
Building trafgen:
LEX trafgen_lexer.l
YAAC trafgen_parser.y
CC xmalloc.c
/bin/sh: ccache: command not found
make: *** [trafgen/xmalloc.o] Error 127

I went to curvetun and did a ./build_nacl.sh ~/nacl
but it did not solve the problem

[Jon Schipp]

What distribution of Linux are you using?

[Jon Schipp]

You're missing ccache for one.
Required packages:
https://github.com/borkmann/netsniff-ng/blob/master/INSTALL

[Daniel Borkmann]

On 03/06/2013 09:24 PM, Jon Schipp wrote:
> What distribution of Linux are you using?
>
> On Wed, Mar 6, 2013 at 3:11 PM, <b...@coco.fr> wrote:
>> Hi,
>>
>> I have this when I make trafgen :
>>
>> make trafgen
>> NACL_LIB_DIR/NACL_INC_DIR is undefined, build libnacl first for curvetun!
>> Building trafgen:
>> LEX trafgen_lexer.l
>> YAAC trafgen_parser.y
>> CC xmalloc.c
>> /bin/sh: ccache: command not found
>> make: *** [trafgen/xmalloc.o] Error 127
>>
>> I went to curvetun and did a ./build_nacl.sh ~/nacl
>> but it did not solve the problem

Either install ccache to speed up compilation or remove it from
the shipped Makefile:

- CCACHE = ccache
+ CCACHE =

Also, I advise you to read INSTALL.

[b...@coco.fr]

I installed ccache but a new error :

make trafgen
NACL_LIB_DIR/NACL_INC_DIR is undefined, build libnacl first for curvetun!
Building trafgen:
LEX trafgen_lexer.l
YAAC trafgen_parser.y

CC trafgen/trafgen_lexer.yy.c
CC trafgen/trafgen_parser.tab.c
LD trafgen
/usr/bin/ld: cannot find -lnl-genl-3
collect2: ld returned 1 exit status
make: *** [trafgen] Error 1

[Jon Schipp]

What distribution of Linux are you using? e.g. Ubuntu, Fedora etc.
You need to install libnl as suggested in the INSTALL file.

[Daniel Borkmann]

On 03/06/2013 10:16 PM, Jon Schipp wrote:
> What distribution of Linux are you using? e.g. Ubuntu, Fedora etc.
> You need to install libnl as suggested in the INSTALL file.

Exactly, please read and follow instructions of the INSTALL file
*CAREFULLY* before you ask questions like this, especially after
we have advised you so. If you would have installed dependencies
as stated in there, then you would not see this error, i.e. that
libnl3 is missing.

(If you have read this carefully and something else is wrong, then
of course ask questions. But this is not the case here.)

[b...@coco.fr]

worked ! despite I had installed libnl-dev it did not work.

now I have trafgen 0.5.8-rc0

tried with --num 500 000 WORKED
tried with --num 5 000 000 crashed my server! had to reboot

but it seems to be faster and to use all cpu.
why did it crashed my server with --num 5M ?

[Daniel Borkmann]

Can you specify what crashed your server means?

Your application broke down or your kernel crashed?

If it's the whole kernel, then your kernel is too old, see also:

http://thread.gmane.org/gmane.linux.network.netsniff-ng/282

[b...@coco.fr]

Hi,

it is my whole dedicated web server that crashed, no Ssh access, had to restart. kernel 3.2.13-grsec
is that too old?

[Daniel Borkmann]

On 03/07/2013 02:19 PM, b...@coco.fr wrote:

> it is my whole dedicated web server that crashed, no Ssh access, had to restart. kernel 3.2.13-grsec
> is that too old?

You have two options:

1) Using your trafgen with added --gap 0 option, this will not use TX_RING
that causes this crash internally.

2) If you are experimenting with your server anyway, then you could try to
install a stable kernel from kernel.org if you know how to build kernels.

For now, I suggest to you, using option 1.

[b...@coco.fr]

Hi Daniel ,

I took a new ubuntu server with kernel 3.7.10
tried 1 M packet with 0.7 and the compiled version of 0.8rc
the 0.7 send 1 M packets and I receive 1 M packets
the 0.8rc send 1 M packets and I rceive only 150 000 .
Below is the command I use with the new trafgen.
What did I do wrong for only 1/8 packets arrive?

/home/netsniff-ng/trafgen/trafgen --in tcpsyn.cfg --out eth0 --num 1000000
8 packets to schedule
432 bytes in total

Running! Hang up with ^C!

1000000 packets outgoing
54000000 bytes outgoing
0 sec, 39748 usec on CPU0 (125000 packets)
0 sec, 27702 usec on CPU1 (125000 packets)
0 sec, 40692 usec on CPU2 (125000 packets)
0 sec, 39536 usec on CPU3 (125000 packets)
0 sec, 44668 usec on CPU4 (125000 packets)
0 sec, 339387 usec on CPU5 (125000 packets)
0 sec, 37306 usec on CPU6 (125000 packets)
0 sec, 40475 usec on CPU7 (125000 packets)

[Daniel Borkmann]

On 03/09/2013 05:13 AM, b...@coco.fr wrote:

> I took a new ubuntu server with kernel 3.7.10
> tried 1 M packet with 0.7 and the compiled version of 0.8rc
> the 0.7 send 1 M packets and I receive 1 M packets
> the 0.8rc send 1 M packets and I rceive only 150 000 .
> Below is the command I use with the new trafgen.
> What did I do wrong for only 1/8 packets arrive?

Maybe your application cannot cannot process so many packets?

Have you tried with:

trafgen --in tcpsyn.cfg --out eth0 --num 1000000 --cpus 1

in order to use only 1 cpu on your server?

[b...@coco.fr]

When using --cpus 1
it works as trafgen 0.5.7
I send 1 M packets and I receive 1 M
but once I specify --cpus 2
I send 1 M and I receive 150k only .
there is something wrong I think

[Daniel Borkmann]

I'm sorry but if you want us to help, then you have to provide
a little more info than just "there is something wrong I think".

Otherwise no-one is willing to give a serious answer.

What is the output of trafgen in that case? Is it the output you've
sent in your previous mail? Also trafgen 0.5.7 does not have an
option --cups, 0.5.8rc0 has it. So which version are you running?

In your previous email, on each CPU 125k packets were sent. You've
mentioned something with 150k. Was that a typo on your side? If you
use --num, then the sent packets are distributed fairly among your
CPUs, so 1 Mio in total.

What is your application doing? Is it written in C? Do you have
access to any kernel drop counters? Is it running on the same node
as trafgen? What are the network driver statistics saying? Did they
see 1M packets?

E.g. on the sender:

before: cat /proc/net/dev | grep ethX
midde: trafgen ....
after: cat /proc/net/dev | grep ethX

E.g. on the receiver:

before: cat /proc/net/dev | grep ethX
midde: <your app>
after: cat /proc/net/dev | grep ethX

[b...@coco.fr]

Ok so Let me explain better
Server A is using trafgen to send the max packet/s
Server B has a program that capture incoming packets ( it is one of my own)

when using trafgen from 0.5.7
all 1 Mpackets are captured

when using trafgen 05.8rc
when sending 1 M packets.
only 150k arrive.
it looks like despite the text above , only few packets leave the server to go to the other server.

any idea?

[Daniel Borkmann]

I don't necessarily think it's an issue with trafgen itself.

Sadly, you didn't take my feedback from my last email into
account, so unless you don't provide some hard data / facts
as I wrote before, I cannot help you otherwise.

Talk is cheap, unless you provide me with some real data!