Tunnels in ns-3

[Antti Mäkelä]

Hey,

Has anyone yet implemented some sort of tunneling logic ( IP =>
[maybesomething] => IP) to NS-3? If yes, could you point me to the
sources...

If not, how would I go around in implementing one?

After scouring the sources and documentation it seems like I
basically need to

Write up code for a "Tunnel" netdevice, where attributes are at least
- Source IP (v4 or v6 - hope to be version-agnostic here)
- Destination IP
- Encapsulation (if I want to support more than one...)

I need to be able to register the tunneling protocols (also
depending on what layer is the tunnel on - whether it's IP over IP, IP
over UDP, GRE over IP and so on....), so that tunnel-demuxer at the
destination can process the incoming traffic, and then yank out the
encapsulated IP packet and put it through normal routing process.

Anyway, how would I go along with implementing this? The tutorial and
manuals at nsnam.org are helpful to a degree - the chapter 15 of the
manual has the "boilerplate" for creating a new object type. However,
creating a completely new protocol seems to include quite many steps,
especially if I intend to insert it into the middle of a stack..

Thanks for any assistance.

[Gustavo Carneiro]

2009/5/27 Antti Mäkelä <zar...@gmail.com>

I am proposing for including TapNetDevice in ns-3-dev:

           http://code.nsnam.org/gjc/ns-3-tap-netdevice/

The TapNetDevice allows one to register a callback for transmitting packets.  There's some documentation in the header file, see:

      http://code.nsnam.org/gjc/ns-3-tap-netdevice/diff/dee7215f0334/src/devices/tap-net-device/tap-net-device.h
 
Writing an example program for this that creates a tunnel is in my TODO list, but I have not found the time to do it yet.  This is incomplete, but I hope it helps a little.

Regards,

--
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert

[Antti Mäkelä]

On May 27, 12:36 pm, Gustavo Carneiro <gjcarne...@gmail.com> wrote:
> The TapNetDevice allows one to register a callback for transmitting
> packets.  There's some documentation in the header file, see:

Sounds excellent - I'll take a look.

[Ismail Amine]

Hi Antti,

In the following Mercurial repository (code.nsnam.org/imaine/ns-3-wimax)
you can find an implementation of a virtual_net_device implementing
IP-in-IP tunneling and GRE tunneling.

The source code of the virtual net device is under
src/devices/Virtual_net_device. We also provide a user level API in
"src/helper/tunnel_helper.h"
The "tunneling.cc" example gives a simple script using the virtual
netdevice API.



Regards
Amine

[Antti Mäkelä]

On May 27, 1:05 pm, Ismail Amine <amine.ism...@sophia.inria.fr> wrote:
> In the following Mercurial repository (code.nsnam.org/imaine/ns-3-wimax)
> you can find an implementation of a virtual_net_device implementing
> IP-in-IP tunneling and GRE tunneling.

This seems even better than the previous option. Looks like this was
last merged with main tree 7 months ago, when ns-3.2 was released...
can I still use this with ns-3.4, or is there chance that something
has broken along the way?

Thanks.

[Antti Mäkelä]

On May 27, 12:36 pm, Gustavo Carneiro <gjcarne...@gmail.com> wrote:

> 2009/5/27 Antti Mäkelä <zar...@gmail.com>

> I am proposing for including TapNetDevice in ns-3-dev:
>
>            http://code.nsnam.org/gjc/ns-3-tap-netdevice/

Well, for tunneling anything over UDP looks like I need this anyway.
Looks relatively straightforward, but can you tell me what are the
parameters that will be passed in your

typedef Callback<bool, Ptr<Packet>, const Address&, const Address&,
uint16_t> SendFromCallback;

When I create the callback function, what happens depending on the
return type (the boolean) being true or false?

Ptr<Packet> is probably just a pointer to the packet itself - I guess
when I'm sending this I'll just push it out as UDP datagram.

Besides wondering what these are and why, wondering why does it
require so many parameters - after all, I just want to load the
octects from the inbound packet into a tunnel...
What's the uint16 ( bool PromiscReceive() suggests it's protocol
number) - what do I need this for?
Also, what are the addresses for? Aren't all the IP headers included
in Ptr<Packet>?

I take it that choosing what actually gets encapsulated is as simple
as setting up a static route with AddNetworkRouteTo and pointing the
network at the other end of a tunnel being reachable via the Tap-Net-
Device...

(Well, if you had that example program, that would probably answer all
of these questions :))

[Gustavo Carneiro]

2009/5/29 Antti Mäkelä <zar...@gmail.com>

I will make an example program tomorrow.

Besides, your questions are very useful for me to know how to improve the documentation ;-)

But I don't have time today to work on the example or answer the question; sorry.
 

[Gustavo Carneiro]

2009/5/29 Gustavo Carneiro <gjcar...@gmail.com>

2009/5/29 Antti Mäkelä <zar...@gmail.com>

On May 27, 12:36 pm, Gustavo Carneiro <gjcarne...@gmail.com> wrote:

> 2009/5/27 Antti Mäkelä <zar...@gmail.com>

> I am proposing for including TapNetDevice in ns-3-dev:
>
>            http://code.nsnam.org/gjc/ns-3-tap-netdevice/

 Well, for tunneling anything over UDP looks like I need this anyway.
Looks relatively straightforward, but can you tell me what are the
parameters that will be passed in your

typedef Callback<bool, Ptr<Packet>, const Address&, const Address&,
uint16_t> SendFromCallback;

When I create the callback function, what happens depending on the
return type (the boolean) being true or false?

Ptr<Packet> is probably just a pointer to the packet itself - I guess
when I'm sending this I'll just push it out as UDP datagram.

The documentation for SendFromCallback will just point to the documentation for NetDevice::SendFrom.

The return boolean means true if the packet was successfully sent, false if transmission failed.  At the moment, NS-3 in general, and the IPv4 stack in particular, is just ignoring this return value, so just return true.
 

Besides wondering what these are and why, wondering why does it
require so many parameters - after all, I just want to load the
octects from the inbound packet into a tunnel...
What's the uint16 ( bool PromiscReceive() suggests it's protocol
number) - what do I need this for?
Also, what are the addresses for? Aren't all the IP headers included
in Ptr<Packet>?

Again, you'll have to read the NetDevice::SendFrom docs to learn why so many parameters and what they mean.
 

I take it that choosing what actually gets encapsulated is as simple
as setting up a static route with AddNetworkRouteTo and pointing the
network at the other end of a tunnel being reachable via the Tap-Net-
Device...

I guess TapNetDevice is slightly more low level than what you wanted.  It can be used to create tunnels, but also can be used for other purposes besides tunneling...
 

(Well, if you had that example program, that would probably answer all
of these questions :))

I will make an example program tomorrow.

The example program is written, see examples/tap-net-device.cc in http://code.nsnam.org/gjc/ns-3-tap-netdevice/
 

Besides, your questions are very useful for me to know how to improve the documentation ;-)

But I don't have time today to work on the example or answer the question; sorry.

Improving the API documentation is still to be done.  Some other day...
 

 

--
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert

[Antti Mäkelä]

> The documentation for SendFromCallback will just point to the documentation
> for NetDevice::SendFrom.

Ok - that wasn't obvious for me - I'll take a look.

> The example program is written, see examples/tap-net-device.cc inhttp://code.nsnam.org/gjc/ns-3-tap-netdevice/

Right...ok, in this example your "Tunnel" class is specifically one
that creates point-to-multipoint tunnel (which is nice example since
it makes more clear how should I go on modifying it).

The tunnel class seems relatively straightforward - you basically
create and bind UDP socket in fixed port 667. I notice that you are
using IPv4Address::GetAny() instead of the specific (physical) point-
to-point interface so I guess it would consider *any* packet recevied
in port 667 as coming in via tunnel.

Anyway, the example makes it clear that SetRecvCallback is already
part of ns-3, your TapDevice adds the SendFromCallback functionality.
I haven''t yet completely grokked the sockets interface yet.

Instead of routing you just use RNG to determine the destination -
using full routing shouldn't be a problem here I guess, interface is
still an interface. Also, you are binding the OnOffapplication
directly to the tunnel so from routing perspective it's directly-
connected. Anyhow, if I want to tunnel stuff that is coming not from
the tunnel endpoint itself but another node, it's should still doable
with routing pointing to the tunnel interface.

All in all, thanks for the example - only stuff that I seem to be
unclear on right now is about the receiving callbacks. Is there any
easy way to make the RecvCallback functions generic? I mean, you have
void N3SocketRecv (Ptr<Socket> socket), void N0SocketRecv (Ptr<Socket>
socket) and void N1SocketRecv (Ptr<Socket> socket), each basically
consisting of m_nXTap->Receive (packet, 0x0800, Address ()).. So
basically, each and every node needs a separate callback function?

(My idea would be to try socket->GetSockName and then maybe have
some data structure that would associate TapDevices with physical i/f
addresses - then you could just do the Receive call through a pointer
to the correct device)

Also, in the m_nXTap->Receive call itself - what purposes do using
protocol 0x0800 (Yes, it's Ethertype for IPv4, but why) and basically
empty address (created by Address())? If I look at your Receive method
it further calls m_RxTrace(), but unfortunalely I cannot decipher
further since the documentation at http://www.nsnam.org/doxygen/functions_0x6d.html#index_m
doesn't include m_RxTrace method...

Anyway, big thanks!

[Gustavo Carneiro]

2009/5/31 Antti Mäkelä <zar...@gmail.com>

> The documentation for SendFromCallback will just point to the documentation
> for NetDevice::SendFrom.

 Ok - that wasn't obvious for me - I'll take a look.

> The example program is written, see examples/tap-net-device.cc inhttp://code.nsnam.org/gjc/ns-3-tap-netdevice/

 Right...ok, in this example your "Tunnel" class is specifically one
that creates point-to-multipoint tunnel (which is nice example since
it makes more clear how should I go on modifying it).

 The tunnel class seems relatively straightforward - you basically
create and bind UDP socket in fixed port 667. I notice that you are
using IPv4Address::GetAny() instead of the specific (physical) point-
to-point interface so I guess it would consider *any* packet recevied
in port 667 as coming in via tunnel.

Yes, I guess we could bind to a specific address instead.  I was just lazy, that's all.
 

 Anyway, the example makes it clear that SetRecvCallback is already
part of ns-3, your TapDevice adds the SendFromCallback functionality.
I haven''t yet completely grokked the sockets interface yet.

 Instead of routing you just use RNG to determine the destination -
using full routing shouldn't be a problem here I guess, interface is
still an interface. Also, you are binding the OnOffapplication
directly to the tunnel so from routing perspective it's directly-
connected. Anyhow, if I want to tunnel stuff that is coming not from
the tunnel endpoint itself but another node, it's should still doable
with routing pointing to the tunnel interface.

 All in all, thanks for the example - only stuff that I seem to be
unclear on right now is about the receiving callbacks.  Is there any
easy way to make the RecvCallback functions generic? I mean, you have
void N3SocketRecv (Ptr<Socket> socket), void N0SocketRecv (Ptr<Socket>
socket) and void N1SocketRecv (Ptr<Socket> socket), each basically
consisting of m_nXTap->Receive (packet, 0x0800, Address ()).. So
basically, each and every node needs a separate callback function?

 (My idea would be to try socket->GetSockName and then maybe have
some data structure that would associate TapDevices with physical i/f
addresses - then you could just do the Receive call through a pointer
to the correct device)

Ideally  we would use MakeBoundCallback, as in the attached patch.  However, I could not get to compile.  Mathieu, do you have any idea why?

 Also, in the m_nXTap->Receive call itself - what purposes do using
protocol 0x0800 (Yes, it's Ethertype for IPv4, but why) and basically
empty address (created by Address())? If I look at your Receive method
it further calls m_RxTrace(), but unfortunalely I cannot decipher
further since the documentation at http://www.nsnam.org/doxygen/functions_0x6d.html#index_m
doesn't include m_RxTrace method...

Calling m_rxTrace is just to make the netdevice consistent with other ones.  All netdevices usually have some form of tracing for Rx/Tx events.  Don't worry about it.
 

 Anyway, big thanks!

[Mohammad Aazam]

Is there any way to simulate IPv6 tunneling techniques, like; 6to4, Teredo, and ISATAP ?

--
"Some cause happiness wherever they go; others whenever they go ------ Oscar Wilde."

Regards,

Mohammad Aazam

[Antti Mäkelä]

On May 27, 12:36 pm, Gustavo Carneiro <gjcarne...@gmail.com> wrote:

> I am proposing for including TapNetDevice in ns-3-dev:
>
>            http://code.nsnam.org/gjc/ns-3-tap-netdevice/
>
> The TapNetDevice allows one to register a callback for transmitting
> packets.  There's some documentation in the header file, see:

Hey, glad to see this got made into VirtualNetDevice and is now part
of the main tree. I have noticed one weird issue - if I have enabled
pcap captures on nodes where there are tunnel devices present, the
capture files turn up zero-length (not the "empty" 24 bytes, but
zero). It's almost as if something is not getting flushed at
Simulator::Destroy(). I have pretty much adapted your tunnel class
example to my purposes.

Anyway, two things - your example as it stands now (virtual-net-
device.cc) does not work anymore, fails with

assert failed. file=../src/routing/list-routing/ipv4-list-routing.cc,
line=108, cond="m_ipv4->GetInterfaceForDevice (idev) >= 0"

thus I cannot check if this happens also with your example,

and second - just my guess - but should I write a deconstructor for
the Tunnel class which would clean up the netdevices? Or what could be
the cause?

[Gustavo Carneiro]

2009/7/1 Antti Mäkelä <zar...@gmail.com>

On May 27, 12:36 pm, Gustavo Carneiro <gjcarne...@gmail.com> wrote:

> I am proposing for including TapNetDevice in ns-3-dev:
>
>            http://code.nsnam.org/gjc/ns-3-tap-netdevice/
>
> The TapNetDevice allows one to register a callback for transmitting
> packets.  There's some documentation in the header file, see:

 Hey, glad to see this got made into VirtualNetDevice and is now part
of the main tree. I have noticed one weird issue - if I have enabled
pcap captures on nodes where there are tunnel devices present, the
capture files turn up zero-length (not the "empty" 24 bytes, but
zero). It's almost as if something is not getting flushed at
Simulator::Destroy(). I have pretty much adapted your tunnel class
example to my purposes.

 Anyway, two things - your example as it stands now (virtual-net-
device.cc) does not work anymore, fails with

assert failed. file=../src/routing/list-routing/ipv4-list-routing.cc,
line=108, cond="m_ipv4->GetInterfaceForDevice (idev) >= 0"

 thus I cannot check if this happens also with your example,

I see it also, now, but unfortunately could find obvious way how to fix.  It used to work at some point in time, I'm sure...
 

 and second - just my guess - but should I write a deconstructor for
the Tunnel class which would clean up the netdevices? Or what could be
the cause?

Probably the assertion above is responsible for premature termination and thus the pcap file is not properly flushed.
 

[Antti Mäkelä]

On Jul 2, 5:09 pm, Gustavo Carneiro <gjcarne...@gmail.com> wrote:
> >  and second - just my guess - but should I write a deconstructor for
> > the Tunnel class which would clean up the netdevices? Or what could be
> > the cause?
> Probably the assertion above is responsible for premature termination and
> thus the pcap file is not properly flushed.

Oh, I'm not using your example as-is, as I'm doing point-to-point
tunnels - and they work. The biggest difference from your example is
which socket to call from VirtualSend function - I can never
understand why it's like this,

N0N1VirtualSend (Ptr<Packet> packet, const Address& source, const
Address& dest, uint16_t protocolNumber)
{
m_n3Socket->SendTo (packet, 0, InetSocketAddress (m_n3Address,
667));
return true;
}

when what I have effectively changed is

m_n0Socket->SendTo (packet, ....

and obviously N1 would require it's own Send function. I never
understood that if you are intending to send from N0, why would you
call N3's socket. Anyway, as I wasn't sure if this is the really
proper way to fix things, I asked first if you know what's broken with
it.

[Gustavo Carneiro]

2009/7/3 Antti Mäkelä <zar...@gmail.com>

Hm.. you don't understand and neither do I because it doesn't make sense! :-P

Pushed a fix now, thanks for spotting the problem.

[Antti Mäkelä]

On Jul 3, 1:15 pm, Gustavo Carneiro <gjcarne...@gmail.com> wrote:

> Hm.. you don't understand and neither do I because it doesn't make sense!
> :-P
>
> Pushed a fix now, thanks for spotting the problem.

Well, apparently the fix doesn't work - assert at ipv4-list-routing
still fails. (Just did hg pull&hg update). Anyway, problem is
apparently elsewhere (and this doesn't help me trace down my zero-
length-captures issue, either). If I turn on logging it appears that
most of the problems appear at the end of the simulation run. I tried
a few tests with logging on on your example, and to save some time I
set the apps to stop (Changed all lines with apps.Stop()) to 1.3
seconds - even so, sim stops at 3.37 to the aforementioned assert()
error.

Oh well, let's see if I can fix my issue by coming up with a proper
deconstructor, although I thought that Ptr<>'s being smart and all
should take care of that by itself.

[Gustavo Carneiro]

2009/7/3 Antti Mäkelä <zar...@gmail.com>

On Jul 3, 1:15 pm, Gustavo Carneiro <gjcarne...@gmail.com> wrote:

> Hm.. you don't understand and neither do I because it doesn't make sense!
> :-P
>
> Pushed a fix now, thanks for spotting the problem.

 Well, apparently the fix doesn't work - assert at ipv4-list-routing
still fails. (Just did hg pull&hg update). Anyway, problem is
apparently elsewhere (and this doesn't help me trace down my zero-
length-captures issue, either). If I turn on logging it appears that
most of the problems appear at the end of the simulation run. I tried
a few tests with logging on on your example, and to save some time I
set the apps to stop (Changed all lines with apps.Stop()) to 1.3
seconds - even so, sim stops at 3.37 to the aforementioned assert()
error.

Grr! I had tested the fix, committed, but then I had to merge with more upstream changes and it stopped working again :-/
 

 Oh well, let's see if I can fix my issue by coming up with a proper
deconstructor, although I thought that Ptr<>'s being smart and all
should take care of that by itself.

 

[Antti Mäkelä]

On Jul 3, 2:48 pm, Gustavo Carneiro <gjcarne...@gmail.com> wrote:

> >   Well, apparently the fix doesn't work - assert at ipv4-list-routing
> > still fails. (Just did hg pull&hg update). Anyway, problem is
> > apparently elsewhere (and this doesn't help me trace down my zero-
> > length-captures issue, either). If I turn on logging it appears that
> > most of the problems appear at the end of the simulation run. I tried
> > a few tests with logging on on your example, and to save some time I
> > set the apps to stop (Changed all lines with apps.Stop()) to 1.3
> > seconds - even so, sim stops at 3.37 to the aforementioned assert()
> > error.
>
> Grr! I had tested the fix, committed, but then I had to merge with more
> upstream changes and it stopped working again :-/

Well, I resolved my issue :) I had used Simulator::Stop(Seconds(30))
to terminate simulation at predetermined time, before calling
Simulator::Destroy(). Removing the Stop() made pcaps print out just
fine. Now I'm just wondering what would be the best way to specify the
maximum running time of a simulation if I don't want to individually
shut down all my apps at that maxtime.

Anyway, as my intention is to simulate creating and shutting down
VPNs (Tunnels), I'm wondering what is the best way to *delete*
tunnels. I mean, you can only AddDevice()s to nodes, and I really
don't want to have gazillion VirtualNetDevices exist at once - they
should be removed completely once shut down.

I can easily close the sockets at deconstructor, but removing the
devices is another matter. Or will smart pointers take care of it
autotically - if no one refers to a NetDevice (the instance of Tunnel
is removed, thus anyone who ever had knowledge (via
Ptr<VirtualNetDevice>s) of the VirtualNetDevice is gone) - it the
device automatically removed from the node when refcount ticks to
zero?

Of course I can handle this by routing as well and keep the devices
up all times, but my intention is to include a keepalive system into
the tunnel protocol, and an error model which forces the tunnels to be
re-established every now and then when the keepalive fails...thus,
simulating the actual tunnel failing by nuking packets and switching
routes sounds a bit cumbersome.

[Gustavo Carneiro]

2009/7/3 Antti Mäkelä <zar...@gmail.com>

On Jul 3, 2:48 pm, Gustavo Carneiro <gjcarne...@gmail.com> wrote:

> >   Well, apparently the fix doesn't work - assert at ipv4-list-routing
> > still fails. (Just did hg pull&hg update). Anyway, problem is
> > apparently elsewhere (and this doesn't help me trace down my zero-
> > length-captures issue, either). If I turn on logging it appears that
> > most of the problems appear at the end of the simulation run. I tried
> > a few tests with logging on on your example, and to save some time I
> > set the apps to stop (Changed all lines with apps.Stop()) to 1.3
> > seconds - even so, sim stops at 3.37 to the aforementioned assert()
> > error.
>
> Grr! I had tested the fix, committed, but then I had to merge with more
> upstream changes and it stopped working again :-/

I fixed the issue, finally.  pushed to ns-3-dev.
 

 Well, I resolved my issue :) I had used Simulator::Stop(Seconds(30))
to terminate simulation at predetermined time, before calling
Simulator::Destroy(). Removing the Stop() made pcaps print out just
fine. Now I'm just wondering what would be the best way to specify the
maximum running time of a simulation if I don't want to individually
shut down all my apps at that maxtime.

 Anyway, as my intention is to simulate creating and shutting down
VPNs (Tunnels), I'm wondering what is the best way to *delete*
tunnels. I mean, you can only AddDevice()s to nodes, and I really
don't want to have gazillion VirtualNetDevices exist at once - they
should be removed completely once shut down.

 I can easily close the sockets at deconstructor, but removing the
devices is another matter. Or will smart pointers take care of it
autotically - if no one refers to a NetDevice (the instance of Tunnel
is removed, thus anyone who ever had knowledge (via
Ptr<VirtualNetDevice>s) of the VirtualNetDevice is gone) - it the
device automatically removed from the node when refcount ticks to
zero?

In NS-3, ns3::Object has a Dispose() method to ensure the object is destroyed even if a smart pointer is referencing it.  But this alone may not be sufficient or adequate, often there exists a higher level API to do something like that specific to each class.  For instance, in case of Socket there is Socket::Close.
 

 Of course I can handle this by routing as well and keep the devices
up all times, but my intention is to include a keepalive system into
the tunnel protocol, and an error model which forces the tunnels to be
re-established every now and then when the keepalive fails...thus,
simulating the actual tunnel failing by nuking packets and switching
routes sounds a bit cumbersome.

[Antti Mäkelä]

On Jul 3, 6:18 pm, Gustavo Carneiro <gjcarne...@gmail.com> wrote:
> I fixed the issue, finally.  pushed to ns-3-dev.

Ok, I was finally able to test something - and looks like the bug
with packet dumps is still there.

Alter your virtual-net-device.cc by adding the line

onoff.SetAttribute ("MaxBytes", UintegerValue(210));

to around line 269. That is, this manifests itself only when you are
sending very small amounts of data.

You'll see zero-sized files.

If you increase the value to something >20000, the files are no
longer empty, but it's apparent not all packets made it. Also,
wireshark complains "The capture file appears to have been cut short
in the middle of a packet".

Problem is that *all* interfaces of a node have these issues. It's
like some flushing is not being done...

[cra...@ee.washington.edu]

> Ok, I was finally able to test something - and looks like the bug
> with packet dumps is still there.
>
> Alter your virtual-net-device.cc by adding the line
>
> onoff.SetAttribute ("MaxBytes", UintegerValue(210));
>
> to around line 269. That is, this manifests itself only when you are
> sending very small amounts of data.
>
> You'll see zero-sized files.
>
> If you increase the value to something >20000, the files are no
> longer empty, but it's apparent not all packets made it. Also,
> wireshark complains "The capture file appears to have been cut short
> in the middle of a packet".
>
> Problem is that *all* interfaces of a node have these issues. It's
> like some flushing is not being done...

This is the classic symptom of a memory leak. What usually happens is that
a pointer reference is held that prevents the reference counts of the nodes
from being decremented to zero, which (after a sometimes long chain of
events) in turn causes the trace writer destructors not to be executed which
means the trace files are not flushed at close.

Try running your simulation under valgrind and I'll bet you'll see a leak.

-- Craig

[Antti Mäkelä]

On 7 heinä, 20:12, <crai...@ee.washington.edu> wrote:
> >   Alter your virtual-net-device.cc by adding the line
> > onoff.SetAttribute ("MaxBytes", UintegerValue(210));
> >   to around line 269. That is, this manifests itself only when you are
> > sending very small amounts of data.

> This is the classic symptom of a memory leak.  What usually happens is that
> a pointer reference is held that prevents the reference counts of the nodes
> from being decremented to zero, which (after a sometimes long chain of
> events) in turn causes the trace writer destructors not to be executed which
> means the trace files are not flushed at close.
>
> Try running your simulation under valgrind and I'll bet you'll see a leak.

I can try, but the whole point I mention in above is that you can
repeat this with your own example with a minor change (limiting the
number of packets) so you can easily testthe issue as well.

[Antti Mäkelä]

This is the output from Valgrind. I don't know, but this tells me
about the fact that something is going wrong with the cleanup of nodes
where there are virtual netdevices present - but we sort of already
knew that.

==7082== Memcheck, a memory error detector.
==7082== Copyright (C) 2002-2008, and GNU GPL'd, by Julian Seward et
al.
==7082== Using LibVEX rev 1878, a library for dynamic binary
translation.
==7082== Copyright (C) 2004-2008, and GNU GPL'd, by OpenWorks LLP.
==7082== Using valgrind-3.4.0, a dynamic binary instrumentation
framework.
==7082== Copyright (C) 2000-2008, and GNU GPL'd, by Julian Seward et
al.
==7082== For more details, rerun with: -v
==7082==
==7082== Conditional jump or move depends on uninitialised value(s)
==7082== at 0x4100ADEE: (within /lib/ld-2.9.so)
==7082== by 0x410039D4: (within /lib/ld-2.9.so)
==7082== by 0x41015129: (within /lib/ld-2.9.so)
==7082== by 0x41001397: (within /lib/ld-2.9.so)
==7082== by 0x410009A6: (within /lib/ld-2.9.so)
==7082==
==7082== Conditional jump or move depends on uninitialised value(s)
==7082== at 0x4100ADF6: (within /lib/ld-2.9.so)
==7082== by 0x410039D4: (within /lib/ld-2.9.so)
==7082== by 0x41015129: (within /lib/ld-2.9.so)
==7082== by 0x41001397: (within /lib/ld-2.9.so)
==7082== by 0x410009A6: (within /lib/ld-2.9.so)
==7082==
==7082== Conditional jump or move depends on uninitialised value(s)
==7082== at 0x4100B185: (within /lib/ld-2.9.so)
==7082== by 0x410039D4: (within /lib/ld-2.9.so)
==7082== by 0x41015129: (within /lib/ld-2.9.so)
==7082== by 0x41001397: (within /lib/ld-2.9.so)
==7082== by 0x410009A6: (within /lib/ld-2.9.so)
==7082==
==7082== Conditional jump or move depends on uninitialised value(s)
==7082== at 0x4100ADEE: (within /lib/ld-2.9.so)
==7082== by 0x41003AF9: (within /lib/ld-2.9.so)
==7082== by 0x41015129: (within /lib/ld-2.9.so)
==7082== by 0x41001397: (within /lib/ld-2.9.so)
==7082== by 0x410009A6: (within /lib/ld-2.9.so)
==7082==
==7082== Conditional jump or move depends on uninitialised value(s)
==7082== at 0x4100ADF6: (within /lib/ld-2.9.so)
==7082== by 0x41003AF9: (within /lib/ld-2.9.so)
==7082== by 0x41015129: (within /lib/ld-2.9.so)
==7082== by 0x41001397: (within /lib/ld-2.9.so)
==7082== by 0x410009A6: (within /lib/ld-2.9.so)
==7082==
==7082== Conditional jump or move depends on uninitialised value(s)
==7082== at 0x4100B185: (within /lib/ld-2.9.so)
==7082== by 0x41003AF9: (within /lib/ld-2.9.so)
==7082== by 0x41015129: (within /lib/ld-2.9.so)
==7082== by 0x41001397: (within /lib/ld-2.9.so)
==7082== by 0x410009A6: (within /lib/ld-2.9.so)
==7082==
==7082== ERROR SUMMARY: 6 errors from 6 contexts (suppressed: 0 from
0)
==7082== malloc/free: in use at exit: 33,601 bytes in 177 blocks.
==7082== malloc/free: 27,118 allocs, 26,941 frees, 963,098 bytes
allocated.
==7082== For counts of detected errors, rerun with: -v
==7082== Use --track-origins=yes to see where uninitialised values
come from
==7082== searching for pointers to 177 not-freed blocks.
==7082== checked 452,268 bytes.
==7082==
==7082==
==7082== 32,545 (180 direct, 32,365 indirect) bytes in 3 blocks are
definitely lost in loss record 27 of 39
==7082== at 0x4007D9E: operator new(unsigned int) (in /usr/lib/
valgrind/x86-linux/vgpreload_memcheck.so)
==7082== by 0x4515390: ns3::Ptr<ns3::Node>
ns3::CreateObject<ns3::Node>() (object.h:484)
==7082== by 0x47AACF1: ns3::NodeContainer::Create(unsigned int)
(node-container.cc:96)
==7082== by 0x804EF35: main (virtual-net-device.cc:220)
==7082==
==7082== LEAK SUMMARY:
==7082== definitely lost: 180 bytes in 3 blocks.
==7082== indirectly lost: 32,365 bytes in 171 blocks.
==7082== possibly lost: 0 bytes in 0 blocks.
==7082== still reachable: 1,056 bytes in 3 blocks.
==7082== suppressed: 0 bytes in 0 blocks.
==7082== Reachable blocks (those to which a pointer was found) are not
shown.
==7082== To see them, rerun with: --leak-check=full --show-
reachable=yes

[Mathieu Lacage]

I can't test this right now but I would try setting m_node to zero in
VirtualNetDevice::DoDispose

Mathieu

[Antti Mäkelä]

Ok, I found the bug - your DoDispose did not clear the pointer to
node properly.

Submitted a bug report and patch.

http://www.nsnam.org/bugzilla/show_bug.cgi?id=628

[Antti Mäkelä]

On Jul 8, 11:25 am, Mathieu Lacage <mathieu.lac...@sophia.inria.fr>
wrote:

> I can't test this right now but I would try setting m_node to zero in
> VirtualNetDevice::DoDispose

Heh, just discovered this as well, I just had finished comparing the
DoDisposes of other NetDevices and found that they always cleared
Ptr<>'s :)