Wired mesh network with RealtimeEthernetProtocol and Qemu VMs as Nodes

[Bernhard Blieninger]

Hey guys,

we try to use ns3 for network simulation for a mesh network between qemu nodes.
I just tried to experiment with the basics but did not succeed.

Can someone explain to me how i should configure my system to fire up tap-csma-virtual-machine.cc example with two bridges at the same host machine connected to two VMs.

So my setup is: VM0 - tap0 - bridge (br0) - tap-left - (ns3) - tap-right - bridge (br1) - tap1 - VM1

VM0 and VM1 have IP addresses on the same network and br0 and br1 are also configured to work on the same network but ping will go trough, because of the functionality of linux bridges, even if i deactivate STP protocol.

My second approach is to use tap-csma example to configure an internal network with more than one external device connected to it.

The Problem here is that i want to use my qemu vm instead of the Linux host interface. Therefore i want to use „UseLocal“ mode in ns-3 because this should listen to the tap device to which qemu is writing and listening. But it tells me that ns-3 cannot fire up because the device is blocked. Starting ns-3 first will block qemu start.

Our main goal is to set up a 5-6 qemu vm’s that will connect via wired mesh through the ns-3 network in realtime( RealtimeEthernetProtoco, e.g. Ethernet Powerlink or TTEthernet), so we can control the network while the simulation is running and see the behavior of our qemu nodes.


Thanks for your answers.


Best Regards,

Bernhard.

[Tom Henderson]

On 11/13/2014 08:17 AM, Bernhard Blieninger wrote:
> Hey guys,
>
> we try to use ns3 for network simulation for a mesh network between qemu
> nodes.
> I just tried to experiment with the basics but did not succeed.

Hi Bernhard, I am not a user of QEMU but I'd be interested in helping to
debug this.

>
> Can someone explain to me how i should configure my system to fire up
> tap-csma-virtual-machine.cc example with two bridges at the same host
> machine connected to two VMs.
>
> So my setup is: VM0 - tap0 - bridge (br0) - tap-left - (ns3) - tap-right
> - bridge (br1) - tap1 - VM1
>
> VM0 and VM1 have IP addresses on the same network and br0 and br1 are
> also configured to work on the same network but ping will go trough,
> because of the functionality of linux bridges, even if i deactivate STP
> protocol.

this is the intended configuration for the example you cite. I just
tested this, and found that if I followed these steps on the wiki:

http://www.nsnam.org/wiki/HOWTO_Use_Linux_Containers_to_set_up_virtual_networks#HOWTO_Use_Linux_Containers_to_set_up_virtual_networks

I could get it to work as expected. In particular, the wiki describes
an lxc use case, but if you ignore that and just follow the steps for:

- configuring tap-left and tap-right on the host
- configuring br-left and br-right on the host
- add tap devices to bridge devices
- disable iptables filtering flags in /proc/sys/net/bridge

and then run the example, you should be able to tcpdump on the bridge
devices or tap devices and look for traffic (assuming you hook your
other VMs to the bridge).

Where do your pings stop getting through, in your topology?

>
> My second approach is to use tap-csma example to configure an internal
> network with more than one external device connected to it.
>
> The Problem here is that i want to use my qemu vm instead of the Linux
> host interface. Therefore i want to use „UseLocal“ mode in ns-3 because
> this should listen to the tap device to which qemu is writing and
> listening. But it tells me that ns-3 cannot fire up because the device
> is blocked. Starting ns-3 first will block qemu start.

this example is designed more for either using the native host with
ns-3, or for VM environments where you can later push the tap device
(which is created by ns-3) into the VM.

Have you looked at 'tap-wifi-virtual-machine.cc' which is a provided
example that uses 'UseLocal" mode?

to help with debugging, it may be useful to enable some logging, so
could you set some environment variables and capture the output?

such as

sudo bash
NS_LOG="CsmaNetDevice:TapBridge" ./waf --run tap-csma-virtual-machine

>
> Our main goal is to set up a 5-6 qemu vm’s that will connect via wired
> mesh through the ns-3 network in realtime( RealtimeEthernetProtoco, e.g.
> Ethernet Powerlink or TTEthernet), so we can control the network while
> the simulation is running and see the behavior of our qemu nodes.

This sounds interesting, I hope we can get it to work and then document
the steps taken.

- Tom

[Bernhard Blieninger]

Hi Tom,

thanks a lot for your very helpful response, i saw the wiki entry before but i didn´t see the part with the bridges.

In the meantime i just fixed all the problems with the bridged network interface and it is now running smoothly.

My setup is now: 4 bridged qemu instances connected to the ns3-csma network.

I figured out that the first bridge created on the host system has some extra priviliges.

So even if I shut down ns-3 it was possible to ping to all other bridges within the instance connected to this bridge.

Tcpdump says that the other bridges are not responding. It shows that only one bridge, the one first created, is responding to all request for all the other bridges.

So i consider this bridge a "root bridge" which is somehow controlling the others and therefore has such a behaviour. Didn´t look it up or investigate it further.

Can you confirmed that or do you know more about it?

As all other bridges are working as they are ment to work i will just exclude the first bridge created to avoid unwanted behaviour.

So the next step is to avoid latency or any unknown behaviour from other system components which means that i actually want to get rid of the bridges

and connect the tap devices of the qemu instances straight to the ns-3 simulation. So no "third party" like the host bridge system is involved anymore.

I could not figure out how this can be done. Because the problem that i cannot connect qemu and ns-3 to the same tap device still remains.

And the wifi-csma example you mentioned also uses the bridging of the host system as the figure in the code shows.

As i see the UseLocal mode, it is just listening and writing to the tap device is it connected via ipc, so this is what i think will be the best for avoiding latency and "third parties" in our simulation.

Can it be done even with qemu or other virtualiser without mutual lock the tap device?

Another next step is that we want to have realtime changes to the simulation, so we want to "disconnect" cables and disturb channels at lifetime, maybe you also know how this can be done best or maybe you have some advice where to look at.

Tanks a lot.

Best Regards,

Bernhard.