Possible bug in CSMA bridge implementation (incomplete routing tables autopopulation)

135 views
Skip to first unread message

neko neko

unread,
Apr 19, 2015, 2:31:14 AM4/19/15
to ns-3-...@googlegroups.com
The full source code follows. It creates a network of bunch of nodes, tied up with CSMA bridges. The model use simple routing (the one that initializes on calling Ipv4GlobalRoutingHelper::PopulateRoutingTables ()). I already tried to run the very similar network model before (the same hosts were just tied together with bus CSMA basic network, the one created with CsmaHelper, no bridges) and it worked perfectly, with routing tables being populated each time. After I redesigned the model to use CSMA bridges instead, most of the routing tables of my host are containing only localhost's and their own network's routes. But, for reason unknown, some of the hosts sometimes have routes to some network beyond their own. Anyway, the model now doesn't work as it should, despite I verified my code dozen of times against official ns3 bridging example, which is, by the way, work too (though, in this example there are only 2 bridges and one router, so it actually doesn't need routing autopopulation)
Or may be I did something wrong? I would really apreciated some assistance asap, I need to submit this piece of work by monday.

#include <iostream>
#include <fstream>
#include "ns3/core-module.h"
#include "ns3/network-module.h"
#include "ns3/csma-module.h"
#include "ns3/bridge-module.h"
#include "ns3/internet-module.h"
#include "ns3/point-to-point-module.h"
#include "ns3/applications-module.h"
#include "ns3/ipv4-global-routing-helper.h"
#include "ns3/flow-monitor.h"
#include "ns3/flow-monitor-helper.h"
#include "ns3/netanim-module.h"
#include "ns3/log.h"
#include <string.h>


using namespace ns3;
using namespace std;

NS_LOG_COMPONENT_DEFINE ("Datacenter_model");

int main(int argc, char *argv[])
{
  bool verbose = false;
  bool with_ECMP = false;
  string ECMP_str="";
  string ttype = "focused";

  CommandLine cmd;
  cmd.AddValue ("ttype", "Connectons\' distribution to simulate", ttype);
  cmd.AddValue ("with_ECMP", "Connectons\' distribution to simulate", with_ECMP);
  cmd.AddValue ("verbose", "Tell echo applications to log if true", verbose);

  if (with_ECMP)
  {
      ECMP_str="-with_ECMP";
  }

  cmd.Parse (argc, argv);

  if (verbose)
    {
      LogComponentEnableAll(LOG_LEVEL_ERROR);
//      LogComponentEnableAll(LOG_LEVEL_INFO);
    }

  LogComponentEnable ("Datacenter_model", LOG_LEVEL_INFO);

  /* Configuration. */

  NS_LOG_INFO ("Create nodes.");

  /* Build nodes. */

  NodeContainer nodes_row1_cl1;
  NodeContainer nodes_row2_cl1;
  NodeContainer sw_Ag_cl1;
  NodeContainer sw_lnk_Ag_to_cl1;

  NodeContainer nodes_row1_cl2;
  NodeContainer nodes_row2_cl2;
  NodeContainer sw_Ag_cl2;
  NodeContainer sw_lnk_Ag_to_cl2;

  NodeContainer sw_Core;
  NodeContainer sw_lnk_Core_to_Ag;

  NodeContainer ether_switches;

  /* Build switches binding the network together. */
  /*sw0 - row1 cl1; sw1 - row2 cl1; sw2 - ag cl1; sw3 - row1 cl2; sw4 - row2 cl2; sw5 - ag cl2; sw6 - core*/

  ether_switches.Create (7);

  /* Build cluster #1 nodes and groups. */

  sw_Ag_cl1.Create (1);

  nodes_row1_cl1.Create (3);
  nodes_row2_cl1.Create (3);

  sw_lnk_Ag_to_cl1.Add (nodes_row1_cl1.Get (2));
  sw_lnk_Ag_to_cl1.Add (nodes_row2_cl1.Get (2));
  sw_lnk_Ag_to_cl1.Add (sw_Ag_cl1.Get (0));

  /* Build cluster #2 nodes and groups. */

  sw_Ag_cl2.Create (1);

  nodes_row1_cl2.Create (3);
  nodes_row2_cl2.Create (3);

  sw_lnk_Ag_to_cl2.Add (nodes_row1_cl2.Get (2));
  sw_lnk_Ag_to_cl2.Add (nodes_row2_cl2.Get (2));
  sw_lnk_Ag_to_cl2.Add (sw_Ag_cl2.Get (0));

  /* Build core network nodes and groups. */

  sw_Core.Create (1);

  sw_lnk_Core_to_Ag.Add (sw_Core.Get (0));
  sw_lnk_Core_to_Ag.Add (sw_Ag_cl1.Get (0));
  sw_lnk_Core_to_Ag.Add (sw_Ag_cl2.Get (0));

  NS_LOG_INFO ("Build Topology");

  /* Build links. */

  CsmaHelper csma;
  csma.SetChannelAttribute ("DataRate", StringValue ("1Mbps"));
  csma.SetChannelAttribute ("Delay", TimeValue (NanoSeconds (500)));

  NetDeviceContainer netdevs_nodes_row1_cl1;
  NetDeviceContainer netdevs_nodes_row2_cl1;
  NetDeviceContainer netdevs_Ag_cl1;

  NetDeviceContainer netdevs_nodes_row1_cl2;
  NetDeviceContainer netdevs_nodes_row2_cl2;
  NetDeviceContainer netdevs_Ag_cl2;

  NetDeviceContainer netdevs_Core;


  NetDeviceContainer switchDevices1;
  NetDeviceContainer switchDevices2;
  NetDeviceContainer switchDevices3;
  NetDeviceContainer switchDevices4;
  NetDeviceContainer switchDevices5;
  NetDeviceContainer switchDevices6;
  NetDeviceContainer switchDevices7;
//-----------------------
  // Create csma links, from each terminal to the switch
  BridgeHelper bridge;
  for (int i = 0; i < 3; i++)
    {
      NetDeviceContainer link = csma.Install (NodeContainer (nodes_row1_cl1.Get (i), ether_switches.Get(0)));
      netdevs_nodes_row1_cl1.Add (link.Get (0));
      switchDevices1.Add (link.Get (1));
    }

  // Create the bridge netdevice, which will do the packet switching
  bridge.Install (ether_switches.Get(0), switchDevices1);

  // Create csma links, from each terminal to the switch

  for (int i = 0; i < 3; i++)
    {
      NetDeviceContainer link = csma.Install (NodeContainer (nodes_row2_cl1.Get (i), ether_switches.Get(1)));
      netdevs_nodes_row2_cl1.Add (link.Get (0));
      switchDevices2.Add (link.Get (1));
    }

  // Create the bridge netdevice, which will do the packet switching

  bridge.Install (ether_switches.Get(1), switchDevices2);

  // Create csma links, from each terminal to the switch

  for (int i = 0; i < 3; i++)
    {
      NetDeviceContainer link = csma.Install (NodeContainer (sw_lnk_Ag_to_cl1.Get (i), ether_switches.Get(2)));
      netdevs_Ag_cl1.Add (link.Get (0));
      switchDevices3.Add (link.Get (1));
    }

  // Create the bridge netdevice, which will do the packet switching

  bridge.Install (ether_switches.Get(2), switchDevices3);
//--------------------------
  // Create csma links, from each terminal to the switch

  for (int i = 0; i < 3; i++)
    {
      NetDeviceContainer link = csma.Install (NodeContainer (nodes_row1_cl2.Get (i), ether_switches.Get(3)));
      netdevs_nodes_row1_cl2.Add (link.Get (0));
      switchDevices4.Add (link.Get (1));
    }

  // Create the bridge netdevice, which will do the packet switching

  bridge.Install (ether_switches.Get(3), switchDevices4);

  // Create csma links, from each terminal to the switch

  for (int i = 0; i < 3; i++)
    {
      NetDeviceContainer link = csma.Install (NodeContainer (nodes_row2_cl2.Get (i), ether_switches.Get(4)));
      netdevs_nodes_row2_cl2.Add (link.Get (0));
      switchDevices5.Add (link.Get (1));
    }

  // Create the bridge netdevice, which will do the packet switching

  bridge.Install (ether_switches.Get(4), switchDevices5);

  // Create csma links, from each terminal to the switch

  for (int i = 0; i < 3; i++)
    {
      NetDeviceContainer link = csma.Install (NodeContainer (sw_lnk_Ag_to_cl2.Get (i), ether_switches.Get(5)));
      netdevs_Ag_cl2.Add (link.Get (0));
      switchDevices6.Add (link.Get (1));
    }

  // Create the bridge netdevice, which will do the packet switching

  bridge.Install (ether_switches.Get(5), switchDevices6);
//--------------------------
  csma.SetChannelAttribute ("DataRate", StringValue ("1.5Mbps"));

  // Create csma links, from each terminal to the switch

  for (int i = 0; i < 3; i++)
    {
      NetDeviceContainer link = csma.Install (NodeContainer (sw_lnk_Core_to_Ag.Get (i), ether_switches.Get(6)));
      netdevs_Core.Add (link.Get (0));
      switchDevices7.Add (link.Get (1));
    }

  // Create the bridge netdevice, which will do the packet switching

  bridge.Install (ether_switches.Get(6), switchDevices7);
//--------------------------

  /* Install the IP stack. */

  InternetStackHelper stack;

  stack.Install (nodes_row1_cl1);
  stack.Install (nodes_row2_cl1);
  stack.Install (sw_Ag_cl1);

  stack.Install (nodes_row1_cl2);
  stack.Install (nodes_row2_cl2);
  stack.Install (sw_Ag_cl2);

  stack.Install (sw_Core);

  /* Assign IP addresses. */

  Ipv4AddressHelper ipv4_address;

  Ipv4InterfaceContainer ipv4_nodes_row1_cl1;
  Ipv4InterfaceContainer ipv4_nodes_row2_cl1;
  Ipv4InterfaceContainer ipv4_Ag_cl1;

  Ipv4InterfaceContainer ipv4_nodes_row1_cl2;
  Ipv4InterfaceContainer ipv4_nodes_row2_cl2;
  Ipv4InterfaceContainer ipv4_Ag_cl2;

  Ipv4InterfaceContainer ipv4_Core;

  ipv4_address.SetBase ("10.0.1.0", "255.255.255.0");
  ipv4_nodes_row1_cl1 = ipv4_address.Assign (netdevs_nodes_row1_cl1);

  ipv4_address.SetBase ("10.0.2.0", "255.255.255.0");
  ipv4_nodes_row2_cl1 = ipv4_address.Assign (netdevs_nodes_row2_cl1);

  ipv4_address.SetBase ("10.1.1.0", "255.255.255.0");
  ipv4_Ag_cl1 = ipv4_address.Assign (netdevs_Ag_cl1);

  ipv4_address.SetBase ("10.0.3.0", "255.255.255.0");
  ipv4_nodes_row1_cl2 = ipv4_address.Assign (netdevs_nodes_row1_cl2);

  ipv4_address.SetBase ("10.0.4.0", "255.255.255.0");
  ipv4_nodes_row2_cl2 = ipv4_address.Assign (netdevs_nodes_row2_cl2);

  ipv4_address.SetBase ("10.2.1.0", "255.255.255.0");
  ipv4_Ag_cl2 = ipv4_address.Assign (netdevs_Ag_cl2);

  ipv4_address.SetBase ("192.168.1.0", "255.255.255.0");
  ipv4_Core = ipv4_address.Assign (netdevs_Core);

  /* Generate Route. */

  Ipv4GlobalRoutingHelper::PopulateRoutingTables ();
  //Ipv4GlobalRoutingHelper::RecomputeRoutingTables ();
  //Ipv4GlobalRouting::RandomEcmpRouting

  /* Generate TCP streams. */

  if (ttype == "mixed")
  {
      /* Create a TCP stream destination. */

      Address LocalAddress (InetSocketAddress (Ipv4Address::GetAny (), 50000));
      PacketSinkHelper packetSinkHelper ("ns3::TcpSocketFactory", LocalAddress);

      ApplicationContainer recvapp;

      recvapp.Add (packetSinkHelper.Install (nodes_row1_cl1.Get (0)));
      recvapp.Add (packetSinkHelper.Install (nodes_row2_cl1.Get (0)));
      recvapp.Add (packetSinkHelper.Install (nodes_row1_cl2.Get (0)));
      recvapp.Add (packetSinkHelper.Install (nodes_row2_cl2.Get (0)));

      recvapp.Start (Seconds (0.3));
      recvapp.Stop (Seconds (152.0));


      /* Create TCP streams' sources. */

      OnOffHelper onOffHelper ("ns3::TcpSocketFactory", Address ());
//      onOffHelper.SetAttribute ("DataRate",StringValue ("5Mbps")); uncomment to control bitrate and packet size of TCP stream generators
//      onOffHelper.SetAttribute ("PacketSize",UintegerValue(512));
      onOffHelper.SetAttribute ("OnTime", StringValue ("ns3::ConstantRandomVariable[Constant=1]"));
      onOffHelper.SetAttribute ("OffTime", StringValue ("ns3::ConstantRandomVariable[Constant=0]"));

      ApplicationContainer sendapp;

      Ptr<Ipv4> ipv4_temp;
      Ipv4InterfaceAddress iaddr_temp;
      Ipv4Address remote_ip_addr;

      /* Create TCP streams from cluster #1 to cluster #2. */

      ipv4_temp = nodes_row1_cl2.Get (0)->GetObject<Ipv4>();
      iaddr_temp = ipv4_temp->GetAddress (1,0);
      remote_ip_addr = iaddr_temp.GetLocal ();

      onOffHelper.SetAttribute ("Remote", AddressValue ((InetSocketAddress (remote_ip_addr, 50000))));
      sendapp.Add (onOffHelper.Install (nodes_row1_cl1.Get (1)));

      ipv4_temp = nodes_row2_cl2.Get (0)->GetObject<Ipv4>();
      iaddr_temp = ipv4_temp->GetAddress (1,0);
      remote_ip_addr = iaddr_temp.GetLocal ();

      onOffHelper.SetAttribute ("Remote", AddressValue ((InetSocketAddress (remote_ip_addr, 50000))));
      sendapp.Add (onOffHelper.Install (nodes_row2_cl1.Get (1)));

      /* Create TCP streams from cluster #2 to cluster #1. */

      ipv4_temp = nodes_row1_cl1.Get (0)->GetObject<Ipv4>();
      iaddr_temp = ipv4_temp->GetAddress (1,0);
      remote_ip_addr = iaddr_temp.GetLocal ();

      onOffHelper.SetAttribute ("Remote", AddressValue ((InetSocketAddress (remote_ip_addr, 50000))));
      sendapp.Add (onOffHelper.Install (nodes_row1_cl2.Get (1)));

      ipv4_temp = nodes_row2_cl1.Get (0)->GetObject<Ipv4>();
      iaddr_temp = ipv4_temp->GetAddress (1,0);
      remote_ip_addr = iaddr_temp.GetLocal ();

      onOffHelper.SetAttribute ("Remote", AddressValue ((InetSocketAddress (remote_ip_addr, 50000))));
      sendapp.Add (onOffHelper.Install (nodes_row2_cl2.Get (1)));

      sendapp.Start (Seconds (0.3));
      sendapp.Stop (Seconds (152.0));

  }
  else
      if (ttype == "focused")
      {
          /* Create a TCP stream destination. */

          Address LocalAddress (InetSocketAddress (Ipv4Address::GetAny (), 50000));
          PacketSinkHelper packetSinkHelper ("ns3::TcpSocketFactory", LocalAddress);
          ApplicationContainer recvapp = packetSinkHelper.Install (nodes_row1_cl1.Get (0));
          recvapp.Start (Seconds (0.3));
          recvapp.Stop (Seconds (152.0));


          /* Create TCP streams' sources. */

          OnOffHelper onOffHelper ("ns3::TcpSocketFactory", Address ());
//          onOffHelper.SetAttribute ("DataRate",StringValue ("5Mbps")); uncomment to control bitrate and packet size of TCP stream generators
//          onOffHelper.SetAttribute ("PacketSize",UintegerValue(512));
          onOffHelper.SetAttribute ("OnTime", StringValue ("ns3::ConstantRandomVariable[Constant=1]"));
          onOffHelper.SetAttribute ("OffTime", StringValue ("ns3::ConstantRandomVariable[Constant=0]"));

          Ptr<Ipv4> ipv4_temp = nodes_row1_cl1.Get (0)->GetObject<Ipv4>();
          Ipv4InterfaceAddress iaddr_temp = ipv4_temp->GetAddress (1,0);
          Ipv4Address remote_ip_addr = iaddr_temp.GetLocal ();

          ApplicationContainer sendapp;

          AddressValue remoteAddress (InetSocketAddress (remote_ip_addr, 50000));
          onOffHelper.SetAttribute ("Remote", remoteAddress);
          sendapp.Add (onOffHelper.Install (nodes_row1_cl1.Get (1)));

          onOffHelper.SetAttribute ("Remote", remoteAddress);
          sendapp.Add (onOffHelper.Install (nodes_row2_cl1.Get (0)));

          onOffHelper.SetAttribute ("Remote", remoteAddress);
          sendapp.Add (onOffHelper.Install (nodes_row2_cl1.Get (1)));

          onOffHelper.SetAttribute ("Remote", remoteAddress);
          sendapp.Add (onOffHelper.Install (nodes_row1_cl1.Get (1)));

          onOffHelper.SetAttribute ("Remote", remoteAddress);
          sendapp.Add (onOffHelper.Install (nodes_row1_cl2.Get (0)));

          onOffHelper.SetAttribute ("Remote", remoteAddress);
          sendapp.Add (onOffHelper.Install (nodes_row1_cl2.Get (1)));

          onOffHelper.SetAttribute ("Remote", remoteAddress);
          sendapp.Add (onOffHelper.Install (nodes_row2_cl2.Get (0)));

          onOffHelper.SetAttribute ("Remote", remoteAddress);
          sendapp.Add (onOffHelper.Install (nodes_row2_cl2.Get (1)));


          sendapp.Start (Seconds (0.3));
          sendapp.Stop (Seconds (152.0));

      }
      else

      {
          Simulator::Destroy ();
          return 1;
      }


/*

          for (int i = 0; i < apps; i++)
          {

              // flow :  t[i] -> r[i]
              onOffHelper.SetAttribute ("Remote", AddressValue(InetSocketAddress(interfaces.GetAddress(r[i]), cbrPort ))); //destination
              cbrApps.Add(onOffHelper.Install(nodes.Get(t[i])));//source
          }

          //Start to generate traffic ..
          cbrApps.Start(Seconds(1.0));
          //Stop generating traffic ..
          cbrApps.Stop(Seconds(m_totalTime));

*/



  /* Simulation. */

  NS_LOG_INFO ("Configure Tracing.");

  /* Pcap output. */

  /* Gathering stats with Flow Monitor, calculating throughput, preparing data for GNU Plot. */


  /* Stop the simulation after x seconds. */

  uint32_t stopTime = 152;
  Simulator::Stop (Seconds (stopTime));

  /* Start and clean simulation. */

  Simulator::Run ();

  Simulator::Destroy ();
  return 0;
}

Tommaso Pecorella

unread,
Apr 19, 2015, 4:00:52 AM4/19/15
to ns-3-...@googlegroups.com
Hi,

please use attachments. It's faster and less cluttered.
Please provide a diagram showing your intended network topology, from what I can see you have a full redundancy through bridges, i.e., you don't really have any router that you have to pass by due to physical network topology. I'm not sure how GlobalRouting could react to this.

Cheers,

T.
.

neko neko

unread,
Apr 19, 2015, 8:13:26 AM4/19/15
to ns-3-...@googlegroups.com
Hi, Tommaso. Thank you for your response. I'm attaching the picture I'm seeing in PyViz with some descriptions added.

So I have 3 "backbone" networks connecting 2 "clusters" each containing 4 hosts (left unmarked on my scheme):
The core network described as NodeContainer sw_lnk_Core_to_Ag .
2 other networks - described by NodeContaners sw_lnk_Ag_to_cl1 and sw_lnk_Ag_to_cl2 - are of "agreggation" kind, each of them connects 2 rows of hosts in its own "cluster" together.
Please note that they are neatly intersected - in marked section of topology they are indeed connected to the same switch devices.
But bridges were installed only on nodes marked with red; those marked with green don't have their internal interfaces connected with bridgehelper, instead there is internet protocol installed (with internetsctackhelper) and they are the ones that should route traffice between the [unmarked] hosts. They shouldn't directly interconnect neighboring network.
19.04.png

neko neko

unread,
Apr 19, 2015, 8:22:26 AM4/19/15
to ns-3-...@googlegroups.com
I also must add that when I start the simulation I only can see host communicating inside their own bridged subnetworks. Even broadcasts won't leave their broadcasting domains (marked as grey rectangulars). If they all were bridged, as you imply, then broadcasts would traverse all network models, from side to side. You can check it for yourself if you run the simulation with next command (from your ns3 directory): "./waf --run "name_of_your_file --ttype=focused" --vis" 

neko neko

unread,
Apr 19, 2015, 8:42:59 AM4/19/15
to ns-3-...@googlegroups.com
I've also decided to add network ranges to picture, for your convinience. Please see new attachment.
I honestly can't see how what I do so much different from what is done in csma-brdige-one-hop.cc example. But, as I said, it only contains one router, so it doesn't need its routing table to be populated with routes to start working immidiately
 
19.04.png

neko neko

unread,
Apr 19, 2015, 8:47:25 AM4/19/15
to ns-3-...@googlegroups.com
I'm sorry for this flurry of messages :) I've also thought you still would like to have the original source codes as an attachment, too
source_code.cc

neko neko

unread,
Apr 19, 2015, 11:10:06 AM4/19/15
to ns-3-...@googlegroups.com
Here is a new picture with some more findings. As you can see, 2 routers of cluster #1 have routes to one of the subnets, but the router connecting this subnet does not have any external routes at all. What is intresting is that the situation is mirrored in cluster #2 too.
19.04.png

neko neko

unread,
Apr 19, 2015, 11:15:51 AM4/19/15
to ns-3-...@googlegroups.com
And also please note than no router of cluster #1 has routes to any cluster's #2 subnet, and vice versa. Host on core network only aware about network it connected to, also. I'm giving up, there is no possible thing I can do here, my code is the same as offered by examples. I probably should try to populate these tables with routes manually as I need this thing to work asap. But the problem is I will need ECMP routing functionality, too, and I don't know wheter or not it will work in case of manual routing setup.

neko neko

unread,
Apr 19, 2015, 2:33:50 PM4/19/15
to ns-3-...@googlegroups.com
I understand that my questions could appear lame and display lack of research into subject, but I'm really in a hurry at the moment, so it would helped a lot if somebody could give me a hint on whether or not it's a bug, or an error in my own code. I've spent several hours on veryfing and googling, and would like to know is it worth to invest more time in it at the moment.

Tommaso Pecorella

unread,
Apr 19, 2015, 3:19:15 PM4/19/15
to ns-3-...@googlegroups.com
Hi,


Thanks for finding it,

T.

neko neko

unread,
Apr 19, 2015, 3:57:13 PM4/19/15
to ns-3-...@googlegroups.com
Hi, Tommaso.

Thank you for answering. Thats unfortunate. Do you think I will be able to circumvent it by setting all routes manually? Or is the bridge implementation is broken in other aspects too, not only the part responsible for route propagation?

воскресенье, 19 апреля 2015 г., 22:19:15 UTC+3 пользователь Tommaso Pecorella написал:

Tommaso Pecorella

unread,
Apr 19, 2015, 4:09:02 PM4/19/15
to ns-3-...@googlegroups.com
Hi,

nope, the manual routing will work. Also RipNg would work, but it's for IPv6.
Summarizing, you have 2 alternatives:
1) use StaticRouting (either the v4 or v6) and manually set all the routes in all the nodes, or
2) use RipNg for IPv6 (but you'll not be able to use IPv4).

Cheers,

T.

PS. while debugging your script, I found another bug. I pushed the fix on ns-3-dev.

neko neko

unread,
Apr 19, 2015, 4:20:49 PM4/19/15
to ns-3-...@googlegroups.com
Hi. Thanks a lot, I will try it this way then.
Best.

воскресенье, 19 апреля 2015 г., 23:09:02 UTC+3 пользователь Tommaso Pecorella написал:
Reply all
Reply to author
Forward
0 new messages