What Makes ns-3 a Complex Thing to Understand and Use?

[igs...@gmail.com]

Dear all,
I am trying to make a post/article on "What Makes ns-3 a Complex Thing to Understand and Use? "

The following is the thing that I discussed in that article/post.
 (Sorry. I could not format this text here - to see formatted text use the above link)

Please share your insights to improve and incorporate things in that article. I think it may be useful for newcomers to understand ns-3.

Please share your experience so that I may fulfil the scope of that post/article

Charles Pandian
-------------------------------------------
What Makes ns-3 a Complex Thing to Understand and Use? 

Article Written by:  CharlesPandian November 23, 2021 |  Edit This

While starting to learn ns-3, most of us find it very difficult to understand. Particularly, if the student already knows to use ns-2 or Omnet++ then it will even become much harder for him to understand/grasp what ns-3 really is.  The student’s or scholar’s preconceived expectation in a simulator will make him misunderstand ns-3. Another major obstacle/misconception in using ns-3 is the way in which python is frequently used along with C++. (Thank Mr Tom Henderson for recent news regarding the removal of python bindings -because it will reduce some of these obstacles )

Of course, the ns-3 Documentations and Manual pages are providing explanations for each and every aspect of this wonderful simulator. But they are huge and their hugeness itself makes it impossible to grasp the very fine, basic aspects of this simulator.
Often, the developer of ns-3 also generously provide answers to the questions or students/learners of ns-3 through the official ns-3 user group.  Even with this kind of help and support, it still, it makes impossible for someone to learn and understand ns-3.

This article tries to explain why it is ambiguously hard to understand ns-3 and an ns-3 simulation (I mean, in most cases, even the student can not express the reason for his poor understanding/misunderstanding on ns-3).

How ns-3 is different from most of the other simulators?

In most of the popular simulators (except ns-3), the simulator will be a different entity and the simulation script that the user will write for that simulator will be a separate entity. While running the simulation script, the simulator will just run/interpret the lines that the user created.

For example,

• Under ns-2, the simulation will be presented in the form of TCL script and the core simulator which is made up of C++ and TCL objects will simply interpret the lines of the TCL simulation script.
• Under Omnet ++, the simulation script will be a custom format text file (not like any programming language) core simulator which is made up of C++ objects will simply interpret the lines of the simulation script.
• Even most of the non-free simulators are following the same ideology (Simulator is different from simulation script)

If a researcher needs to add a new protocol/application agent (for example in the case of ns-2), then they will write code for that protocol/application agent, by adopting the standards of the core simulator and compiling the core simulator itself with the added new protocol/application agent model.  After that, a user can use that new object within their simulations. The simulation script can be modified as per the requirement and can be run without recompiling the core simulator code.

So in other simulators, the simulator will interpret/run the simulation script.

So there will be only one “main()” function through which the core simulator starts running and start interpreting/running the simulation script. No other C++ code module will have “main()” function.

The Organization of ns-2 Simulator and an ns-3 SimulationA. The ns-2 Way of Doing Simulation and Analysis

a. The Simulator

The following explains psuedo code explain’s simulator ns-2’s  architecture :
The ns-2 Simulator (made up of C++ and TCL Object)

main() {
//the core simulator 
// and some core functionalities
}
OtherProtocolModule1() {
//the protocol module code
}
OtherProtocolModule2() {
//the protocol module code
}
…
…
ANewCustomProtocolModule() {
//the protocol module code added by the user/researcher
}

b. The Simulation

 

A Typical User-level Simulation Code (made up of TCL)

// The simulation script will Use the above ns-2 modules and do the following:

//Initializing some common simulation parameters
//Creating simulation Object
//Initializing event trace logs and nam Trace ouput
//Configuring Nodes and Protocols
//Setting traffic Flows
//Tunning the simulation

c) The Trace Analysis

Trace Analysis is generally done separately using a scripting language (using scripting languages/(advance filters from UNIX perspective) such as tcl, awk, perl, shell etc.,)

 

A Typical Trace Analysis Script 

// Read the even trace output that was generated during the simulation
// Phrase them line by line and get statistics out of it
// prepare tables and graphs using the extracted statistics

So, generally, under ns-2, the simulator is a different entity – so without compiling its code, again and again, a tcl level simulation script can be modified and run several times.  Further, as a standard, the trace analysis done by another independent script, it can be run again and again (with little modification during trial and error) without even disturbing or running the simulation code.

The ns-3 Way of Doing Simulation and Analysis

ns-3’s way is entirely different. Here, the simulation script that we write in C++ will itself become a separate simulator because it will have the “main()” function. In other words, each and every simulation script that we write under ns-3 will become a separate simulator. So whenever we need to change a single line in our simulation script (C++ code), then we need to recompile our simulator.

a. The Simulation Modules

The following is the way in which the ns-3 modules and other modules can be visualized. Here, there is no module with a main() function. One model can inherit from another and may use the functionalities of another module.

CoreSimulatorModule{
     //the core simulator modules
     // and some core functionalities
}
OtherProtocolModule1() {
     //the protocol module code
     // functionalities inherited from other modules
}
OtherProtocolModule2() {
     //the protocol module code
     // may also contain some minor modification done by a user/researcher
}
…
…
ANewCustomProtocolModule() {
     //the protocol module code added by the user/researcher
}

b) The Simulation or the Custom Simulator

A Typical ns-3 Simulation Script (made up of ns-3 Standard c++ Code). Here, it is the gate way of start of a ns-3 simulation.

// The simulation script will Use the above ns-3 core modules and other modules and do the following:
main() {
      //Initializing some common simulation parameters
      //Creating simulation Objects
      //Configuring Nodes and Protocols
      //Setting traffic Flows
      //Initializing different kinds of trace logs and NetAnim Trace ouput
      //running the simulation
}

So, after compiling this ns-3 simulation script(C++  code), the compiled output itself will become an independent simulator on its own.  (Omnet++ simulations can also be compiled and used as separate custom simulators – but ns-2 simulation script will always depend on the core simulator and only can be run through that)

c) The Trace Analysis

There are so many standards (non-standards) in doing trace analysis in the case of ns-3.  These standards (non-standards) is another main obstacle in front of a student or researcher that makes it difficult to understand and do research under ns-3.

One may do trace analysis in the following ways :

1) We can use the default logging functionality of ns-3 protocol agents /models to log different events of simulation in a text file and then do Trace Analysis just like in the case of ns-2. Like analyzing the trace logs separately using a scripting language (using scripting languages/(advance filters from UNIX perspective) such as tcl, awk, perl, shell etc.,). This approach mostly fails or will not be reliable because of the standards (non-standards) in triggering event logs by different ns-3 protocol agents /models. Further, such creation of logs by a particular protocol agent may differ from one ns-3 version to another. (my articles  “Event Trace File Generation in a ns-3 Simulation” and “Trace Analysis of ns-3 Ascii Traces using TraceMetrics Tool – An elementary Trace Analysis Solution for ns-3“ are dealing with this kind of tracking and analysis.)

 

2) We can generate tcpdump format trace outputs while running a simulation, and later we may do trace analysis using tcpdump analysers such as “Wireshark”. This approach also will not be useful in most cases of a scholarly research-level simulation – because, in such cases, the trace analysis should be done on the output trace files of several batch runs of the simulation.  So applying Wireshark based trace analysis on that huge number of separate trace files will become a meaningless operation. So, this kind of approach of trace analysis will be useful in understanding an event from a single run of simulation – for example while teaching something in classroom/lab or finding a critical event that may be causing unexpected behaviour in a simulation.

 

3) The most reliable way of doing trace analysis on a batch of ns-3 simulations is using the FlowMonitor facility of ns-3. Further, that cal also is done more than one way.

i)  The most often used way is: using FlowMonitor inside the ns-3 simulation, and iterating FlowMonitor::FlowStats with a for loop and calculate and to log throughput, delay etc.,  using different parameters of FlowMonitor::FlowStats. Again, this will need additional scripting to collect and parse output from the logged outputs of several batch runs of the simulation.

ii) Another way using FlowMonitor is: to instruct the FlowMonitor to log all the events in an XML file and we may later do trace analysis on such  XML flow monitor log files. This way is much suitable for doing scholarly research. (my article  ”

Trace Analysis of TCP Flows Under ns-3 MANET/ FANET/ VANET/ WSN Scenario” uses this approach)

iii) There may be some other technique to use FlowMonitor inside a simulation and using the XML file of FlowMonitor outside the simulation – I do not know – the researcher may explore it as per their need of analysis of the ouputs of batch runs of the simulation.

So, what is ns-3 now?

ns-3 is nothing but a collection of C++ objects of Protocols/Application Agents and the simulation core modules. So the separate code of all of these objects of Protocols/Application Agents and the simulation core modules, will not have “main()” function.  So ns-3 is technically a collection of Network Simulation objects – not a simulator by itself.  (please correct me if I am wrong)

So, simply, the simulation script (the C++ code) that we write will have the main() function, which will be the gateway/starting point of the simulator. So, simply, after compiling our code, using all other referred/used modules of ns-3(including the core simulator modules),  our code itself gets compiled and will become a new simulator. (please correct me if I am wrong)

Technically, what we conventionally call ns-3 is not at all a simulator by itself. The simulation script that we write itself will become a custom simulator. So each and every simulation script that we write will become a separate, custom network simulator. (please correct me if I am wrong)

Why ns-3 simulation run faster than most of the simulators?

The main reason is; the simulation code is not interpreted by any other entity. The simulation code itself is the simulator. So it will run at the native speed of the hardware and software – just like other running executable code/process of the operating system.

Another important reason is: While writing the ns-3 simulation script, we will only add the C++ objects of Protocols/Application Agents that are needed for that particular simulation. So, the final compiled version of the simulator (compiled version of our code) will only have the used/referred modules. So, each final compiled version of the simulator (compiled version of our code) will be smaller in size and the size may vary according to the number of used/referred C++ objects of Protocols/Application Agents in each simulation scripts (our main c++ simulation code)

 

Other obstacles in front of Understanding ns-3

The more advanced way of handling C++ objects/functions/constants/variables makes it even harder to understand or grasp a typical ns-3 simulation model and even a simple ns-3 simulation script.

Even though some of these kinds of advanced C++ conventions are taught during an object-oriented programming classroom, here and there, they are not commonly practised in programming; at least by a student or an amateur programmer.

For example, the following line will set the default WifiMacQueue delay

Config::SetDefault(“ns3::WifiMacQueue::MaxDelay”, TimeValue(Seconds(WifiMacQueueMaxDelay)));

Of course, there may a good reason for doing like this from the Object-Oriented Programming Point of View. But these kinds of long, explicit references in declarations simply make it impossible to grasp what actually we are doing here(and where). Further, if a typo mistake is made anywhere in this line the compiler will raise an error message in a “foreign language” which will not be at all inhuman readable form. It may advise you to use gdb to trace that error – then again the gdb will raise another error message in another “foreign language” which will not also be at all inhuman readable form.  (Please think about a poor learner who is trying to learn ns-3 here)

What restricts the developers to using simple conventions/declarations like the following? In this case, understanding the line and even tracking an error may be an easy task for a human.

WifiMacQueue->MaxDelay=10
or
WifiMacQueue.MaxDelay=10
or even
WifiMacQueue.SetMaxDelay(10)

Of course, this kind of declaration may lead to some errors/problems from an Object-Oriented Programming Point of View.  But will be a more human-understandable way of doing things in a computer programme.

To understand it further, I hereby give another example :

An XML way of handling data may be easy for a machine to handle and parse data. But definitely, the same XML format data will not be good for human understanding.

So, from this perspective, an ns-3 simulation script is more friendly towards a machine-readable form. Definitely not friendly towards a human-readable form.

While developing a huge system like ns-3 using Object-Oriented Programming techniques, the level of attraction and hierarchy of organizing/doing things should hide the complex things behind the scene from the end-user (the ns-3 simulation writer/student/researcher).  The interfaces that are provided to handle the objects and change the properties of the objects should be human-understandable forms.  But according to my view, from the first day of development itself, the developers ignored this point and now the ns-3 becomes a huge, complex machine only understandable form. 

Once I heard that Unix was originally developed by programmers for programmers.

Just like that, I think, ns-3 was developed by brilliant developers for brilliant developers.

But I think, changing the way ns-3 does these kinds of things can not be altered/rectified now – because, ns-3 become huge with all kinds of these complexities.  Just expressing my feelings/disabilities that’s all.

Solution(?):

And I think, using ns-3 within an IDE like Eclipse or QTCreator may hide some of this complexity. But, definitely, it also will make a user not to understand what is really happening in a particular line of code.

(Final Note: My scope is not offending the generous developers who are sacrificing their precious time on the development and maintenance of ns-3 – just telling my  opinion from my idiotic point of view -that’s all)

Charles Pandian. 

[Gabriel]

Just my opinion on a few points:

> Why ns-3 simulation run faster than most of the simulators?

> The main reason is; the simulation code is not interpreted by any other entity. 

As far as I know, other simulators parse the simulation script and schedule a few events, which is exactly what the Python interpreter did with the help of the ns-3 bindings.

The reason simulators are slow include:

1) the compiler and the processor don't know at compile time nor at runtime which paths will really be taken by each simulation, so their prediction fails more often than normal programs, which results in more branch and cache misses, pipeline stalls, etc;

2) how things are organized in memory can drastically affect performance due to cache collisions and misses, etc.

Even ns-3 has these problems. This is where profile guided optimizations (PGO), link time optimization (LTO),  multi-level JIT, software speculation and other techniques may help.

Regarding trace results: there is work on that front. 

And SEM probably does what you want: https://github.com/signetlabdei/sem

> Config::SetDefault(“ns3::WifiMacQueue::MaxDelay”, TimeValue(Seconds(WifiMacQueueMaxDelay)));

> What restricts the developers to using simple conventions/declarations like the following? In this case, understanding the line and even tracking an error may be an easy task for a human.

> WifiMacQueue->SetMaxDelay=10

> So, from this perspective, an ns-3 simulation script is more friendly towards a machine-readable form. Definitely not friendly towards a human-readable form.

As far as I know you can do it both ways. The first one will make that value the default value for instances created after it is called and the latter will only change for the instance/object that called it.

> And I think, using ns-3 within an IDE like Eclipse or QTCreator may hide some of this complexity.

We are working on that.

>  But, definitely, it also will make a user not to understand what is really happening in a particular line of code.

How so? Some IDEs bring up the docs, expand macros showing the underlying code, highlight orthographic and syntax mistakes, plus additional c++ checks.

It can also help you understand the chain of events as you can click to go to the definition of specific functions. 

[Charles Pandian]

Thank you, Mr. Gabriel for sharing your insights.
It will help me and others to understand ns-3 and bring us a little closer to it.

 Mr Tom Henderson recently posted a news regarding the removal of python bindings .
Could you elaborate what is "removal of Python bindings?"

Is it meaning entire removal of Python? (including the python based guild process?)

Will it improve the performance of the simulator and will it reduce some of the python related complexities in the code (I mean, users are trying to learn both and simply wasting their time - of course, before settling with C++ only conventions) 

Charles Pandian,

--
Posting to this group should follow these guidelines https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting
---
You received this message because you are subscribed to the Google Groups "ns-3-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ns-3-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/daefd1f9-9661-402c-aa92-7231e0074806n%40googlegroups.com.

[Charles Pandian]

I mean
"Is it meaning entire removal of Python? (including the python-based build process?)

Charles Pandian,

[Gabriel]

Bindings are usually wrappers written in a different language which call a standard API (in the case of ns-3 it is c++). 

The python bindings are doing nothing more than loading the c++ libraries, calling functions (just like any linked program would do), but it needs to translate the compiled types into to something Python can represent and from Python back to the compiled structures to use with the C++ calls. 

These wrappers can be written by hand or be scanned and built (semi-)automatically. 

The current generator being used is pybindgen, which doesn't support a few features being used, so maintainers need to use workarounds to keep it working. This is what is changing if there isn't any demand for it. 

I'm working on a replacement for the python based buildsystem (waf). It still needs a lot of testing, some refactoring and updating the documentation, but using IDEs and some other tools will be easier. However, this has nothing to do with the python bindings. 

There's no python code in the simulator itself, so no reduction in complexity. 

Neither the removal of the python bindings nor the replacement of the buildsystem will affect the simulator performance in any way. 

In the best case scenario, the build phase of the project will be a bit faster. 

[Tom Henderson]

Charles, thanks for your thoughts; some responses inline below.

On 11/23/21 12:39 AM, igs...@gmail.com wrote:

Dear all,
I am trying to make a post/article on "What Makes ns-3 a Complex Thing to Understand and Use? "

The following is the thing that I discussed in that article/post.
 (Sorry. I could not format this text here - to see formatted text use the above link)

Please share your insights to improve and incorporate things in that article. I think it may be useful for newcomers to understand ns-3.

Please share your experience so that I may fulfil the scope of that post/article

Charles Pandian
-------------------------------------------
What Makes ns-3 a Complex Thing to Understand and Use? 

Article Written by:  CharlesPandian November 23, 2021 |  Edit This

While starting to learn ns-3, most of us find it very difficult to understand. Particularly, if the student already knows to use ns-2 or Omnet++ then it will even become much harder for him to understand/grasp what ns-3 really is.  The student’s or scholar’s preconceived expectation in a simulator will make him misunderstand ns-3. Another major obstacle/misconception in using ns-3 is the way in which python is frequently used along with C++. (Thank Mr Tom Henderson for recent news regarding the removal of python bindings -because it will reduce some of these obstacles )

Of course, the ns-3 Documentations and Manual pages are providing explanations for each and every aspect of this wonderful simulator. But they are huge and their hugeness itself makes it impossible to grasp the very fine, basic aspects of this simulator.
Often, the developer of ns-3 also generously provide answers to the questions or students/learners of ns-3 through the official ns-3 user group.  Even with this kind of help and support, it still, it makes impossible for someone to learn and understand ns-3.

'impossible' is a bit drastic, but I will grant you that it can have a steep learning curve.  My impression is that it is no worse than ns-2, however.

This article tries to explain why it is ambiguously hard to understand ns-3 and an ns-3 simulation (I mean, in most cases, even the student can not express the reason for his poor understanding/misunderstanding on ns-3).

How ns-3 is different from most of the other simulators?

In most of the popular simulators (except ns-3), the simulator will be a different entity and the simulation script that the user will write for that simulator will be a separate entity. While running the simulation script, the simulator will just run/interpret the lines that the user created.

For example,

• Under ns-2, the simulation will be presented in the form of TCL script and the core simulator which is made up of C++ and TCL objects will simply interpret the lines of the TCL simulation script.

With ns-3, the idea was that Python could take the place of OTcl, but unlike ns-2, it was possible to write directly in C++.  Over time, there hasn't been much use of the Python API, however.

• Under Omnet ++, the simulation script will be a custom format text file (not like any programming language) core simulator which is made up of C++ objects will simply interpret the lines of the simulation script.
• Even most of the non-free simulators are following the same ideology (Simulator is different from simulation script)

Many simulators have a high-level modeling language, and a GUI for simulation composition.  ns-3 did not preclude adding this, but the decision early on was to just code directly into C++, assuming that typical users would need to know C++ anyway to modify or write new models.  A GUI or high-level language could be added on top of ns-3, and it would be very helpful to new users or those less skilled with C++; the problem is that this is a lot of work to develop and maintain.

If a researcher needs to add a new protocol/application agent (for example in the case of ns-2), then they will write code for that protocol/application agent, by adopting the standards of the core simulator and compiling the core simulator itself with the added new protocol/application agent model.  After that, a user can use that new object within their simulations. The simulation script can be modified as per the requirement and can be run without recompiling the core simulator code.

So in other simulators, the simulator will interpret/run the simulation script.

I am not really following how an interpreted vs. compiled script is making things difficult for ns-3.  In any case, we do (at the moment) have Python for this if you want.

It is not hard to use ns-3 command-line arguments to avoid recompiling the program when conducting a large simulation campaign, though.

Further, as a standard, the trace analysis done by another independent script, it can be run again and again (with little modification during trial and error) without even disturbing or running the simulation code.

The ns-3 Way of Doing Simulation and Analysis

ns-3’s way is entirely different. Here, the simulation script that we write in C++ will itself become a separate simulator because it will have the “main()” function. In other words, each and every simulation script that we write under ns-3 will become a separate simulator. So whenever we need to change a single line in our simulation script (C++ code), then we need to recompile our simulator.

Please keep in mind that the simulator is really a set of shared libraries that are less frequently compiled.  I am not sure that calling every simulation script a separate 'simulator' is a helpful way to put it.

We have felt that the 'non-standards' was a feature, not an obstacle.  There are many choices, some built-in (ascii, pcap, flowmon) and the rest (trace sources, logs) customizable.

One may do trace analysis in the following ways :

1) We can use the default logging functionality of ns-3 protocol agents /models to log different events of simulation in a text file and then do Trace Analysis just like in the case of ns-2. Like analyzing the trace logs separately using a scripting language (using scripting languages/(advance filters from UNIX perspective) such as tcl, awk, perl, shell etc.,). This approach mostly fails or will not be reliable because of the standards (non-standards) in triggering event logs by different ns-3 protocol agents /models. Further, such creation of logs by a particular protocol agent may differ from one ns-3 version to another. (my articles  “Event Trace File Generation in a ns-3 Simulation” and “Trace Analysis of ns-3 Ascii Traces using TraceMetrics Tool – An elementary Trace Analysis Solution for ns-3“ are dealing with this kind of tracking and analysis.)

 

2) We can generate tcpdump format trace outputs while running a simulation, and later we may do trace analysis using tcpdump analysers such as “Wireshark”. This approach also will not be useful in most cases of a scholarly research-level simulation – because, in such cases, the trace analysis should be done on the output trace files of several batch runs of the simulation.  So applying Wireshark based trace analysis on that huge number of separate trace files will become a meaningless operation. So, this kind of approach of trace analysis will be useful in understanding an event from a single run of simulation – for example while teaching something in classroom/lab or finding a critical event that may be causing unexpected behaviour in a simulation.

 

3) The most reliable way of doing trace analysis on a batch of ns-3 simulations is using the FlowMonitor facility of ns-3. Further, that cal also is done more than one way.

i)  The most often used way is: using FlowMonitor inside the ns-3 simulation, and iterating FlowMonitor::FlowStats with a for loop and calculate and to log throughput, delay etc.,  using different parameters of FlowMonitor::FlowStats. Again, this will need additional scripting to collect and parse output from the logged outputs of several batch runs of the simulation.

ii) Another way using FlowMonitor is: to instruct the FlowMonitor to log all the events in an XML file and we may later do trace analysis on such  XML flow monitor log files. This way is much suitable for doing scholarly research. (my article  ”

Trace Analysis of TCP Flows Under ns-3 MANET/ FANET/ VANET/ WSN Scenario” uses this approach)

iii) There may be some other technique to use FlowMonitor inside a simulation and using the XML file of FlowMonitor outside the simulation – I do not know – the researcher may explore it as per their need of analysis of the ouputs of batch runs of the simulation.

So, what is ns-3 now?

ns-3 is nothing but a collection of C++ objects of Protocols/Application Agents and the simulation core modules. So the separate code of all of these objects of Protocols/Application Agents and the simulation core modules, will not have “main()” function.  So ns-3 is technically a collection of Network Simulation objects – not a simulator by itself.  (please correct me if I am wrong)

ns-3 is a set of simulation libraries that can be linked to many simulation programs that configure the scenario, run the simulation, and perform any necessary post-processing.  I think you can clearly call ns-3 a simulator; I don't think people would object to that term.

So, simply, the simulation script (the C++ code) that we write will have the main() function, which will be the gateway/starting point of the simulator. So, simply, after compiling our code, using all other referred/used modules of ns-3(including the core simulator modules),  our code itself gets compiled and will become a new simulator. (please correct me if I am wrong)

Technically, what we conventionally call ns-3 is not at all a simulator by itself. The simulation script that we write itself will become a custom simulator. So each and every simulation script that we write will become a separate, custom network simulator. (please correct me if I am wrong)

I'm sorry but I do not really understand the emphasis here on whether or not to call ns-3 or its programs a simulator.

Why ns-3 simulation run faster than most of the simulators?

The main reason is; the simulation code is not interpreted by any other entity. The simulation code itself is the simulator. So it will run at the native speed of the hardware and software – just like other running executable code/process of the operating system.

Another important reason is: While writing the ns-3 simulation script, we will only add the C++ objects of Protocols/Application Agents that are needed for that particular simulation. So, the final compiled version of the simulator (compiled version of our code) will only have the used/referred modules. So, each final compiled version of the simulator (compiled version of our code) will be smaller in size and the size may vary according to the number of used/referred C++ objects of Protocols/Application Agents in each simulation scripts (our main c++ simulation code)

 

I think Gabriel had some feedback on these performance points.

Other obstacles in front of Understanding ns-3

The more advanced way of handling C++ objects/functions/constants/variables makes it even harder to understand or grasp a typical ns-3 simulation model and even a simple ns-3 simulation script.

Even though some of these kinds of advanced C++ conventions are taught during an object-oriented programming classroom, here and there, they are not commonly practised in programming; at least by a student or an amateur programmer.

For example, the following line will set the default WifiMacQueue delay

Config::SetDefault(“ns3::WifiMacQueue::MaxDelay”, TimeValue(Seconds(WifiMacQueueMaxDelay)));

Of course, there may a good reason for doing like this from the Object-Oriented Programming Point of View. But these kinds of long, explicit references in declarations simply make it impossible to grasp what actually we are doing here(and where). Further, if a typo mistake is made anywhere in this line the compiler will raise an error message in a “foreign language” which will not be at all inhuman readable form. It may advise you to use gdb to trace that error – then again the gdb will raise another error message in another “foreign language” which will not also be at all inhuman readable form.  (Please think about a poor learner who is trying to learn ns-3 here).

This is a consequence of the low-level C++ that users need to deal with.  However, most typos are easily deduced and fixed, in my experience.

What restricts the developers to using simple conventions/declarations like the following? In this case, understanding the line and even tracking an error may be an easy task for a human.

WifiMacQueue->MaxDelay=10
or
WifiMacQueue.MaxDelay=10
or even
WifiMacQueue.SetMaxDelay(10)

Of course, this kind of declaration may lead to some errors/problems from an Object-Oriented Programming Point of View.  But will be a more human-understandable way of doing things in a computer programme.

I agree, but as Gabriel pointed out, you can do this with similar syntax already if you just need to change the value on a single queue.   But of course, '10' is imprecise, so you need units, and then you need to parse the strings like 'ms' that make up the units, and for various reasons we found that it was much more solid to encode time in integers within an ns3::Time class, leading to the use of time objects.  This means that you can, in fact, write:

wifiMacQueue->SetMaxDelay (MilliSeconds (10));

which is close to what you are suggesting.

I think that there is a facility that provides more friendly syntax for the default values, which is the ConfigStore subsystem:

https://www.nsnam.org/docs/release/3.35/manual/html/attributes.html#configstore

To understand it further, I hereby give another example :

An XML way of handling data may be easy for a machine to handle and parse data. But definitely, the same XML format data will not be good for human understanding.

So, from this perspective, an ns-3 simulation script is more friendly towards a machine-readable form. Definitely not friendly towards a human-readable form.

ConfigStore allows formatting attribute values in either XML or text formats.

While developing a huge system like ns-3 using Object-Oriented Programming techniques, the level of attraction and hierarchy of organizing/doing things should hide the complex things behind the scene from the end-user (the ns-3 simulation writer/student/researcher).  The interfaces that are provided to handle the objects and change the properties of the objects should be human-understandable forms.  But according to my view, from the first day of development itself, the developers ignored this point and now the ns-3 becomes a huge, complex machine only understandable form. 

Once I heard that Unix was originally developed by programmers for programmers.

Just like that, I think, ns-3 was developed by brilliant developers for brilliant developers.

I would phrase it as 'ns-3 was developed by and for power users'; specifically, people who are inclined to do networking research.  That is definitely true.  The idea was that at some point, more could be added on top of the low level, command-line API to make it simpler for more beginning students, and a GUI can be provided, etc.

The other key thing to remember is that ns-3 is fully open source and contribution driven; there is no company behind it.  Therefore, things get done by virtue of what is contributed, not what any maintainer is told to work on.  We are on the verge of adding CMake build system because a contributor (Gabriel) showed up with the patches and perseverance to get it done-- no one asked him to do this work (as far as I know).  The main contributors of ns-3 are generally people who work on networking research either in academia or industry.  Therefore, the type of code contributions reflect this, and we acknowledge that it could be easier to use if contributors had more time to focus on usability and more examples, etc.  We also seldom get example programs contributed from the community; we are usually trying to come up with (basic) examples ourselves.

But I think, changing the way ns-3 does these kinds of things can not be altered/rectified now – because, ns-3 become huge with all kinds of these complexities.  Just expressing my feelings/disabilities that’s all.

Solution(?):

And I think, using ns-3 within an IDE like Eclipse or QTCreator may hide some of this complexity. But, definitely, it also will make a user not to understand what is really happening in a particular line of code.

As Gabriel mentioned, this is around the corner once we add CMake support.

(Final Note: My scope is not offending the generous developers who are sacrificing their precious time on the development and maintenance of ns-3 – just telling my  opinion from my idiotic point of view -that’s all)

I take no offense at what you wrote.  In fact, I welcome feedback on the usability of ns-3, and it is a concern of mine-- I actually plan to be focusing more on usability once some of my current efforts wrap up.  I wish more people gave us constructive criticism or suggestions for improvements.

- Tom

Charles Pandian. 

--
Posting to this group should follow these guidelines https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting
---
You received this message because you are subscribed to the Google Groups "ns-3-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ns-3-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/8a2baab6-2005-4ce0-8b39-ccc30d84201cn%40googlegroups.com.

[Durgesh Kshirsagar]

First of all, I really appreciate the time and efforts put by all respected authors of not so easy NS3 simulator. I agree upon the fact that for beginners it is quite difficult to understand the underneath code. I myself am working in NS2 since last 7 years. Definitely NS3 has an edge to the NS2 over advanced things like 802.11AX, multiple wifi interfaces, LTE, 5G etc. As Tom has stated rightly, it is pretty tough to lear NS3 from scratch at beginner level. My cent to this discussion will be, if authors can add a prerequisites(C++ Advanced language topics) to NS3 as a beginner it can surely boost the learning process.  
Thank you!

Regards,

Durgesh Kshirsagar

Author of Udemy Course - Network Simulation using NS2

(https://www.udemy.com/course/network-simulation-using-ns2/?src=sac&kw=network+simulation+u)

[Yufei Yan]

Finally, I'm glad that not only me who thinks NS3 is almost impossible to learn. 

To understand every bit of NS3 is impossible in my mind, because it's just a so huge project and the program is written so well in C++ in terms of efficiency, such that many many modern advanced C++ programing techniques are used. Only in case that you are extremely proficient in C++, you will really have a hard time understanding the underlying architecture and how each module communicates with other. That being said, if you want to implement some new protocols, you will need to spend a really good amount of time.

However, writing some scripts to run simulation with already existing protocols won't be that intimidating. As long as the modules are already in there, take some time to learn the language and study the examples in the tutorial, 8 of 10 times, you will fine.

That's just my two cents.  

[igs...@gmail.com]

Yes. as mentioned by Yufei Yan,  the ns-3 code and its flow is very hard to understand because it is using many modern advanced C++ programming techniques and concepts in its design. I realized the same fact while trying to develop a 'Simple Mobility Model' for ns-3. In fact, ns-3 is the only software in where I am seeing the usage of lot modern features and concepts of modern C++.

 Yufei Yan, thank you for your wonderful comments and feedback.

Charles Pandian

[Tommaso Pecorella]

I take "the ns-3 code and its flow [...] is using many modern advanced C++ programming techniques and concepts in its design" as a compliment :)

Yes, this can be intimidating and a barrier for some, but look at the bright side, by reading the code you'll discover many programming tricks.

Btw, the complexity is (also) because the codebase is very large, and it must be as efficient as possible - you don't want your simulations to last forever...

On the bright side (for real) writing a script should be fairly easy. It's when you have to modify a model that things become... complicated. However, the trick in this case is: learn by copying. Don't try to understand everything at once, sometimes you have to trust the fact that it will work. After a while you'll discover why it works.

And if this can make you feel better, I think that parts of the code are pure black magic. I trust that they work, and I don't even dare to touch them.

[Charles Pandian]

Yes, this can be intimidating and a barrier for some, but look at the bright side, by reading the code you'll discover many programming tricks.

Btw, the complexity is (also) because the codebase is very large, and it must be as efficient as possible - you don't want your simulations to last forever...

Of course, the organization of ns-3 or the idea of organizing and integrating each and every module seems to be more efficient. In fact it reminds me the elegant way the  "WordPress CMS" is working - no one will ever able to collapse the WordPress CMS by installing a plugin (but if someone remembers the old version of PhpNuke CMS, just installing a new module may simply collapse the entire system). Just like that, ns-3 rocks - just adding an extension module could not spoil the existing ns-3 system.  If we try to add a new ns-3 extension module, and if it has some problems and the compile process fails, the just removing that extension folder and recompiling ns-3 will take the system back to its original condition. But in the days of ns-2, if we failed during adding a ns-2 extension such as a new routing protocol, then removing the failed module was not a easy job to do. Because, ns-2 components were not developed like a plug and play like modules.

On the bright side (for real) writing a script should be fairly easy. It's when you have to modify a model that things become... complicated. However, the trick in this case is: learn by copying. Don't try to understand everything at once, sometimes you have to trust the fact that it will work. After a while you'll discover why it works.

In fact, in this way, only I am (and hopefully most of the ns3 users) using ns-3.
But frankly saying, even though ns-3 is having such an elaborate documentation, tutorials and examples, grasping its internals and the way it is working is not easy for an average human being to grasp. 

For example, eventhough the documentation about "Mobility Models" is abstractly explaining it, an average person will never able to understand the elegant way it is working—especially the way in which every simple mobility model is 'seamlessly' working along with group mobility - anyway such kind of thigs are amazing in ns-3.

🤔🤔Is there any other secret documentation circulating among the ns-3 developers for understanding ns-3's whole design and working?😊😊😊 

And if this can make you feel better, I think that parts of the code are pure black magic. I trust that they work, and I don't even dare to touch them.

For me, even the simple techniques used to handle the trace events and callbacks are mere “black magic”.

Even though I can write simple modules such as mobility and routing and easily integrate them with ns-3, still I could not understand how they are really getting merged with the existing code base of ns-3- Amazing, 
(I mean, in ns-2 days, for incorporating a new protocol, we have to edit a lot of existing files and include few lines- and then compile—then only it will be possible to integrate it with existing code base) 
Brilliant design (like rock solid WordPress CMS) 

Charles Pandian,

--

Posting to this group should follow these guidelines https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting
---
You received this message because you are subscribed to the Google Groups "ns-3-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ns-3-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/05cbc315-1b36-4af8-9025-306d5b0112a8n%40googlegroups.com.

[Tommaso Pecorella]

Well, let's say that the documentation is the weakest part of any project. Everybody loves to code, nobody loves to write the documentation (and tests). Well, almost nobody.

That said, we're always open to suggestions on what parts of the documentation can be improved. Manual, tutorial, anything. Even better, we'd love to have more documentation written, possibly with the help of the users - because what is totally clear to the devs is often completely obscure to others.

[Engr_ahmed]

Sorry I am very late to the conversation. 
I have been working in ns-2 simulator for almost 8 years now. I know in and out of ns-2 as I worked in it as a PhD student. After doing PhD I have been teaching computer networks in university to undergrade/postgraduate students. I wanted to teach simulations in ns-3 to my students. I worked on preparing lectures for a couple of months but failed to deliver them. I observed that it is easy to understand scripts because all you have to do is to understand the working of APIs at abstract level. However, running scripts is not the sole purpose of working on a simulator. Sooner or later the student would have to add a new protocol in the ns-3 simulator and that's when he faces some very serious problems. 
We teach some very good in depth object oriented programming in c++ covering templates and STL as well at the undergrade level. Even then the researcher is bound to fail when it comes to contributing to the ns-3 core. Imagine an MSc student who has only two years to submit his thesis. In these two years he will first come up with some research idea by reading a lot of papers, after that he will run some of the already existing ns-3 scripts and in the last he will add his proposed idea into the simuator. It is more like doing research in programming rather than doing research in computer networks. 
I believe an intermediate level proficiency in programming should be sufficient for someone to specialize in computer networks. Ultimately many of my students either leave the MSc degree right in the middle of the program or switch to some kind of customized simulator by using for example matlab. 
If a student is as good in c++ programming as required by working in ns-3 then why does he even need to do research in computer networks by investing his money and time on msc/phd degree, he would be better off to work as a senior programmer in a company where his programming skills are required. 
I have also some very serious concerns about the documentation of the simulator and not very easy to understand even for someone who has doctorate degree. There should be exclusively one chapter to guide the reader on what c++ skills are required and should recommend a list of c++ books. Again I am saying working in ns-3 looks more like a journey towards mastering c++ programming skills rather than learning computer networks.
Unfortunately,  It looks like I may not be spending more time learning ns-3.
Good Luck
Dr. Bilal

[faisal Lone]

Hello, to jump into the topic, which is the better simulator out of NS3 and Omnet++ considering all aspects?

To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/b5e310b2-55a8-407c-9a9a-2af0cd214156n%40googlegroups.com.

[Tom Henderson]

Thank you for taking the time to provide feedback like this. I read
your message a couple of times, from the perspective of trying to
understand specifically what we can prioritize to improve. Would a
tutorial dedicated to "how to add a new protocol" be helpful? What
aspects did students stumble on in doing this-- was it trying to work in
the context of existing modules like wifi or lte, or was it in figuring
out how to write protocol headers and state machines for something
entirely new? Or was it that they invariably have to try to extend the
core and it becomes too much for them?

When you compare your experience with ns-2, do you find that ns-2 was
easier because of a higher degree of abstraction in the protocol models,
or that it used a simpler version of C++, or because students worked at
the OTcl level? I think that ns-3's documentation is more
comprehensive and thorough than ns-2's, so I don't think it is that aspect.

- Tom

> efficient. */In fact it reminds me the elegant way the

> "WordPress CMS" is working - no one will ever able to collapse
> the WordPress CMS by installing a plugin (but if someone
> remembers the old version of PhpNuke CMS, just installing a

> new module may simply collapse the entire system). /*Just like

> that, ns-3 rocks - just adding an extension module could not
> spoil the existing ns-3 system.  If we try to add a new ns-3
> extension module, and if it has some problems and the compile
> process fails, the just removing that extension folder and
> recompiling ns-3 will take the system back to its original
> condition. But in the days of ns-2, if we failed during adding a
> ns-2 extension such as a new routing protocol, then removing the
> failed module was not a easy job to do. Because, ns-2 components
> were not developed like a plug and play like modules.
>
> On the bright side (for real) writing a script should be
> fairly easy. It's when you have to modify a model that
> things become... complicated. However, the trick in this
> case is: learn by copying. Don't try to understand
> everything at once, sometimes you have to trust the fact

> that it will work. After a while you'll discover /why/ it works.

>
>
> In fact, in this way, only I am (and hopefully most of the ns3
> users) using ns-3.
> But frankly saying, even though ns-3 is having such an
> elaborate documentation, tutorials and examples, grasping its
> internals and the way it is working is not easy for an average

> human being to grasp.**
> *For example*, eventhough the *documentation
> <https://www.nsnam.org/docs/models/html/mobility.html>* about

> that it will work. After a while you'll discover /why/ it works.

> I am trying to make a post/article on "*What

> Makes ns-3 a Complex Thing to Understand and
> Use?

> <https://www.projectguideline.com/what-makes-ns-3-a-complex-thing-to-understand-and-use/>"*

> The following is the thing that I discussed in
> that article/post.

> * (Sorry. I could not format this text here - to

> see formatted text use the above link)

> *
> *Please share your insights to improve and

> incorporate things in that article. I think it
> may be useful for newcomers to understand ns-3.
>
> Please share your experience so that I may

> fulfil the scope of that post/article*
>
> Charles Pandian
> -------------------------------------------
> *What Makes ns-3 a Complex Thing to Understand
> and Use?
> <https://www.projectguideline.com/what-makes-ns-3-a-complex-thing-to-understand-and-use/>*
> Article Written by: CharlesPandian
> <https://www.projectguideline.com/author/charlespandian/> November 23, 2021 | Edit This <https://www.projectguideline.com/wp-admin/post.php?post=4666&action=edit>

>
> While starting to learn ns-3, most of us find it
> very difficult to understand. Particularly, if
> the student already knows to use ns-2 or Omnet++
> then it will even become much harder for him to
> understand/grasp what ns-3 really is.  The
> student’s or scholar’s preconceived expectation
> in a simulator will make him misunderstand ns-3.
> Another major obstacle/misconception in using
> ns-3 is the way in which python is frequently
> used along with C++. (Thank Mr Tom Henderson for
> recent news regarding the removal of python
> bindings

> <https://mail.google.com/mail/u/0/?zx=tn7fvveoczrb#inbox/WhctKKXHDXfdBzqmrFKtdKmjwXtzRHRxvlnkNDNKPFGfntHvlxXKBZHqpzVCRjKTJTKCsTB> -because it will reduce some of these obstacles )

>
> Of course, the ns-3 Documentations and Manual
> pages are providing explanations for each and
> every aspect of this wonderful simulator. But
> they are huge and their hugeness itself makes it
> impossible to grasp the very fine, basic aspects
> of this simulator.
> Often, the developer of ns-3 also generously
> provide answers to the questions or
> students/learners of ns-3 through the official
> ns-3 user group

> <https://groups.google.com/g/ns-3-users>.  Even

> with this kind of help and support, it still, it
> makes impossible for someone to learn and
> understand ns-3.
>
> This article tries to explain why it is
> ambiguously hard to understand ns-3 and an ns-3
> simulation (I mean, in most cases, even the
> student can not express the reason for his poor
> understanding/misunderstanding on ns-3).
>
> How ns-3 is different from most of the other
> simulators?
>
> In most of the popular simulators (except ns-3),
> the simulator will be a different entity and the
> simulation script that the user will write for
> that simulator will be a separate entity. While
> running the simulation script, the simulator
> will just run/interpret the lines that the user
> created.
>
> For example,
>

> * Under ns-2, the simulation will be presented

> in the form of TCL script and the core
> simulator which is made up of C++ and TCL
> objects will simply interpret the lines of
> the TCL simulation script.

> * Under Omnet ++, the simulation script will

> be a custom format text file (not like any
> programming language) core simulator which
> is made up of C++ objects will simply
> interpret the lines of the simulation script.

> * Even most of the non-free simulators are

> <https://www.projectguideline.com/event-trace-file-generation-in-a-ns-3-simulation/>” and “Trace Analysis of ns-3 Ascii Traces using TraceMetrics Tool – An elementary Trace Analysis Solution for ns- <https://www.projectguideline.com/trace-analysis-of-ns-3-ascii-traces-using-tracemetrics-tool-an-elementary-trace-analysis-solution-for-ns-3/>*3 <https://www.projectguideline.com/trace-analysis-of-ns-3-ascii-traces-using-tracemetrics-tool-an-elementary-trace-analysis-solution-for-ns-3/>“* are dealing with this kind of tracking and analysis.)

> *Trace Analysis of TCP Flows Under ns-3 MANET/
> FANET/ VANET/ WSN Scenario”
> <https://www.projectguideline.com/trace-analysis-of-tcp-flows-under-ns-3-manet-fanet-vanet-wsn-scenario/>* uses this approach)

> Once I heard that *Unix was originally developed
> by programmers for programmers.*
>
> Just like that, I think, *ns-3 was developed by
> brilliant developers for brilliant developers.*

> https://groups.google.com/d/msgid/ns-3-users/05cbc315-1b36-4af8-9025-306d5b0112a8n%40googlegroups.com <https://groups.google.com/d/msgid/ns-3-users/05cbc315-1b36-4af8-9025-306d5b0112a8n%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> Posting to this group should follow these guidelines
> https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting
> <https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting>
> ---
> You received this message because you are subscribed to the Google
> Groups "ns-3-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to ns-3-users+...@googlegroups.com

> <mailto:ns-3-users+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/ns-3-users/b5e310b2-55a8-407c-9a9a-2af0cd214156n%40googlegroups.com <https://groups.google.com/d/msgid/ns-3-users/b5e310b2-55a8-407c-9a9a-2af0cd214156n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[faisal Lone]

Respected Sir, 

As I am also a beginner in NS3, I also faced a lot of problems dealing with it. A simple basic gui to start with basic scenarios and the capability to edit the auto generated code for these simple scenarios would help every beginner immensely.

To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/c3d099b5-81ea-0813-05a4-8a81738c7449%40tomh.org.

[Gabriel Ferreira]

Thinks that I think we could change in ns-3 to make things more accessible.

1. Reference scenarios.

Instead of letting each user try to reinvent the wheel with new scenarios, we could have benchmark scenarios

for certain aspects of wifi/lte/mesh/etc (handover, throughput, indoor, outdoor, interference, etc).

If you want to improve some aspect of the standards, you change the algorithm that handles that aspect 

and run the respective simulation campaign.

Now you have results that can be directly compared to other implementations in the same scenario.

2. Easier and more compact way to setup the network. 

The mobility part is fairly easy, but could be easier. Just read a YAML or JSON file with the coordinates, mobility type, simulation range and done.

Same for the network interfaces. I haven't touched in wifi in a while, and while I was preparing an example for WNS-3 I realized that the setup has become quite complicated. 

I believe anyone can specify and understand what a YAML like the following means:

- nodes:

  - node0:

    - name: wifi access point

  - node1:

    - name: gabriel's phone

  - node2:

    - name: gabriel's laptop
- mobility:

  - node0:

    - mobilityModel: ConstantPosition

    - position:

      - x: 10

      - y: 10

  - node1:

    - mobilityModel: RandomWalk

    - mobilityModelAttributes:

      - x: [Min=0|Max=30]

      - y: [Min=0|Max=30]

    - position:

      - x: 15

      - y: 15

  - node2:

    - mobilityModel: ConstantPosition

    - position:

      - x: 20

      - y: 20

- networks:

  - network0:

    - type: wifi

    - ssid: exampleNetworkSsid

    - wifiStandard: 802.11AX

    - band: 6GHz

    - bandwidth: 160MHz

    - channel: 1

    - networkAddress: 192.168.0.0

    - subnetMask: 255.255.255.0

    - ap:

      - node: node0

    - stas:

      - node: node1

      - node: node2

It shouldn't be that hard to write a small GUI to setup these and to parse them in the simulator. 

Extracting all attributes from the code and checking if all settings are sane before actually trying to run the simulations is the hard part.

[Charles Pandian]

Dear  Gabriel Ferreira,

           "2. Easier and more compact way to setup the network. "

Is there any way to set up the network using a YAML or JSON file? (I could not find it in the documentation).
Please explain the second point in the previous answer.

Charles Pandian,

To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/99d745ae-1de4-4f31-8558-a394733de31an%40googlegroups.com.

[Gabriel Ferreira]

Not yet. Working on that. 

[Giovanni Grieco]

Hi there, 

I have tried a while ago to implement high-level JSON configurations on a ns-3 module called IoD-Sim. Parsing is made straightforward with rapidjson.

Some useful links in case you are interested in how it is implemented:

1. Some JSON examples: https://github.com/telematics-lab/IoD_Sim/tree/main/scenario

2. Parser implementation: https://github.com/telematics-lab/IoD_Sim/tree/main/src/configuration

3. More high-level details to understand how it works: https://doi.org/10.1109/JIOT.2022.3207324

If you have any feedback to share, this may be a starting point for a (more mature) reference implementation in ns-3.

Cheers,

Giovanni