Request for information on: NS3 configuration for Python bindings, upgrading packages

[Haraesh Jayasethu Ramachandran]

Hello,

I am new to NS3, and I am trying to simulate some networks using the tool. I have two questions I would like to ask at your availability:

Question 1

I have a C++ package I am trying to generate Python bindings for using NS3. The package is as follows: https://github.com/signetlabdei/lorawan

I used the following command to clean, configure and build the package (I'm running it from an Ubuntu laptop):

./ns3 clean && ./ns3 configure --enable-examples --enable-tests --enable-python-bindings && ./ns3 build

The build succeeds and the tests do pass, but I'm not sure how to use the Python bindings in programs (as I am trying to use the C++ package to execute Python simulations). I tried using cppyy.cppdef, for example, and I directly tried to call it in a Python program such as:

lora_helper = ns.lorawan.LoraHelper

I typically used the NS3 documentation for 3.41, alongwith the documentation for Python bindings and for creating a new module to try to understand what I might have to do:

https://www.nsnam.org/docs/manual/html/python.html

https://www.nsnam.org/docs/manual/html/new-modules.html

I don't know what I'm missing, so I would appreciate any guidance on this.

Question 2

I would also like to know if you had any general recommendations/tips on upgrading an existing package? For example, I am trying to use packages created with older versions of NS3 (which used PyBindGen for the bindings and which used waf and wscript files for configuration, for instance), and I would like to use them in newer versions (where cppyy is used alongwith CMake for configuration). I'd like to know what to look out for in terms of facing build errors, any specific changes I would have to make etc. So far, I have a lot of build errors, and I'm not sure on how to proceed resolving them.

Thank you in advance for any help you can provide.

[Gabriel Ferreira]

Q1) 

To use the bindings, you create a python script, then import the bindings with

from ns import ns

And that is it. Now you can create

lora_helper = ns.LoraHelper()
ns.Simulator.Stop(ns.Seconds(10))
ns.Simulator.Start()

ns.Simulator.Destroy()

If you want to use a debugger to debug your python script, which is a little bit easier, you need to add this preamble before importing the ns-3 bindings

import sys 

sys.path.append("./ns-3-dev/build/bindings/python")

sys.path.append("./ns-3-dev/build/lib")
from ns import ns

Much more examples here:  https://github.com/Gabrielcarvfer/ns3_for_education/blob/master/000_introduction_to_ns3.ipynb

Q2) The pybindgen part usually isn't an issue, since cppyy takes care of almost everything. Porting the wscript to cmake is what can be tricky, depending on external dependencies on libraries, etc. https://www.nsnam.org/docs/manual/html/working-with-cmake.html#migrating-a-waf-module-to-cmake

But we have a section for that in the manual https://www.nsnam.org/docs/manual/html/working-with-cmake.html#linking-third-party-libraries.

[Haraesh Jayasethu Ramachandran]

Hello,

Thank you for your response.

Before I explain - I'm trying to execute this on NS3 3.41.

Here's what I tried to do. I executed the following commands:

./ns3 clean

./ns3 configure --enable-examples --enable-tests --enable-python-bindings

./ns3 build
./test.py (When I execute this, it actually fails a couple of tests, but those tests aren't within the scope of what I want to execute in NS3 for now, so I usually ignore it)
./ns3 shell

And then, what I tried to do was write the following Python simulation program (this is saved in examples/lorawan/simple-network-simulator.py):

try:
    from ns import ns
except ModuleNotFoundError:
    raise SystemExit(
        "Error: ns3 Python module not found;"
        " Python bindings may not be enabled"
        " or your PYTHONPATH might not be properly configured"
    )

lora_helper = ns.LoraHelper()
ns.Simulator.Stop(ns.Seconds(10))
ns.Simulator.Start()
ns.Simulator.Destroy()

When I try to execute this in the command line using:

$ ./ns3 run examples/lorawan/simple-network-simulator.py

It gives me the following error:

[0/2] Re-checking globbed directories...
ninja: no work to do.
[runStaticInitializersOnce]: Failed to materialize symbols: { (main, { _ZN3ns3L16g_TimeStaticInitE, $.cling-module-140.__inits.0, __orc_init_func.cling-module-140, __cxx_global_var_initcling_module_140_, _GLOBAL__sub_I_cling_module_140 }) }
[runStaticInitializersOnce]: Failed to materialize symbols: { (main, { __orc_init_func.cling-module-140 }) }
[runStaticInitializersOnce]: Failed to materialize symbols: { (main, { $.cling-module-170.__inits.0, __cxx_global_var_initcling_module_170_, _ZN3ns3L27UE_MEASUREMENT_REPORT_DELAYE, __orc_init_func.cling-module-170, _ZGVN3ns3L27UE_MEASUREMENT_REPORT_DELAYE, _ZN3ns34TimeD1Ev, _ZGVZN3ns34Time14PeekResolutionEvE10resolution, _ZZN3ns34Time14PeekResolutionEvE10resolution }) }
[runStaticInitializersOnce]: Failed to materialize symbols: { (main, { __orc_init_func.cling-module-170 }) }
Traceback (most recent call last):
  File "/home/user/Downloads/ns-3-dev-for-lorawan/ns-3-dev/examples/lorawan/simple-network-simulator.py", line 10, in <module>
    lora_helper = ns.LoraHelper()
AttributeError: <namespace cppyy.gbl.ns3 at 0x5c1893c17560> has no attribute 'LoraHelper'. Full details:
  type object 'ns3' has no attribute 'LoraHelper'
  'ns3::LoraHelper' is not a known C++ class
  'LoraHelper' is not a known C++ template
  'LoraHelper' is not a known C++ enum
Command 'python3 examples/lorawan/simple-network-simulator.py' returned non-zero exit status 1.

I'm not sure if I've configured something wrong, or if I have to configure anything extra to make this work.

Do let me know of the same. I'll try to port the wscript files using the guides you've given. Thanks again!

[Gabriel Ferreira]

ns-3.41 is compatible with cppyy==2.4.2 
ns-3.42 is compatible with cppyy==3.1.2

[Haraesh Jayasethu Ramachandran]

Hello,

Thank you for letting me know.

I tried setting up a conda environment with the following versions (not all inclusive) based on your response:

cppyy=2.4.2

python=3.11.9

ns3=3.41

My ultimate goal is to use https://github.com/signetlabdei/lorawan/tree/v0.3.0 (LoRaWAN) and https://github.com/sns3/sns3-satellite (SNS3) together to perform LoRa and SNS3 simulations in the same NS3 project. The module LoRaWAN builds as expected (without the Python bindings), so I thought I would add the SNS3 files and then do another integrated build with C++ with the Python bindings.

When I try to run ./ns3 configure --enable-tests --enable-examples --enable-python-bindings (And I usually follow these properties to test the examples and tests also), I get the following message when the config code runs:

-- NS3_BINDINGS_INSTALL_DIR was not set. The python bindings won't be installed with ./ns3 install. This setting is meant for packaging and redistribution.
-- Set NS3_BINDINGS_INSTALL_DIR="/home/user/.local/lib/python3.11/site-packages" to install it to the default location.

Furthermore, I get this following warning. It's generated multiple times after the section where it specifies which modules are ready to be built (and the lorawan and sns3 modules are inside that list):

CMake Warning at build-support/custom-modules/ns3-module-macros.cmake:155 (add_library):
  Cannot generate a safe runtime search path for target libmagister-stats
  because files in some directories may conflict with libraries in implicit
  directories:

    runtime library [libsqlite3.so.0] in /home/user/anaconda3/envs/ns3-lora-sns3/lib may be hidden by files in:
      /usr/lib/x86_64-linux-gnu

  Some of these libraries may not be found correctly.
Call Stack (most recent call first):
  contrib/magister-stats/CMakeLists.txt:50 (build_lib)

I don't want to make any untoward changes on any of the existing configurations, and I don't know if I need to do any specific changes for executing the ns3 project in this context, inside conda.

Do let me know if you have any guidance on resolving the issue I specified above, or if you would recommend I fully establish the Python bindings for LoRaWAN first before integrating it with SNS3 (because my understanding is that SNS3 seems to be configured as expected) at your convenience.

Much appreciated!

[Gabriel Ferreira]

For the NS3_BINDINGS_INSTALL_DIR, this setting is meant for packaging and redistribution, like the message says. You're not doing that, so ignore.

The second warning about safe runtime search path is related to conda shipping their own libsqlite. You may end up loading a header from the system and the library from conda, or vice-versa. The way to solve this would be to ignore the system library directories, but we haven't merged support for that just yet (untested and can break stuff). But you could try applying this commit https://gitlab.com/Gabrielcarvfer/ns-3-dev/-/commit/8a8e6b00e3ccdc24cda04bd9153115c16e56ce09

[Haraesh Jayasethu Ramachandran]

Hello,

I tried applying the changes you suggested in https://gitlab.com/Gabrielcarvfer/ns-3-dev/-/. It doesn't appear to have any effect, as I still get the numerous CMake warnings. Since I've configured them in conda, I imagine that could potentially be the issue, since I never had this problem when I tried to execute NS3 with the LoRaWAN package without a conda environment.

I'm not sure how to resolve them in theory, because if there's a conflict between headers and libraries, I wouldn't know if there is one until the build, when the warnings/errors get displayed. Do you have any other pointers I can follow to get the build running? I'm not concerned with the tests passing since my priority right now is to get the build working.

Do let me know at your convenience. Much appreciated.

[Gabriel Ferreira]

I don't really test with conda, and I know conda typically ships their own compiler, library, etc.
Not too sure what you can do to isolate yourself other than the patch I provided.
My advice would be not to use conda.

[Haraesh Jayasethu Ramachandran]

Hello,

I tried making the changes you suggested outside of conda in NS3 3.41 with the LoRaWAN package and the SNS3 package. I don't get any CMake errors, but I get a lot of build errors, regardless of if I try to integrate the SNS3 package into LoRaWAN or vice versa. I can't integrate the LoRa package into the latest SNS3 project with the code changes you suggested before because I imagine your changes are for a later version.

For now, I'll have to be working with SNS3, so I'll try my hand at LoRaWAN-SNS3 integration later, depending on if I need to. Considering that it seems to always expect the headers to come from different places (as in one package seems to get the headers from a different place than the other), I imagine I need some more input on resolving it, but not for now.

Thank you in any event for your help! Much appreciated.