Generating python bindings for your own (new) module

[nat...@gmail.com]

Good day

How can I automatically generate python bindings for a new ns3 module that I created, so that I can call it from a python3.7 script?

I saw the instructions: "To cover a new module, you have to create a bindings/python/ns3_module_<name>.py file, similar to the what is described in the previous section, and register it in the variable LOCAL_MODULES in bindings/python/ns3modulegen.py" 

but the bindings for existing modules are done differently and do not have a file like bindings/python/ns3_module_<name>.py so I am not sure how to generate that file and I would presume there is a way to generate at least part of it automatically?

Could someone please outline in clear instructions what needs to be done to generate the python bindings for the new module and if there is a way to generate the bindings automatically?

[Tom Henderson]

The instructions are here:

https://www.nsnam.org/docs/manual/html/python.html#working-with-python-bindings

In summary, there are a few steps (Linux machine required):

1) the new module wscript file must have this statement added:

bld.ns3_python_bindings()

2) you have to install the CastXML, pygccxml, and pybindgen tools as prerequisites.
The bake tool can be used for this (as described in the documentation).

You have to get to the point where the Waf configuration report says:

Python API Scanning Support   : enabled

3) Assuming your new module name is called 'my-module' (for example), you can now scan the APIs
using:

./waf --apiscan=my-module

If the C++ API in my-module is all supported by pybindgen, this should complete successfully.

- Tom

[nat...@gmail.com]

Hi Tom

Thanks for the response. I am able to go through the bake steps until the point of `./bake.py build` and then I always get a problem. The debug output does not clarify what the problem is. Are you able to help and let me know what you think is going wrong? Please see my terminal output below. (It might be worth noting that bake.py looks for python-dev and setuptools and not python3-dev or python3-setuptools even though I have both of them installed.)

xxxxxxxxxxx@xxxxxxxxxx:~/repos/ns-3-dev-git/bake$ ./bake.py show

module: pygraphviz (enabled)

  No dependencies!

module: python-dev (enabled)

  No dependencies!

module: llvm-dev (enabled)

  No dependencies!

module: clang-dev (enabled)

  No dependencies!

module: g++ (enabled)

  No dependencies!

module: cxxfilt (enabled)

  No dependencies!

module: setuptools (enabled)

  No dependencies!

module: libgoocanvas2 (enabled)

  No dependencies!

module: gi-cairo (enabled)

  No dependencies!

module: pygobject (enabled)

  No dependencies!

module: qt (enabled)

  No dependencies!

module: cmake (enabled)

  No dependencies!

module: castxml (enabled)

  depends on:

     clang-dev (optional:False)

     llvm-dev (optional:False)

     cmake (optional:False)

module: pybindgen (enabled)

  depends on:

     python-dev (optional:True)

     setuptools (optional:False)

module: pyviz-gtk3-prerequisites (enabled)

  depends on:

     python-dev (optional:True)

     pygraphviz (optional:True)

     pygobject (optional:True)

     gi-cairo (optional:True)

     libgoocanvas2 (optional:True)

module: netanim (enabled)

  depends on:

     qt (optional:False)

     g++ (optional:False)

module: pygccxml (enabled)

  depends on:

     castxml (optional:False)

     setuptools (optional:False)

     cxxfilt (optional:False)

module: ns-3-dev (enabled)

  depends on:

     netanim (optional:True)

     pybindgen (optional:True)

     pyviz-gtk3-prerequisites (optional:True)

-- System Dependencies --

 > clang-dev - OK

 > cmake - OK

 > cxxfilt - OK

 > g++ - OK

 > gi-cairo - Missing

   >> Python gobject components missing from system.

   >> Try: "sudo apt-get install python-gi-cairo", if you have sudo rights.

 > libgoocanvas2 - Missing

   >> GooCanvas2 support missing from system.

   >> Try: "sudo apt-get install gir1.2-goocanvas-2.0", if you have sudo rights.

 > llvm-dev - OK

 > pygobject - OK

 > pygraphviz - Missing

   >> The pygraphviz is not installed, try to install it.

   >> Try: "sudo apt-get install python-pygraphviz", if you have sudo rights.

 > python-dev - OK

 > qt - OK

 > setuptools - OK

xxxxxxxxxxx@xxxxxxxxxx:~/repos/ns-3-dev-git/bake$ ./bake.py download

 >> Searching for system dependency pygraphviz - Problem

 > Problem: Optional dependency, module "pygraphviz" not available

   This may reduce the  functionality of the final build. 

   However, bake will continue since "pygraphviz" is not an essential dependency.

   For more information call bake with -v or -vvv, for full verbose mode.

 >> Searching for system dependency python-dev - OK

 >> Searching for system dependency llvm-dev - OK

 >> Searching for system dependency clang-dev - OK

 >> Searching for system dependency qt - OK

 >> Searching for system dependency g++ - OK

 >> Searching for system dependency cxxfilt - OK

 >> Searching for system dependency setuptools - OK

 >> Searching for system dependency libgoocanvas2 - Problem

 > Problem: Optional dependency, module "libgoocanvas2" not available

   This may reduce the  functionality of the final build. 

   However, bake will continue since "libgoocanvas2" is not an essential dependency.

   For more information call bake with -v or -vvv, for full verbose mode.

 >> Searching for system dependency gi-cairo - Problem

 > Problem: Optional dependency, module "gi-cairo" not available

   This may reduce the  functionality of the final build. 

   However, bake will continue since "gi-cairo" is not an essential dependency.

   For more information call bake with -v or -vvv, for full verbose mode.

 >> Searching for system dependency pygobject - OK

 >> Searching for system dependency cmake - OK

 >> Downloading castxml - OK

 >> Downloading pybindgen - OK

 >> Downloading netanim - (Nothing to do, source directory already exists) - OK

 >> Downloading pygccxml - (Nothing to do, source directory already exists) - OK

 >> Downloading ns-3-dev - OK

xxxxxxxxxxx@xxxxxxxxxx:~/repos/ns-3-dev-git/bake$ ./bake.py --debug build

 >> Building castxml - Problem

TaskError

  File "/home/natasha/repos/ns-3-dev-git/bake/bake/Module.py", line 427, in build

    self._build.build(env, jobs)

  File "/home/natasha/repos/ns-3-dev-git/bake/bake/ModuleBuild.py", line 538, in build

    directory=env.objdir)

  File "/home/natasha/repos/ns-3-dev-git/bake/bake/ModuleEnvironment.py", line 540, in run

    raise TaskError('Subprocess failed with error %d: %s' % (retcode, str(args)))

 > Error:  Critical dependency, module "castxml" failed 

DependencyUnmet

  File "/home/natasha/repos/ns-3-dev-git/bake/bake/Bake.py", line 687, in _iterate

    deps.resolve(targets)

  File "/home/natasha/repos/ns-3-dev-git/bake/bake/Dependencies.py", line 198, in resolve

    self._resolve_serial(targets, callback)

  File "/home/natasha/repos/ns-3-dev-git/bake/bake/Dependencies.py", line 433, in _resolve_serial

    finished = self._resolve_one_iteration(targets, callback)

  File "/home/natasha/repos/ns-3-dev-git/bake/bake/Dependencies.py", line 411, in _resolve_one_iteration

    raise DependencyUnmet(i.dst(), tailError)

Please assist if you have an idea what is going wrong? I did also add the paths to my PATH env variable.

Regards

[nat...@gmail.com]

I also repeated the whole process from a clean bake clone and having removed all other instances of pygccxml and castxml from my machine using python3 explicitly, but then get the error `>>> Error: missing Python 'distro' package; please install`

[Tom Henderson]

On 5/2/21 11:21 PM, nat...@gmail.com wrote:
> Hi Tom
>
> Thanks for the response. I am able to go through the bake steps until
> the point of `./bake.py build` and then I always get a problem. The
> debug output does not clarify what the problem is. Are you able to help
> and let me know what you think is going wrong? Please see my terminal
> output below. (It might be worth noting that bake.py looks for
> python-dev and setuptools and not python3-dev or python3-setuptools even
> though I have both of them installed.)

The above (python and not python3) suggests to me that you are not using
a current version of bake.

Please try this:

$ git clone https://gitlab.com/nsnam/bake.git
$ cd bake
$ ./bake.py configure -e ns-3-dev -e pygccxml
$ ./bake.py show

-- System Dependencies --
> g++ - OK
> gi-cairo - OK
> gir-bindings - OK

> pygobject - OK
> pygraphviz - Missing
>> The pygraphviz is not installed, try to install it.

> python3-dev - OK
> python3-setuptools - OK
> qt - OK

The important ones above are g++, python3-dev, python3-setuptools. The
other tool reports can be ignored (for Python bindings).

$ export PATH=`pwd`/build/bin:$PATH
$ export PYTHONPATH=`pwd`/build/lib
$ ./bake.py download
$ ./bake.py build

Now cd into ns-3-dev and check that API scanning support is enabled

$ cd source/ns-3-dev
$ ./waf --check-config

Python API Scanning Support : enabled

Now you should be able to run the --apiscan command successfully.

$ ./waf --apiscan=all

>> I saw the instructions: "/To cover a new module, you have to

>> create a bindings/python/ns3_module_<name>.py file, similar to the
>> what is described in the previous section, and register it in the

>> variable LOCAL_MODULES in bindings/python/ns3modulegen.py/"

>>
>> but the bindings for existing modules are done differently and do

>> not have a file like /bindings/python/ns3_module_<name>.py /so I

>> am not sure how to generate that file and I would presume there is
>> a way to generate at least part of it automatically?
>>
>> Could someone please outline in clear instructions what needs to
>> be done to generate the python bindings for the new module and if
>> there is a way to generate the bindings automatically?
>
> The instructions are here:
>
> https://www.nsnam.org/docs/manual/html/python.html#working-with-python-bindings
>
> In summary, there are a few steps (Linux machine required):
>
> 1) the new module wscript file must have this statement added:
>
> bld.ns3_python_bindings()
> 2) you have to install the CastXML, pygccxml, and pybindgen tools as prerequisites.
> The bake tool can be used for this (as described in the documentation).
>
> You have to get to the point where the Waf configuration report says:
>
> Python API Scanning Support : enabled
>
> 3) Assuming your new module name is called 'my-module' (for example), you can now scan the APIs
> using:
>
> ./waf --apiscan=my-module
>
> If the C++ API in my-module is all supported by pybindgen, this should complete successfully.
>
> - Tom
>

> --
> Posting to this group should follow these guidelines
> https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting
> ---
> You received this message because you are subscribed to the Google
> Groups "ns-3-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to ns-3-users+...@googlegroups.com
> <mailto:ns-3-users+...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ns-3-users/f1b29800-9a59-4102-9655-84fb7098a451n%40googlegroups.com
> <https://groups.google.com/d/msgid/ns-3-users/f1b29800-9a59-4102-9655-84fb7098a451n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Tom Henderson]

On 5/3/21 1:05 AM, nat...@gmail.com wrote:
> I also repeated the whole process from a clean bake clone and having
> removed all other instances of pygccxml and castxml from my machine
> using python3 explicitly, but then get the error `>>> Error: missing
> Python 'distro' package; please install`

$ pip3 install --user distro

should fix this.

- Tom

[nat...@gmail.com]

Thanks for the response and help, Tom.

I have done this and at least the packages installed are now python3-dev and python3-setuptools but I still have a problem in the build step

xxxxx@xxxxx:~/repos/ns-3-dev-git/bake$ ./bake.py download

 >> Searching for system dependency cxxfilt - OK

 >> Searching for system dependency setuptools - OK

 >> Searching for system dependency python3-setuptools - OK

 >> Searching for system dependency gi-cairo - Problem

 > Problem: Optional dependency, module "gi-cairo" not available

   This may reduce the  functionality of the final build. 

   However, bake will continue since "gi-cairo" is not an essential dependency.

   For more information call bake with -v or -vvv, for full verbose mode.

 >> Searching for system dependency gir-bindings - OK

 >> Searching for system dependency pygobject - OK

 >> Searching for system dependency pygraphviz - Problem

 > Problem: Optional dependency, module "pygraphviz" not available

   This may reduce the  functionality of the final build. 

   However, bake will continue since "pygraphviz" is not an essential dependency.

   For more information call bake with -v or -vvv, for full verbose mode.

 >> Searching for system dependency python3-dev - OK

 >> Searching for system dependency llvm-dev - OK

 >> Searching for system dependency clang-dev - OK

 >> Searching for system dependency qt - OK

 >> Searching for system dependency g++ - OK

 >> Searching for system dependency cmake - OK

 >> Downloading pybindgen - OK

 >> Downloading castxml - OK

 >> Downloading netanim - OK

 >> Downloading pygccxml - OK

 >> Downloading ns-3-dev - OK

xxxxx@xxxxx:~/repos/ns-3-dev-git/bake$ ./bake.py build

 >> Building pybindgen - Problem

 > Problem: Optional dependency, module "pybindgen" failed

   This may reduce the  functionality of the final build. 

   However, bake will continue since "pybindgen" is not an essential dependency.

   For more information call bake with -v or -vvv, for full verbose mode.

 >> Building castxml - Problem

 > Error:  Critical dependency, module "castxml" failed 

   For more information call Bake with --debug and/or -v, -vvv, for full verbose mode (bake --help)

[nat...@gmail.com]

Hi Tom 

Sorry to keep asking but I am still struggling with this. Is there something else I should try?

Regards

[Tom Henderson]

On 5/5/21 1:40 AM, nat...@gmail.com wrote:
> Hi Tom
>
> Sorry to keep asking but I am still struggling with this. Is there
> something else I should try?

> >> Building pybindgen - Problem
> > Problem: Optional dependency, module "pybindgen" failed
> This may reduce the functionality of the final build.
> However, bake will continue since "pybindgen" is not an
> essential dependency.
> For more information call bake with -v or -vvv, for full verbose
> mode.
>
> >> Building castxml - Problem
> > Error: Critical dependency, module "castxml" failed
> For more information call Bake with --debug and/or -v, -vvv, for
> full verbose mode (bake --help)

e --help)

Try building with verbose flags as it suggests and see what it reports.

$ ./bake.py build -vvv

[nat...@gmail.com]

Thanks, I did not get much useful debug output when running ./bake build previously but now it is more verbose, but I cannot make out what to do about it. Please see below:

 >> Building castxml -     > mkdir -p /home/natasha/repos/ns-3-dev-git/bake/build/bin

 mkdir /home/natasha/repos/ns-3-dev-git/bake/source/castxml/objdir dir=/home/natasha/repos/ns-3-dev-git/bake/source/castxml

mkdir: cannot create directory ‘/home/natasha/repos/ns-3-dev-git/bake/source/castxml/objdir’: File exists

 cmake /home/natasha/repos/ns-3-dev-git/bake/source/castxml -DCMAKE_INSTALL_PREFIX:PATH=/home/natasha/repos/ns-3-dev-git/bake/build dir=/home/natasha/repos/ns-3-dev-git/bake/source/castxml/objdir

-- The C compiler identification is GNU 5.4.0

-- The CXX compiler identification is GNU 5.4.0

-- Check for working C compiler: /usr/bin/cc

-- Check for working C compiler: /usr/bin/cc -- works

-- Detecting C compiler ABI info

-- Detecting C compiler ABI info - done

-- Detecting C compile features

-- Detecting C compile features - done

-- Check for working CXX compiler: /usr/bin/c++

-- Check for working CXX compiler: /usr/bin/c++ -- works

-- Detecting CXX compiler ABI info

-- Detecting CXX compiler ABI info - done

-- Detecting CXX compile features

-- Detecting CXX compile features - done

CMake Error at /usr/share/llvm-3.8/cmake/LLVMConfig.cmake:178 (include):

  include could not find load file:

    /usr/share/llvm/cmake/LLVMExports.cmake

Call Stack (most recent call first):

  CMakeLists.txt:83 (find_package)

CMake Error at /usr/share/llvm-3.8/cmake/LLVMConfig.cmake:181 (include):

  include could not find load file:

    /usr/share/llvm/cmake/LLVM-Config.cmake

Call Stack (most recent call first):

  CMakeLists.txt:83 (find_package)

CMake Error at CMakeLists.txt:107 (message):

  Clang_DIR or LLVM_DIR refers to a LLVM/Clang installation:

    Clang_DIR=Clang_DIR-NOTFOUND

    LLVM_DIR=/usr/share/llvm-3.8/cmake

  that does not have `clang/` headers in any include directory:

    /usr/include

-- Configuring incomplete, errors occurred!

See also "/home/natasha/repos/ns-3-dev-git/bake/source/castxml/objdir/CMakeFiles/CMakeOutput.log".

 >> Building castxml - Problem

   > Subprocess failed with error 1: ['cmake', '/home/natasha/repos/ns-3-dev-git/bake/source/castxml', '-DCMAKE_INSTALL_PREFIX:PATH=/home/natasha/repos/ns-3-dev-git/bake/build']

 > Error:  Critical dependency, module "castxml" failed 

   For more information call Bake with --debug and/or -v, -vvv, for full verbose mode (bake --help)

I have libclang-dev installed (Ubuntu) and the clang checks were OK in previous steps so I don't know why this would be a problem now. Do you perhaps have some insight?

Thanks.

Regards

[Tom Henderson]

On 5/5/21 12:16 PM, nat...@gmail.com wrote:
> Thanks, I did not get much useful debug output when running ./bake
> build previously but now it is more verbose, but I cannot make out
> what to do about it.

I am not sure what system you are building on, but you need the clang
development packages installed,   For Ubuntu, it would be 'libclang-dev'
and 'llvm-dev' packages.    For Fedora, it is 'clang-devel' and
'llvm-devel'.

- Tom

[nat...@gmail.com]

Thanks, again.

As I said, I have an Ubuntu system (16.04) that has libclang-dev installed. I also have llvm-dev installed. The error seems to indicate that the clang directory cannot be found but it is installed and the following directories exist on my system:

/usr/include/clang

/usr/lib/clang

/usr/lib/llvm-3.8/include/clang

/usr/lib/llvm-3.8/lib/clang

I cannot work out from the debug output where this Clang_DIR variable that cannot be found needs to be set. The error:

CMake Error at CMakeLists.txt:107 (message):

  Clang_DIR or LLVM_DIR refers to a LLVM/Clang installation:

    Clang_DIR=Clang_DIR-NOTFOUND

    LLVM_DIR=/usr/share/llvm-3.8/cmake

  that does not have `clang/` headers in any include directory:

    /usr/include

might be because the headers are in fact in /usr/include/clang/3.8/include and /usr/include/clang/3.8.0/include? I don't know where to tell cmake that this is where the headers are that seem to be missing. Do you know what needs to be changed in the makefiles so the necessary directories are correctly specified and can be found? Or why the default installation is not putting files in the right places?

This is the full error output:

Curious that error that the error line mkdir: cannot create directory ‘/home/natasha/repos/ns-3-dev-git/bake/source/castxml/objdir’: File exists persist even when I delete the directory /home/natasha/repos/ns-3-dev-git/bake/source/castxml/objdir, but I don't think that is the main issue.

Any further advice?

[Tom Henderson]

On 5/5/21 10:25 PM, nat...@gmail.com wrote:

Thanks, again.

As I said, I have an Ubuntu system (16.04) that has libclang-dev installed. I also have llvm-dev installed. The error seems to indicate that the clang directory cannot be found but it is installed and the following directories exist on my system:

/usr/include/clang

/usr/lib/clang

/usr/lib/llvm-3.8/include/clang

/usr/lib/llvm-3.8/lib/clang

OK, there is a particular issue with Ubuntu packages of clang and llvm, version 3.8; they are broken.  This issue is flagged on our wiki in the Ubuntu Installation section:

https://www.nsnam.org/wiki/Installation#Ubuntu.2FDebian.2FMint

Please try to remove these two:

libclang-dev llvm-dev

and instead install these two:

libclang-6.0-dev llvm-6.0-dev

and try again.

Tom

p.s. I perform API scan on an Ubuntu 16.04 system frequently so I know it works, once the above issue is dealt with.

However, our support for Ubuntu 16.04 tools may end soon (as Ubuntu 16.04 has reached end-of-life).

[nat...@gmail.com]

Ok, thanks. That got me further. The build worked.

But I think there is a bigger issue here. What is the directory structure between bake and your existing ns3 directory supposed to be? I put bake inside my cloned ns-3-dev-git directory, which has my new modules for which I want python bindings. But bake creates another ns-3-dev directory in bake/source. And I assume that is the one for which we are doing the bindings in all the subsequent steps. Now when I run ./waf configure in bake/source/ns-3-dev it is missing pygccxml again, even though that was successfully built by waf. So Python Bindings are enabled but api scanning support is not because "(Missing 'pygccxml' Python module)". Running ./waf configure from my top directory ns-3-dev-git nothing is enabled.

Thanks

[nat...@gmail.com]

I also tried cloning bake into my "repos" directory along ns-3-dev-git and then running ./bake.py configure -e ns-3-dev-git -e pygccxml but that does not work either

[nat...@gmail.com]

So the question is really how to make bake use an existing ns3 directory instead of creating a new one

[Tom Henderson]

On 5/6/21 9:02 AM, nat...@gmail.com wrote:
> So the question is really how to make bake use an existing ns3
> directory instead of creating a new one

When I do not want a fresh ns-3 download with bake, but instead want to
use it with an existing repo, I do one of two things.

1) I configure bake and download the packages I need, but then before
building, I remove or move out of the way the 'ns-3-dev' that it
downloaded, and either move or symlink the other ns-3 directory that I
care about in its place under source/ directory, before I next call
'./bake.py build'.

2) I just use bake to install the dependencies I want, and then I set my
environment variables (PATH, LD_LIBRARY_PATH, PYTHONPATH) to include
also the bake build directories, and then I go to the other directory
location (somewhere else on my file system) where my other ns-3 is, and
Waf configure from there.  Waf will pick up the location of castxml,
pygccxml, and pybindgen from the environment variables.

A third option would be to install these supporting packages to
well-known system locations that are already in the path, but I don't
tend to like to do that.

I don't try to run bake from within ns-3's directory, though.

Think of bake as an ns-specific package manager that builds supporting
packages from source and installs them to the bake build directory.  I
think of bake as creating my supporting environment (such as for Python
bindings, or DCE) and then because I am editing the ns-3 code, once this
first stage of (bake) building is done, I typically stop using bake and
move into ns-3 to use Waf for the rest of my work.

- Tom

[nat...@gmail.com]

Hi Tom

Thanks for the pointers. I tried the symlink route and ./bake build works fine and builds everything but now ./waf configure still does not enable python bindings because it says Pybindgen is missing, even though it is installed, and I added --with-pybindgen=~/repos/bake/source/pybindgen to the command. This is whether I run it from the symlinked ns-3-dev or the original ns-3-dev-git repo. I really don't know where to go from here. Please could you assist with how to get ./waf configure to find PyBindGen.

[Tom Henderson]

In that directory you are referencing, do you have a file named
'version.py' found in the 'pybindgen' subdirectory? This file is what
Waf is looking for (to check if the pybindgen installation is fresh
enough). It must be generated after download; bake.py build should do
this for you.

It is not sufficient to simply download pybindgen; one must either run

python3 setup.py clean

or

./waf configure && ./waf

to generate it.

- Tom

> --
> Posting to this group should follow these guidelines
> https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting
> <https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting>
> ---
> You received this message because you are subscribed to the Google
> Groups "ns-3-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to ns-3-users+...@googlegroups.com
> <mailto:ns-3-users+...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/ns-3-users/acd0e92a-9f3f-4b75-beaa-550eeb1379b2n%40googlegroups.com
> <https://groups.google.com/d/msgid/ns-3-users/acd0e92a-9f3f-4b75-beaa-550eeb1379b2n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[nat...@gmail.com]

Thanks. No, there is no version.py inside the pybindgen directory. I downloaded pybindgen with bake and ran ./bake.py build so I would think it should have been created if bake is supposed to do it.

Anyway, I then ran ./waf configure && ./waf inside the pybindgen directory and this was the outcome (there is still no version.py):

xxxxxx@xxxxxxxxx:~/repos/bake/source/pybindgen$ ./waf configure && ./waf

Setting top to                           : /home/natasha/repos/bake/source/pybindgen 

Setting out to                           : /home/natasha/repos/bake/source/pybindgen/build 

Checking for program 'python'            : /usr/bin/python 

Checking for python version >= 2.3       : 2.7.12 

Couldn't find index page for 'setuptools_scm' (maybe misspelled?)

No local packages or download links found for setuptools_scm

Traceback (most recent call last):

  File "setup.py", line 27, in <module>

    'Programming Language :: Python :: 3',

  File "/usr/lib/python2.7/distutils/core.py", line 111, in setup

    _setup_distribution = dist = klass(attrs)

  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 269, in __init__

    self.fetch_build_eggs(attrs['setup_requires'])

  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 313, in fetch_build_eggs

    replace_conflicting=True,

  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 826, in resolve

    dist = best[req.key] = env.best_match(req, ws, installer)

  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1092, in best_match

    return self.obtain(req, installer)

  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1104, in obtain

    return installer(requirement)

  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 380, in fetch_build_egg

    return cmd.easy_install(req)

  File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 657, in easy_install

    raise DistutilsError(msg)

distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('setuptools_scm')

Checking for 'gcc' (C compiler)          : /usr/bin/gcc 

Checking for 'g++' (C++ compiler)        : /usr/bin/g++ 

python-config                            : /usr/bin/python-config 

Asking python-config for pyembed '--cflags --libs --ldflags --embed' flags : not found 

Asking python-config for pyembed '--cflags --libs --ldflags' flags         : yes 

Testing pyembed configuration                                              : yes 

Asking python-config for pyext '--cflags --libs --ldflags' flags           : yes 

Testing pyext configuration                                                : yes 

Checking for program 'castxml'                                             : /home/natasha/.local/bin/castxml 

Checking for python module 'pygccxml'                                      : not found 

Checking for compilation flag -fvisibility=hidden support                  : ok 

Checking for header stdint.h                                               : yes 

Checking for header boost/shared_ptr.hpp                                   : not found 

Boost is not available as a python extension, trying native compilation.

Checking for header boost/shared_ptr.hpp                                   : not found 

Checking for compilation flag -std=c++11 support                           : ok 

'configure' finished successfully (1.010s)

Waf: Entering directory `/home/natasha/repos/bake/source/pybindgen/build'

Waf: Leaving directory `/home/natasha/repos/bake/source/pybindgen/build'

'build' finished successfully (0.029s)

When running `python3 setup.py clean` there is no error but also still no version.py. Then, when trying to run ./waf configure in ns-3-dev it still does not pick up pybindgen.

So I am still stuck I am afraid.

[Tom Henderson]

I just tried these steps; can you try to repeat?

$ git clone https://github.com/gjcarneiro/pybindgen.git

$ cd pybindgen

$ find . -name 'version.py'

$ python3 setup.py clean

running clean

$ find . -name 'version.py'

./pybindgen/version.py
./.eggs/setuptools_scm-5.0.2-py3.5.egg/setuptools_scm/version.py

$ cat pybindgen/version.py
# coding: utf-8
# file generated by setuptools_scm
# don't change, don't track in version control
version = '0.22.0'
version_tuple = (0, 22, 0)

If this does not work for you, can you please tell us more about your
system?

- Tom

[Tom Henderson]

>
> If this does not work for you, can you please tell us more about your
> system?

Or, if it does not work, can you investigate whether your setuptools
needs to be upgraded or if there is a problem specific to setuptools_scm
on your system? Searching for the error string

distutils.errors.DistutilsError: Could not find suitable distribution
for Requirement.parse('setuptools_scm'

yields some hits; perhaps it is something to do with this.

- Tom

[nat...@gmail.com]

Hello again, Tom

Thanks so much for all your help! I have eventually been able to get Python API Scanning Support and Python Bindings enabled, so that's a big step forward.

However, when I try to scan a module, e.g. ./waf --apiscan=wifi, I get this output:

Modules to scan:  ['wifi']

api-scan-gcc_LP64

[ 969/1958] Compiling src/core/model/des-metrics.cc

[ 970/1958] Compiling src/core/model/ascii-file.cc

[ 971/1958] Compiling src/core/model/unix-system-mutex.cc

<ns3modulegen_core_customizations.SmartPointerTransformation object at 0x7f01bc8769b0>

Traceback (most recent call last):

  File "/home/natasha/repos/ns-3-dev-git/bindings/python/ns3modulescan-modular.py", line 31, in <module>

    import ns3modulescan

  File "/home/natasha/repos/ns-3-dev-git/bindings/python/ns3modulescan.py", line 7, in <module>

    from pybindgen.gccxmlparser import ModuleParser, PygenClassifier, PygenSection, WrapperWarning

ModuleNotFoundError: No module named 'pybindgen.gccxmlparser'

Waf: Leaving directory `/home/natasha/repos/ns-3-dev-git/build'

Build failed

 -> task in '' failed with exit status 1 (run with -v to display more information)

Do you have any further advice, please?

Kind regards

[nat...@gmail.com]

Hi Tom

I read that pybindgen.gccxmlparser only works in python2 because the pygccxml package has not been ported to Python 3. I then installed pybindgen and pygccml using python2 instead of python3 and ran ./waf configure in ns-3-dev with that version and then tried to do api scanning with that, but now when running /usr/bin/python ./waf --apiscan=wifi the output is:

Waf: Entering directory `/home/natasha/repos/ns-3-dev-git/build'

Traceback (most recent call last):

  File "/home/natasha/repos/ns-3-dev-git/.waf-2.0.18-ff4ae9f5cc05353d3dc3aeff8854ae69/waflib/Scripting.py", line 119, in waf_entry_point

    run_commands()

  File "/home/natasha/repos/ns-3-dev-git/.waf-2.0.18-ff4ae9f5cc05353d3dc3aeff8854ae69/waflib/Scripting.py", line 182, in run_commands

    ctx=run_command(cmd_name)

  File "/home/natasha/repos/ns-3-dev-git/.waf-2.0.18-ff4ae9f5cc05353d3dc3aeff8854ae69/waflib/Scripting.py", line 173, in run_command

    ctx.execute()

  File "/home/natasha/repos/ns-3-dev-git/.waf-2.0.18-ff4ae9f5cc05353d3dc3aeff8854ae69/waflib/Scripting.py", line 375, in execute

    return execute_method(self)

  File "/home/natasha/repos/ns-3-dev-git/.waf-2.0.18-ff4ae9f5cc05353d3dc3aeff8854ae69/waflib/Build.py", line 93, in execute

    self.execute_build()

  File "/home/natasha/repos/ns-3-dev-git/.waf-2.0.18-ff4ae9f5cc05353d3dc3aeff8854ae69/waflib/Build.py", line 96, in execute_build

    self.recurse([self.run_dir])

  File "/home/natasha/repos/ns-3-dev-git/.waf-2.0.18-ff4ae9f5cc05353d3dc3aeff8854ae69/waflib/Context.py", line 126, in recurse

    user_function(self)

  File "/home/natasha/repos/ns-3-dev-git/wscript", line 965, in build

    add_scratch_programs(bld)

  File "/home/natasha/repos/ns-3-dev-git/wscript", line 818, in add_scratch_programs

    obj = bld.create_ns3_program(name, all_modules)

  File "/home/natasha/repos/ns-3-dev-git/wscript", line 768, in create_ns3_program

    program.use = program.ns3_module_dependencies.copy()

AttributeError: 'list' object has no attribute 'copy'

and when running just ./waf --apiscan=wifi, the output is:

Waf: Entering directory `/home/natasha/repos/ns-3-dev-git/build'

Modules to scan:  ['wifi']

api-scan-gcc_LP64

  File "/home/natasha/repos/ns-3-dev-git/bindings/python/ns3modulescan-modular.py", line 113

    print(pygccxml_definition, file=sys.stderr)

                                   ^

SyntaxError: invalid syntax

Please let me know if you have any further suggestions. I really need to get python bindings to work and I have really tried everything I can think of.

Thanks

Regards,