ns3-gym vs ns3-ai

[Halefom Kahsay]

Dear ns-3- users,

I am on my way  to implement  AI algorithms on NS-3. Some of the Toolkits or softwares modules that  help   release these types of projects are ns3-gym and ns3-ai. They both have some pros and cons but I am not sure which one to choose. 

Since ns3-gym was the first, I was inclined to use it ,but the problem is this may not work for NS3 beyond ns3.29. At the same time I find ns3-ai difficult to use because I have to deal with some memory issues.

I really  need your suggestions.

Thank you very much .

[Eduardo Almeida]

Hi,

I have used both frameworks.

ns3-gym is a framework specifically developed for reinforcement learning, which enables the development of an OpenAI Gym interface for ns-3. It does work on the current version of ns-3. You just need to download it from the ns-3 App Store and place it on the contrib/ folder. ns3-ai is a more flexible framework that allows the integration of ns-3 with external processes, not limiting to AI. It uses shared memory to exchange information between the ns-3 simulation process and another process running on your PC.

Both frameworks work as intended, although ns3-gym does have some limitations if you plan to use more advanced reinforcement learning models (I tweaked ns3-gym to overcome its limitations). Depending on your objectives, you can see what framework suits you the best. For instance, if you plan to work on reinforcement learning, ns3-gym would ease the integration with common reinforcement learning Python libraries. If you plan to work on supervised learning, then ns3-ai is probably the easiest (only?) option.

To the best of my knowledge, both apps are not yet compatible with ns-3.36 and beyond, since they lack CMake support. However, it is very easy to convert the wscript script into the corresponding CMakeLists.txt.

Regards,

Eduardo Almeida

[Mubanga Nsofu]

Hi Halefom,

And what if you consider writing your own custom environment as described in gym documentation?

Mubanga

--
Posting to this group should follow these guidelines https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting
---
You received this message because you are subscribed to the Google Groups "ns-3-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ns-3-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/CAJZRbsGuW_KH0c1s5uqr_Cm1MW-FAMgUZvyfYSTt6cPXgjGEnQ%40mail.gmail.com.

[Anandakrishnan Anil]

Hi Eduardo,

I wanted to ask whether it would be possible to use q learning to do an adaptive congestion control project with ns3-gym, where it would try to find the optimal congestion window.

Regards,

Anandakrishnan 

[Shruthi Sharan]

Hi Halefom,

Can I use ns3-gym even for underwater sensor networks?

Regards

Shruthi K R

Assistant Professor

Dept. of ISE

Global Academy of Technology

To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/CANFUzA4sgP20xUYhhzk1Quf8_UD37cfH%3DOuNBt_CBqt68z8t1Q%40mail.gmail.com.

[Eduardo Almeida]

Hi all,

Yes, you can use ns3-gym to work with Q-Learning (I've done it myself, but on another Wi-Fi research topic). You have to develop an ns3-gym environment in ns-3, which will act as the reinforcement learning environment. In that ns-3 environment, you will have to define how to collect the observations, apply the actions and calculate the reward. The main advantage of using ns3-gym is that you may use common Python RL libraries, such as TF-Agents, OpenAI Baselines, and many others.

Yes, you can also use ns3-gym for underwater sensor networks.

Regards,

Eduardo Almeida

[Shruthi Sharan]

Thanks a lot @ Eduardo Almeida

Regards

Shruthi K R

Assistant Professor

Dept. of ISE

Global Academy of Technology

To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/3f10fbd9-8572-4796-b6af-564c58b49b70n%40googlegroups.com.

[Halefom Kahsay]

Thank you Eduardo Almeida. The information you wrote is helpful.

To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/3f10fbd9-8572-4796-b6af-564c58b49b70n%40googlegroups.com.

[Ibrahim Sammour]

You might write your own messaging service between ns-3 and python, I used ZeroMQ to exchange messages between python and ns-3 using the json format. 

It is quite easy, straightforward, and fun to make.

[Anandakrishnan Anil]

Can you provide any open source code or resources for the same?(using ZeroMQ)

Thanks

Anandakrishnan

[Mubanga Nsofu]

Hi Eduardo,

You mention some limitations of ns3-gym when it comes to more advanced state of the art RL algorithms. May you please shed some light on these limitations and are you referring to e.g. SEED RL, etc. implementations ?

Kind regards & Thanks

Mubanga

[Shruthi Sharan]

Hi  @ Eduardo Almeida

Can you help me how to create an underwater environment in ns3gym or ns3-ai? It would be more helpful