RL-FEC - A Self-Contained Reinforcement Learning Code(?) of FEC for ns-3

[Charles Pandian]

I come across a recent work of  Feifan Zhang et.al. called RL-FEC.

I thank  Feifan Zhang  et.al. for providing their entire work on GitHub.

It seems to be good work on ML-based algorithm implementation on ns-3. The reason is : This code is a self-contained RL implementation. Unlike ns3-AI and ns3-Gym, the RL-FEC provides a simple way of implementing reinforcement learning under ns-3. The same idea can be used to implement some other elementary ML/AI algorithm under ns-3.

I tried to install it under ns-3 and posted my chroot based installation procedure at the following link:

RL-FEC – A Self-Contained Reinforcement Learning Code of FEC for ns-3

My questions are:
1) Did anyone use this code to design any other RL based algorithm under ns-3?
2)Did anyone test the code on a somewhat big topology with more nodes?

The authors of RL-FEC and others can share their experiences and ideas for using this code for some other Networking related ML tasks under ns-3.

Charles Pandian,

[Shilpi Mittal]

Thanks Charles for sharing such great information. I want to do some test on RL in NS3. 

Could it be sufficient for?

--
Posting to this group should follow these guidelines https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting
---
You received this message because you are subscribed to the Google Groups "ns-3-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ns-3-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/CAHXeiM-3qFvu9R_hwvUBTXZuWoKfPAMbT%2BtPwmRcRw4bhfPKwg%40mail.gmail.com.

[Shilpi Mittal]

Hi

I tried to run this concept as per the given steps in the reference link. Installation went smoothly but failed at the end step when I tried to run this in the scratch folder, It started asking for header files and I copied all the asked header files in scratch folder, but still showing the same error.

ubuntu@ubuntu:~/ns-3-dev$ ./waf --run scratch/"rlfec-stream-server-client-sim --nPackets=5 --lossRate=0.05 --training=false --cbrMode=true --cbrRate=10Mbps"
Waf: Entering directory `/home/ubuntu/ns-3-dev/build'
[1943/1995] Compiling scratch/rlfec-stream-server-client-sim.cc
In file included from ../scratch/rlfec-stream-server-client-sim.cc:18:
../scratch/rlfec-stream-server.h:17:10: fatal error: ns3/windowed-filter.h: No such file or directory
   17 | #include "ns3/windowed-filter.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

Waf: Leaving directory `/home/ubuntu/ns-3-dev/build'
Build failed
 -> task in 'rlfec-stream-server-client-sim' failed with exit status 1 (run with -v to display more information)

Kindly give me a solution where I am wrong. 

[Charles Pandian]

The authors said that they tested it only on ns-3.35.

Charles Pandian,

To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/CADeTpUeMf_z_-nzc2YdhOiG7-wmO6mC7f6hhJJ2KhtAQmOLfcg%40mail.gmail.com.

[Shruthi Sharan]

Hi Charles,

I tried running RL-FEC on ns3.36.1. But I am getting this error.

usr/bin/ld: CMakeFiles/scratch_rl-fec_rlfec-stream-server-client-sim.dir/rl-fec/rlfec-stream-server-app.cc.o: in function `ns3::RlFecStreamServerApp::PacketAckRecv(ns3::Ptr<ns3::Packet>)':
rlfec-stream-server-app.cc:(.text+0x3198): undefined reference to `flush_acked_packets'
/usr/bin/ld: CMakeFiles/scratch_rl-fec_rlfec-stream-server-client-sim.dir/rl-fec/rlfec-stream-server-app.cc.o: in function `ns3::RlFecStreamServerApp::StartApplication()':
rlfec-stream-server-app.cc:(.text+0x5b2d): undefined reference to `initialize_encoder'
/usr/bin/ld: CMakeFiles/scratch_rl-fec_rlfec-stream-server-client-sim.dir/rl-fec/rlfec-stream-server-app.cc.o: in function `ns3::RlFecStreamServerApp::DoSendDataPackets()':
rlfec-stream-server-app.cc:(.text+0x6db4): undefined reference to `output_source_packet'
/usr/bin/ld: rlfec-stream-server-app.cc:(.text+0x6dd6): undefined reference to `serialize_packet'
/usr/bin/ld: rlfec-stream-server-app.cc:(.text+0x709f): undefined reference to `free_packet'
/usr/bin/ld: rlfec-stream-server-app.cc:(.text+0x7138): undefined reference to `output_repair_packet'
/usr/bin/ld: CMakeFiles/scratch_rl-fec_rlfec-stream-server-client-sim.dir/rl-fec/stream-client-app.cc.o: in function `ns3::StreamClientApp::ParameterRecv(ns3::Ptr<ns3::Packet>, ns3::Address)':
stream-client-app.cc:(.text+0x1254): undefined reference to `initialize_decoder'
/usr/bin/ld: CMakeFiles/scratch_rl-fec_rlfec-stream-server-client-sim.dir/rl-fec/stream-client-app.cc.o: in function `ns3::StreamClientApp::PacketRecv(ns3::Ptr<ns3::Packet>, ns3::Address)':
stream-client-app.cc:(.text+0x1530): undefined reference to `deserialize_packet'
/usr/bin/ld: stream-client-app.cc:(.text+0x1801): undefined reference to `receive_packet'
collect2: error: ld returned 1 exit status
make[3]: *** [scratch/CMakeFiles/scratch_rl-fec_rlfec-stream-server-client-sim.dir/build.make:177: ../build/scratch/rl-fec/ns3.36.1-rlfec-stream-server-client-sim-optimized] Error 1
make[2]: *** [CMakeFiles/Makefile2:16788: scratch/CMakeFiles/scratch_rl-fec_rlfec-stream-server-client-sim.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:16794: scratch/CMakeFiles/scratch_rl-fec_rlfec-stream-server-client-sim.dir/rule] Error 2
make: *** [Makefile:5324: scratch_rl-fec_rlfec-stream-server-client-sim] Error 2

I think its not able to link library i.e. libstream.a. Can you let me know where have i gone wrong?

Regards

Shruthi K R

[Li, Ye]

Hi Shruthi,

Thanks for your interest. I'm one of the authors of RL-FEC and the paper. 

Please note that RL-FEC was developed in ns-3.35, which used Waf. Since you are using ns-3.36, you need to write a CMakeList.txt to indicate where to find libstreamc.a. Assume that you put rl-fec under ns-3.36.1/examples/rl-fec/. You need to write CMakeList.txt like this:

[04:23:14] [ns-3.36.1] $ cd examples/rl-fec/
[04:23:28] [rl-fec] $ cat CMakeLists.txt

find_library(

  libstreamc

  libstreamc.a

  PATHS /Users/yeli/Documents/ns-3.36.1/examples/rl-fec

 )

build_example(

  NAME rlfec-stream-server-client-sim

  SOURCE_FILES rlfec-stream-server-client-sim.cc

                              rlfec-stream-server-app.cc

                              stream-client-app.cc

                              handlepacket.cc

                              infoqueue.cc

                             qlearning.cc

                             tiles.cc

  LIBRARIES_TO_LINK

    ${libcore}

    ${libstats}

    ${libpoint-to-point}

    ${libinternet}

    ${libapplications}

    ${libflow-monitor}

    ${libpoint-to-point-layout}

    ${libstreamc}

)

Please refer to ns-3 documentation for details on how to translate wscript to CMakeList.txt. Hope it helps.

Best Regards,

To view this discussion on the web visit https://groups.google.com/d/msgid/ns-3-users/f7537818-d89c-4b2d-a19c-5920e88f7e3an%40googlegroups.com.

--

LI, Ye

School of Information Science and Technology

Nantong University

Nantong 226019, Jiangsu Province, China

GitHub: https://github.com/yeliqseu