Help with Integrating Kafka/Spark with ns-3

[Elio Rojo]

Hi everyone,

I’m working on my master’s thesis using ns-3 and looking for ways to integrate big data tools like Apache Kafka or Apache Spark with ns-3. My goal is to simulate a telecommunication network and handle large amounts of data in real-time.

I’d appreciate any help or advice on the following:

1. How to connect ns-3 to Kafka for sending or receiving data streams.
2. How to process ns-3 simulation data with Spark for analysis.
3. Any recommendations or resources for combining ns-3 with big data tools.

Any pointers or examples would be really helpful. Thanks a lot!

[Tommaso Pecorella]

I'd say that the very first thing to do is to define what you want to pass to Kafka to process (which is rather non-obvious). This is strictly related to your research goal, so we can't help on that.

Once you have a clearer picture of the events you want to process, we can suggest a way to extract them from ns-3. Examples might be generating PCAP files, using logs, etc. However, we'll need first to know what you're interested to, or any suggestion might send you in the wrong direction.