network layer questions

[李文杰]

Dear ASTRA-sim Users and Developers ,

I am currently using Chakra and ASTRA-sim for network simulations, and I have some questions regarding trace generation and network layer operations.

1. Does the trace generated by Chakra include the detailed contents of network communication packets? If it does, I would like to understand more about how the packet details are captured and represented in the trace files.

2. Regarding the network layer in ASTRA-sim, is it possible to use ns-3 to perform different operations on packets based on their content, similar to what programmable switches allow? Specifically, I am interested in performing operations such as:

   • Dropping packets conditionally
   • Caching or performing computations based on packet contents
   • Any other custom actions that can be triggered based on packet characteristics

I would appreciate any guidance or examples on how to achieve this in ASTRA-sim.

Thank you for your time and support!

Best regards,
Wenjie Li
University of Chinese Academy of Sciences  

[Yoo, Jinsun]

Hi Wenjie, thank you for reaching out!

Please find my answers below.

 

Best,

Jinsun

 

1. Does the trace generated by Chakra include the detailed contents of network communication packets? If it does, I would like to understand more about how the packet details are captured and represented in the trace files.

No, Chakra trace does not contain such information. The nodes in Chakra trace only contain whether this is an AllGather or an AllReduce, and the process groups that are included in that rank. It is upto the system layer to split this collective into multiple send/receive messages.*)

That is, the system layer will simply tell the network layer to simulate a send/receive, of xx bytes from src to dst. (Please refer to the sim_send and sim_recv in the Network API: https://github.com/astra-sim/astra-sim/blob/master/astra-sim/common/AstraNetworkAPI.hh)

It is upto the network layer to decide how to simulate that network message. For example, the Analytical backend doesn’t even split a send/receive message into packets. So far, only the ns-3 backend does.

 

2. Regarding the network layer in ASTRA-sim, is it possible to use ns-3 to perform different operations on packets based on their content, similar to what programmable switches allow? Specifically, I am interested in performing operations such as:

Your three bullet points depend on an understanding of the internals of the ns-3 simulator.

• Dropping packets conditionally

What does ‘conditionally’ mean here? NS-3 has a knob to drop packets probablistically (e.g. drop 0.5% of packets), but it does not have the feature to leverage human-defined conditions (e.g. drop if source is rank 0 and xxx…). Hence, I’d like to understand your ‘conditionally’ a bit better.

• Caching or performing computations based on packet contents

ASTRA-sim is a simulator and does not perform any actual computation. Hence, it does not simulate what the ‘content’ would be. (One could probably extend the simulator to take note of the tensor address that is being communicated, and build a ‘cache’ based on that tensor address, but this is not something the developers intend to actively develop)

• Any other custom actions that can be triggered based on packet characteristics

For each packet one could record the route it has taken, for example, and perform network actions based on that? Again, a detailed discussion of your usecase would be helpful.

 

 

 

From: astrasi...@googlegroups.com <astrasi...@googlegroups.com> on behalf of 李文杰 <craft...@gmail.com>
Date: Sunday, January 19, 2025 at 11:15 PM
To: astrasi...@googlegroups.com <astrasi...@googlegroups.com>
Subject: network layer questions

You don't often get email from craft...@gmail.com. Learn why this is important

--
You received this message because you are subscribed to the Google Groups "ASTRA-sim Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to astrasim-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/astrasim-users/CAOmHS-bULAewv2vEdM94wOacUb8iHuxa%2BuQBnyzif2LRkZX_Pg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.