Question about how STG determines the collective communication type

18 views
Skip to first unread message

Bozhen Wei

unread,
Nov 4, 2024, 1:39:24 AM11/4/24
to ASTRA-sim Users

Hello, I've been reading through the STG code and have some confusion and questions about the step where the Chakra communication node is generated. 


In the match_comms() function, what is the meaning of the shape and hidden parameters? In the get_parallel_dims() function, why does dim from shape correspond to the PARTITION type, while dim from hidden corresponds to the REDUCED type? What do these two types represent? Additionally, in the following section, why is the collective communication type (AllGather, AllToAll, etc.) determined based on the PARTITION or REDUCED type of from_comm and to_comm? What is the mechanism for this?


Thanks for your answer!
截屏2024-11-04 14.36.48.png截屏2024-11-04 14.37.18.png

Reply all
Reply to author
Forward
0 new messages