Dest IP in the sai_tunnel_attr structure

[Rajesh Sankaran]

Moving this as a separate discussion.

Hi Marian,

  

    Thanks for the mail. 

     I see that the SAI_VLAN_FLOOD_CONTROL_TYPE_COMBINED has been introduced in SAI version 1.5 and that 

     helps in keeping the access port and tunnel side members be in the same flooding domain. 

     The API allows for separate access side (vlan members) flooding group with the 

      network (tunnel) side flooding group through this newly introduced flood control type. 

     For devices which have a single flooding group for both the access and network side interfaces this poses

     some challenges when the first/last tunnel member is added/removed to the dot1q/dot1d bridge. 

     when the dot1q/dot1d bridge is created there would be a flood group created. 

   

     This could mean, 

     a) inefficient use of resources in terms of L2MC group usage. OR

     b) introducing new knob in sonic to create a dot1q/dot1d bridge with an L2MC group and to support

         coexistence of both. 

 

     So in essence the API doesn't abstract the individual implementations well. 

     Apart from the above we would also like to have the ability,

   a) To support per remote IP counters.

   b) To support remote IP+ VNI counters.  

   c) To support operational status for the network side much like the access side. 

     

     These are possible with a p2p version of the tunnel object. (i.e with DEST_IP). 

     The p2p version models the tunnel like a physical port/LAG and hence allows for similar behavior. 

    dot1q bridge case

    ==============

/* Tunnel bridge port created for each remote IP. */

create_bridge_port( &tun_bport, default_dot1q_bridge, SAI_BRIDGE_PORT_TYPE_TUNNEL, tunnel_id) 

/* vlan_member_port created for each VLAN+remote IP */

/* This adds the tunnel to the flood group and works well for single flood group and multiple flood group 

    device implementations */

create_vlan_member_port(&tun_vlan_member_port, tun_bport, tagging_mode) 

/* Get stats per remote IP */

get_tunnel_stats_ext(tunnel_id,....)

/* operational status which reflects the data forwarding status towards a remote ip */

similar mechanism as port but using the tunnel_bport. 

   dot1d bridge case

   ==============

/* Tunnel bridge port corresponds to remote IP + VNI  like PORT+VLAN for access side*/

/* This allows for the tunnel to be added to the dot1d bridge's flooding group */

create_bridge_port( &tun_bport, dot1d_bridge_id, SAI_BRIDGE_PORT_TYPE_TUNNEL, tunnel_id) 

/* Get stats per remote IP */

get_tunnel_stats_ext(tunnel_id,....)

/* Get stats per remote IP + VNI */

get_bridge_port_stats_ext(tun_bport,...)

/* operational status which reflects the data forwarding status towards a remote ip */

similar mechanism as port but using the tunnel_id */ 

       

Regards,

Rajesh 

Hi,

Going back to the DEST_IP tunnel attribute, below is the pseudo-code that sets flood control to multiple endpoints. Any reason not to use this API?

// add tunnel endpoints 1.1.1.2, 1.1.1.3, 1.1.1.4 to flood vector

// vlan, tunnel, tunnel_bridge_port objects are pre-created

sai_object_id_t l2mc_group;

sai_object_id_t member2, member3, member4;

sai_attribute_t attr;

vector<sai_attribute_t> attrs;

// create L2MC group

sai_l2mc_group_api->cerate_l2mc_group(&l2mc_group, g_switch_id, attrs.size(), attrs.data());

// create L2MC group members

attr.id = SAI_L2MC_GROUP_MEMBER_ATTR_L2MC_GROUP_ID;

attr.value.oid = l2mc_group;

attrs.push_back(attr);

attr.id = SAI_L2MC_GROUP_MEMBER_ATTR_L2MC_OUTPUT_ID;

attr.value.oid = tunnel_bridge_port;

attrs.push_back(attr);

attr.id = SAI_L2MC_GROUP_MEMBER_ATTR_L2MC_ENDPOINT_IP;

attr.value.ipaddr.addr = 0x01010102;

attrs.push_back(attr);

sai_l2mc_group_api->cerate_l2mc_group_member(&member2, g_switch_id, attrs.size(), attrs.data());

attrs.clear();

attr.id = SAI_L2MC_GROUP_MEMBER_ATTR_L2MC_GROUP_ID;

attr.value.oid = l2mc_group;

attrs.push_back(attr);

attr.id = SAI_L2MC_GROUP_MEMBER_ATTR_L2MC_OUTPUT_ID;

attr.value.oid = tunnel_bridge_port;

attrs.push_back(attr);

attr.id = SAI_L2MC_GROUP_MEMBER_ATTR_L2MC_ENDPOINT_IP;

attr.value.ipaddr.addr = 0x01010103;

attrs.push_back(attr);

sai_l2mc_group_api->cerate_l2mc_group_member(&member3, g_switch_id, attrs.size(), attrs.data());

attrs.clear();

attr.id = SAI_L2MC_GROUP_MEMBER_ATTR_L2MC_GROUP_ID;

attr.value.oid = l2mc_group;

attrs.push_back(attr);

attr.id = SAI_L2MC_GROUP_MEMBER_ATTR_L2MC_OUTPUT_ID;

attr.value.oid = tunnel_bridge_port;

attrs.push_back(attr);

attr.id = SAI_L2MC_GROUP_MEMBER_ATTR_L2MC_ENDPOINT_IP;

attr.value.ipaddr.addr = 0x01010104;

attrs.push_back(attr);

sai_l2mc_group_api->cerate_l2mc_group_member(&member4, g_switch_id, attrs.size(), attrs.data());

// set the L2MC group on a vlan

attr.id = SAI_VLAN_ATTR_UNKNOWN_UNICAST_FLOOD_CONTROL_TYPE;

attr.value.oid = SAI_VLAN_FLOOD_CONTROL_TYPE_COMBINED;

sai_vlan_api->set_vlan_attribute(vlan, &attr);

attr.id = SAI_VLAN_ATTR_UNKNOWN_UNICAST_FLOOD_GROUP;

attr.value.oid = l2mc_group;

sai_vlan_api->set_vlan_attribute(vlan, &attr);

Marian

[Rajesh Pukhraj Jain]

Just to add further on this aspect.

As of now SAI as well as SONiC, do not have any representation of the evpn discovered tunnels.

The below design facilitate to track the dest-ip based tunnels (It can be static or evpn discovered) and can have the following operations on the tunnel.

Add tunnel as a member of VLAN.

SAI indicates the tunnel operation status (up/down) to application.

SAI will provide support for the counters (bytes/packets) on the corresponding tunnel. Application can fetch the counters.

Mac/route can be associated with the tunnel

It also maintains the current portsorch/fdborch to add tunnel as part of vlan and associate the Mac corresponding to the tunnel bridge-port.

Thanks

Rajesh P Jain

--
You received this message because you are subscribed to the Google Groups "sonic-evpn-workgroup" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonic-evpn-workg...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonic-evpn-workgroup/946c50ba-9bae-4a4c-a4c5-cf383d839d46%40googlegroups.com.

[maryan...@gmail.com]

Hi Rajesh,

regarding counters,

   a) To support per remote IP counters - SW summarization of counters ni item (b) by remote IP

   b) To support remote IP+ VNI counters - we already have this in current API - SAI_FDB_ENTRY_ATTR_COUNTER_ID 

   c) To support operational status for the network side much like the access side - can you please define what do you mean by tunnel operational status? How it is determined logically? Today the bridge port API has only the admin state API.

Regarding the main part, for the combined flood type the SAI implementation can merge two MC groups into one if the ASIC allows so. This is what our implementation is going to do AFAIK. This API was introduced for the user convenience and SAI implementation will have to deal with the details.

So assuming I can describe the flooding logic with the existing API without suboptimal HW utilization, I think introducing another version of API that intends to resolve the flooding problem will push it into a direction where there is API per vendor, which is not a where we would like to see SAI and SONiC.

[Rajesh Sankaran]

Hi Marian:

    On the per FDB entry counter id the following are some observations. 

    a) This being attached to a Dest MAC is by definition a Tx counter. There is no Rx counting.

    b) Assuming one counter-id per MAC entry, this is a scalability challenge for very large number of MACs. 

    c) For BUM traffic this doesn't help and we will need to introduce a counter id in the l2mc_group_member. 

    d) Assuming there is a counter id allocated per remote ip and all FDB entries point to this single counter 

        how do devices which natively count at a per remote ip basis know that this counter is a per remote IP counter 

        and not a per FDB counter ?

        Looks like we might have to add a counter_type_shared_per_remote_ip and shared_per_vni  to the 

        sai_counter_type_t ?

   On the operational status, the sai_port_oper_status_notification_t can be used. The port_id field is a 

   of type sai_object_id_t and currently supports objects of type - port, lag and bridge_port. 

   We might have to add object_type tunnel (the p2p version) 

   The operational status(p2p version of tunnel)  should reflect the data forwarding capability to the remote ip. 

   This could be,

    a) IP underlay reachability to the remote IP - the operstatus here can toggle between up and down.

    b) device programming status. This is creation time and could be reflective of either implementation issues or 

        resource availability issues. This is very device specific and by providing an operational status we can 

        provide a clean abstraction. 

   

   Introducing the dest_ip field and giving a p2p connotation solves a lot of requirements cleanly and with clean 

   abstraction. The SONiC code continues to have only one implementation.

   This is as follows.

/* Create Mapper and Mapping entries */

a) map_oid = create_mapper() 

b) mapping_entry_oid = create_mapping_entry (map_oid,...)  

/* Create a tunnel and term as existing i.e the P2MP version of the tunnel */

c) sip_tunnel_oid  = create_tunnel(SIP, map_oid)  

d) sip_tunnel_term_oid = create_tunnel_terminator(sip_tunnel_oid, SIP) /* P2MP */

---> Till this point things are the same as the existing implementation.

/* Create a P2P version of the tunnel with the proposed destip field as part of the tunnel_attr */

/* There are no changes to the terminator API or attrs */

e) per_dip_tunnel_oid = create_tunnel(SIP, DIP, map_oid) /* uses the map_oid created in the first step */

d) per_dip_tunnel_term_oid = create_tunnel_terminator(per_dip_tunnel_oid, SIP, DIP)  /* P2P type terminator */

/* per dip tunnel learning enable/disable */

/* For EVPN tunnels we disable hardware learning and for static tunnels we can enable hardware learning  like access

   ports. */

f) per_dip_bridge_port_oid = create_bridge_port(default_dot1q_bridge, SAI_BRIDGE_PORT_TYPE_TUNNEL, per_dip_tunnel_oid, learning_disable) 

/* If there are devices which need to create a separate L2MC group and attach it to the vlan then it could be

   done within SAI implementation based on the bridge_port type.  */

g) tunnel_vlan_member_port = create_vlan_member_port(vlanid, per_dip_bridge_port_oid)

/* FDB adds */

h) add_fdb_entry(mac, bv_id, per_dip_bridge_port_oid, dest_ip) 

/* tunnel nexthop creation for L3VXLAN*/

Remains the same as existing implementation i.e it uses the sip_tunnel_oid created in step (c) 

Regards,

Rajesh