vxlan tomahawk

[Алексей Брагин]

Hello, 

I'm trying to build a small PoC using a few of DX010 switches (BCM56960_B1, Broadcom Tomahawk).

I got some basic functionality working just fine (breakouts, lacp, bgp) but I'm really struggling with the configuration of L2 VxLAN.

From what I can see in the logs:

config vxlan add vtep <loopback_ip>

config vxlan evpn_nvo add nvo vtep

config vlan add 129

applies without any error.

But once I issue:

config vxlan map add vtep 129 10129

I get following error messages:

Sep  8 10:59:40.947338 sonic NOTICE swss#orchagent: :- create_tunnel: create_tunnel:encapmaplist[0]=0x29000000000625

Sep  8 10:59:40.948024 sonic NOTICE swss#orchagent: :- create_tunnel: create_tunnel:encapmaplist[1]=0x29000000000627

Sep  8 10:59:40.957544 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:_brcm_sai_vxlan_create_vpn:872 create tunnel initiator setup for net port failed with error Invalid parameter (0xfffffffc).

Sep  8 10:59:40.958129 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:_brcm_sai_vxlan_enable:1164 create a vxlan decap tunnel failed with error -5.

Sep  8 10:59:40.961262 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:brcm_sai_create_tunnel_map_entry:2633 Can't create brcm vxlan tunnel

Sep  8 10:59:40.961262 sonic ERR syncd#syncd: :- sendApiResponse: api SAI_COMMON_API_CREATE failed in syncd mode: SAI_STATUS_FAILURE

Sep  8 10:59:40.962152 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE: SAI_TUNNEL_MAP_TYPE_VNI_TO_VLAN_ID

Sep  8 10:59:40.962741 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP: oid:0x29000000000624

Sep  8 10:59:40.964363 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_MAP_ENTRY_ATTR_VLAN_ID_VALUE: 129

Sep  8 10:59:40.966217 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_MAP_ENTRY_ATTR_VNI_ID_KEY: 10129

Sep  8 10:59:40.969094 sonic INFO systemd-udevd[9982]: Using default interface naming scheme 'v247'.

Sep  8 10:59:40.970653 sonic INFO systemd-udevd[9982]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.

Sep  8 10:59:40.978113 sonic ERR swss#orchagent: :- create: create status: SAI_STATUS_FAILURE

Sep  8 10:59:40.978113 sonic WARNING swss#orchagent: :- addOperation: Error adding tunnel map entry. Tunnel: vtep. Entry: map_10129_Vlan129. Error: Can't create a tunnel map entry object

Sep  8 10:59:40.978113 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:_brcm_sai_vxlan_create_vxlan_vpn:77 vxlan vpn create failed with error Entry exists (0xfffffff8).

Sep  8 10:59:40.978113 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:_brcm_sai_vxlan_create_vpn:858 create vxlan vpn failed with error -6.

Sep  8 10:59:40.978113 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:_brcm_sai_vxlan_enable:1164 create a vxlan decap tunnel failed with error -6.

Sep  8 10:59:40.978113 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:brcm_sai_create_tunnel_map_entry:2633 Can't create brcm vxlan tunnel

Sep  8 10:59:40.978113 sonic ERR syncd#syncd: :- sendApiResponse: api SAI_COMMON_API_CREATE failed in syncd mode: SAI_STATUS_FAILURE

Sep  8 10:59:40.978113 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE: SAI_TUNNEL_MAP_TYPE_VNI_TO_VLAN_ID

Sep  8 10:59:40.978113 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP: oid:0x29000000000624

Sep  8 10:59:40.978113 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_MAP_ENTRY_ATTR_VLAN_ID_VALUE: 129

Sep  8 10:59:40.978113 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_MAP_ENTRY_ATTR_VNI_ID_KEY: 10129

Sep  8 10:59:40.978113 sonic ERR swss#orchagent: :- create: create status: SAI_STATUS_FAILURE

Sep  8 10:59:40.978113 sonic WARNING swss#orchagent: :- addOperation: Error adding tunnel map entry. Tunnel: vtep. Entry: map_10129_Vlan129. Error: Can't create a tunnel map entry object

Sep  8 10:59:40.979782 sonic INFO kernel: [  267.028692] Bridge: port 2(vtep-129) entered blocking state

Sep  8 10:59:40.979822 sonic INFO kernel: [  267.028696] Bridge: port 2(vtep-129) entered disabled state

Sep  8 10:59:40.979830 sonic INFO kernel: [  267.029856] device vtep-129 entered promiscuous mode

Sep  8 10:59:41.015781 sonic INFO kernel: [  267.067912] Bridge: port 2(vtep-129) entered blocking state

Sep  8 10:59:41.015817 sonic INFO kernel: [  267.067916] Bridge: port 2(vtep-129) entered forwarding state

Sep  8 10:59:41.015823 sonic INFO kernel: [  267.068202] IPv6: ADDRCONF(NETDEV_CHANGE): Vlan129: link becomes ready

Sep  8 10:59:41.250863 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:_brcm_sai_vxlan_create_vxlan_vpn:77 vxlan vpn create failed with error Entry exists (0xfffffff8).

Sep  8 10:59:41.251433 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:_brcm_sai_vxlan_create_vpn:858 create vxlan vpn failed with error -6.

Sep  8 10:59:41.251996 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:_brcm_sai_vxlan_enable:1164 create a vxlan decap tunnel failed with error -6.

Sep  8 10:59:41.251996 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:brcm_sai_create_tunnel_map_entry:2633 Can't create brcm vxlan tunnel

Sep  8 10:59:41.251996 sonic ERR syncd#syncd: :- sendApiResponse: api SAI_COMMON_API_CREATE failed in syncd mode: SAI_STATUS_FAILURE

Sep  8 10:59:41.252113 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP_TYPE: SAI_TUNNEL_MAP_TYPE_VNI_TO_VLAN_ID

Sep  8 10:59:41.252113 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_MAP_ENTRY_ATTR_TUNNEL_MAP: oid:0x29000000000624

Sep  8 10:59:41.252161 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_MAP_ENTRY_ATTR_VLAN_ID_VALUE: 129

Sep  8 10:59:41.252193 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_TUNNEL_MAP_ENTRY_ATTR_VNI_ID_KEY: 10129

Sep  8 10:59:41.252193 sonic ERR swss#orchagent: :- create: create status: SAI_STATUS_FAILURE

I've tried latest and 202106 branches without any luck.

If anyone got this working on tomahawk chip I would really appreciate if you can share your conifg.

Thank you !

[Rajesh Sankaran]

Hi,

  There is no support for EVPN VXLAN on Tomahawk platforms.

Thanks.

[Алексей Брагин]

Thanks, Rajesh

Will it be added in the future or there are hardware limitations?

Is there a list anywhere which I can reference on which platforms EVPN/VXLAN is supported ?

четверг, 9 сентября 2021 г. в 06:49:11 UTC-4, Rajesh Sankaran:

[Sudharshan S]

Hi All,

      I am trying to bring up EVPN-VXLAN on the following boards  for a POC

x86_64-accton_as7716_32x-r0 (Tomahawk)
x86_64-accton_as5712_54x-r0 (Trident2)

and end up with the same error message as mentioned in the above thread.

Jul 20 16:07:25.498047 sonic NOTICE swss#orchagent: :- create_tunnel: create_tunnel:encapmaplist[0]=0x29000000000687
Jul 20 16:07:25.498047 sonic NOTICE swss#orchagent: :- create_tunnel: create_tunnel:encapmaplist[1]=0x29000000000689
Jul 20 16:07:25.501310 sonic NOTICE swss#orchagent: :- addBridgePort: Add bridge port Port_SRC_VTEP_10.0.0.1 to default 1Q bridge
Jul 20 16:07:25.502197 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:_brcm_sai_vxlan_create_vpn:890 create tunnel initiator setup for net port failed with error Invalid parameter (0xfffffffc).
Jul 20 16:07:25.502197 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:_brcm_sai_vxlan_enable:1182 create a vxlan decap tunnel failed with error -5.
Jul 20 16:07:25.502197 sonic ERR syncd#syncd: [none] SAI_API_TUNNEL:brcm_sai_create_tunnel_map_entry:2682 Can't create brcm vxlan tunnel

In one of the discussions, I found that the following needs to be added in the .bcm file.

#vxlan
use_all_splithorizon_groups=1
riot_enable=1
sai_tunnel_support=1
riot_overlay_l3_intf_mem_size=4096
riot_overlay_l3_egress_mem_size=32768
riot_overlay_ecmp_resilient_hash_size=16384
flow_init_mode=1

https://github.com/Azure/sonic-buildimage/pull/9491/commits/f2142c15574fe0f305a699c6617ef0dff3e21e48

Is this applicable from td3 onwards? I still face the crash eventhough the board supports VXLAN. Is this is a SAI restriction. Kindly provide your valuable thoughts.

Thanks,

Sudharshan