Cisco Nexus switches question

[Panagiotis Christias]

Hello,

I have seen that currently Emulab is supporting Cisco Nexus switches (or at least some models/series, model 3172 is mentioned in the code).

We are currently trying to set up Emulab with our Nexus 5600 series (models 5672UP and 56128P) and we are facing various problems.

We managed to create and delete VLANs using the same SNMP commands used in Emulab but we failed to assign ports to VLANs. It looks like the specific OID (vmVlan) is read-only (we hope that we are wrong).

This is what we get when we try to set interface with ifIndex 526647360 to VLAN 11:

root@boss:~ # snmpset -v 2c -m all -c private $SWITCH vmVlan.526647360 i 11

Error in packet.

Reason: (notWritable) That object does not support modification

Failed object: enterprises.cisco.ciscoMgmt.ciscoVlanMembershipMIB.ciscoVlanMembershipMIBObjects.vmMembership.vmMembershipTable.vmMembershipEntry.vmVlan.526647360

Are we missing something obvious? Is there someone that has some experience on Nexus switches and Emulab or even better using them successfully in their testbed?

Thank you very much,

Panagiotis

[Kirk Webb]

Hi Panagiotis,

I spoke with the individual that tweaked the Cisco snmpit module
(snmpit_cisco.pm) to work with NX-OS switches. His recollection is
that the `feature vtp` setting unlocked (perhaps through some fluke)
part of the OID space that was read-only before enabling this feature.
If you have this setting in place on your switches, then perhaps the
version of NX-OS on them has different OID restrictions. I'm afraid
we don't have any specific experience with these switches. They run
NX-OS version ~3.03.

I am guessing based on where things are failing for you that snmpit is
able to create vlans OK, but then fails to set port membership?

-Kirk

> --
> You received this message because you are subscribed to the Google Groups
> "emulab-admins" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to emulab-admin...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Kirk Webb]

Correction: The NX-OS version running at the other site is ~6.0.

-Kirk

[Michael Blodgett]

Greetings Panagiotis,  do you have the ports in question in layer 2 access mode,   

 

interface Ethernet1/10

  no switchport

 

if your port configs have the ‘no switchport’ you’ll get an error like you described, from the switch cli, just put a ‘switchport’ on each port,

 

Mike

--

[Panagiotis Christias]

Hello Kirk,

Yes, I read the comment in the code mentioning `feature vtp` and added it before testing. I suppose that this allowed me to create/delete vlans through snmp, the way snmpit does but it seems it is not enough for port membership to work in my case (different mib for starters). I was wondering whether there was another extra setting required that was not mentioned.

Is the following command working without any extra settings on your nexus-based testbed?

snmpset -v 2c -m all -c $COMMUNITY $SWITCH vmVlan.$IFINDEX i $VLAN

That is the University of Wisconsin-Madison cluster of CloudLab, right?

We are running NX-OS version 7.1(4)N1(1), the latest recommended by Cisco. There is also 7.3(2)N1(1) and I don't know if it's worth a try.

In case I don't have any luck with snmp, I'm evaluating the alternatives, such as netconf and nx-api. Any suggestions or comments?

Cheers,

Panagiotis

[Panagiotis Christias]

Hello Michael,

I don't think that I have them in layer 2 access mode. I haven't seen any 'no switchport' commands around but I'll check again first thing in the morning and add explicitly a ‘switchport’ on every port before testing again.

Is this a Nexus specific thing?

Thank you,

Panagiotis

[Kirk Webb]

On Wed, May 31, 2017 at 4:59 PM, Panagiotis Christias
<chri...@gmail.com> wrote:
> Hello Michael,
>
> I don't think that I have them in layer 2 access mode. I haven't seen any
> 'no switchport' commands around but I'll check again first thing in the
> morning and add explicitly a ‘switchport’ on every port before testing
> again.
>
> Is this a Nexus specific thing?

I don't know about IOS or CatOS, but a similar mode has to be set on
Dell Force10 switch ports too, otherwise similar set operations fail.

-Kirk

[Kirk Webb]

On Wed, May 31, 2017 at 4:50 PM, Panagiotis Christias
<chri...@gmail.com> wrote:

> In case I don't have any luck with snmp, I'm evaluating the alternatives,
> such as netconf and nx-api. Any suggestions or comments?

We have code that uses netconf for setting up Openflow on the HP
FlexFabric platform. It may be possible to re-use or extend it to
work with Netconf on NX-OS. The code is in "snmpit_libNetconf.pm".
Note that it makes use of the ssh protocol for interacting with
Netconf.

-Kirk

[Panagiotis Christias]

I've just checked. There aren't any 'no switchport' commands and 'switchport' is implied. Adding a 'switchport' command will not show up and will not affect the switch's SNMP behavior. Even on ports that have been already assigned to a VLAN, changing the VLAN fails.

Looks like I've hit wall :/

Panos

[Panagiotis Christias]

 
I will take a look at it and probably use it as a reference in case I decide to go the netconf way.

Thank you,
Panos

[Michael Blodgett]

This part of the MIB would be a pretty big change for Cisco to deprecate,  what version of code were you running?  For the interface your trying to assign to a vlan, what does the cli output of ‘show int ethernet <interface-number> switchport’ look like,

 

Mike

 

example,

 

Name: Ethernet1/13

  Switchport: Enabled

  Switchport Monitor: Not enabled

  Switchport Isolated : Not enabled

  Switchport Block Multicast: Not enabled

  Switchport Block Unicast: Not enabled

  Mac learning: Disabled

  Operational Mode: access

  Access Mode VLAN: 1 (default)

  Trunking Native Mode VLAN: 1 (default)

  Trunking VLANs Allowed: 1-4094

  Administrative private-vlan primary host-association: none

  Administrative private-vlan secondary host-association: none

  Administrative private-vlan primary mapping: none

  Administrative private-vlan secondary mapping: none

  Administrative private-vlan trunk native VLAN: none

  Administrative private-vlan trunk encapsulation: dot1q

  Administrative private-vlan trunk normal VLANs: none

  Administrative private-vlan trunk private VLANs: none

  Operational private-vlan: none

--

[Michael Blodgett]

And also the output ‘show running-config interface ethernet <interface-number> all’

 

Mike

 

From: <emulab...@googlegroups.com> on behalf of Panagiotis Christias <chri...@gmail.com>

Reply-To: "emulab...@googlegroups.com" <emulab...@googlegroups.com>
Date: Thursday, June 1, 2017 at 4:00 AM
To: emulab-admins <emulab...@googlegroups.com>

--

[Panagiotis Christias]

Hello Mike,

Sorry for the delayed reply. Friday and Monday were holidays around here and I had no access to my setup.

The switches are running NX-OS 7.1(4)N1(1). Here is the output of both commands:

expsw1# show interface Ethernet101/1/2 switchport 

Name: Ethernet101/1/2

  Switchport: Enabled

  Switchport Monitor: Not enabled 

  Operational Mode: access

  Access Mode VLAN: 1 (default)

  Trunking Native Mode VLAN: 1 (default)

  Trunking VLANs Allowed: 1-4094

  Pruning VLANs Enabled: 2-1001

  Voice VLAN: none

  Extended Trust State : not trusted [COS = 0]

  Administrative private-vlan primary host-association: none

  Administrative private-vlan secondary host-association: none

  Administrative private-vlan primary mapping: none

  Administrative private-vlan secondary mapping: none

  Administrative private-vlan trunk native VLAN: none

  Administrative private-vlan trunk encapsulation: dot1q

  Administrative private-vlan trunk normal VLANs: none

  Administrative private-vlan trunk private VLANs: none

  Operational private-vlan: none

  Unknown unicast blocked: disabled

  Unknown multicast blocked: disabled

expsw1# show running-config interface Ethernet101/1/2 all

!Command: show running-config interface Ethernet101/1/2 all

!Time: Tue Jun  6 11:30:53 2017

version 7.1(4)N1(1)

interface Ethernet101/1/2

  description node5 NIC2

  priority-flow-control mode auto

  lldp transmit

  lldp receive

  no switchport block unicast

  no switchport block multicast

  hardware multicast hw-hash

  no hardware vethernet mac filtering per-vlan

  cdp enable

  switchport

  switchport mode access

  no switchport dot1q ethertype

  no switchport priority extend

  spanning-tree port-priority 128

  spanning-tree cost auto

  spanning-tree link-type auto

  spanning-tree port type edge

  spanning-tree bpduguard enable

  no spanning-tree bpdufilter

  speed auto

  duplex auto

  flowcontrol receive off

  flowcontrol send on

  linkdebounce time 100

  link debounce link-up time 0

  no beacon

  delay 1

  snmp trap link-status

  logging event port link-status default

  logging event port trunk-status default

  mdix auto

  storm-control broadcast level 100.00

  storm-control multicast level 100.00

  storm-control unicast level 100.00

  no shutdown lan

  load-interval counter 1 30

  load-interval counter 2 300

  no load-interval counter 3

  medium broadcast

  vtp

  switchport trunk pruning vlan 2-1001

  no shutdown

Thank you,

Panagiotis

[Michael Blodgett]

I’ll see if I can get that rev running on hardware here, though one question, I noted your interface is Ethernet101/1/2, I’ve previously only seen the higher number that “101” for connected FEX units, is that the expansion slot on the 56128?

Mike

emulab-admin...@googlegroups.com <javascript:>.
For more options, visit
https://groups.google.com/d/optout <https://groups.google.com/d/optout>.

[Panagiotis Christias]

Hello Mike,

Yes, Ethernet101/1/2 is a port on a FEX unit connected to the 56128. In our setup, all 10G SFP+ ports on the 56128 and on both its expansion modules are used to connect FEX units (2248TP-E fabric extenders) and then connect nodes on the their 1000BASE-T ports.

Just to be sure, I've just tried the same snmpset command on one of 'local' interfaces (Ethernet1/5) and it failed the same way, while the output of the two 'show' commands was identical to that of the other interface.

Panagiotis