Cimc Installation Guide
Grethe Presnar
The following guide will serve as a living document for deploying and troubleshooting the vMX solution on Cisco's Enterprise NFV Infrastructure Software (NFVIS) platform. After completing these steps outlined in this document, you will have a virtual MX appliance running on a UCS compute platform in your own datacenter leveraging the NFVIS software. CSP UCS hardware is the recommended UCS platform for running NFVIS and the Meraki vMX.
*Meraki fiber SFP modules are not currently compatible with the Intel X-520 NIC and the CSP-SFP-1WS/LR modules above are not currently supported by the Meraki switches. We recommend using the Meraki twinax cables when using Meraki MS switches or if fiber is needed then we recommend the CSP-SFP modules and non-Meraki switches.
**In testing, the CSP-SFP modules did not work with the MS425 switch but do work with the MS350 series switches however the only officially supported SFP modules for Meraki switches are those documented here.
Additionally, for optimal memory performance having more smaller size DIMMs is better than fewer larger size ones. For example, if you need 128GB of memory for your CSP then it's better to use 8 x 16GB DIMMs instead of 4 x 32GB DIMMs
When the server reboots, the KVM console automatically installs Cisco Enterprise NFVIS from the virtual CD/DVD drive. The entire installation might take 30 minutes to one hour to complete. During the install process it may look like nothing is happening but the install procedure is executing.
The system prompts you to change the default password at the first login attempt. You must set a strong password as per the on-screen instructions to proceed with the application. You cannot run API commands or proceed with any tasks unless you change the default password at the first login. The API commands will return 401 unauthorized error if the default password is not reset.
The Cisco Enterprise NFVIS upgrade image is available as a .nfvispkg file. Currently, downgrade is not supported. All RPM packages in the Cisco Enterprise NFVIS upgrade image are signed to ensure cryptographic integrity and authenticity. In addition, all RPM packages are verified during Cisco Enterprise NFVIS upgrade.
You will see the following message in your browser while the upgrade takes place. Upgrades can take anywhere from 30 - 60 minutes to complete. The host will reboot as well when the upgrade is complete.
NOTE: We will not enable SR-IOV on Eth0-1/Eth0-2 since these are the on-board management interfaces and we recommend not running vMXs from these interfaces (keep management traffic segmented from VM traffic) so we will just clean this up first. We need to delete all other SRIOV networks on the other NICs so we can enable more of the sub-interfaces in the next step.
The Intel X-520 cards support 61 VF's (sub-interfaces) for VMs to connect to. By default, only 4 are enabled in NFVIS. To enable more VF's we first need to disable SR-IOV. Change SRIOV Status to disabled. Click Submit.
Navigate to VM Life Cycle > Networking and click the plus icon () next to SRIOV Networks. In the modal that expands at the top of the page, select the interface which you want to create the SRIOV VF's for and enter the range 1-61 to create all 61 VF's for the interface. Set them to trunk or access mode depending on the desired option (setting to access mode allows you to define a VLAN for this VF) and click Submit.
The package that you upload must be named Meraki.tar.gz just how it was named when downloaded from Cisco. The datastore selected must be datastore1. Deploying the package as a different name or on a different datastore (ex. NFS) will result in an error
The Meraki vMX package must be named Meraki.tar.gz just as how it was when downloaded from Cisco. If the name is changed, the registration will fail and you will see the following error
VIM Error Code for creating image object: 500, VIM Response: "Could not untar: data/intdatastore/uploads/test.tar.gz \ngzip: stdin: not in gzip format\ntar: Child returned status 1\ntar: Error is not recoverable: exiting now\n"
First we need to generate a vMX node inside the Meraki dashboard which we will then instantiate on NFVIS. Begin by creating a new Security Appliance network in your organization. This guide will walk you through creating a new network in the Meraki Dashboard.
Click the new OTHER object and drag a link to the NIC of choice. Click the newly created link and enter the desired SRIOV VF number in the Network Name on the vNIC Details on the right hand modal that opens. The vNIC id can be left as 0 or blank and Model as VIRTIO.
When a vMX first connects to a network it will do so via DHCP unless a static IP config is provided in the user-data. Once a vMX connects to dashboard (step 4 above) then a static IP can be applied through dashboard just as it can with any Meraki product.
NOTE: NFVIS is the only platform that currently supports static network configuration via user-data for the initial vMX provisioning process (pre-dashboard checkin). Public cloud environments such as AWS, Azure, GCP and Alicloud rely on DHCP from their VPC.
Once a vMX has successfully connected to a network, it will then attempt to obtain its user-data (vMX auth token). There are different user-data mechanisms in each platform that the vMX currently runs on to provide the token to the vMX. In AWS, Azure, GCP and Alicloud there are user-data fields in the VM config where this can be provided. In NFVIS we use the user-data mechanism to get this token to the vMX.
NOTE: Unlike the network config above, the token is not displayed on the console for security and usability reasons (the token is a very long string that is meaningless to anyone looking at it). If you see a token value on the console it means that the token was not provided in the format "token " (note that token should be lowercase).
vMX auth tokens have a lifetime of only 1 hour for security purposes. If you see the following message on your vMX console it means the token you provided is no longer valid. Please generate a new one in dashboard, update the Day0 config and restart your vMX. The vMX will attempt to authenticate against dashboard with the provided token 3 times. After 3 failures, the provisioning process stops and the "provisioning failed" message is displayed.
If the vMX is unable to reach dashboard on TCP port 7734 then the initial provisoning phase will fail and an "Unable to reach Meraki Dashboard" message will be displayed on the console. Please refer to this document on the correct ports/IP's that need to be opened for Meraki Dashboard communication.
Contents hide1Install Cisco Enterprise NFVIS2Install NFVIS Through CIMC3Default System Configuration on the Cisco ENCS4Install NFVIS on USC C-Series Servers and CSP Platforms5Default System Configuration on the Cisco UCS C220 M4 Server and Cisco CSP 21006Install NFVIS on UCS-E Series Servers7Default System Configuration on the Cisco UCS E-Series Servers8Install NFVIS Through USB9Documents / Resources9.1References10Related PostsInstall Cisco Enterprise NFVISThis chapter describes how to install Cisco NFVIS through Cisco IMC and USB for the supported hardware platforms.
Step 8
Verify the installation using the System API, CLI, or by viewing the system information from the Cisco Enterprise NFV portal.Step 9
Configure hostname and assign a management IP address to access NFVIS.
Connect ethernet management port to the network for management access. To enable IP address based access over ethernet for NFVIS, use the serial console connection port.
The diagram below illustrates the default network configuration of Cisco Enterprise NFVIS with the Cisco ENCS.
Figure 1: Default Network Configuration of Cisco Enterprise NFVIS with the Cisco ENCS 5400
Note
The following networks and bridges are automatically configured. You can configure more as required.
- A LAN network (lan-net) and a LAN bridge (lan-br)
- A WAN network (wan-net) and a WAN bridge (wan-br) wan2-net and wan2-br are the default configurations for ENCS 5400 and ENCS 5100.
The default networks and bridges cannot be deleted.
Note
Starting from NFVIS 4.6 release, USC C-Series Servers and CSP Platforms support upto 3 RAID groups.
The first raid group is reserved for OS installation and the other RAID groups can be used as external storage drives.
Step 1 Log in to CIMC.
The recommended CIMC version for USC-C Series Servers and Cisco CSP platforms is 3.0(3c) or later version.Step 2
To launch KVM Console, Select Launch KVM from the CIMC homepage.
You can choose Java or HTML based KVM. It is recommnded to use HTML based KVM. Ensure that the pop-up blocker is disabled as KVM Console will open in a separate window.
Figure 4: Default Network Configuration with a Cisco UCS E-Series ServerThe following networks and bridges are created by default, and cannot be deleted. You can configure more as required.
Step 2 Insert USB device in one of USB slot in ENCS5104.
Step 3 Power on system.
Step 4 During system boot up, press F6 key.Press or to enter setup, Boot Menu, Network Boot in 5 seconds or press any key to continue.Step 5 Once you press F6, you will see the following screenshot to select which device you want to boot from. Select your USB device.
In the following screenshot example, there is STEC USB being used. That display will vary depending on your usb device vendor. Use the arrow key to select that device.