Cisco CSR 1000v Deployment Guide for GCE ???

1,323 views
Skip to first unread message

Mark Leonard Reynolds

unread,
Aug 31, 2016, 10:12:40 PM8/31/16
to gce-discussion
Cisco CSR 1000v Deployment Guide for GCE

Is there any such thing?

It seems that the Cisco Cloud Services Router will run on AWS, AZURE, VMware and KVM generally.

Any one successful with this?


We're trying to setup something like Figure 4 on this page.

Any pointers or suggestions welcome!

thanks
Mark

George (Google Cloud Support)

unread,
Sep 1, 2016, 2:02:49 PM9/1/16
to gce-discussion
Hello Mark,

In order to accomplish a similar setup as in Figure 4, you can use Google Cloud Router which enables dynamic Border Gateway Protocol (BGP) route updates between your Google Cloud Platform network and your on-premise network which also works with both legacy networks and Subnetworks.

For the initial release, Cloud Router supports BGP for Cloud VPN only.

You can refer to the VPN Interoperability Guides for more information about setting up a VPN between GCP and other vendors or providers.

I hope this helps.

Sincerely,
George

Mark Leonard Reynolds

unread,
Sep 1, 2016, 7:23:01 PM9/1/16
to gce-discussion
Hi George!

Thanks for responding!

Yes, we had looked at Google Cloud Router, but discarded it as an option, since it did not support DMVPN.

I suspect DMVPN is much quicker than BGP at making routing decisions to remote offices with 'variable' network connectivity.

Any chance you will be adding DMVPN to your Google Cloud Router offering? I suspect not, as it appears to be Cisco proprietary.

Is google _not_ supporting the cloud based Cisco CSV1000V since they are delivery their own cloud router option?

Seems a shame, since the CSR1000V seems to be able to run on nearly every other cloud provider. There seem to be few reasons why it can't be run there either.

regards
Mark

Jesse Scherer (Google Cloud Support)

unread,
Sep 1, 2016, 7:32:03 PM9/1/16
to gce-discussion
Hi Mark,

Cloud Support and former Google datacenter team member here (so I've seen this stuff firsthand).

I don't really know a lot about CSV1000V, but if you're interested in Google's networking technology you might check out this article or the linked paper on Google's production networks. It's fascinating reading, and might help explain our cloud router option.

Also, you mention that you're worried BGP might be slower; have you given it a try? It might be hard to support a whole stack like CSV1000V but if you have applications which perform badly on Cloud Router we'd definitely entertain feature requests to improve perf on those use cases.

Hope this helps (or entertains)!

- Jesse

Ron Royston

unread,
Oct 31, 2016, 3:46:31 PM10/31/16
to gce-discussion
I'm trying to load the Cisco CSR1000v on GCP as well.  I stood up a RHEL 1 vCPU, 0.6 GB VM w assigned public IP and default private IP and uploaded the csr1000v-universalk9.03.11.00.S.154-1.S-std.iso

I was able to mount it with :

[ron@csr1000 ~]$ sudo mkdir /mnt/tmp[ron@csr1000 ~]$ sudo mount -t iso9660 -o loop csr1000v-universalk9.03.11.00.S.154-1.S-std.iso /mnt/tmpmount: /dev/loop0 is write-protected, mounting read-only

then cd'd over to /mnt/tmp but nothing seems to be there...  So, I'm kinda stuck.  I imagine it's my lack of nix skills.  Anyone know how to stand this CSR1000v up on GCP?

[ron@csr1000 ~]$ sudo cd /mnt/tmp[ron@csr1000 ~]$ sudo ls -a. .bash_history .bash_profile csr1000v-universalk9.03.11.00.S.154-1.S-std.iso.. .bash_logout .bashrc .ssh[ron@csr1000 ~]$ dircsr1000v-universalk9.03.11.00.S.154-1.S-std.iso

There appears to be something called virt-install discussed in the Cisco CSR 1000v Series Cloud Services

Ron Royston

unread,
Oct 31, 2016, 6:05:19 PM10/31/16
to gce-discussion
I got a couple of steps closer...

I simply CD'd wrong (via sudo).  Once I did cd /mnt/tmp I saw the boot and boot.catalog files.

So, after some light reading it seems like what to do next is modify the fstab file in /etc/fstab so as to tell the box to boot the CSR (the .iso) upon startup then restart the box.

...Still working on this part.

Ron Royston

unread,
Oct 31, 2016, 7:09:18 PM10/31/16
to gce-discussion
After using sudo su -, I was able to switch to user root.  Once that was done, I could use VI editor to tell the box to boot the CSR .iso.

[root@csr1000 ~]# ls -l /etc/fstab-rw-r--r--. 1 root root 318 Oct 31 20:58 /etc/fstab

this tells me that root owner may read and write a file, while all others may only read the file.  This seems to be sufficient for fstab.  so

vi /etc/fstaband

and once in VI hit i for enter mode and add below with a new line / empty line after it

/home/ron/csr1000v-universalk9.03.11.00.S.154-1.S-std.iso /mnt/tmp iso9660 loop 0 0

Then exit with save
:w
:q

reboot

...and ....nothing.  Still stuck.  So now I'm trying to figure out how to create or use a vSerial port to get into the CSR. 

This is fun!

Ron Royston

unread,
Nov 3, 2016, 10:31:27 AM11/3/16
to gce-discussion
After a couple of days I now accept that it is currently not possible to run the CSR1000v on a Google Compute Engine (VM), linux (including qemu) or windows.  I am moving on to another cloud provider that rhymes with bamazon for my csr1000 instance.  All I really wanted to do is stand up an IOS router/switch for testing/development.

:-(

Virtualization is not enabled in any of the machine which is created under GCE. Currently Google don't have this functionality in their VMs.

On Wednesday, August 31, 2016 at 9:12:40 PM UTC-5, Mark Leonard Reynolds wrote:

Scott Van Woudenberg

unread,
Nov 4, 2016, 1:41:28 AM11/4/16
to Ron Royston, gce-discussion
Hi Ron,

I admit I don't know much about CSR1000v either, but did you discover/try connecting to the interactive serial console of the VM that failed to boot?

It looks like KVM is a supported hypervisor for CSR1000v, which is what GCE uses, so it seems like it should be possible to get it working on GCE...

---
Scott Van Woudenberg
Product Manager
Google Compute Engine


--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-discussion@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussion+unsubscribe@googlegroups.com.
To post to this group, send email to gce-discussion@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/6702f631-cc10-405b-ae6a-00f302763539%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Ron Royston

unread,
Nov 4, 2016, 12:25:44 PM11/4/16
to Scott Van Woudenberg, gce-discussion
Scott, thanks for reaching out.  So, after getting way off in the weeds in a linux instance, I launched a Windows instance and noticed that I could not launch HyperV.  Came to the conclusion that running a VM on a GCP VM is not supported.  So, I went back to the GCP console thinking I should be able to create a Custom virtual instance.  Added the csr1000v.iso to a Google Storage bucket.  However, the custom instance failed to accept the .iso.  It wanted a tar.gz.  So, I converted the .iso to a tar.gz (with IZArc) and that failed too.  What probably needs to happen is one of your linux guru's should take a look at Cisco's published Free CSR1000v doc which points to their download site.  There are 3 flavors - the .ova, .iso, and the .bin.

Surely this should work.

Standing it up on AWS was rather easy - the hardest part was finding the login username which they fail to publish.  The exact steps I published here if you care to glance at it.

Thanks again, Scott.

To post to this group, send email to gce-dis...@googlegroups.com.




--

Scott Van Woudenberg

unread,
Nov 4, 2016, 12:46:28 PM11/4/16
to Ron Royston, gce-discussion
Thank you for the pointers, Ron, especially your detailed steps. I agree that it seems like this should work, so we'll spend some time on this ourselves and update this thread with what we find.

Ron Royston

unread,
Dec 1, 2016, 3:34:43 PM12/1/16
to Scott Van Woudenberg, gce-discussion
After about 40 hours of experimentation (see here for where to send payment) I am getting close, but can't get it all the way stood up.  Here's what I've gotten done.  Looking at GCE documentation, I think what might be missing is an MS-DOS MBR?

1. Create Client Side VM on VirtualBox
    1a. Create a New VM, Select Linux (Other 64bit).
    1b. Set memory to 4096
    1c. Create a New Virtual Disk, Select Advanced > qcow format, Set to 10G
2. Settings
    2a. Select KVM Accelleration
    2b. Deselect floppy
    2c. Disable/Deselect Audio
    2d. Enable Network Interface(s), Select Paravirtualized (vitrio)
    2e. Enable Serial Port
    2f. Click Storage, Click CD, Click CD Icon, select your csr.iso
3. Start VM
    3a. Let it boot up, all the way up.  It restarts itself once.
    3b. Shut down machine.  Exit VirtualBox.
4. Convert Virtual Disk to GCE Allowed Format
    4a. Using VBoxManage, convert the qcow to "disk.raw", e.g., VBoxManage clonehd -format RAW "C:\Users\me\VirtualBox VMs\csr\csr.qcow" "C:\Users\me\VirtualBox VMs\disk.raw"
    4b. Using Cygwin (windows users), create sparse tarball of disk.raw, e.g. tar -cSzf csr1000v.tar.gz disk.raw --format=oldgnu
5. Create GCE Image
    5a. Upload your csr1000v.tar.gz from Step 4 to a Google Storage Bucket
    5b. Create new GCE Image by importing from File (GCE Storage)
6. Create VM
    6a. Create a GCE VM with 4.0G memory
    6b. Choose custom machine > custom image > Select csr1000v.tar.gz
    6c. Enable serial port under metadata with serial-port-enable=1
    6d. Enable IP routing
7. Start GCE VM
    7a. Once the VM is fired up, click it, scroll to bottom Click Serial Port
    7b. Watch the VM fail to boot over and over again with "%IOSXEBOOT-1-BOOTFLASH_FAILED_MISSING: (rp/0): Required Bootflash disk failed or missing, reloading system"

On Fri, Nov 4, 2016 at 12:41 AM, Scott Van Woudenberg <sco...@google.com> wrote:
To post to this group, send email to gce-dis...@googlegroups.com.

Zach Marano

unread,
Dec 1, 2016, 4:57:44 PM12/1/16
to gce-discussion, sco...@google.com
Hi Roy,
I did a little investigation (although its hard to decipher all of the release notes). I believe the image version you are using (3.11S) does not support KVM virtio_scsi drivers. The lack of drivers would produce the errors you are seeing in that there isn't a driver for the kernel to see any disks on GCE. Can you try using either the most recent version 3.17S or at least 3.12S. I believe that 3.12S may have a new enough kernel to work with KVM virtio_scsi (although i can't find decisive information in the release notes). I think you are on the right path with regards to converting the ISO to a bootable GCE image however.
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.

---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussio...@googlegroups.com.

To post to this group, send email to gce-dis...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages