The Hybrid Cloud Multiverse (IPv6 VLANS)
Reuven Cohen
His post actual was a kind of epiphany which lead me to think that one of the great things about cloud computing is in its ability to virtualize everything. The cloud is a kind of "multiverse" where the rules of nature can continually be rewritten using quarantined virtual worlds within other virtual worlds (aka virtualization). The need for a traditional physical piece of hardware is no longer a requirement or necessary.
For example VLANs don't need to differentiate between IPv4 and IPv6; the deployment is just dual-stack, as Ethernet is without VLANs. So why not just use modern VLAN technology to "overlay" IPv6 links onto existing IPv4 links? This can be achieved without needing any changes to the IPv4 configuration and allows for seamless and secure cloud networking while also allowing for all the wonders that IPv6 brings. It's in a sense the best of both worlds, the old with the new.
A VLAN based IPv6 overlay offers several interesting aspects such network security that is directly integrated into the design of the IPv6 architecture. (Security being one of the biggest limitations to broader cloud adoption) IPv6 also implements a feature that simplifies aspects of address assignment (stateless address autoconfiguration) and network renumbering (prefix and router announcements) when changing Internet connectivity providers. It's almost like the designers of IPv6 envisioned the hybrid cloud model.
Thanks for the inspiration Hoff, looking forward to trying this out.
Reuven Cohen
CCIF Instigator
David Bernstein (daberns)
Reuven and all
Interesting post on the idea that IP addressing is sort of broken when it comes to Clouds. A lot of people in my company have been thinking about this (needless to say). I am going to paraphrase some thoughts and write-ups we’ve had in this space. Although extending Layer 2 as widely as possible solves a lot of problems, it doesn’t solve the general “private to public” or “public to public” problem. You always get back to routing in the capital-I Internet.
It all starts with the fact that, in a highly virtualized environment, IP address space explodes. Everything has multiple IP addresses; servers have IP addresses for management, for the physical NICs, for all of the virtual machines and the virtual NIC therein, and if any virtual appliances are installed they have multiple IP addresses as well.
Several areas are of concern here, on the one hand, the IPv4 address space simply starts to run out. Consider an environment inside the Cloud which has 1M actual servers. As explained above, assuming a 16 core server, each server could have 32 VM’s, and each VM could have a handful of IP addresses associated with it (virtual NICs, etc). That could easily explode to a Cloud with well over 32M IP addresses. Even using Network Address Translation (NAT), the 24-bit Class A reserved Private Network Range provides a total address space of only 16M unique IP addresses!
For this reason many Cloud operators are considering switching to IPv6 which provides for a much larger local address space in the trillions of unique IP addresses. Switching to IPv6 is quite an undertaking, and some believe that switching from one static addressing scheme to another static addressing scheme (eg IPv4 to IPv6) might not be the right approach in a large highly virtualized environment such as Cloud Computing. If one is reconsidering addressing, one should consider the Mobility aspects of VMs in Cloud.
VM Mobility provides for new challenges in any static addressing scheme. When one moves a running VM from one location to another, the IP address goes with the running VM and any application runtimes hosted by the VM. IP addresses (of either traditional type) embody both Location and Identity in the IP address, eg, routers and switches use the form of the IP address not only to identify uniquely the endpoint, but by virtual of decoding the address, infer the Location of the endpoint (and how to reach that endpoint using switching and routing protocols). So while an addressing scheme is being reconsidered, let’s consider two schemes which embody Mobility.
You might think that Mobile IPv4 and Mobile IPv6 mechanisms can be used in this case. Because IP addresses in either case are still provider-supplied and follow top level address allocations, we still find VM mobility issues when a VM attempts more general mobility from one Cloud provider to another for example.
In an attempt to completely generalize the addressing solution, a completely dynamic scheme where Location and Identification have been separated has been developed. This new scheme is called Location Identity Separation Protocol (LISP). LISP based systems can interwork with both IPv4 and IPv6 based networks, through protocol support on edge routers. However, internal to a Cloud, which may in itself span several geographies, LISP addressing may be used.
The basic idea behind the Loc/ID split is that the current Internet routing and addressing architecture combines two functions: Routing Locators (RLOCs), which describe how a device is attached to the network, and Endpoint Identifiers (EIDs), which define “who” the device is, in a single numbering space, the IP address. Proponents of the Loc/ID split argue that this “overloading” of functions places the constraints on end-system use of addresses that we detailed. Splitting these functions apart by using different numbering spaces for EIDs and RLOCs yields several advantages, including improved scalability of the routing system through greater aggregation of RLOCs. To achieve this aggregation, we must allocate RLOCs in a way that is congruent with the topology of the network. EIDs, on the other hand, are typically allocated along organizational boundaries.
Because the network topology and organizational hierarchies are rarely congruent, it is difficult (if not impossible) to make a single numbering space efficiently serve both purposes without imposing unacceptable constraints (such as requiring renumbering upon provider changes) on the use of that space. LISP, as a specific instance of the Loc/ID split, aims to decouple location and identity. This decoupling will facilitate improved aggregation of the RLOC space, implement persistent identity in the EID space, and hopefully increase the security and efficiency of network mobility.
Although LISP isn’t in routers yet, it is alive and open, and we think it may be just what the doctor ordered for the IP addressing ‘challenge’ in Clouds. Sorry for the long post, but IP addressing is something near and dear to us around here.
- David Bernstein
(acknowledgements to Dino F, Doug G, and the whole Cisco LISP team)
Gary Mazzaferro
All those IPv4 routers need to be upgraded..
How does lisp handle(d) inter-map domain handover ?
How does lisp handle multiple connected (together) mobile end points ?
BTW, bravo -- a very slick design !
-gary
> and Mobile IPv6 <http://www.ietf.org/rfc/rfc3775.txt> mechanisms can
> provider-supplied and follow top level address allocations, we still
> find VM mobility issues when a VM attempts more general mobility from
> one Cloud provider to another for example.
>
> In an attempt to completely generalize the addressing solution, a
> completely dynamic scheme where Location and Identification have been
> separated has been developed. This new scheme is called Location
> Identity Separation Protocol
> <http://gforge.info.ucl.ac.be/projects/openlisp>, and we think it may
David Bernstein (daberns)
Yes, it's a software upgrade for edge routers, indeed.
On the questions, check out http://www.lisp4.net/ which has good links
to docs and tutorials.
David.
Diego Parrilla Santamaría
So probably their experience can help us to shape these new 'Cloud Multiverse'
Regards
Diego
Gary Mazzaferro
I have a question, "when an agent is mobilized and then resumed, what
type of impact will it have on network switches ? "
If we start juggling around VMs, containers and the like, each with at
least one IP address it may adversely effect the network. Lets say you
have 1,000 of these guys moving around in 5 seconds. I'm assuming the
switches won't be too happy. Some may even route traffic incorrectly.
There will also be a bunch of NDP traffic. If its an issue, it may be a
show stopper for effervescent mobile agents (very short life cycles).
Then very short life cycle mobile agents would only be able to be
deployed within the context of a single image cluster.
gm
Diego Parrilla Santamaría wrote:
> I think some of the members of the list have already mentioned it, but
> this is the same problem you can find in the massive deployment of
> connected devices. That's the problem that Mobile Operators are
> dealing with when deploying M2M (machine-to-machine) services. With
> IPv4 this can be a nightmare of VLANs, Firewalls, NATs and all kind of
> networking 'challenges'. With IPv6 you can avoid some of these
> networking 'challenges', but at these moment, as far as I know only
> Japanese Mobile operators are using IPv6 for M2M deployments.
>
> So probably their experience can help us to shape these new 'Cloud
> Multiverse'
>
> Regards
> Diego
>
>
> On Wed, Feb 11, 2009 at 4:42 AM, Reuven Cohen <r...@enomaly.com
>
> Christofer Hoff
> <http://rationalsecurity.typepad.com/blog/2009/02/incomplete-thought-support-of-ipv6-in-cloud-providers.html>
> of the cloud providers (IaaS, PaaS) support IPv6 natively or
> support tunneling without breaking things like NAT and firewalls?
> As part of all this Infrastructure 2.0 chewy goodness, from a
> networking (and security) perspective, it's pretty important."
>
> His post actual was a kind of epiphany which lead me to think that
> one of the great things about cloud computing is in its ability to
> virtualize everything. The cloud is a kind of "multiverse
Tom Deckers
Mike DiPetrillo
just bridging everything together on a flat layer 2 network. Either
that or re-write the networking stack and the applications that use
anything over the network. It's very important that the IP address and
MAC address follow the service that's using them.
This is very similar to conversations we have with customers doing DR
today. It's relatively easy to move data today. It's even easy to move
the VMs (they go with the storage). The last trick has always been how
to get to the VM once it's moved. A lot of times the IP addressing is
completely different in the recovery space than the production site.
For some apps it's easy to just change the IP address and then rely on
global DNS load balancing to direct the user to the new location of
the app. For other apps (Websphere, SAP, etc) it can be almost
impossible to quickly change the IP address. Ironically these are the
apps that are sometimes the most critical in a DR environment. These
are also the same apps that I could see being targeted for moving in
and out of provider clouds during heavy use periods (think retail or
financial at the end of the year or quarter).
I don't think we should really be trying to boil water before we get
the pot on the stove. We need to figure out what this whole cloud
taxonomy looks like first and then we can start to troubleshoot the
issues with gluing the pieces together. There are a lot of issues
we'll come across and they're the typical ones - what happens with
software licensing, what happens to networking, what happens to
storage and in particular databases, what about compliance, etc. These
are the usual roadblocks in cloud and actually in hosted DR as well.
This is an interesting conversation and all but back to the basics of
the problem.
-Mike
Drue Reeves
1. Getting a cloud definition
2. Creating a diagram that we can use for a taxonomy
All of us have burning questions we want to ask/solve, but trying to solve every cloud question without some type of cloud framework AND organizational structure is probably impossible.
Like Mike said "back to basics of the problem".
Drue
-----Original Message-----
From: cloud...@googlegroups.com [mailto:cloud...@googlegroups.com] On Behalf Of Mike DiPetrillo
Sent: Wednesday, February 11, 2009 7:14 AM
To: Cloud Computing Interoperability Forum (CCIF)
Subject: Re: The Hybrid Cloud Multiverse (IPv6 VLANS)
scott radeztsky
already semi-settled:
Have we defined the space so we can show how a scenario or use case (and
its associated constraints) fits into the whole?
like RUP or SunTone, talking about a particular subset of an
architecture (e.g., "the scalability of the HW layer in the database
tier") helps build up a taxonomy of the whole, while also allowing a
very detailed conversation and set of decisions about each particular
element.
what is our "cube"? one thought:
axis 1: type: private vs. public vs. hybrid cloud
axis 2: layer: SaaS, PaaS, IaaS
axis 3: domain (known optimizations of HW/SW): HPC, Analytics,
Healthcare, Telco
constraints will help reduce the multiple scenarios in our spectrum
where one size will not fit all. even in a particular place in the cube
there are options:
- app space contributing: my app can use dynamic elements and
rediscover some things or it has hardwired/hardcoded things
- infra space contributing: my infra contains dynamic things that can
help apps or it does not (DDNS, identity)
- NW space contributing: interop between the various cloud types,
high rates of shuffling vs. lower, etc
there are pieces of technology and typical optimizations and places in /
pieces of the cube where solns are perhaps simpler, and places where
things are still not known and therefore must remain more abstract or
academic in out definitions.
what do people think about breaking this out into focussed chunks while
still acknowledging the whole?
Scott.
Scott Radeztsky, Ph.D.
Sun Principal Engineer
Chief Technologist, Americas Systems Engineering
Sun Microsystems, Inc.
(312) 952-6761 cell
* Customer First * Integrity * Respect for the Individual *
* Teamwork * Financial Success * Openness * Fun *
Patrick J Kerpan
Here is an opportunity to answer and experiment with this question yourself.
You can do so by beta testing our (CohesiveFTs) latest version of VPN-Cubed. We are looking for BETA testers for an EC2-specific packaging of our overlay network product "VPN-Cubed for EC2".
One of the neat artifacts of using a hybrid virtual switch/router/bridge/vpn like this is the use of cryptographic credentials mapped to IP addresses.
It lets you do things like this (lets suspend state management questions for now):
- In a small time window (the length of a executing a couple line bash script), you revoke "certificate 6" which is mapped to 172.31.1.40 in the overlay network and actually sitting at 10.x.x.something at EC2 USA, and invoke certificate 6 which is mapped to 172.31.1.40 on an identical virtual machine which is sitting at EC2 EU at 10.x.x.somethingelse. Within seconds any of the devices that used to talk to VM 1 at EC2 USA - now seamlessly talk to VM 2 at EC EU. (or wherever the end nodes of the overlay network are)
THAT SAID...give it a try - email me to try it or "@reply elasticserver" or @reply pjktech" to follow up.
Cheers
pat k