I would like to take this opportunity to share my ideas as well as get
some feedback on some of the key pain points I see for the creation of
common cloud computing reference API or standard.
* Cloud Resource Description
The ability to describe resources is (in my opinion) the most
important aspect of any standardization effort. One potential avenue
might be to use the Resource Description Framework proposed by the
W3C. The Resource Description Framework (RDF) is a family of
specifications, originally designed as a metadata data model, which
has come to be used as a general method of modeling information
through a variety of syntax formats. The RDF metadata model is based
upon the idea of making statements about Web resources (or Cloud
Resources) in the form of subject-predicate-object expressions, called
triples in RDF lingo. This standardized approach could be modified as
a primary mechanism for describing cloud resources both locally and
* Cloud Federation (Cloud 2 Cloud)
The holy grail of cloud computing may very well be the ability to
seamlessly bridge both private clouds (datacenters) and remote cloud
resources such as EC2 in a secure and efficient manor. To accomplish
this a federation standard must be enabled. One of the biggest hurdles
to over come in federation is the lack of clear definition to what
So let me take a stab at defining it.
Cloud federation manages consistency and access controls when two or
more independent geographically distinct clouds share either
authentication, files, computing resources, command and control or
access to storage resources. Cloud federations can be classified into
three categories: peer-to-peer, replication, and hierarchical. Peer 2
peer seems to be the most logical first step in creating a federation
spec. Protocols like XMPP, P4P and Virtual Distributed Ethernet may
make for good starting points.
* Distributed Network Management
The need for a distributed and optimized virtual network is an
important aspect in any multi-cloud deployment. One potential
direction could be to explore the use of VPN or VDE technologies. My
preference would be to use VDE, (Virtual Distributed Ethernet). A
quick refresher, a VPN is a way to connect one or more remote
computers to a protected network, generally tunnelling the traffic
through another network. VDE implements a virtual ethernet in all its
aspects, virtual switches, virtual cables. A VDE can also be used to
create a VPN.
VDE interconnects real computers running (through a tap interface),
virtual machines as well as the other networking interfaces through a
common open framework. VDE supports heterogeneous virtual machines
running on different hosting computers and could be the ideal starting
point. Network shaping and optimization may also play an important
role in the ability to bridge two or cloud resources.
Some network optimization aspects may include;
* Compression - Relies on data patterns that can be represented
* Caching/Proxy - Relies on human behavior , accessing the same
data over and over.
* Protocol Spoofing - Bundles multiple requests from chatty
applications into one.
* Application Shaping - Controls data usage based on spotting
specific patterns in the data and allowing or disallowing specific
* Equalizing - Makes assumptions on what needs immediate priority
based on the data usage.
* Connection Limits - Prevents access gridlock in routers and
access points due to denial of service or peer to peer.
* Simple Rate Limits - Prevents one user from getting more than a
fixed amount of data.
* Memory Management
When looking at the creation of compute cloud memory tends to be a
major factor in the performance of a given virtual environment,
whether a virtual machine or some other application component. Cloud
memory management will need to involve ways to allocate portions of
virtual memory to programs at their request, and freeing it for reuse
when no longer needed. This is particularly important in "platform as
a service" cloud deployments.
Several key memory management aspects may include;
* Provide memory space to enable several processes to be executed
at the same time
* Provide a satisfactory level of performance for the system users
* Protect each program's resources
* Share (if desired) memory space between processes
* Make the addressing of memory space as transparent as possible
for the programmer.
* Distributed Storage
I've been working on creating a cloud abstraction layer called "cloud
raid" as part of our ElasticDrive platform and have been looking at
different approaches for our implementation. My initial idea is to
connect multiple remote cloud storage services (S3, Nirvanix, CloudFS)
for a variety of purposes. During my research the XAM specification
began to look like the most suitable candidate. XAM addresses storage
interoperability, information assurance (security), storage
transparency, long-term records retention and automation for
Information Lifecycle Management (ILM)-based practices.
XAM looks to solve key cloud storage problem spots including;
* Interoperability: Applications can work with any XAM conformant
storage system; information can be migrated and shared
* Compliance: Integrated record retention and disposition metadata
* ILM Practices: Framework for classification, policy, and implementation
* Migration: Ability to automate migration process to maintain
* Discovery: Application-independent structured discovery avoids
Potential Future Additions to the API
The virtualization of I/O resources is a critical part of enabling a
set of emerging cloud deployment models. In large scale cloud
deployments a recurring issue has the ability to effectively
management I/o resources whether on a machine level or network. One of
the problems a lot of users are encountering is that of the "nasty
neighbor" or a user who has taken all available system I/o resources.
A common I/o API for sharing, security, performance, and scalability
will need to be addressed to help resolve these issues. I've been
speaking with several hardware vendors on how we might be able to
address this problem. This will most like have to be done at a later
point after a first draft has been released.
* Monitoring and System Metrics
One of the best aspects of using cloud technology is the ability to
scale applications in tandem to the underlying infrastructure and the
demands placed on it. Rather then just scaling on system load, users
should have the ability to selectively scale on other metrics such as
response time, network throughput or other metrics made available.
Having a uniform way to interact with system metrics will enable cloud
providers and consumers a common way to scale applications.
Security & Auditability.
In my conversations with several wall street CIO's the questions of
both security and cloud transparency with regards to external audits
has come up frequently.
My list of requirements is by no means a complete list. Cloud
computing encompasses a wide variety of technologies, architectures
and deployment models. What I am attempting to do is address the
initial pain points whether you are deploying a cloud or just using
it. A lot of what I've outlined may be better suited to a reference
implementation then a standard, but none the less I thought I'd put
these out ideas out for discussion.
(Original Post: http://elasticvapor.com/2008/08/standardized-cloud.html)
- Most of the resources requiring description via Cloud Resource Description
are computer system resources that are already well described via CIM. Any
standards proposal that does not base itself on such a widely accepted
resource model would need significant technical justification.
- The W3C appears to be pretty committed to WS-Federation. Again, why not
leverage an existing standard to build a Cloud Federation standard on?
- Your use of terminology for "Distributed Network Management" is more about
implementation than management. The established standard for network
management is SNMP and the most widely accepted management standard for
managing remote systems is WBEM which is easily extended to a cloud via
WS-Man. Using a VPN to implement a secure network makes perfect sense,
however, there is a wide variety of alternatives within the VPN umbrella. A
key starting point would be to establish whether to use Layer 2 or Layer 3
tunnels. Layer 2 (e.g. L2TP) is more complex but has the advantage of
creating a clear channel for IP. Layer 3 (e.g. TLS) is trendy these days
but is problematic when tunnelling TCP unless UDP is used to create the
Layer 3 tunnel (as in OpenVPN).
VDE is indeed appealing because it presents a virtual network to virtual
machines and uses a Layer 2 approach (e.g. Slirp is based on PPP) without
security to interconnect physical servers. That would work well in a cloud
that is not distributed across the Internet; e.g. a data-center resident
facility that offers cloud services to the internet. But federation implies
that a cloud standard should support distributed members, and the virtual
channels between those members would need to be secure since relying on
security at the virtual network level would not guarantee the integrity of
the interconnecting channels. There is no reason why VDE could not be mapped
onto another Layer 2 implementation, but that work has not yet been done by
the open source team. A good choice would be L2TP/IPSec tunnel mode for
- Your list of API elements should also include resource scheduling and
reservation. There are well-established grid standards here to leverage.
- As per my previous posts on the topic, cloud security/auditibity is
critical to mainstream acceptance by customers. Closely tied to this is a
management interface standard that can be used by external customers to
obtain metrics and audit information. Again, WS-Man is a well established
candidate for this purpose.
In summary, I think your idea will gain the most support if it is anchored
in a set of existing and well accepted standards.
I would love an introduction to the folks at the W3C. I've also been
speaking to some of the guys at IEEE about similar cloud
standardization efforts. As part of this, I've been asked to join the
Program Committee for International Workshop on Cloud Computing being
held with the IEEE International Symposium on Cluster Computing and
the Grid (CCGRID 2009) during May 18-21, 2009, in Shanghai, China. I
hope to unveil our efforts at this event. >
I've also got some other standard related activities in the works, but
I'm not ready to publicly say what, just yet.
I'm also not totally sold on the whether or not cloud computing is
ready for a cloud standard just yet. What I do think we need is a
reference implementation (Platform & Infrastructure) and common
extensible API. "CloudVirt" This API may someday form the basis for a
standard, but in the mean times gives us a uniform API to work
against., so whether you're using Google App Engine or Force.com,
GoGrid or EC2, Nirvanix or S3, you'll have a central point of
programmatic contact. I personally don't want to have to rewrite my
platform for every new cloud providers API., which is exactly what
we're doing now.
I just don't see the big clouds being motivated towards a standard at this point (unless, of course, it's their own standard). makes it too easy for customers to escape. Someone will make a killing tying disparate clouds together and this will probably morph into the "standard".
The South bound interface of this mid-layer API will change (as more
"serious and substantial" cloud providers (anything comparable to
AWS), appear on the horizon) for some more time to come.
But large majority of the cloud users can be protected from these
changes by adding cloud-provider specific plug-ins at the bottom of
this normalized mid-layer.
i.e. creating a provider for each distinct cloud-provider.
An important hurdle will still remain until and unless there is a
convergence into a single data-model paradigm. If the Cloud Providers
each use their own data model, providing a uniform North-bound API
will still be a big hassle and will not be worth doing.
For want of anything better, I would throw in my vote for CIM for this
uniform data model (if anyone will listen :-))
Ruv et al,
We need standardization of some sort. We can draw parallels of the current state of cloud computing to the early days of networking - independent islands of clouds with little interoperability, no standards to speak of and proprietary management interfaces. Naturally as the domain matures (or in other words, for the domain to mature) it will follow a path that will unify the control and management plane.
In most of these cases, what we need is a declarative deployment and programmable model. How the underlying infrastructure implements them is out of the cloud consumers’ hands. For example we might say I need round robin between these instances and the load balancer figures out how. And when we add more instances or delete an instance, the load balancer should know what to do, without doing a CLI for every change. Most probably this is what we mean by Cloud standards.
Would the locus and trajectory of the cloud computing take us through things like inter-cloud IM(XMPP, anyone?), peering standards, gossip and P2P protocols, facilities for classless VM migration across a hierarchy consisting of VMs-Clouds-Cloud Federation and so forth ? Time will tell.
From my blog http://doubleclix.wordpress.com/2008/08/21/the-standard-cloud/. For some reason few of my e-mails got eaten by Google. May be they will show up along with the single socks that I lost in my washing machine ;o)
Alas, most recent standards committees have turned good efforts into mush.
That is because most players participating in those standards committees
have only 1 objective: Preserve their own competitive advantage and
posture as leaders.
Take WS-I Basic Profile, how many years wasted to achieve what?
Interoperable request/reply over HTTP, with XML as a payload?
When pundits were busy shoving SOAP and the rest of the alphabet soup down
Amazon's throat, Amazon decided to go with REST and other less politicized
approaches. It served them well.
I believe they'll welcome standards when said standards will prove to be
more than crippling tools "designed" by wanting latecomers.
(a bit too harsh and vitriolic...sorry)
Taking the datacenter paradigm to the cloud is a fundamental error in
my opinion. The cloud isn't a datacenter and I'm not sure you want it
to be. Besides which, if you look at where the datacenter seems to be
headed, it's more towards a cloud model. Large pools of commodity
hardware, largely throw away. The datacenter is in the same position
as Big Iron in the late 90s. It's about to expire as a model for
In the future we'll see a clean schism between the physical datacenter
(run by completely different folks) and the 'cloud' (internal or
external) where virtualization is the enabling technology (server,
storage, and network virtualization) to completely disassociate the
folks who run the cloud from the hardware and the datacenter itself.
They will just run pools of resources. You can see this happening
every where right now.
> IMHO, the base API needs to start with virtual datacenter resources
> covering virtual servers, virtual networks, and virtual storage. It's
> easy to go too deep and too broad. I would resist getting into
> anything running on the guests. I'd like to see an API that allows a
> user to create a "Virtual Private Datacenter" including multiple
> network zones, vpns, virtual NAS & SAN interconnected with virtual
> hosts of various capabilities.
I think this brings a lot of older datacenter-centric ideas to the
table. Network zones and VLANs are out (too limiting). VDE and
similar tech will be in. If you want scale, then old-school polling-
based monitoring is out and distributed instrumentation is in. If you
want storage, NAS/SAN are out, and block devices on tap are in.
Let me give you an example for why this matters. NAS failover (NFS or
iSCSI) is painful or impossible to do properly. But once virtualized
and presented as a block device on demand, a la Amazon EBS, the
integration issues and management for the individual servers go away.
Now it's just a reliable piece of hardware that shows up when you want
it as a 'real' device.
> Some day we can come back and create a CIM profile that somewhat
> matches the API created in libcloud. Who knows, someone may even want
> to author some a CML library in SML, but open source software is the
> place to start IMHO ;-)
I agree. Open source is the place to start and de facto standards
will emerge. My big thing is that I think datacenter-centric notions
are fundamentally flawed. We need new paradigms for the cloud.
Let's not mix up a technology concept (CIM) with a process concept
We have to accept that the resources in the cloud and resources in the
users local data center will coexist for a long time (most likely for
So far CIM has well in standardizing storage management (SMIS). CIM is
also being used in SMASH (for both physical and virtual) servers.
It will help speed up the process (and will not be SLOOOOW) if we do
not spend time in devising a new modeling paradigm for Cloud resorces
(just because we can) which will be different from the modeling
paradigm for the local resources and then spend even more time trying
to integrate the two.
To take your example further, the SimpleDB/AppEngine models are unnecessarily simple and significantly limited, so even if they converged I would question the value of standardizing on primitive patterns. For example, I am working with some new algorithm technologies at the moment that can index-organize arbitrarily high-dimensional spaces on massive clusters, which translated means that something resembling conventional relational models can be supported on Google-scale clusters plus some additional features that are currently lacking in conventional systems as well (like scaling of geospatial datasets). While you could standardize on the trivial "distributed B-tree" model the above data stores use, I think it would be premature given how fast many of these models are being obviated by better technology.
A reality is that existing cluster technology sucks pretty badly in many cases but it is being rapidly improved, so standardizing on patterns predicated on medieval and poorly suited technology will serve us poorly over the long-term. At the very minimum these standards will get subverted by de facto standards that reflect what people are actually doing, which has killed more than a couple standards.
In selecting standardization targets, you generally want a target that will not significantly change or become obsolete in the near term. I think some cloud computing aspects *are* pretty static, but like your apparent impression of cloud databases, that may reflect my naive understanding of the state of the technology. While it would be convenient if all of this was standardized, that convenience will not return value if the target is constantly moving. At this stage, I suspect just about every aspect of cloud technology is a moving target.
Developing a standard is not only getting it standard, but realizing
what you've got to do.
What is the target audience? The cloud users? Or providers?
2008/8/21 Reuven Cohen <r...@enomaly.com>:
What is the competency of a federation? Why would a cloud user care if
there is a federation or a single provider?
> So let me take a stab at defining it.
> Cloud federation manages consistency and access controls when two or
> more independent geographically distinct clouds share either
> authentication, files, computing resources, command and control or
> access to storage resources. Cloud federations can be classified into
> three categories: peer-to-peer, replication, and hierarchical. Peer 2
> peer seems to be the most logical first step in creating a federation
> spec. Protocols like XMPP, P4P and Virtual Distributed Ethernet may
> make for good starting points.
> * Distributed Network Management
> The need for a distributed and optimized virtual network is an
> important aspect in any multi-cloud deployment. One potential
> direction could be to explore the use of VPN or VDE technologies. My
> preference would be to use VDE, (Virtual Distributed Ethernet). A
> quick refresher, a VPN is a way to connect one or more remote
> computers to a protected network, generally tunnelling the traffic
> through another network. VDE implements a virtual ethernet in all its
> aspects, virtual switches, virtual cables. A VDE can also be used to
> create a VPN.
Is this for the cloud user to use to shape their own network? Should
they be aware of the mapping of virtual hardware to physical hardware?
Will they be able to define a network without knowing this mapping?
Otherwise, looks like you are "standardising" the wrong thing about
the cloud, i.e. the implementation.
Isn't this addressed by hardware and VMs? Why should a cloud user be
bothered about this?
I would tend to agree with you. All the pieces are to be coming
together for cloud computing.
I would we are in the trough of two elephants.