The Universal Amazon EC2 API Adapter (UEC2)
Reuven Cohen
conversations in regards to standardizing cloud based API's. As you
know I am a big proponent of the concept of semantic cloud
abstraction. Our Unified Cloud Interface project (UCI) has attracted
more then 350 members in a little over a month. Before I get into my
latest scheme, I want to assure you I still feel that a singular cloud
abstraction interface that can encompass the entire infrastructure
stack as well as emerging cloud centric technologies through a
semantic application interface is truly the future of cloud
computing. We hope to have a functional UCI demo ready for
presentation at the upcoming Wall Street Interoperability Forum, so
stay tuned for more news on that front.
I'm also realistic, most users who have deployed to the cloud have
written their applications specifically for the Amazon Web Service API
, making it currently the De facto standard. So it occurred to me,
that a potentially big opportunity might be to create an open
universal EC2 API adapter / abstraction layer (UEC2). Unlike
EUCALYPTUS, the EC2 API adapter can work with your existing
infrastructure tools and is completely platform agnostic.
At the heart of this concept would be a universal EC2 abstraction,
similar to ODBC, a platform-independent database abstraction layer.
Like ODBC a user could install the specific EC2 api-implementation,
through which a cloned EC2 API is able to communicate with traditional
virtual infrastructure platforms such as VMware using the standardized
EC2 API. The user then has the ability to have their EC2 specific
applications communicate directly with any infrastructure using this
EC2 Adapter. The adapter then relays the results back and forth
between the the other various infrastructure platforms & API's.
I admit the downside of a universal EC2 abstraction layer is the
increased overhead to transform statements into constructs understood
by the target management platforms.
The Universal EC2 API adapter complements our current unified cloud
interface efforts because in a sense it is a logical inverse. Where
UCI is a semantic representation for all API's, (One API to Rule them
all) the EC2 API is very specific to an infrastructure as a service
environment. The EC2 adapter could easily utilize UCI as an
interchange format allowing for a one to many deployment methodology.
An EC2 abstraction layer will reduce the amount of developer work by
providing a consistent API . To put it another way, rather then coming
at the problem from the top down, your coming at from the bottom up
with UCI in the middle.
To be clear I don't have the time or resources to make this project
happen myself, between my various cloud advocacy efforts and a new
baby, I'm totally overwhelmed. So I'd like to propose we crowd source
this idea. Make it an open source project governed by an enterprise
friendly open source license such as BSD.
If others think this is a good idea, I created a UEC2 Google Group,
(http://groups.google.com/group/UEC2) please go ahead and signup.
Next we round up some sponsors (Starting with CloudCamp / CCIF
Sponsors), put together a Google code project and jointly get to work.
If it's a bad idea, well just ignore my ramblings.
Comments please.
--
Reuven Cohen
CCIF Instigator
tluk...@exnihilum.com
>> "their applications specifically for the Amazon Web Service API"
You yourself just ran Salesforce's $1 Billion a year business up the flagpole! In the first quarter of 2008 alone they signed up nearly 3,000 new companies - and it never stops. And how about Intuit's and Oracles and Caspio's 10's of thousands of Cloud customers? Or all of the other Cloud/XaaS vendors that you ought to be aware of?
I have a few ideas of why you might have made such a statement, but I'd like to hear you defend this pseudo-fact that you present as a reason to buy into all of the rest of your statements?
TL
Reuven Cohen
<tluk...@exnihilum.com> wrote:
>>> "most users who have deployed to the cloud have written"
>>> "their applications specifically for the Amazon Web Service API"
>
> Holy Cow, Reuven.. do you just make this stuff up as you go along?
>
> You yourself just ran Salesforce's $1 Billion a year business up the
> flagpole! In the first quarter of 2008 alone they signed up nearly 3,000 new
> companies - and it never stops. And how about Intuit's and Oracles and
> Caspio's 10's of thousands of Cloud customers? Or all of the other
> Cloud/XaaS vendors that you ought to be aware of?
>
> I have a few ideas of why you might have made such a statement, but I'd like
> to hear you defend this pseudo-fact that you present as a reason to buy into
> all of the rest of your statements?
>
> TL
>
>
Sure, a salesforce API abstraction is cool too.
And yes, I do make this stuff up as I go.
r/c
Matthew Zito
I was more curious as to why we neeeded an EC2 abstraction layer, since there's already an API for EC2, and libraries for it for the major development languages I'm aware of.
Perhaps I'm missing the goal.
Thanks,
Matt
--
Matthew Zito
Chief Scientist
GridApp Systems
P: 646-452-4090
mz...@gridapp.com
http://www.gridapp.com
-----Original Message-----
From: cloud...@googlegroups.com on behalf of Reuven Cohen
Sent: Thu 3/5/2009 2:58 PM
To: cloud...@googlegroups.com
Subject: Re: The Universal Amazon EC2 API Adapter (UEC2)
Reuven Cohen
>
> I was more curious as to why we neeeded an EC2 abstraction layer, since
> there's already an API for EC2, and libraries for it for the major
> development languages I'm aware of.
>
> Perhaps I'm missing the goal.
>
> Thanks,
> Matt
>
> --
> Matthew Zito
> Chief Scientist
> GridApp Systems
> P: 646-452-4090
> mz...@gridapp.com
> http://www.gridapp.com
>
>
All the tools and libraries for EC2 are specific to the Ec2 cloud.
What if you wanted to take your application which was built to scale
against the EC2 API and move it somewhere else?
r/c
David Bernstein (daberns)
Hi Folks
I think what Reuven was trying to say is, there are a lot of folks who are developing various tools, environments, and applications on top of AWS. Some use just EC2, some use EC2 and S3, some use more of the API’s that AWS offers like SimpleDB and so on.
Please consider tools like Rightscale, Elastra, Hyperic and many others.
Other cloud operators have other APIs such as GoGrid, ECP, and VMware. Again not an exhaustive list.
The applications which use EC2 and S3 and (…) run only on AWS. Suppose you want to build your own cloud and run EC2 and S3 apps. Well your only option today is to use Eucalyptus. And I can tell you from personal experience this is not an easy solution. I am sure Eucalyptus will improve, but then Eucalyptus requires the Xen hypervisor. Maybe your company has a stated direction to use VMware. You are stuck. Or you wanted to run your AWs application on GoGrid.
All of these point to the usefulness of a layer, which on the top looks like as uch of AWS as we can implement, and underneath can clue onto a variety of clouds as long as that cloud has some way to issue a VM and some way to support storage mdels which can be mapped to S3.
Having UEC2 would enable you to do this.
Of course, maybe the AWS programming model is not our perfect choice for a “common API”, maybe AWS does not want it to be used in this way (maybe they do however). So there are those open issues.
Irrespective, I believe that this would be of great interest to developers as portability layers always are. And to the platform vendors who have access now to a bunch of applications intended for AWS, now they can run them, that is good for platform vendors.
David Bernstein, Cisco.
tluk...@exnihilum.com
If you're making a joke of Salesforce's "Cloud" because you don't think it has the stature of, say Amazon's Cloud, then I don't believe that you can ever be the thought leader that you'd like to be while harboring such un-varnished biases.
If you're not and are serious, then let me just say that CCIF isn't existing to judge the "coolness" of one Cloud vs. another, but to advocate for those business that will put their success and continuity in the hands of a Cloud vendor, which we can only do by mitigating, reducing and minimizing the risk of developing and hosting business critical applications in the Cloud.
>> "And yes, I do make this stuff up as I go."
Reuven Cohen
You are not the first and probably not the last to question my bias. I
certainly don't expect everyone to agree with my ideas.
If you're interested, I've previously posted an overview of my
opinions & biases.
Cloud Interoperability and The Neutrality Paradox
http://groups.google.com/group/cloudforum/browse_thread/thread/e4dce7e0df1e046b
As for the word coolness, I suppose I could use a more suitable
adjective next time, no sarcasm was intended.
r/c
Srinivas Vedula
I don't see Amazon cloud as the mainstream yet. It is just one of the first movers. Do you think we need to concentrate on it specifically? Sure we can use the experience in coming up with the Amazon abstraction to create the actual generic abstraction definition and API but that is the only reason I can think of to go down this path. But the flip side is for an interoperability group we will be pushing for one model as the default :)
Am I missing something?
- Srinivas
Reuven Cohen
> Reuven,
>
> I don't see Amazon cloud as the mainstream yet. It is just one of the first
> movers. Do you think we need to concentrate on it specifically? Sure we can
> use the experience in coming up with the Amazon abstraction to create the
> actual generic abstraction definition and API but that is the only reason I
> can think of to go down this path. But the flip side is for an
> interoperability group we will be pushing for one model as the default :)
>
> Am I missing something?
>
> - Srinivas
>
> On Thu, Mar 5, 2009 at 12:24 PM, David Bernstein (daberns)
> <dab...@cisco.com> wrote:
>>
>> Hi Folks
>>
>>
Your totally right, this is why I'm focusing on the UCI model. The EC2
abstraction concept is just an alternative way to look at the problem.
r/c
Geva Perry
EUCALYPTUS? The fact that Eucalyptus only supports Xen at this point,
and not VMWare, seems irrelevant. That is where they are at this point
in time and I am sure if it catches on and evolves it will add
additional environments. This proposed project won't start by
supporting everything under the sun from day one either.
Plus, an intiative like this may pull the rung from under the feet of
under important vendors. TL gave a few good examples. There are others
like GoGrid who open sourced their API and are trying to organize
other cloud vendors, such as Rackspace/Mosso to support it.
On Mar 5, 12:58 pm, Reuven Cohen <r...@enomaly.com> wrote:
Rafat Alvi
If you are really talking about cloud ( including saas/paas ) then the statement below is inaccurate.
Perhaps the statement should have been rewritten as
"most users who have deployed to an IaaS cloud have written their applications specifically for the AWS API."
The assessment resting on top of eucalyptus then makes sense.
Also: does the AWS API offer anything more than management of the instances ( and of course S3 calls? )
Rafat
>> "most users who have deployed to the cloud have written"Holy Cow, Reuven.. do you just make this stuff up as you go along?
>> "their applications specifically for the Amazon Web Service API"
You yourself just ran Salesforce's $1 Billion a year business up the flagpole! In the first quarter of 2008 alone they signed up nearly 3,000 new companies - and it never stops. And how about Intuit's and Oracles and Caspio's 10's of thousands of Cloud customers? Or all of the other Cloud/XaaS vendors that you ought to be aware of?
I have a few ideas of why you might have made such a statement, but I'd like to hear you defend this pseudo-fact that you present as a reason to buy into all of the rest of your statements?
TL
- Show quoted text -
tluk...@exnihilum.com
I read the document that you referred me to, which concluded nicely with the statement:
>> "The first step in creating a fair and balanced interoperable cloud ecosystem is "
>> "to in fact state our biases and take steps to offset them by including a broad "
>> "swath of the greater cloud community, big or small, vendor, analyst or journalist."
This would be a step in the right direction, however I'd lobby for enlarging it to also include stating our individual perspectives on the problems of 'Cloud Interoperabilty' and 'Portability across Clouds', and clearly and simply describing the "Acceptance Criteria" that could (from our POV) be used as a means of accurately recognizing successful and therefor "acceptable" candidate solutions.
In fact, I'd bet that this exercise itself would do a fairly good job of identifying (exposing?) individual biases even if there were no other explicit effort made to do so. IMHO, most people are not aware of or in touch with their biases enough to enumerate them for CCIF anyway. (Excepting of course, you Reuven ;-)
It's really not much to ask of members, for example:
My perspective - the one I'm adopting and advocating for in my work here within CCIF - is of someone who's developing, deploying and maintaining both enterprise and situational end-user applications in a Cloud/XaaS environment.
My acceptance criteria would be whether or not a proposed solution could and would materially reduce or minimize the likelihood of a business facing catastrophic costs, total rewrites, or serious business disruption in the event that it needed to relocate one or more of it's XaaS applications from one Cloud to another - for any reason.
See simple, easy, not complicated or technical. But having this basic awareness of each other as individuals is useful both as a way to better understand where each other's posts are coming from, and eventually as a source of raw metrics for quantifying and qualifying our progress.
There's a lot more that I could say about both the technical and social aspects of CCIF, but I'd bet that everyone's heard quite enough from me today.
TL
-----Original Message-----
From: "Reuven Cohen" [r...@enomaly.com]
Date: 03/05/2009 03:53 PM
To: cloud...@googlegroups.com
Subject: Re: The Universal Amazon EC2 API Adapter (UEC2)
>
> remark, but since IMHO in either case it demands a response, let me give it
> a shot.
>
> If you're making a joke of Salesforce's "Cloud" because you don't think it
> has the stature of, say Amazon's Cloud, then I don't believe that you can
> ever be the thought leader that you'd like to be while harboring such
> un-varnished biases.
>
> If you're not and are serious, then let me just say that CCIF isn't existing
> to judge the "coolness" of one Cloud vs. another, but to advocate for those
> business that will put their success and continuity in the hands of a Cloud
> vendor, which we can only do by mitigating, reducing and minimizing the risk
> of developing and hosting business critical applications in the Cloud.
>
>>> "And yes, I do make this stuff up as I go."
>
> Kudos for the honesty.. but it would help if you didn't.
>
> TL
>
>
>
>
>
> From: "Reuven Cohen" [r...@enomaly.com]
> Date: 03/05/2009 02:59 PM
> To: cloud...@googlegroups.com
> Subject: Re: The Universal Amazon EC2 API Adapter (UEC2)
>
>
> <tluk...@exnihilum.com> wrote:
>>>> "most users who have deployed to the cloud have written"
>>>> "their applications specifically for the Amazon Web Service API"
>>
>> Holy Cow, Reuven.. do you just make this stuff up as you go along?
>>
>> You yourself just ran Salesforce's $1 Billion a year business up the
>> flagpole! In the first quarter of 2008 alone they signed up nearly 3,000
>> new
>> companies - and it never stops. And how about Intuit's and Oracles and
>> Caspio's 10's of thousands of Cloud customers? Or all of the other
>> Cloud/XaaS vendors that you ought to be aware of?
>>
>> I have a few ideas of why you might have made such a statement, but I'd
>> like
>> to hear you defend this pseudo-fact that you present as a reason to buy
>> into
>> all of the rest of your statements?
>>
>> TL
>>
>>
>
>
Tony Lucas
work in this area, and has definately already established it's place,
and I really can't see the difference with what Ruv is proposing.
We should concentrate on trying to help push existing projects on
further, rather than fragment things even more.
Regards,
Tony Lucas
Chief Executive Officer
XCalibre Communications Ltd / FlexiScale
www.flexiscale.com