The Business of Building Clouds

[Reuven Cohen]

There is an old saying in the venture capital world that consulting doesn't scale. As an entrepreneur I'm continually walking the line between making the short term buck (consulting revenue) versus the long-tail (recurring revenue on product based licensing and support). Given our platform is open source, consulting is typically a major part of our revenue model. The dilemma is a fairly straight forward one. I'm in business to make money, in our case, from as many different opportunities as possible.

Lately it seems everyone is in need of assistance with their clouds, from architecture, setup and deployment there seems to be real need for the "Cloud Consultant". For us these jobs range from your dedicated hosting firms and large telecoms looking to create EC2 like utilities to software & traditional enterprises looking to deploy their new "as a service" offerings in a scalable way. A lot of people talk about the cloud killing the traditional system administrator's job, but in my opinion there has never been a better time to working in IT. Those so see this paradigm shift toward cloud computing will prosper.

Defining what cloud computing is in itself a tough job, the lack of common cloud methodologies and best practices is making the job even harder. Trying to find experienced people with knowledge on how to build out a 30,000 machine cloud is nearly impossible, finding someone who's deployed hundreds is proving to be almost as difficult.  We the pioneers in the cloud computing space must take steps to create an open development ecosystem, one where we share our failures and successes so others can learn the trade.

One way may be to create a common cloud specification. David Young over at Joyent, attempted to do this, he has called for a common cloud specification called "Cloud Nine". In his modest proposal, he calls for an open specification based on nine core components.

1) Virtualization Layer Network Stability

2) API for Creation, Deletion, Cloning of Instances

3) Application Layer Interoperability

4) State Layer Interoperability

5) Application Services (e.g. email infrastructure, payments infrastructure)

6) Automatic Scale (deploy and forget about it)

7) Hardware Load Balancing 

8) Storage as a Service

9) "Root", If Required

Although I'm not sure about the need for root access or hardware based load balancing his post raises some interesting ideas. In particular he says "a developer should be able to move between Joyent, the Amazon Web Services, Google, Mosso, Slicehost, GoGrid, etc. by simply pointing the "deploy gun" at the cloud and go." I think he nailed it dead on with this statement. 

At the end of the day our job as cloud builders is about creating simplicity and making IT easier to manage and faster to scale.

--------
(Original Post: http://elasticvapor.com/2008/06/business-of-building-clouds.html)

Cloud Nine: Specification for a Cloud Computer. A Call to Action.
http://www.joyeur.com/2008/05/08/cloud-nine-specification-for-a-cloud-computer-a-call-to-action

--
--

Reuven Cohen
Founder & Chief Technologist, Enomaly Inc.
www.enomaly.com :: 416 848 6036 x 1
skype: ruv.net // aol: ruv6

blog > www.elasticvapor.com
-
Get Linked in> http://linkedin.com/pub/0/b72/7b4

[James Urquhart]

Reuven,

I'm not sure I agree with you or David.  Here is my problem: different platforms fill different needs in different spaces.  Google AppEngine is intentionally a completely different animal than Amazon EC2/S3.  So "pointing the deploy gun" makes no sense whatsoever *unless* you build a AppEngine clone on EC2/S3.  But then you are just building a new stack.  Do we say everyone must develop to the lowest common denominator?  In that case, SmugMug, et. al. are wildly out of compliance, as it is clear you need to develop in python to Google's libraries to make the "point and deploy" scenario work between these two providers.

Look, that kind of compatibility between any two vendors serving the same layer of David's model makes some sense to me.  However, what we have today is a primordial ooze of technologies, target markets and business models.  I think we need a lot more time before the dominant complex life forms evolve, and the kind of portability mentioned here is "boil the ocean" complex.  Let's focus on getting portability within platform "solar systems" and worry about connecting those systems once we conquered that.

James

[Stuart Charlton]

> Cloud Nine: Specification for a Cloud Computer. A Call to Action.
> http://www.joyeur.com/2008/05/08/cloud-nine-specification-for-a-cloud-computer-a-call-to-action

I think there are a number of assumptions here that are worth
discussing.

- "Cloud computers must operate on some sort of virtualization
technology for many of the following features to even be feasible."

Really? Why can't it just work on pre-installed capacity? Blade
arrays have been doing this for many years.

Certainly virtualization is important long term, but, in the open
systems world at least, it's still _very_ new in the data centre.
What happens if enterprises just aren't ready to adopt VMWare or Xen
for a variety of technical or political reasons? To paraphrase what
one person here asked, "will you trust your hypervisor more than your
OS?"

- "API for Creation, Deletion, Cloning of Instances"

I agree here, though I could see this as contentious, depending on
whether people disagree on what an 'instance' is'.

BTW, wasn't this the point of the Open Virtual Machine Format (OVF)
from VMWare, Xen, etc? [ Of course, it's yet not ratified by the DMTF
to my knowledge. Oh, and also, DMTF... not exactly a well known
standards body, yet. ]

- "Application Layer Interoperability"

It'll be interesting to see if user-packaging standards start to form
here, like WAR files. Or wouldn't a tarball be enough for many of
these frameworks?

- "State Layer Interoperability"

There are billions of dollars thrown at interoperability, semantics,
and integration annually by both vendors and customers, flowing like a
river into a tarpit of epic proportions. A large neon sign hangs
over this tarpit that says "Abandon All Hope, Ye Who Enter Here."

There is some hope, of course - cue talk of the Semantic Web. It's
still too soon to tell what pieces remain to make it work, but a
standard XMPP-based information bus is probably low on the list of
priorities -- it's too specific a solution. (The technical challenges
in financial markets don't apply to every industry). I'd say data
model interoperability probably is a more important nut to crack.

- "Application Services".

Yup, I agree here. But It'll be hard-fought street fighting to get
the standards in place over the next few years.

- Automatic Scale (deploy and forget about it)

Funny, I keep looking for the "fast=true" parameter in my
infrastructure, and I still haven't found it.

Joking aside, I do agree that 'adding & removing resources on demand'
should be a feature of how one provisions cloud infrastructure. (Not
coincidentally, that's one of Elastra's features in our preview
release). Though I'll go back to our conversation on performance
management a couple of weeks ago: many times, the best way to scale
is to fix the application.

Barring that, the cloud would have to detect and rewrite the
developer's shitty code for them. This isn't completely out of the
question, btw -- recent Oracle releases can rewrite a developers' SQL
to perform better, and a DBA can choose to "swizzle" the statement at
runtime without requiring a code change. They added this feature so
shops could workaround the crappy code in many packaged applications
that somehow never seem to be fixed.

"- Hardware Load Balancing"

This seems very much aimed at web application hosting. While clearly
important, that would be pretty limiting, especially if we're trying
to shift the enterprise onto the cloud. I'd go further and suggest
that the network capacity and (virtual) topology itself needs to be a
cloud service.

"8) Storage as a Service"

WebDAV?! Why not AtomPub?

On another note, has anyone seriously considered iSCSI? Certainly we
can't think of storage just as web resources -- there also need to be
mount points as a service, like Amazon EBS. Or is this just a pipe
dream?

Cheers
Stu Charlton
Elastra

[Larry Ludwig]

 

In particular he says "a developer should be able to move between Joyent, the Amazon Web Services, Google, Mosso, Slicehost, GoGrid, etc. by simply pointing the "deploy gun" at the cloud and go." I think he nailed it dead on with this statement.   

Hi Reuven,

I've read that article also,  I too think it would be great to move service between different providers.  I think reality will set in how different each cloud provider will be.   Yes I think you will find some application to convert between providers, but I also think that will be the key differentation between providers. One provider will only have feature X, while another provider will only have feature Y.  In order for each cloud provider to exist, they has to be some barrier of entry from other providers and also a method to not make it too easy for customers to leave.  Otherwise why would a cloud provider exist and offer service?  Geolocation of the cloud isn't enough of a reason.

-L

--
Larry Ludwig
Empowering Media
1-866-792-0489 x600
Managed and Unmanaged Xen VPSes
http://www.hostcube.com/

[Khazret Sapenov]

Larry,

In theory most cloud vendors(given they offer considerably homogenuos services) should have some common set of features, that comply to some protocol. I would compare it to webservers using HTTP. So what Joyent proposes is something like HTTP (GET/PUT/DELETE etc) for cloud computing. 

--
Khaz Sapenov,
Director of Research & Development
Enomaly Labs

US Phone: 212-461-4988 x5
Canada Phone: 416-848-6036 x5
E-mail: kh...@enomaly.net
Get Linked in> http://www.linkedin.com/in/sapenov

[Michael Moran]

Khaz,

 

By HTTP Get/Put/Delete/etc., you mean RESTful web services support?

 

Thanks,

-Michael

[Khazret Sapenov]

Michael,

Not exectly, I would draw a parallel between START/STOP/PAUSE and GET/PUT/DELETE (in a sense of HTTP's simplicity), so I'm just making assumptions on possible match. Not sure if this is valid or viable comparison, goal was just to convey my viewpoint of Joyent's suggestion.

 

KS

[Ray Nugent]

This question is for all the platform folks out there. Just out of curiosity, how many of your customers are asking for these portability features? Is there really a pent up demand to move apps from one cloud platform to another?

Ray

[Khazret Sapenov]

Ray,

I guess this is nice-to-have feature to manage risk of cloud provider going down (just one possible scenario).

 

KS

[arculeo]

Ray,

As a client perspective, I really like the fact to be able to compare
cloud features to what I am doing without the cloud. I take the example
of being able to say

before : how many HTTP request handling in GET/POST
after : how many HTTP request handling in GET/POST

before : how many SQL request handling in SELECT/INSERT
after : how many SQL request handling in SELECT/INSERT

I can compare a "cost per scalable INSERT" and evaluate what is better
for me. Not to move from cloud X to cloud Y but to able to move from
non-cloud to cloud and understand the offer in terms I already use with
our present platform.

Adrien

Ray Nugent a écrit :

> This question is for all the platform folks out there. Just out of
> curiosity, how many of your customers are asking for these portability
> features? Is there really a pent up demand to move apps from one cloud
> platform to another?
>
> Ray
>
> ----- Original Message ----
> From: Larry Ludwig <larr...@gmail.com>
> To: cloud-c...@googlegroups.com
> Sent: Thursday, June 5, 2008 5:44:17 AM
> Subject: RE: The Business of Building Clouds
>
>
>
> In particular he says "a developer should be able to move between
> Joyent, the Amazon Web Services

> <http://finance.google.com/finance?q=amzn>, Google
> <http://finance.google.com/finance?q=goog>, Mosso, Slicehost,

[Sassa NF]

How do you move apps from the cloud that went down?.. ;-)

Sassa

2008/6/5 Khazret Sapenov <sap...@gmail.com>:

[Tony Lucas]

There is two problems here, firstly (and most simply), Cloud providers
providing an API interface that does allow this degree of control. Some
do at the moment, some don't, but I'm sure it's something most will do
in time. The bigger problem is making them work to a common standard,
which is unlikely to happen (and that is a good and a bad thing). I
would imagine this is where people like RightScale, CohesiveFT, Enomaly,
Elastra etc come in. They are providing a management layer in one way
or another over cloud providers, and thus can be independant of them.

We at FlexiScale fully support the adoption of open standards and
interoperability, but I can't see any provider making it ridiculously
easy to move away, as it is a danger to their revenue model (not that
I'm a fan of lock in problems at all).

Some people would also say it's a benefit to the revenue model, as it
enables people to start using these services with less of a risk, what
if provider x goes bust/stops providing services/doubles pricing etc/has
major technical issues etc, and I can certainly see their point, and is
a question we have been asked on a number of occasions.

Just my 2c

Look forwarding to seeing a lot of you at Structure 08.

Regards,

Tony Lucas
Chief Executive Officer
XCalibre Communications Ltd / FlexiScale
www.flexiscale.com

> E-mail: kh...@enomaly.net <mailto:kh...@enomaly.net>

> Get Linked in> http://www.linkedin.com/in/sapenov
> On Thu, Jun 5, 2008 at 8:44 AM, Larry Ludwig
> <larr...@gmail.com <mailto:larr...@gmail.com>> wrote:
>
>
>
> In particular he says "a developer should be able to

> move between Joyent, the Amazon Web Services
> <http://finance.google.com/finance?q=amzn>, Google
> <http://finance.google.com/finance?q=goog>, Mosso,

[Khazret Sapenov]

Adrien,

This might depend on specific configuration of hardware layer and configuration of cloud (in many cases virtualized environment). There's always a tradeoff in performance (around 10-15%), when using hypervisor, but this also might be optimized by using eficient drivers, written for known hardware (I/O is pretty crucial one). However virtualization brings very important features, not present in barebone layout.

 

KS

[Khazret Sapenov]

you don't do it after event, but prepare for it.

[Ignacio Martin Llorente]

Hi,

In the context of the RESERVOIR project (http://www.reservoir-
fp7.eu/), one of the main aims is to define an API specification to
access both an infrastructure site (cloud system) from, for example, a
service management layer and to federate infrastructure sites. The
promotion of this specification as standard will be done in the
context of the OGF WG on Grid and Virtualization chaired by SAP and
IBM. The first deliverable with the architecture of the infrastructure
and a brief description of the APIs will be released in few days.

Regards,
--
Ignacio M. Llorente, Full Professor (Catedratico): web http://dsa-research.org/llorente
and blog http://imllorente.dsa-research.org
DSA Research Group: web http://dsa-research.org and blog http://blog.dsa-research.org
Globus GridWay Metascheduler: http://www.GridWay.org
OpenNebula Virtual Infrastructure Engine: http://www.OpenNebula.org

[Mike Culver]

James is on the money, IMHO. Building a spec is about like building a standardized spec for intergalactic space travel. You don't even know what life form will appear in front of you when coming out of hyperdrive, so how can you build a spec before travel is required?

________________________________

From: cloud-c...@googlegroups.com on behalf of James Urquhart
Sent: Wed 6/4/2008 10:15 PM
To: cloud-c...@googlegroups.com
Subject: Re: The Business of Building Clouds

Reuven,

I'm not sure I agree with you or David. Here is my problem: different platforms fill different needs in different spaces. Google AppEngine is intentionally a completely different animal than Amazon EC2/S3. So "pointing the deploy gun" makes no sense whatsoever *unless* you build a AppEngine clone on EC2/S3. But then you are just building a new stack. Do we say everyone must develop to the lowest common denominator? In that case, SmugMug, et. al. are wildly out of compliance, as it is clear you need to develop in python to Google's libraries to make the "point and deploy" scenario work between these two providers.

Look, that kind of compatibility between any two vendors serving the same layer of David's model makes some sense to me. However, what we have today is a primordial ooze of technologies, target markets and business models. I think we need a lot more time before the dominant complex life forms evolve, and the kind of portability mentioned here is "boil the ocean" complex. Let's focus on getting portability within platform "solar systems" and worry about connecting those systems once we conquered that.

James

----- Original Message ----
From: Reuven Cohen <r...@enomaly.com>
To: cloud-computing <cloud-c...@googlegroups.com>
Sent: Wednesday, June 4, 2008 2:54:06 PM
Subject: The Business of Building Clouds

There is an old saying in the venture capital world that consulting doesn't scale. As an entrepreneur I'm continually walking the line between making the short term buck (consulting revenue) versus the long-tail (recurring revenue on product based licensing and support). Given our platform is open source, consulting is typically a major part of our revenue model. The dilemma is a fairly straight forward one. I'm in business to make money, in our case, from as many different opportunities as possible.

Lately it seems everyone is in need of assistance with their clouds, from architecture, setup and deployment there seems to be real need for the "Cloud Consultant". For us these jobs range from your dedicated hosting firms and large telecoms looking to create EC2 like utilities to software & traditional enterprises looking to deploy their new "as a service" offerings in a scalable way. A lot of people talk about the cloud killing the traditional system administrator's job, but in my opinion there has never been a better time to working in IT. Those so see this paradigm shift toward cloud computing will prosper.

Defining what cloud computing is in itself a tough job, the lack of common cloud methodologies and best practices is making the job even harder. Trying to find experienced people with knowledge on how to build out a 30,000 machine cloud is nearly impossible, finding someone who's deployed hundreds is proving to be almost as difficult. We the pioneers in the cloud computing space must take steps to create an open development ecosystem, one where we share our failures and successes so others can learn the trade.

One way may be to create a common cloud specification. David Young over at Joyent, attempted to do this, he has called for a common cloud specification called "Cloud Nine <http://www.joyeur.com/2008/05/08/cloud-nine-specification-for-a-cloud-computer-a-call-to-action> ". In his modest proposal, he calls for an open specification based on nine core components.

1) Virtualization Layer Network Stability

2) API for Creation, Deletion, Cloning of Instances

3) Application Layer Interoperability

4) State Layer Interoperability

5) Application Services (e.g. email infrastructure, payments infrastructure)

6) Automatic Scale (deploy and forget about it)

7) Hardware Load Balancing

8) Storage as a Service

9) "Root", If Required

Although I'm not sure about the need for root access or hardware based load balancing his post raises some interesting ideas. In particular he says "a developer should be able to move between Joyent, the Amazon Web Services <http://finance.google.com/finance?q=amzn> , Google <http://finance.google.com/finance?q=goog> , Mosso, Slicehost, GoGrid, etc. by simply pointing the "deploy gun" at the cloud and go." I think he nailed it dead on with this statement.

At the end of the day our job as cloud builders is about creating simplicity and making IT easier to manage and faster to scale.

--------
(Original Post: http://elasticvapor.com/2008/06/business-of-building-clouds.html)

Cloud Nine: Specification for a Cloud Computer. A Call to Action.
http://www.joyeur.com/2008/05/08/cloud-nine-specification-for-a-cloud-computer-a-call-to-action

--
--

Reuven Cohen
Founder & Chief Technologist, Enomaly Inc.

www.enomaly.com <http://www.enomaly.com/> :: 416 848 6036 x 1
skype: ruv.net <http://ruv.net/> // aol: ruv6

blog > www.elasticvapor.com <http://www.elasticvapor.com/>

[Stuart Charlton]

On 5-Jun-08, at 7:35 AM, Khazret Sapenov wrote:

> Larry,
> In theory most cloud vendors(given they offer considerably
> homogenuos services) should have some common set of features, that
> comply to some protocol. I would compare it to webservers using
> HTTP. So what Joyent proposes is something like HTTP (GET/PUT/DELETE
> etc) for cloud computing.
> --

How about "Just HTTP", and work on media type(s) for cloud
resources? That's the approach that we're taking.

Cheers
Stu Charlton
Elastra

[Reuven Cohen]

I should note I'm note actually for David's nine points. In a lot of ways it is impossible to cover everyone's needs in 9 basic components. He seems to have found a catchy marketing gimic in his "cloud nine" and then tried to fit nine components into it.  For example his spec does little to define chargeback / metering requirements or common cpu measurements. What he does do a good job of is explaining some of the major pain points he has encountered within the context of his "cloud " service.

I also think an attempt at a specification is better then nothing at all. At the very least it give us something to discuss.

ruv

www.enomaly.com :: 416 848 6036 x 1
skype: ruv.net // aol: ruv6

blog > www.elasticvapor.com

[arculeo]

Khazret,

I fully agree with you, that is the reason why I said I am comparing
what I could do "by hand" on my own with what a vendor can do, I compare
everything on one question "price per scalable X".

Means

1/ Price (the lowest being a single machine with a very specialised
monitoring but not scalable)
2/ Scalable (how close to linear is the price/performance curve?)

For now I admit that I saw linear curves only in companies that are more
expensive that doing myself.

So I am wondering if I could re-align my question from "cost per linear
performance over scale" to "cost per linear performance for one
particular piece of infrastructure over scale" and explode the
infrastructure into multiple smaller vendor to take into account.

Vendors have the problem to answer all infrastructure questions, I have
the problem to answer only one complex infrastructure question but fine.

Adrien

Khazret Sapenov a écrit :

[Chandra Shekhar Tekwani]

There is another angle to “building clouds” that I am seeing…..

 

1. Enterprises consolidated data centers and implemented virtualization or at least that is the plan they are working with.

 

2. They are running into space and power limitations so they are putting them up in separate buildings in same area (metro area).

 

3. These metro area networks are connected by fiber (and copper) but then they have trouble implementing layer 3 networks so they want a flat layer 2 architecture.

 

4. The layer 2 architecture with virtualization (primarily VMware) requires Virtual center and Cisco Enterprise manager to work together.

 

5. The other part is that they want to put VLANs over the same fiber and have the fiber equipment separate the traffic (somehow) using DWDM or other technologies.

 

Now this is an entirely different angle. Does this mean VMware will have to get an infrastructure partner that provides optical ether switching equipment?

 

Is any one else seeing this trend/need?

 

Chandra

 

---

<br

[randall]

Anyone building business applications will be asked about moving their
system out of the cloud. It's one of the first questions I'm asked
about our platform. Public clouds are fine for consumer apps and
public websites, but the majority of businesses want to keep their
information behind their firewall.

We need standardization to move from cloud to cloud or to simply move
out of the cloud.

The comparison of cloud computing and electrical utilities is a good
one and applicable in many senses, including standardization, but
somewhere in that analogy we are forgetting there is no information in
electricity.

- randall

On Jun 5, 9:47 am, Ray Nugent <rnug...@yahoo.com> wrote:
> This question is for all the platform folks out there. Just out of curiosity, how many of your customers are asking for these portability features? Is there really a pent up demand to move apps from one cloud platform to another?
>
> Ray

> In th

[Reuven Cohen]

The article got picked up on sys-con 
http://virtualization.sys-con.com/read/584130.htm

ruv

[Alan Ho]

Picking a provider that has data center failover is critical - but it does mean that you write your application in a way that can failover gracefully. Cloud providers need to provide the base infrastructure to do so OR constrain the user to a particular programming paradigm (like the limitations of google app engine)

I expect in the future that cloud computing systems will provide the concept of "cloud events" in case of major datacenter failures. I just don't see any way round it.

Regards,
Alan Ho

---

From: Khazret Sapenov <sap...@gmail.com>
Sent: June 05, 2008 8:18 AM
To: cloud-c...@googlegroups.com
Subject: Re: The Business of Building Clouds

you don't do it after event, but prepare for it.

On Thu, Jun 5, 2008 at 10:59 AM, Sassa NF <sassa.nf@gmail.com> wrote:

How do you move apps from the cloud that went down?.. ;-)


Sassa

2008/6/5 Khazret Sapenov <sap...@gmail.com>:

> Ray,
> I guess this is nice-to-have feature to manage risk of cloud provider going
> down (just one possible scenario).
>

> KS
>

> On Thu, Jun 5, 2008 at 10:47 AM, Ray Nugent <rnu...@yahoo.com> wrote:
>>

>> This question is for all the platform folks out there. Just out of
>> curiosity, how many of your customers are asking for these portability
>> features? Is there really a pent up demand to move apps from one cloud
>> platform to another?
>>
>> Ray
>>
>> ----- Original Message ----
>> From: Larry Ludwig <larr...@gmail.com>
>> To: cloud-c...@googlegroups.com
>> Sent: Thursday, June 5, 2008 5:44:17 AM
>> Subject: RE: The Business of Building Clouds
>>
>>
>>
>> In particular he says "a developer should be able to move between Joyent,

>> the Amazon Web Services, Google, Mosso, Slicehost, GoGrid, etc. by simply

[Khazret Sapenov]

Adrian,

Now I see what you mean, thanks for explaining.

I think you also have to consider more dimensions like time, quality, elasticity etc, breaking out of price/scale viewpoint.

While certain hardware layout can show linear scalability at cetain point, introducing additional nodes might take time and overall degradation. Simple example is one of our clients (large insurance company), a request to upgrade RAM takes 2-3 months :).

 

KS

On Thu, Jun 5, 2008 at 11:28 AM, arculeo <arc...@arculeo.com> wrote:

[Stuart Charlton]

Hi James,

On 4-Jun-08, at 10:15 PM, James Urquhart wrote:

  Google AppEngine is intentionally a completely different animal than Amazon EC2/S3.  So "pointing the deploy gun" makes no sense whatsoever *unless* you build a AppEngine clone on EC2/S3.  But then you are just building a new stack.  Do we say everyone must develop to the lowest common denominator? 

 I'd turn it around and suggest that perhaps its greatest strength is also its greatest problem.  It's a "Big Monolithic Black Box".    Now, in my experience, black boxes are quite valuable, in they focus you on "getting things done" at a usable level of granularity.   But sometimes you need to be able to see what's behind the curtain to diagnose problems, or to change the way things work a bit.   Hopefully it's well layered or abstracted enough that you don't have to dig into a rat's nest to get anything done.     

It could be that Google wants to be in the "full stack support outsourcing business", but given how laborious that can be, I'd bet they would want to automate a lot of that and turn many of the options into self-service controls.   

Look, that kind of compatibility between any two vendors serving the same layer of David's model makes some sense to me.  However, what we have today is a primordial ooze of technologies, target markets and business models.  I think we need a lot more time before the dominant complex life forms evolve, and the kind of portability mentioned here is "boil the ocean" complex. 

Well, not really, given there's already an OVF draft spec out there that covers a big chunk of what would be useful, though it's virtualization-centric.   

 I would expect every instance-level cloud provisioner out there will have some kind of spec or API published now or in short order, leading to lots of silo's and a need for interoperability.

In short, sometimes one has to be naive and try to drive agreement where the domain is well-understood, like provisioning *nix instances.    This does, of course, make one take pause and ask in hushed tones,  "what about Microsoft?" ;-)

On the other hand, I do think compatibility at the application state layer is likely a tarpit.

Let's focus on getting portability within platform "solar systems" and worry about connecting those systems once we conquered that.

I'm not quite sure what you mean by that.

Cheers

Stu

[Sassa NF]

:-) There is no problem to move the services... the problem is to move
the service state = resource + data.

but I like the view that the standard would allow interop of clouds,
i.e. super-cloud or federation as a business model? So to speak, buy
data as a service from Amazon, buy search as a service from Google,
and make them work together. However, for this to make sense, there
must be a task that the other clouds cannot solve sufficiently well
alone, or where the integrity level of a cloud is not sufficient
(integrity level here being like the level of detail seen in the
Universe). This is what drove the Grid standardisation efforts to
where they are now.

Someone already complained here that the Grid standardisation effort
is not heading in the direction of applied IT. But this is exactly the
reason why: do we see the same scale of the problems in applied IT as
seen in the academic community? (use CERN Hydron Collider as the data
provider, and NESC Condor pool as the CPU provider for my simulations)

Sassa

2008/6/5 Khazret Sapenov <sap...@gmail.com>:

> you don't do it after event, but prepare for it.
>

[Khazret Sapenov]

which is basically clouds mashup, isn't it ?

On Thu, Jun 5, 2008 at 11:57 AM, Sassa NF <sassa.nf@gmail.com> wrote:

:-) There is no problem to move the services... the problem is to move
the service state = resource + data.

but I like the view that the standard would allow interop of clouds,
i.e. super-cloud or federation as a business model? So to speak, buy
data as a service from Amazon, buy search as a service from Google,
and make them work together. However, for this to make sense, there
must be a task that the other clouds cannot solve sufficiently well
alone, or where the integrity level of a cloud is not sufficient
(integrity level here being like the level of detail seen in the
Universe). This is what drove the Grid standardisation efforts to
where they are now.

Someone already complained here that the Grid standardisation effort
is not heading in the direction of applied IT. But this is exactly the
reason why: do we see the same scale of the problems in applied IT as
seen in the academic community? (use CERN Hydron Collider as the data
provider, and NESC Condor pool as the CPU provider for my simulations)

Sassa

2008/6/5 Khazret Sapenov <sap...@gmail.com>:
> you don't do it after event, but prepare for it.
>

[Ignacio Martin Llorente]

What we try to do is to provide infrastructure on-demand for the
execution of any service. The interface provides functionality for the
submission and control of service, where a service is a set of VMs
with requirements and rules for affinity, interdependencies... So, the
infrastructure provider manages the execution of the VMs according
infrastructure policies that could eventually delegate the execution
of the VM to a remote infrastructure site.

Regards

--
Ignacio M. Llorente, Full Professor (Catedratico): web http://dsa-research.org/llorente
and blog http://imllorente.dsa-research.org
DSA Research Group: web http://dsa-research.org and blog http://blog.dsa-research.org
Globus GridWay Metascheduler: http://www.GridWay.org
OpenNebula Virtual Infrastructure Engine: http://www.OpenNebula.org

[Stuart Charlton]

On 5-Jun-08, at 8:35 AM, Alan Ho wrote:

Picking a provider that has data center failover is critical - but it does mean that you write your application in a way that can failover gracefully. Cloud providers need to provide the base infrastructure to do so OR constrain the user to a particular programming paradigm (like the limitations of google app engine)

That's a very astute observation, Alan.    Constraining an architecture to induce certain properties (guarantees?) is likely the right approach.  

 Though I wonder if AppEngine is a bit too "Nanny-ish" that limit its audience in ways that don't really impact the big picture qualities.   

For example, the choice of Python was easy because it was a standard Google language, but that doesn't seem to be inherently a more applicable language than say C#, Java or Ruby.

I expect in the future that cloud computing systems will provide the concept of "cloud events" in case of major datacenter failures. I just don't see any way round it.

I wonder if Google actually provides this sort of failover for AppEngine today.   Certainly, they could, though they provide no such guarantees at the moment.

As for "cloud events" - yup.   In the traditional data centre, it's likely SNMP or JMX traps.   On the cloud, it's not entirely clear if/where SNMP would play.   Or WS-Man.   Or something newer (?).

Cheers

Stu

[Sassa NF]

...or grids. :-)

Sassa

2008/6/5 Khazret Sapenov <sap...@gmail.com>:

> which is basically clouds mashup, isn't it ?
>

[Michael Moran]

Ignacio,

 

So then does that mean the language used to write-apps-for your cloud would be irrelvant because the infrastructure provider would handle that detail by providing multi-language support (say, via VMs that support one or more languages, or by routing any language-specific call to the appropriate sub-cloud that supports that language in question)?

Thanks,

Michael

[wyim wyim]

In regards to failover, does Google App Engine have some sort of a LoadBalancer API?
 
thanks
Wayne Yim

---

From: stuartc...@gmail.com

To: cloud-c...@googlegroups.com
Subject: Re: The Business of Building Clouds

Date: Thu, 5 Jun 2008 09:07:12 -0700

[Reuven Cohen]

From what I've seen of Google App Engine, they distribute your python code to dozens of servers and then use some kind of round robin to spread the load. Nothing ground breaking.

r/c

[Andre Merzky]

Quoting [Stuart Charlton] (Jun 05 2008):
>
> BTW, wasn't this the point of the Open Virtual Machine Format (OVF)
> from VMWare, Xen, etc? [ Of course, it's yet not ratified by the DMTF
> to my knowledge. Oh, and also, DMTF... not exactly a well known
> standards body, yet. ]

Out of curiosity, what consider people on this list to be a
relevant standards body flor cloud related technologies?

DMTF? IETF? OASIS? OGF? ...

Thanks, Andre.

--
Nothing is ever easy.

[Larry Ludwig]

>
>
> > Cloud Nine: Specification for a Cloud Computer. A Call to Action.
> >
>
http://www.joyeur.com/2008/05/08/cloud-nine-specification-for-a-cloud-comput
er-a-call-to-action

The intresting thing I think is missing is the need for private/secure
communications. Yes you can do it via software but why not have dedicated
hardware for ipsec communcation?

[Larry Ludwig]

> The comparison of cloud computing and electrical utilities is a good
> one and applicable in many senses, including standardization, but
> somewhere in that analogy we are forgetting there is no information in
> electricity.

In some instances it is. You can't take a US lamp and plug it into a UK
outlet with out a converter. So just like utilities worldwide there will be
different standards.

[Larry Ludwig]

> I expect in the future that cloud computing systems will provide the
concept
> of "cloud events" in case of major datacenter failures. I just don't see
any way round it.

Forgive the pun but couldn't they be called "storms"?

[Greg Pfister]

On Jun 4, 4:54 pm, "Reuven Cohen" <r...@enomaly.com> wrote:
> There is an old saying in the venture capital world that consulting doesn't
> scale.

Like Accenture, IBM Global Services, ...

[snip]

> Lately it seems everyone is in need of assistance with their clouds, from
> architecture, setup and deployment there seems to be real need for the
> "Cloud Consultant".

Thank you, Reuven, for spontaneously verifying a thought I've been
having: Clouds are erector sets. Ya got'cher DB, load-balancer, DNS,
authenticator, auto-deployer, etc., etc., and All You Have To Do is
connect 'em up with a little glue -- your VALUE ADD -- and off you go,
scaling to the heavens. Right.

Why do you think IBM loves Clouds? The hardware and software revenue
is fine, but in addition this is a brand new opportunity to grow
services -- and not just SaaS, traditional-style consulting services.

[snip part about how everybody's cloud facilities being different
makes this even harder, which is true, of course.]

> One way may be to create a common cloud specification. David Young over at
> Joyent, attempted to do this, he has called for a common cloud specification

> called "Cloud Nine<http://www.joyeur.com/2008/05/08/cloud-nine-specification-for-a-cloud...>".

Seems to me that Dennis Richie (or maybe Ken Thompson) had a follow-on
to Unix he called Cloud 9, back in the late 80s or earl 90s. But even
Google can't find it, so what the hey.

[snip things about core components in Joyent's Cloud Nine]

> ...he

> says "a developer
> should be able to move between Joyent, the Amazon Web

> Services<http://finance.google.com/finance?q=amzn>,
> Google <http://finance.google.com/finance?q=goog>, Mosso, Slicehost, GoGrid,

> etc. by simply pointing the "deploy gun" at the cloud and go." I think he

> nailed it dead on with this statement. At the end of the day our job as

> cloud builders is about creating simplicity and making IT easier to manage
> and faster to scale.

Well, yes, it would certainly be simpler, easier, faster, etc. But
this is exactly parallel to how it would be easier to move
applications if there were a standard OS. There isn't one, for any
number of reasons, and there isn't likely to be one --- except,
possibly, and only in the server context, Linux is a fair imitation.
Because it's free.

Anybody up for becoming the Linus Torvalds of Clouds?

Otherwise, isn't this exactly the realization that produced the Globus
Alliance for Grids? The big difference is that Google, Amazon, (?
Microsoft?), and a pile of smaller players weren't actively engaged in
making money from their own grid walled gardens.

> Reuven Cohen

--
Greg Pfister

[Ignacio Martin Llorente]

I think, it depends on which cloud solution you have in mind. If you
see a cloud service as a provider of resources, all those bodies
provide or are working in any of the components. In the case of OGF,
its role is on the "grid part" of the virtualization (http://www.ogf.org/gf/group_info/view.php?group=gridvirt-wg
), that is the remote interfaces for accessing and federating the
sites, while the rest of bodies focus on components of the internal
virtualization solution.

Regards
--
Ignacio M. Llorente, Full Professor (Catedratico): web http://dsa-research.org/llorente
and blog http://imllorente.dsa-research.org
DSA Research Group: web http://dsa-research.org and blog http://blog.dsa-research.org
Globus GridWay Metascheduler: http://www.GridWay.org
OpenNebula Virtual Infrastructure Engine: http://www.OpenNebula.org

[Ignacio Martin Llorente]

Well, you know that there are many definitions for clouds. I see cloud
as a provider on resources (Amazon EC2, RESERVOIR,...). In such a case
the interfaces allow the on-demand scaling of a local infrastructure.
The services running on each virtual machine are not seen by the
infrastructure manager.

Regards
--
Ignacio M. Llorente, Full Professor (Catedratico): web http://dsa-research.org/llorente
and blog http://imllorente.dsa-research.org
DSA Research Group: web http://dsa-research.org and blog http://blog.dsa-research.org
Globus GridWay Metascheduler: http://www.GridWay.org
OpenNebula Virtual Infrastructure Engine: http://www.OpenNebula.org

> >> On Thu, Jun 5, 2008 at 10:59 AM, Sassa NF <sass...@gmail.com>

[Noah Campbell]

Greg: are you thinking of Plan 9? http://plan9.bell-labs.com/plan9/about.html

-Noah

[Greg Pfister]

On Jun 6, 12:36 pm, Noah Campbell <noahcampb...@gmail.com> wrote:
> Greg: are you thinking of Plan 9? http://plan9.bell-labs.com/plan9/about.html

That's it! Plan, Cloud, whatever. :-)

--
Greg Pfister