Cloud Economies and Standards

[Geoffrey Fox]

Here is a new thread spinning off Introducing the Virtual Private Cloud
(VPC)

Perhaps we could discuss
1) Is it practical to establish Cloud standards allowing different
clouds to interoperate?
2) Maybe this will enable concepts like Cloud Economies?
Grid economies are well studied with high quality ideas but
complexity (IMHO) has led to little use of Grid economies
3) Maybe this will make clouds as complicated as Grids?
The explicit attention to interoperating heterogeneity in Grids has
led to difficult complex systems which clouds have so far avoided

--
:
: Geoffrey Fox g...@indiana.edu FAX 8128567972 http://www.infomall.org
: Phones Cell 812-219-4643 Home 8123239196 Lab 8128567977
: SkypeIn 812-669-0772 with voicemail, International cell 8123910207

[Khazret Sapenov]

Just reposting my message from mentioned thread, so it resides in proper topic:

 

As a next logical step in evolution of cloud computing, I would see
some universal unit of computing used for trading of excessive
resources (computing as a commodity). Thus analogy with electric grid
or, say, public transport system is very viable and initially be
limited by grid locality.

Khaz Sapenov

[Khazret Sapenov]

On Fri, May 9, 2008 at 4:30 PM, Geoffrey Fox <g...@grids.ucs.indiana.edu> wrote:

2) Maybe this will enable concepts like Cloud Economies?
   Grid economies are well studied with high quality ideas but
complexity (IMHO) has led to little use of Grid economies

 

Geoffrey,
This is what I have off top the head, sorry if it is too muddled:

It is hard to say whether and when it'll use grid economy.
Currently we have sporadic cases of entire chain (like Amazon Compute Cloud), where you have supplier of compute capacity (S), third parties (R), consumers (C) charged flat or variable fee for amount of resources (CPU-hour or a combination of metrics).

Lets consider why almost all entities of cloud computing ecosystem might be interested in it.
For simplicity, I omitted an imported hidden entity - hardware vendors (H), who sell more units, necessary to power the cloud. 
Suppliers (S) benefit from selling at wholesale prices and having margin.
Third parties (R) serving as retailers, resell compute capacity in different forms, such as hosting websites, SAAS etc. This area is growing, since it gives huge cost savings to resellers. Consumers (C) also benefit from cheaper services.

Going further, there's a question whether this economy would use trading model(using market-based resource management and scheduling) or keep proprietary chargeback schemes.
In my opinion, once it enters the stage where retailers of compute capacity (or it's large clients) need to manage the risk of availability of compute capacities, there will be obvious need for regulated market.

regards,
Khaz Sapenov

[Ian Marr]

To continue the analogy... we need to find the equivalent of the Kilowatt-Hour for the "universal unit of computing".  So let's discuss the characteristics of the kWh.

 

My first immediate thought is that the kWh is relatively easy to control. Each electrical appliance and light bulb I buy for my house has a clear watt rating so I know what I'll consume when I use it and the electricity supplier gives me a clear price per kWh. And I can reduce my consumption and bill by finding more efficient appliances (witness the evolution in light bulbs) or by turning stuff off! 

 

It's so much harder to do this with compute consumption but then that's the point Khazret is making and leads to the need for a "universal unit of computing" that's easily controllable by the consumer who foots the bill.

 

Sorry, no stunning insights or solutions from me, just $0.02.

 

Ian.

--
Ian Marr.
http://www.linkedin.com/in/ianmarr

[randall]

I'll take a stab at this...

For standards, I think either a committee will sprout up to manage
protocols (better) or the leading supplier will become the defacto
standard (more likely). Boutique providers will distinguish themselves
from the pack with support, proximity, price and environmental
friendliness among others, which I'll mention later.

As cloud services approach commodity status, they will all have up-
time of five nines, about the same performance, network hops, etc, so
then price will be a distinguishing factor. Because consumers want a
standard pricing metric for commodity products we will likely see grid
economies evolving.

Right now pricing is varied and complicated. Amazon's pricing
structure is based on server capacity, MediaTemple uses a GPU and
Mosso uses requests. As customers already find these pricing schemes
too unpredictable, we will start to see tools arrive to help us
calculate our application demands in Grid Units. These "Grid Units"
could be based on process scheduling priority, processor and network
utilization, disk space, memory, etc. "Your application consumes 14
Grid Units." Multiply 14 by the vendor price per Grid Unit and you
know what you'll be paying before you move to any particular cloud.
That's an oversimplification, but I think the vendors who provide this
kind of predictability will win out.

With thousands of customers using the same cloud, eventually demand
will outstrip supply and affluent consumers will pay more for quicker
apps. I think here, we will see competing vendors catering to
different customers by offering different resource management
mechanisms. Some customers will want to bid for scheduling priority,
others will want to pay a higher fixed rate for higher priority
processing. There are lots of algorithms and I don't know which ones
customers or vendors will prefer. It is fairly certain though that
vendors will not leave revenue on the table when some customers will
pay more than others.

Back to standards, I think suppliers using proprietary APIs will
present a barrier to service adoption. "Will I have to rewrite my code
to work with your storage solution?" If the answer is yes, forget it.
When electricity was deregulated, customers could switch their
providers to save money or buy greener power, but if they had to
rewire their homes or buy new devices -- no one would. ISVs are going
to build cloudable products and they'll want to write them once and
let the customer choose and pay for the cloud services independently.

[Geoffrey Fox]

So here are some questions which some might be able to answer!
1) Do we have experience in switching between clouds. How easy is that?
    This involves the actual hosting of service and in some cases (common for Grids) of data-compute affinity i.e.
2) If one needs extensive associated data does one need to move or replicate data between clouds. e.g. if I store my data in S3, when is it practical to compute on clouds other than EC2. This involves both bandwidth and cost of bandwidth
3) The issue of affinity is also relevant within a single cloud. Does/should cloud provider assign compute-nodes "near" the needed data?
4) and in a different class of application; can I specify node-node affinity as needed by parallel computing jobs that need a few microsecond latency for communication between nodes.

[Geoffrey Fox]

Following up my previous note, do we need to include cost of data and communication in economy?
These are some of complexities that bedevil grids!

[Reuven Cohen]

I think you nailed it with "ISVs are going to build cloudable products

and they'll want to write them once and let the customer choose and
pay for the cloud services independently".

You should be able to install and or use cloud software with the same
ease you currently do when utilizing desktop software. Plug and play
(except in the cloud).

Reuven
www.elasticvapor.com

--
--

Reuven Cohen
Founder & Chief Technologist, Enomaly Inc.
www.enomaly.com :: 416 848 6036 x 1
skype: ruv.net // aol: ruv6

blog > www.elasticvapor.com
-
Get Linked in> http://linkedin.com/pub/0/b72/7b4

[Tarry Singh]

Cloud or Grid enabled applications, like that of what Reuben en other guys from 3Tera, Electric Cloud, Nirvanix etc etc, will heavily undermine the current trend of Siloed Clouds, meaning virtualization.

Switching between cloud is done pretty well if you want to check out the 3Tera guys (I don't work for them, neither do I get anything in return) or other parties who are into it. It is about GDN or Global Delivery Networks. I have written several articles on GDM and how a typical Cloud enabled, Globally spread (and therefore , the risks spread as well, not like the siloed cloud, which is the virtualization today, where you carve up a server and hook all servers as one entity but when connecting 2 or more data centers, then we start seeing complications as it only ties the infra stack, not really the app stack.

Anyways great discussion going on and I just wanted to get this out (If it is a bit off-tangents then please forgive me)

/TarrySingh

--
Kind Regards,

Tarry Singh
______________________________________________________________
Founder, Avastu: Research-Analysis-Ideation
"Do something with your ideas!"
http://www.avastu.com
Business Cell: +31630617633
Private Cell: +31629159400
LinkedIn: http://www.linkedin.com/in/tarrysingh
Blogs: http://tarrysingh.blogspot.com

[Daniel Feygin]

I certainly agree that "plug-and-play" is the way to go, it's the
essence of cloud computing.

However I view customer-driven assembly of the "whole product" out of
separate platform and application components as counterproductive
(assuming it will some day become a possibility as PaaS standards
emerge, which is far from given), because it:
1) effectively kills multi-tenancy at application layer;
2) requires testing and certification of apps on each vendor's cloud;
3) carries deployment risks at time of "purchase" (assembly-time);
4) spreads support responsibility and diminishes vendor liability;
5) complicates software update rollouts for application developers;
6) forces application developers to maintain ongoing dependency
compliance with multiple externally controlled environments.

Indeed, that would be more like desktop computing, which is precisely
what I thought we are trying to move away from.

Daniel

[Geoffrey Fox]

I believe what you call customer-driven assembly is a typical Grid goal and ends up quite complicated so I agree this is not so reasonable/easy.

Using desktop analogy, do we expect people to use applications that link "desktops" (clouds) or that are just plug and play within a single desktop (cloud) but you are allowed to choose multiple desktops (clouds). It is is not so easy today to move between Linux, Windows and Mac etc. so this might suggest it will be hard to move between clouds -- let alone link clouds?

[Daniel Feygin]

There would definitely be less unproductive effort if a linking
economy emerged rather than developers making the same app for
different clouds and maintaining it in everywhere it's deployed. Each
application should be deployed to a single vendor's cloud, but clients
would be able to manage application portfolios spanning multiple
clouds.

Again, if applications become easily portable across clouds, which
requires a standardized set of cloud capabilities and APIs, then
co-locating the entire application portfolio on one cloud would be the
most appealing option. But real portability, despite some standards,
never happened with packaged platforms (beyond POSIX, if even that)
and I have no expectation it would happen with hosted ones.

Daniel

[Nathan Fiedler]

On the contrary, I find as a developer that I can easily compile
various daemons to run on Solaris, BSD, Linux, Mac OS X, and if I'm
lucky, even Windows via Cygwin, all thanks to the commonly available
tools (e.g. autoconf, gcc) and standard C APIs. Likewise, if I were to
write a Java EE application that sticks to the published standard, it
is easy to deploy on any of a number of Java EE containers. Yes, there
are plenty of tempting APIs within a specific system that would render
my application non-portable, but that's the trade-off I make as a
developer.

Regarding the desktop analogy, I think that's a poor fit. Desktops
have no standard whatsoever, with each one doing their own thing, each
with a completely different API. That, if applied to clouds, would be
a disaster for developers. I'd sooner compare clouds to Java EE
containers: the standards are driven by a committee (with Sun as the
main driver) and all the providers typically support the standard very
well. Portability, without even recompiling the Java source, is fairly
easy.

In any case, a standard for cloud computing needs to be in place, even
if it will not be implemented perfectly everywhere. Otherwise we'll
see one provider become dominant and turn into the de facto standard
(i.e. the "Windows" of cloud computing).

n

[dowe...@aol.com]

I like the cloud economy approach.  Today, we see infrastructure raised in the service oriented context especially from a strategic perspective.  As stated earlier " as a service". 

Basic Question - aside from app issues - when we attempted to design a solution spanning existing data centers we also struggled with network security and performance guarantees (few years ago)).  Basically we had to break through all of the silos.  In a service oriented view we may build a contract to establish this "agreement" -- is this pattern accepted in the cloud economy? 

Realizing I may need to "catch up"  - any pointers to research appreciated.

-Scott

---

Plan your next roadtrip with MapQuest.com: America's #1 Mapping Site.