Cloudbursting

[Reuven Cohen]

Last week Jeff Barr from Amazon Web services came up with a great new
term to describe the need to handle sudden and extreme spikes in
demand by enabling a hybrid cloud computing model which combines both
private data center resources and remote cloud resources such as
Amazon Ec2. He called this model "cloudbursting". This term seems to
be quickly becoming the preferred way to describe a major problem or
for some the perfect "opportunity" in the emerging cloud computing
space (myself included).

In doing some research earlier I came across the original definition
of a cloudbust. In meteorology a cloudburst is an extreme form of
rainfall, which normally lasts no longer than a few minutes but is
capable of creating flood conditions. Similarly in IT a sudden and
unexpected rise in demand can quickly overwhelm a data center. In
coming up with the term cloudbursting, Jeff has give a simple name to
a rather complex problem. At Enomaly this is a problem we've been
debating for awhile; How do you effectively enable a kind of cloud
overflow in a secure yet efficient manor?

Provisioning instances in Amazon EC2 for example is relatively easy,
moving live workloads across a wide area is not. In most modern
dynamic applications the idea of having a "hot cloud standby" or a
prebuilt virtual machine that is basically waiting in the wings would
solve a lot of problems. But in reality there are a number of
complexities that need to be overcome. These complexities range from
network optimization to secure data transfer & replication to load
balancing across geographically diverse hosting environments, just to
name a few.

To truly enable a capable cloudbursting infrastructure, I feel there
needs to be a common consensus on how this may be archived and by what
means. So the question in the short term is; what are some of the
practical approaches, technologies and architectures needed to make
this kind of hybrid cloud infrastructure feasible?

Please feel free to give us your two cents.

You can see Jeff's original post here >
http://aws.typepad.com/aws/2008/08/cloudbursting-.html

--
--

Reuven Cohen
Founder & Chief Technologist, Enomaly Inc.
blog > www.elasticvapor.com
--
Open Source Cloud Computing - www.enomalism.com

[Jian Zhen]

Not to take away credit from Jeff, but 451 used this term on a Q-layer impact report from July 23rd, "In addition to direct sales to enterprises, going forward it hopes that extending out from private clouds to public ones – what we like to call 'cloudbursting' – will become a prevailing IT weather pattern and provide it with additional opportunities."

And again on a RightScale impact report from August 1st. To quote: "ISV virtual appliances should underpin a new surge in cloud use followed by self-service mechanisms and enterprise connectors enabling organizations to 'cloudburst' to using cloud services."

RightScale rolls its on-ramp toward other cloud systems

[ via Cloud Computing Terminology ]

--

CloudFeed.net - Blogging about the SaaS and cloud computing world

OnSaaS.info - Providing a continuous stream of SaaS and cloud computing news

Follow on http://twitter.com/onsaas, http://friendfeed.com/onsaas

[Khazret Sapenov]

On Wed, Sep 3, 2008 at 12:13 PM, Reuven Cohen <r...@enomaly.com> wrote:

...

To truly enable a capable cloudbursting infrastructure, I feel there
needs to be a common consensus on how this may be archived and by what
means. So the question in the short term is; what are some of the
practical approaches, technologies and architectures needed to make
this kind of hybrid cloud infrastructure feasible?

Please feel free to give us your two cents.

 

Traditional approach is mostly reactive to monitoring metrics or other constraints. In some cases you can use euristics to predict supply and demand in your real-time infrastructure, making adjustments.

 

However, there's also pro-active approach, when you allocate jobs, based on probability of some level of resource utilization.

 

Khaz Sapenov

[Reuven Cohen]

I'm not sure who came up with the term, but regardless I think it's a
good one. I spoke to William Fellows of 451group this morning, he
mentioned the term several times in our conversation which gave me the
inspiration to do a little more digging. The article by Jeff was the
only result I could find when I did a quick search after the call. But
now that I think of it, I remember William mentioning the term during
his presentation at CloudCamp London back in July.

ruv

[Barr, Bill]

I’m feeling really old … these problems were solved and solutions built, last century. Most of these products and tools have pretty much been sitting on the shelf for over a decade, waiting for the industry to catch up.

[Reuven Cohen]

Most of the technology we discuss on this group isn't revolutionary on
its own. But mashed up and repurposed I would beg to differ.

ruv

[Barr, Bill]

Oh, they don't even have to be mashed up and re-purposed, they just need
to be used and not poorly re-invented. All of this unnecessary
re-invention is just sucking resources away from solving the really,
really hard problems. Seamless interoperability is one of those problems
and even that is 99% political and 1% technical.

-----Original Message-----
From: cloud-c...@googlegroups.com
[mailto:cloud-c...@googlegroups.com] On Behalf Of Reuven Cohen
Sent: Wednesday, September 03, 2008 8:22 PM
To: cloud-c...@googlegroups.com
Subject: Re: Cloudbursting

On Wed, Sep 3, 2008 at 7:25 PM, Barr, Bill <Bill...@tectura.com>
wrote:

> I'm feeling really old ... these problems were solved and solutions

[Ray Nugent]

Bill, every generation of technologist does this. Great example of late - the Google Chrome cartoon had a panel with a 20 something software engineer asking "why not start over?"... :-)

Ray

[Reuven Cohen]

@bill

I would be interested in hearing about some examples of technology
from 10 years ago that would enable this type of solution?

ruv

[Christopher Steel]

Not 10 years old (about 8) Project Rio (http://www.rio-project.org/)
provides exactly this capability. It has the ability to define service
levels and dynamically reallocate "compute resources" to accommodate
cloudburst spikes. It is written in Java and leverages Jini. I think Jini,
in general, fits the bill of "sitting on the shelf", waiting for the
industry to catch up.

-Chris

[Paul Kamp]

Chris, In general I agree, Jini is technology before its time.  It is something that is "prematurely right".  It is probably being used in a number of efforts like Sun's project Caroline and others who do not want you to know they are using it.  I believe that GigaSpaces uses Jini.  So, Jini is being used in the cloud. It will be interesting to watch the uptake with the renewed interest in cloud computing. -Paul --- On Thu, 9/4/08, Christopher Steel <cst...@fortmoon.com> wrote:

[Barr, Bill]

That's certainly one of them, and is the underpinning of several
commercial products already. Research into remote dynamic proxies,
agents, tuple spaces, mobile code, blackboards and whiteboards,
federated databases, mobile objects, P2P networks, ad nauseum. It's all
there. Apollo, DEC, IBM and Sun (plus lots of others I'm certainly
forgetting) have all walked down this path before.

The more interesting question to find an answer to is, why haven't they
been successful or at least hyped as much?

- far too specialized?
- too small a market/need?
- unusable by mere mortals or the average corporate developer?
- lack of zealotry?
- never provided any real, concrete benefits, only different
implementations?

[Reuven Cohen]

On Thu, Sep 4, 2008 at 12:45 PM, Barr, Bill <Bill...@tectura.com> wrote:
>
> That's certainly one of them, and is the underpinning of several
> commercial products already. Research into remote dynamic proxies,
> agents, tuple spaces, mobile code, blackboards and whiteboards,
> federated databases, mobile objects, P2P networks, ad nauseum. It's all
> there. Apollo, DEC, IBM and Sun (plus lots of others I'm certainly
> forgetting) have all walked down this path before.
>
> The more interesting question to find an answer to is, why haven't they
> been successful or at least hyped as much?
>
> - far too specialized?
> - too small a market/need?
> - unusable by mere mortals or the average corporate developer?
> - lack of zealotry?
> - never provided any real, concrete benefits, only different
> implementations?
>
>

The answer is they were too early. I first came up with elastic
computing 5 years ago, nobody cared. Now the phone rings off the hook.
What's changed?

ruv

[Paul Kamp]

Bill, One reason that the technologies may not have been successful to date was the business model.  Note the companies that you named, all hardware companies.  When implemented correctly with an appropriate business model these technologies cannibalize the sales of their bread and butter, hardware.  the simplest example is virtualization that increases server utilization.  Today, these technologies are being used in quite a different way than any of the hardware vendors had forseen (or perhaps would have liked).  The technology is cannibalizing their hardware sales and the value is migrating over to the services oriented companies like AWS, AppNexus, GoGrid, GigaSpaces, etc.  It is the efficiency and flexibility that enable these services to undercut the pricing of owning a server of your own.  Anyway, that is my rationale, FWIW. Regards, Paul Kamp pg...@yahoo.com 401-261-5421 (mobile) --- On Thu, 9/4/08, Barr, Bill <Bill...@Tectura.com> wrote:

From: Barr, Bill <Bill...@Tectura.com> Subject: RE: Cloudbursting To: cloud-c...@googlegroups.com

[Barr, Bill]

Too early directly translates either into a solution in search of a
problem to solve or, the problem isn't yet painful enough to solve.

[Reuven Cohen]

On Thu, Sep 4, 2008 at 1:40 PM, Barr, Bill <Bill...@tectura.com> wrote:
>
> Too early directly translates either into a solution in search of a
> problem to solve or, the problem isn't yet painful enough to solve.
>
>
> >
>

Nicely said.

r/c

[Barr, Bill]

Rent vs. buy, I get that. I don’t think some of the cloud vendors do, though.

 

 

 

From: cloud-c...@googlegroups.com [mailto:cloud-c...@googlegroups.com] On Behalf Of Paul Kamp
Sent: Thursday, September 04, 2008 10:07 AM
To: cloud-c...@googlegroups.com
Subject: RE: Cloudbursting

 

Bill,

[Christopher Steel]

Or... too different for the marketing people to understand and the sales
people to sell.

-Chris

-----Original Message-----
From: Barr, Bill [mailto:Bill...@Tectura.com]
Sent: Thursday, September 04, 2008 1:41 PM
To: cloud-c...@googlegroups.com
Subject: RE: Cloudbursting

[Paul Kamp]

Or the solution is quite good but it may work against a certain parties best interest.

Regards, Paul Kamp pg...@yahoo.com 401-261-5421 (mobile) --- On Thu, 9/4/08, Barr, Bill <Bill...@Tectura.com> wrote:

From: Barr, Bill <Bill...@Tectura.com> Subject: RE: Cloudbursting To: cloud-c...@googlegroups.com

[Dan Kearns]

On Thu, Sep 4, 2008 at 10:07 AM, Paul Kamp <pg...@yahoo.com> wrote:

One reason that the technologies may not have been successful to date was the business model.  Note the companies that you named, all hardware companies. When implemented correctly with an appropriate business model these technologies cannibalize the sales of their bread and butter, hardware.  the simplest example is virtualization that increases server utilization.

I'd be shocked, completely and utterly shocked even, if cloud computing is anything but a huge driver for hardware sales for the next ten years, much the same as the "paperless office" was a bonanza for printer and paper sales. The primary emergent effect of virtualization is to reduce the friction in bringing new software projects online, which makes businesses more agile. It doesn't reduce the need for computing power, it increases it at a huge rate. The sales pitch of reducing cost is completely disingenuous. It would only be true if your business never changed (and good luck with that).

If there's a hardware-company angle, it's probably just that hardware companies aren't great at selling software. However, I suspect it's more that people just didn't have the types of computing problems then that they do now, and for practical reasons, eg 1tb of storage didn't cost $200 and fit in 5.25", it was more like $1million and  15 cubic feet.

-d

[Paul Kamp]

I agree that it will be a driver only it will be a driver for a different type of company now.  I am pretty certain that all of the cloud vendors will be buying lots of hardware and they will be pushing improved server utilization.  This is a bit different than a single purpose built system which has been the primary driver of purchases in the past, though not the only one. Then again, this could be viewed as a re-incarnation of time sharing.

Regards, Paul Kamp pg...@yahoo.com 401-261-5421 (mobile)

--- On Thu, 9/4/08, Dan Kearns <d...@thekearns.org> wrote:

From: Dan Kearns <d...@thekearns.org> Subject: Re: Cloudbursting To: cloud-c...@googlegroups.com

[timothy norman huber]

Paul,

I'd like to coin the term Business Model Collision, or BMC.   For the hardware companies seeking to make servers more efficient often times find their business model collides with those licenses are based on CPU count or memory per node.   For instance, MySpace yanked RAM from their servers in order to avoid triggering Enterprise pricing for their .net deployment.   Good for reducing the cost of their licenses, bad for server efficiency.

Timothy Huber

Strategic Account Development

tim.huber (at) metaram.com

cell 310 795.6599

MetaRAM Inc.

181 Metro Drive, Suite 400 

San Jose, CA 95110 

http://www.linkedin.com/in/timhuber

[Tross]

On September 11, it will be 8 years ago that we incorporated
ThinkDynamics which based on dynamic provisioning to provide server
capacity on demand. At the end of the first internet bubble we aimed
to provide burstable server capacity by "moving" servers across
network and storage fabrics into isolated dedicated infrastructures
for each application. There was a lot of interest back then, when
"utility computing" or "on demand" were the terms du jour.

Ejasent was an interesting company also playing on the same message
They had a product called upscale that was the first product I'd seen
to do "live migration". They were focused on solaris and did vertical
scaling as opposed to the horizontal scaling we did. I think they
were the first company I remember to use the "elastic" term - back in
2001.

Interestingly, IBM had a research project called oceano which was very
similar to our approach at think.

I just looked over our original VC pitch early 2001 - man it takes me
back, yet it's the same vision. So what's changed all these years
later? IMHO, widespread adoption of server virtualization
particularly due to x86 and VMWare, has made it much easier and more
reliable to perform dynamic provisioning. Multi-core SMP intel boxes
have also made single hosts much more powerful and able to be
partitioned.

In my opinion, the technology plays a significant role. I do,
however, recognize that the messaging for the past 8 years has
definitely warmed the industry up to the concept of dynamic scaling.
I'd peg it at 75/25 in favor of technology, with virtualization being
a critical catalyst to unleash the potential.

Tross

On Sep 4, 12:13 pm, "Christopher Steel" <cst...@fortmoon.com> wrote:
> Not 10 years old (about 8) Project Rio (http://www.rio-project.org/)
> provides exactly this capability. It has the ability to define service
> levels and dynamically reallocate "compute resources" to accommodate
> cloudburst spikes. It is written in Java and leverages Jini. I think Jini,
> in general, fits the bill of "sitting on the shelf", waiting for the
> industry to catch up.
>
> -Chris
>
>
>
> -----Original Message-----
> From: Reuven Cohen [mailto:r...@enomaly.com]
> Sent: Thursday, September 04, 2008 11:45 AM
> To: cloud-c...@googlegroups.com
> Subject: Re: Cloudbursting
>

> On Thu, Sep 4, 2008 at 11:31 AM, Barr, Bill <Bill.B...@tectura.com> wrote:
>
> > Oh, they don't even have to be mashed up and re-purposed, they just need
> > to be used and not poorly re-invented. All of this unnecessary
> > re-invention is just sucking resources away from solving the really,
> > really hard problems. Seamless interoperability is one of those problems
> > and even that is 99% political and 1% technical.
>
> > -----Original Message-----
> > From: cloud-c...@googlegroups.com
> > [mailto:cloud-c...@googlegroups.com] On Behalf Of Reuven Cohen
> > Sent: Wednesday, September 03, 2008 8:22 PM
> > To: cloud-c...@googlegroups.com
> > Subject: Re: Cloudbursting
>

> > On Wed, Sep 3, 2008 at 7:25 PM, Barr, Bill <Bill.B...@tectura.com>

[Gerrit Huizenga]

This sounds a lot like the effect we call "slashdotting" - referring
to the sudden increase in load to a site brought on by a sudden and
sharp increase in the number of references to site. It sounds like we'd
handle getting slashdotted by setting up a cloudburst. Well, the
metaphors don't work perfectly but a cloudburst is the IT/data center
solution to deal with a phenomenon like getting slashdotted...

gerrit