Fwd: Rackspace Cloud Service Interruption

[Khazret Sapenov]

This kind of issues seem to be re-occurring with different cloud providers.

Could anything be done from data center technology standpoint to minimize downtime?

Perhaps they could have sent notification of scheduled maintenance well before, so I temporarily migrate my processes to another 'availability zone'. Even better scenario would be automatic migration without my intervention, thus my processes were temporarily relocated to an 'oceanview' room until maintenance is over. 

---------- Forwarded message ----------
From: The Rackspace Cloud <notifi...@rackspacecloud.com>
Date: Tue, Nov 3, 2009 at 8:58 PM
Subject: Rackspace Cloud Service Interruption
To: Khazret Sapenov <my email here>


Dear Khazret,

At approximately 12:29am CST this morning, our Dallas - Fort Worth (DFW) data center experienced a power disruption, and consequently an interruption of our services.  The power disruption was the result of issues during a maintenance effort that was scheduled and expected to be non-impacting.

This summer our DFW facility had power issues, and as a result, we invested significant resources to improve all aspects of our power systems.  Last night, during one of these steps, we encountered issues and had a brief loss in power.  The power disruption was approximately 5 minutes in duration.  Despite this short power disruption, many customers experienced downtime that was significantly longer.  Since the power disruption hit the core of many of our cloud services, recovery of full operations required more effort than simple recovery of power.   The experience you had last night is not acceptable to us.

Here is what we know about the events:

 · The scheduled maintenance was planned to occur from 12:05am - 6:05am CST in our DFW data center.  This maintenance is part of a preventative maintenance schedule for several PDUs in UPS Cluster G at the DFW datacenter.  The PDUs were down for a total of 5 minutes before power was restored.  At approximately 12:29am CST, all PDUs behind UPS Cluster G lost power.

 · Although the power outage was very brief (5 minutes), it forced a hard re-boot to occur on a portion of our cloud infrastructure.  As our engineers worked to bring hardware back online, we experienced several unforeseen hardware failures.  Further complicating our recovery effort, the incident also created internal DNS issues, which caused additional delays.  With that said, the vast majority of cloud customers affected by this outage had service restored within one hour's time (many in as little as five minutes); however, depending upon the service, a few customers experienced service interruptions for up to few hours.

Here is how we plan to deal with it:

 · We have invested massively in the DFW facility to ensure it delivers at a level you expect from Rackspace - despite last night, we feel very good about our plan and have high confidence in the DFW facility - clearly we have to prove it.

 · We are reviewing our maintenance notifications - we typically do not share information on expected non-impacting events, but clearly we need to ensure we calibrate these events and are fully transparent.

 · We are reviewing our procedures and systems for quickly resuming cloud operations when an unexpected event like this occurs - unexpected events will happen, our job is to minimize their impacts.

We live by high standards and clearly have not lived up to them.  We welcome any feedback.  If you would like a call from me, or anyone on our senior team to discuss these issues personally, please reply with a phone number.

We have work to do to earn back your trust.  We will not rest until we have.

Thank you,

Emil Sayegh
General Manager, The Rackspace Cloud

[Rao Dronamraju]

“Although the power outage was very brief (5 minutes), it forced a hard re-boot to occur on a portion of our cloud infrastructure.”

 

Actually this surprises me a bit. This says that they do not have redundant power source/supply for that portion of the cloud infrstructure?....

I would certainly expect name brands in CC space like Rackspace and Amazon to take care of such basic DR&BC issues.

Redundancy in power supply is as basic as reserve gas in your gas tank.

 

---

[bro...@netgate.net]

On Tue, 3 Nov 2009, Rao Dronamraju wrote:

> Actually this surprises me a bit. This says that they do not have redundant
> power source/supply for that portion of the cloud infrstructure?....
>
> I would certainly expect name brands in CC space like Rackspace and Amazon
> to take care of such basic DR&BC issues.
>
> Redundancy in power supply is as basic as reserve gas in your gas tank.

You design and deploy your architecture with a certain amount of risk.
Service based architectures, even cloud based, are driven by economics.
Given enough money we can build a machine to take men into space, yet the
failure of a few protective tiles can have catastrophic effects. With
enough money we can build a compute service that "almost" never fails, but
would anyone be willing to pay for it? Unfortunately, "good enough" is
often the best we can build given customer pricing expectations. Even so,
I can't defend their position and I do agree with the concept of moving
cloud parts to a less risky cluster, or whatever you want to call it, but
that's easier said than done.

And who decided to add "never fail" into the definition of "cloud
computing"? :-)

[Tim M. Crawford]

Unfortunately, it's not that simple. It doesn't say that they don't have power redundancy for their cloud infrastructure. It states that it failed. I've seen cases where the right processes and technologies were put in place for full power redundancy... only to have it bring a power interruption.

Further, it's not simple to bring a DC back online after a power failure. It's not like turning on a switch and everything is ok. There is a specific process that systems need to be brought back online, validated and tested in a phased approach.

I'm not trying to defend Rackspace. But at the same time, they have had two major power incidents over the course of the year. Would this cause me to avoid using Rackspace? No. Would I consider using other facilities (other than their DFW/ Grapevine facility)? Probably so. At least until I had built in redundancy for my apps/ svcs across facilities/ providers.

-t

____________________________________

Tim M. Crawford

+1.650.804.1300

Blog: http://timcrawford.org/

Twitter: http://twitter.com/tcrawford

[Peglar, Robert]

Brooks said:
" And who decided to add "never fail" into the definition of "cloud
computing"? :-)"

Who, indeed? The answer is simple - the consumers (in particular, CIOs)
that buy such services. To them, cloud compute is a way to save money
and get rid of a perceived problem. They of course expect it to be
perfect, as does the media, given the hype.

So when a cloud provider loses its ability to service its customers,
much as an electric power utility when it goes down, it's news. We all
might as well get used to it; it won't change anytime soon.

Rob

-----Original Message-----
From: cloud-c...@googlegroups.com
[mailto:cloud-c...@googlegroups.com] On Behalf Of
bro...@netgate.net
Sent: Tuesday, November 03, 2009 11:48 PM
To: cloud-c...@googlegroups.com
Subject: [ Cloud Computing ] Re: Fwd: Rackspace Cloud Service
Interruption

No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 8.5.423 / Virus Database: 270.14.45/2476 - Release Date:
11/02/09 07:51:00

[cloud-computing]

Correct me if I'm wrong but shouldn't the right architecture with
fault tolerance have a secondary image that is available if the
primary host is unavailable assuming the additional physical hosts in
the cluster have separate UPS'? I'm not a vm admin but we have two
vCenter platforms going up in two datacenters currently and I would
hope we are architecting properly (separate UPS for separate clusters)
to avoid downtime with power issues like this. I dont' know if they
provide you the ability to manage your VM's, resource pools, etc or
provide you the ability to create secondary images on separate
phyiscal hosts (VMWare Fault Tolerance) but I guess I'm making the
assumption they do and that you would be able to prepare for this
scenario as standard practice in case a phyisical host unexpectedly
dies.

Brian

On Nov 3, 10:44 pm, "Rao Dronamraju" <rao.dronamr...@sbcglobal.net>
wrote:

> "Although the power outage was very brief (5 minutes), it forced a hard
> re-boot to occur on a portion of our cloud infrastructure."
>
> Actually this surprises me a bit. This says that they do not have redundant
> power source/supply for that portion of the cloud infrstructure?....
>
> I would certainly expect name brands in CC space like Rackspace and Amazon
> to take care of such basic DR&BC issues.
>
> Redundancy in power supply is as basic as reserve gas in your gas tank.
>
>   _____  
>

>  . The scheduled maintenance was planned to occur from 12:05am - 6:05am CST

> in our DFW data center.  This maintenance is part of a preventative
> maintenance schedule for several PDUs in UPS Cluster G at the DFW
> datacenter.  The PDUs were down for a total of 5 minutes before power was
> restored.  At approximately 12:29am CST, all PDUs behind UPS Cluster G lost
> power.
>

>  . Although the power outage was very brief (5 minutes), it forced a hard

> re-boot to occur on a portion of our cloud infrastructure.  As our engineers
> worked to bring hardware back online, we experienced several unforeseen
> hardware failures.  Further complicating our recovery effort, the incident
> also created internal DNS issues, which caused additional delays.  With that
> said, the vast majority of cloud customers affected by this outage had
> service restored within one hour's time (many in as little as five minutes);
> however, depending upon the service, a few customers experienced service
> interruptions for up to few hours.
>
> Here is how we plan to deal with it:
>

>  . We have invested massively in the DFW facility to ensure it delivers at a

> level you expect from Rackspace - despite last night, we feel very good
> about our plan and have high confidence in the DFW facility - clearly we
> have to prove it.
>

>  . We are reviewing our maintenance notifications - we typically do not

> share information on expected non-impacting events, but clearly we need to
> ensure we calibrate these events and are fully transparent.
>

>  . We are reviewing our procedures and systems for quickly resuming cloud

> operations when an unexpected event like this occurs - unexpected events
> will happen, our job is to minimize their impacts.
>
> We live by high standards and clearly have not lived up to them.  We welcome
> any feedback.  If you would like a call from me, or anyone on our senior
> team to discuss these issues personally, please reply with a phone number.
>
> We have work to do to earn back your trust.  We will not rest until we have.
>
> Thank you,
>
> Emil Sayegh

> General Manager, The Rackspace Cloud- Hide quoted text -
>
> - Show quoted text -

[Gilad Parann-Nissany]

I see it like Tim. I've had the "privilege" of looking deep into several power outages. Even when proper redundancy is in place, Murphy's law still applies. For example some fuse can jump in your UPS, a small component can blow, a surge when the grid power comes back can actually cause trouble, and so on.

A solution is to have redundancy on top of redundancy, but that gets expensive.

Regards
Gilad
__________________
Gilad Parann-Nissany
CEO, Founder
http://www.porticor.com/

[Philip Cox]

> And who decided to add "never fail" into the definition of "cloud
> computing"? :-)

This is an interesting statement. I would agree that if one thinks about it
realistically, that is true. However, I would argue that MOST cloud
consumers or people considering the cloud have exactly that underlying
expectation, "it will never fail", however misguided it is.

Phil

[Rao Dronamraju]

I do not understand, what are the chances of both power sources (primary and the secondary) failing at the same time?...if they have redundancy built in their DC.

 

Also here is what a N+1 data center operator says on their website as to how they provide for power redundancy.

 

“Once the power transfer switches sense a decline in power from the local utility, they automatically signal diesel-powered generators to start up and switch power flow to the generators. In the few seconds between the decline in power from the utility and switch over to our backup generators, our UPS units are supplied with power through a battery system that ensures the power supply to servers is uninterrupted.”

 

So why should the servers have a total power outage for 5 minutes if you had the above or similar redundancy?....probably they do not for that part of the infratsructure.

May be they felt it does not host their most important critical cleints. I am not being negative on Rackspace. It just surprised me a bit because they are in Gartner’s magic quadrant (right top) for CC providers.

 

---

[Ray DePena]

How many people here are aware that all of TX is on its own independent power grid?

--
Ray DePeña
Director, Stealth Startups
Strategic Business Advisor

http://www.linkedin.com/in/raydepena
Sacramento, CA 95630
(916) 941-5558

[davidnavetta]

Strawman. Nobody ever claimed that failproof was the goal. Some level
of reasonable security is the goal. If they dont have basic DR&BC
controls in place that is a problem.

[Bernard Golden]

Part of the value proposition for commercial cloud providers is that
they have the expertise and capital to invest in best-of-breed
facilities to improve availability vis a vis internal data centers.
Certainly one would expect cloud providers to implement Tier 4 data
centers, which explicitly call for redundant functionality and paths
(although Microsoft published a very interesting cloud computing paper
discussing the opportunity to achieve redundancy via multiple data
centers, each of which was lower quality in and of itself).

Not to jump on Rackspace, but a cloud provider losing power is pretty
serious and definitely impinges the core value proposition for the
provider.

[Peglar, Robert]

Yes, aware. Been this way for quite a while. 

Sent via iPhone. Please forgive any typos.

Robert Peglar
Vice President, Technology, Storage Systems Group
Xiotech Corporation
Robert...@xiotech.com
952 983 2287 (Office)
314 308 6983 (Mobile)
636 532 0828 (Fax)
www.xiotech.com : Toll-Free 866 472 6764

[Peglar, Robert]

+1 Phil. Well said

Sent via iPhone. Please forgive any typos.

On Nov 4, 2009, at 6:31 PM, "Philip Cox" <phil...@systemexperts.com>
wrote:

---

Robert Peglar
Vice President, Technology, Storage Systems Group

Email: mailto:Robert...@xiotech.com
Office: 952 983 2287
Mobile:314 308 6983
Fax: 636 532 0828
Xiotech Corporation
1606 Highland Valley Circle
Wildwood, MO 63005 http://www.xiotech.com/ : Toll-Free 866 472 6764

[Jan Klincewicz]

This is similar to Mark Hurd of HP admonishing his listeners not to go to Public Clouds until they are "100% secure."   Why is the Public Cloud held to a heretofore unachievable standard ?
I've seen electricity out for hours on end in downtown Philadelphia, and it barely makes the papers.  ANYTHING that remotely RESEMBLES CC has a hiccup, and it's all over the news that CC
is not ready for prime time, dangerous, unstable, etc.

Hype is a double-edged sword.

<<Brooks said:
" And who decided to add "never fail" into the definition of "cloud
computing"? :-)"

Who, indeed?  The answer is simple - the consumers (in particular, CIOs)
that buy such services.  To them, cloud compute is a way to save money
and get rid of a perceived problem.  They of course expect it to be
perfect, as does the media, given the hype.

So when a cloud provider loses its ability to service its customers,
much as an electric power utility when it goes down, it's news.  We all
might as well get used to it; it won't change anytime soon.

Rob >>

--
Cheers,
Jan

[Rao Dronamraju]

Not to forget the fact that clouds are supposed to be mega data centers with
multi-tenancy. This obviously means that these are single points of
(concentration) of business / wealth creation or sustainace entities. If
such facilities are brought down either due to internal systems failure or
external attacks there is going to be massive economic loss. That is why
these facilities are supposed to have lot more rigid and rigarous DR&BC
built into them. Since clouds are still in their infancy, the impact of
power failure is not yet felt widely. But once they are widely deployed it
will be expected that they be LOT more reliable in their DR&BC than present
day data centers. Ofcourse, I would assume the way these mega data
centers/clouds will be built in such a way that they will be segmented to
minimize the impact of such a failure.

[Peglar, Robert]

No doubt about it.  Hype is always double-edged.  So, it is left to us, the cloud community, to educate the consumers that clouds, while a great solution for several use cases, are not perfect.

 

Rob

--
Cheers,
Jan

Version: 9.0.698 / Virus Database: 270.14.52/2483 - Release Date: 11/05/09 13:52:00

[Peglar, Robert]

I've seen several cloud datacenters and can tell you that they are not that much more advanced - if at all - than a good commercial enterprise datacenter. So, while it's natural to make the assumption you did below, it's not always the case, not by a long shot. Some of these cloud datacenters are running on the ragged edge - they are trying to get away with the least infrastructure resilience they can without losing customers due to outage. It's a constant cat-and-mouse game, fueled by toothless SLAs. Other cloud datacenters are doing it right, segmenting, using resilient hardware, and not relying on one source of anything, either external or internal. A few forward-looking datacenters are also exploring getting off the grid entirely and self-generating all their own power.

Rob

[Ray]

Why do they have to be "mega centers"?

[Rao Dronamraju]

For economies of scale.

THE RISE OF THE MEGA-DATA CENTER
http://www.nytimes.com/2009/06/14/magazine/14search-t.html?pagewanted=all

[Rao Dronamraju]

I think you are right. But the reason clouds are just a bit better than
commercial data centers is because they are just in their nascent stage.
Once they pickup the CSP will have no choice but to provide the best
possible servcies if they want to remain competitive and continue in the
business. I think next 3 to 5 years when we will see some excellent mega
data centers with highly resilient facilities.

[Roland Rambau]

Rao, vaporistas,

to me it appears obvious that in order to use clouds in an
enterprise setting, I often need to utilize at least 2 clouds
so that I have a standby backup in case one of them fails.

However, what is not obvious to me: how can a cloud *user*
ensure and control that the infrastructure behind 2 different
clouds has no common point of failure ?

It would be useless to engage with 2 separate cloud offerings
if then both end up using the same data center, maybe even the
same set of servers. Note: its clearly not enough to check this
once at the first setup - since the innards behind the cloud can
in principle change at any time, this needs continuous monitoring
for possible single point of failure between independent clouds.

Is there already a method (maybe even API) to provide such a feature ?

-- Roland

PS: as a possible approach for this I can imagine the concept of
hardware sets used by clouds that are guaranteed to be disjunct
- think "a cloud in US" vs. "a cloud in europe" vs. "a cloud in asia".
Would that be good enough to exclude single points of failure ???



Rao Dronamraju schrieb:

--

**********************************************************
Roland Rambau Platform Technology Team
Principal Field Technologist Global Systems Engineering
Phone: +49-89-46008-2520 Mobile:+49-172-84 58 129
Fax: +49-89-46008-2222 mailto:Roland...@sun.com
**********************************************************
Sitz der Gesellschaft: Sun Microsystems GmbH,
Sonnenallee 1, D-85551 Kirchheim-Heimstetten
Amtsgericht München: HRB 161028; Geschäftsführer:
Thomas Schröder, Wolfgang Engels, Wolf Frenkel
Vorsitzender des Aufsichtsrates: Martin Häring
******* UNIX ********* /bin/sh ******** FORTRAN **********

[Roland Rambau]

Ray,

Ray schrieb:

> Why do they have to be "mega centers"?

a) to provide elasticity
b) to have any efficiency advantages

-- Roland

[Peglar, Robert]

I agree @Rao. 3-5 years from now, we will have some outstanding CSP facilities. I look forward to that day.

I can see a bifurcation occurring, fairly quickly; megacenters which are highly resilient, providing outstanding service with a strong SLA and a price to match, and others which are not resilient, provide spotty service with a weak SLA but are cheap.

One side races to the top, the other races to the bottom. Much like commercial centers today; they are either viewed at the C-level as an investment/strategic to the business, something to be proud of and leverage to make or save $$, or as a cost center/tactical, a necessary evil.

It's a stark contrast to be sure, but that contrast absolutely exists. Unfortunately, the latter view is dominant today.

[Ray Nugent]

Roland, while you can certainly gain these qualities from a mega, you can also derive them from all other size data centers as well. I'm just saying that size does not necessarily matter.

Ray

---

From: Roland Rambau <Roland...@Sun.COM>
To: cloud-c...@googlegroups.com
Sent: Fri, November 6, 2009 3:36:05 AM

[Ray Nugent]

Rao, put down the newspaper and go look at some "cloud" centers. While you'll certainly find some like in the article, I think you'll find more that are not. You might be surprised to find some very compact, efficient centers that fly in the face of this article.

Ray

---

From: Rao Dronamraju <rao.dro...@sbcglobal.net>
To: cloud-c...@googlegroups.com
Sent: Thu, November 5, 2009 8:11:55 PM

[Rao Dronamraju]

Amazon provides availability zones in its Cloud architecture.

"However, what is not obvious to me: how can a cloud *user*
ensure and control that the infrastructure behind 2 different
clouds has no common point of failure ?"

I think this is ensured by Amazon. I would assume they will design their
infrastructure in such a way that there isn't a common point of failure
between primary and secondary zones. Otherwise they are not provding REAL
availability/fault-tolerance any way. You may also want to take a look at
the following blog post that has some good information about Amazon
availability zones. It answers some or most of your questions.

Setting up a fault-tolerant site using Amazon’s Availability Zones
http://tinyurl.com/2ybok5

"Is there already a method (maybe even API) to provide such a feature ?"

Also straight from Amazon's mouth....
http://tinyurl.com/ylcewqa

Hope that helps.

Regards,
Rao

[Roland Rambau]

Ray,

but here I disagree: I think that for elasticity and for efficiency
size DOES necessarily matter a lot

-- Roland

Ray Nugent schrieb:

[Rao Dronamraju]

Ray,

 

What you are seeing is not CLOUDS!...they are data centers quickly rebranded as clouds to get into the action.

How many provide infinite resource capacity when they are compact?...How many have multi-tenancy implemented even at compact scale?...What is their CapEx and OpEx returns to CIOs?..of the cloud scale?..

Can you forward any URLs?...

 

 

---

[Nik Simpson]

Rao Dronamraju wrote:
>
> Amazon provides availability zones in its Cloud architecture.
>
> "However, what is not obvious to me: how can a cloud *user*
> ensure and control that the infrastructure behind 2 different
> clouds has no common point of failure ?"
>
> I think this is ensured by Amazon. I would assume they will design their
> infrastructure in such a way that there isn't a common point of failure
> between primary and secondary zones.

I don't think "I assume" will cut it in conversations with the CIO ;-)
The cloud providers are going to have to be very open about their
architectures and approach to high-availability if they want to be taken
as a serious alternative to traditional IT for critical applications.

--
Nik Simpson

[Ray DePena]

Rao,

Ok, I don't want to debate technicalities, I'm fast forwarding a bit looking into the future. 

Lets just call them "aggregation points" so as to not get caught up in definitions (and the subsequent debate regarding their non-existence).  I'm having difficulty seeing beyond that trend of "aggregation points", and at the moment, those are a bit like black holes sucking in all surrounding IT hw/sw/svcs and growing larger in the process.

-RD

[Rao Dronamraju]

"I don't think "I assume" will cut it in conversations with the CIO"

Yes, I agree. Since I am not the CIO, they have not opened it up to me:-).

But, you are right, the CTOs or EAs of the enterprises will learn about the
availability zone internals from the likes of Amazon and Racksapce etc
before making the major decision of migrating their data centers.

-----Original Message-----
From: cloud-c...@googlegroups.com
[mailto:cloud-c...@googlegroups.com] On Behalf Of Nik Simpson
Sent: Friday, November 06, 2009 5:30 PM
To: cloud-c...@googlegroups.com
Subject: [ Cloud Computing ] Re: Fwd: Rackspace Cloud Service Interruption

[cloud-computing]

I don't understand the need for two clouds. If your partner provides
you the ability to utilize/manage (vmware) High Avail/Fault Tolerance
and you have a VMs running simulateneously on separate hosts, if one
rack goes down due to poor power planning you should have zero
downtime as your other mirror should take over. Assuming you have the
ability to perform these features or manage them yourself which I hope
you would. Additionally, the same should go for diversity at the data
center level. If you have the proper ability to manage the DR
planning at the "user" level then if the data center goes dark
instantly you have already done your DR planning and should be able to
bring up your services elsewhere w/o much issue.

Brian

> Fax:   +49-89-46008-2222      mailto:Roland.Ram...@sun.com

> **********************************************************
>      Sitz der Gesellschaft: Sun Microsystems GmbH,
>      Sonnenallee 1, D-85551 Kirchheim-Heimstetten
>      Amtsgericht München: HRB 161028;  Geschäftsführer:
>      Thomas Schröder, Wolfgang Engels, Wolf Frenkel
>      Vorsitzender des Aufsichtsrates:   Martin Häring

> ******* UNIX ********* /bin/sh ******** FORTRAN **********- Hide quoted text -

[Ray Nugent]

Well I guess that's your definition of cloud. It's not mine. Cloud will not require One Big Room loaded with gear.That's just how they are implemented today because the software dictates it. Rather, they will have the ability to span resources, wherever they are, seamlessly in the not to distant future.

Ray (the original one)

---

From: Rao Dronamraju <rao.dro...@sbcglobal.net>
To: cloud-c...@googlegroups.com

Sent: Fri, November 6, 2009 3:19:39 PM

[Ray Nugent]

If you talk to a venture capitalist they will tell you they would never back a Huge Data center with an  investment. If, on the other hand you brought to them the concept of a small, highly efficient DC or a string of them that where integrated by software, I think you would get some action.

Ray (the original one)

---

From: "Peglar, Robert" <Robert...@xiotech.com>
To: cloud-c...@googlegroups.com
Sent: Fri, November 6, 2009 5:49:05 AM

[Alan Ho]

My personal recommendation is that if you are building new apps on the
public clouds, try to not assume "gold plated" datacenters such as
redundant generators when building your apps. Public clouds are
suppose to save money - it's usually cheaper to build redundancy in
the software layer than the hardware layer.

Regards,
Alan Ho

>> >> phyiscal hosts (VMWare Fault Tole> Fax:   +49-89-46008-2222      mailto:Roland.Ram...@sun.com

[Roland Rambau]

Brian,

cloud-computing schrieb:

> I don't understand the need for two clouds.

what if your single cloud provider goes bankrupt ? or gets
involved in a criminal investigation and their equipment
gets taken away by investigators ?

I have seen happen: a bomb is found and police evacuates
surrounding buildings immediately and noone is allowed return
into them for hours - and that included the single datacenter
of a bank ... ( ok, that was more than 10 years ago )

I still think it is obvious that one needs at least 2 clouds.

> If your partner provides
> you the ability to utilize/manage (vmware) High Avail/Fault Tolerance
> and you have a VMs running simulateneously on separate hosts, if one
> rack goes down due to poor power planning you should have zero
> downtime as your other mirror should take over. Assuming you have the
> ability to perform these features or manage them yourself which I hope
> you would. Additionally, the same should go for diversity at the data
> center level. If you have the proper ability to manage the DR
> planning at the "user" level then if the data center goes dark
> instantly you have already done your DR planning and should be able to
> bring up your services elsewhere w/o much issue.

well, yes, but only if not all of your data are in the first cloud
and unaccessible now.

You maybe can get by with computing in only one cloud, but your data
( the part that is required for a restart only ) needs to be
held in 2 coulds ( or inside and outside of the single cloud, but
would that be any better ? ).

-- Roland

Fax: +49-89-46008-2222 mailto:Roland...@sun.com

[Rao Dronamraju]

So Google, Yahoo and MS are all wrong but VC’s are right?....VC’s cannot afford maga data centers, that is why they don’t fund.

 

http://tinyurl.com/la7tte

 

http://tinyurl.com/ye8kw45

 

---

[Rao Dronamraju]

“Well I guess that's your definition of cloud. It's not mine.”

 

So your cloud definition does not have

 

1)       Infinite resource elasticity

2)       No multi-tanancy

3)       No CapEx, OpEx savings as the main business driver for CIOs

 

“That's just how they are implemented today because the software dictates it. Rather, they will have the ability to span resources, wherever they are, seamlessly in the not to distant future.”

 

Software has nothing to do with it. It is the WAN bandwidths!.

 

“Rather, they will have the ability to span resources, wherever they are, seamlessly in the not to distant future.”

 

Spawning resources is not such a big deal….it is getting to your data….it is either in the data center or you access it over WAN.

[Mabuse68]

I'd like to add that the cloud provider itself should offer an option
for contingency at another provider. But this could be asking too much
to the Cloud industry, too much busy selling smoke to understand
customer's needs: this is a rare case where competition has
detrimental effect on the offer.
Cloud means availability and no lock-in? So why are IaaS providers not
giving us an interperable environment?
I am a cloud backer but I do not necessarly agree with this best
effort economy that is taking on

gabriele b

On Nov 6, 4:19 pm, Ray Nugent <rnug...@yahoo.com> wrote:
> Roland, while you can certainly gain these qualities from a mega, you can also derive them from all other size data centers as well. I'm just saying that size does not necessarily matter.
>
> Ray
>
> ________________________________

> From: Roland Rambau <Roland.Ram...@Sun.COM>

> Fax:  +49-89-46008-2222     mailto:Roland.Ram...@sun.com

[cloud-computing]

I failed to give consideration to financial uncertainty. We are debt
free so I wasn't thinking about the very real possiblity of funded
companies that may have funding dry up or any other reason. I see
your point and concur. This brings up the challenge of platform
compatibility. For example, we are running VMWare's platform so if
you had the keys (literally and figurately) to control your data's
fate from two providers and were able to bridge the two platforms it
would be ideal. As a method of operation we are going to utilize our
two data centers with 50% production at each site and 50% DR at each
site so no one site has all production or DR but 50% of each. I'd
be more than willing to allow a customer the ability to protect
themselves in this fashion just like we do but using another vendor in
the mix. In fact I believe this presents a very good opportunity.
However, it would be quite difficult to put this into action with two
"clouds" running disparate platforms.

Brian

[cloud-computing]

I disagree. This would be a great selling point from my perspective
and I am very willing to embrace this opportunity. The only thing I
need is code to bridge our platfrom to the others so they work
properly. Does anyone know of anyone who is developing this? Of
course clear DR processes would need to be tested between platforms.
Thinking on this further....why would this need to be limited to DR?
Why couldn't the HA/FT options be leveraged as well? The only problem
here of course would be WAN connectivity limitations?

Brian

> ...
>
> read more »- Hide quoted text -

[Jan Klincewicz]

Check out what Marathon Technologies does ... even with WANs.  They are based on XenServer (and I understand developing for Hyper-V as well.)

http://www.marathontechnologies.com/

It is not as latency-constrained as you might think once things get synchronized ....

I disagree.  This would be a great selling point from my perspective
and I am very willing to embrace this opportunity.  The only thing I
need is code to bridge our platfrom to the others so they work
properly.  Does anyone know of anyone who is developing this?  Of
course clear DR processes would need to be tested between platforms.
Thinking on this further....why would this need to be limited to DR?
Why couldn't the HA/FT options be leveraged as well?  The only problem
here of course would be WAN connectivity limitations?

--
Cheers,
Jan

[cloud-computing]

I'm reading www.rightscale.com right now that manages multiple cloud
deployments. Interesting