Something worth discussing...

[Chris Marino]

I agree w/Jose here.  It's been a while since I've anything on this group worth participating in.  The occasional topic that is interesting quickly become personality driven and IHMO suppresses legitimate criticism and commentary.

Also, it seems that some approaches/providers/alternatives seem to have a Most Favored Vendor status, while other's, inexplicably, can't seem to get a break.

I'm generalizing, of course, but it seems that Amazon can do no wrong, and nobody seems to care about Microsoft. 

What's with that?

Here's an example.  Yesterday Microsoft began fleshing out the details of their SQL Data Services and judging by the interest on this group you'd never know it ever happened.

http://blogs.msdn.com/ssds/archive/2009/03/10/9469228.aspx

IMHO, this is a pretty big freak'n deal. 

I was going to post something here last night, but then waited to see if anyone else would bring it up. Hasn't happened....

Compare that to the chatter we often see posted here about the latest and greatest DB/researchy/shiny thingy that's often not much more than a research project. CouchDB, SimpleDB, Hadoop, Haskell, BigTable, and on and on...

Something happens here and withing a second or two there are threads and replies galore.

Don't get me wrong, I think these topics are interesting and worth discussing, it's just that there's a lot more to Cloud Computing than you'd realize if all you ever read were the threads on this group.

CM

On Wed, Mar 11, 2009 at 10:06 AM, Jose Gonzalez <jed...@yahoo.com> wrote:

Is it me, or is this list becoming more and more like the school yard that is ruled by a couple of bullies whom always have to have the last word?

This type of comments and behavior is really of a low taste and very unprofessional. The purpose of this forum is for us to bring ideas and talk about them like real professionals. Not to be making jokes at someone else's expense.

jeg

---

From: Jan Klincewicz <jan.kli...@gmail.com>
To: cloud-c...@googlegroups.com
Sent: Wednesday, March 11, 2009 7:07:20 AM
Subject: [ Cloud Computing ] Re: Cloud Computing: How many servers does the world need?

On Tue, Mar 10, 2009 at 4:39 PM, John D. Mitchell <jdmit...@gmail.com> wrote:

 [and now I really am
going to stop participating in this part of this thread -- my doctor
says that I need to stop debating with fanatics :-).]

John

It's a good step seeking professional help.  Perhaps he can get your meds right..  Being a fanatic can be an asset in this industry.  Being delusional is never a good thing.


--
Cheers,
Jan

[Christopher Steel]

Chris,

                I agree that this does look like a big deal, thanks for posting the blog link. I am not that familiar with TDS or its performance. Is there a link somewhere that shows some performance benchmarks of TDS versus ODBC and other local connections? What would be the approximate amount of time to upload a 50Gb database?

 

-Chris

[Khazret Sapenov]

Is there any pricing/charge model for SDS yet?

The only information, I could find is below:

How will Microsoft charge for the Azure Services Platform?

Subject to certain limits, Azure Services will be available without charge during our Community Technology Preview (CTP). Once Microsoft Azure launches for commercial use, we will offer a portfolio of services and you will be billed according to your actual consumption of each service. Based on feedback during the CTP period, pricing offers may be provided based on the following parameters.

• Windows Azure - Compute and Storage services
• .NET Services - Access Control, Service Bus, and/or Workflow Services
• SQL Services- Database service for LOB applications
• SharePoint Services (future) - SharePoint components that developers can utilize and build into their application

How will resource utilization be measured?

Monitoring agents in the Azure platform will measure specific resource utilization. However, no specific pricing or consumption models will be announced until we have received sufficient input from the user community and partners during the CTP period. This will include:

• CPU time, measured in CPU-hours
• Bandwidth for ingress/egress from the data center, measured in GBs
• Storage, measured in GBs
• Transactions, measured as requests likes Gets & Puts

While technology may be very advanced and novel, I'd like to know how it fits my budget.

regards,

Khazret Sapenov

[Dave Graham]

You'd prolly get a kick out of this then:
http://flickerdown.com/2009/03/a-vision-of-failure/

;)

Enjoy (and yes, it's about the cloud)

Dave

>> ------------------------------
>> *From:* Jan Klincewicz <jan.kli...@gmail.com>
>> *To:* cloud-c...@googlegroups.com
>> *Sent:* Wednesday, March 11, 2009 7:07:20 AM
>> *Subject:* [ Cloud Computing ] Re: Cloud Computing: How many servers does

>> the world need?
>>
>>
>>
>> On Tue, Mar 10, 2009 at 4:39 PM, John D. Mitchell
>> <jdmit...@gmail.com>wrote:
>>
>>> [and now I really am
>>> going to stop participating in this part of this thread -- my doctor
>>> says that I need to stop debating with fanatics :-).]
>>>
>>>
>>> John
>>>
>>>
>>>
>> It's a good step seeking professional help. Perhaps he can get your meds
>> right.. Being a fanatic can be an asset in this industry. Being
>> delusional
>> is never a good thing.
>>
>>
>> --
>> Cheers,
>> Jan
>>
>>
>>
>>
>> >
>>
>
> >
>

--
Sent from my mobile device

Dave Graham
1207 Main St. #2
Holden, MA 01520
978.239.2489

[Adwait Ullal]

Chris,

 

TDS [1] is the low-level protocol that ODBC, etc utilize to talk to the database.

 

[1] http://en.wikipedia.org/wiki/Tabular_Data_Stream

- Adwait
--
Adwait Ullal

w: http://www.adwait.com
p: (408) 898-2581

[Christopher Steel]

I went back and reread the blog after doing a little research on TDS. I now understand that it is the proprietary (now open) protocol that runs over top of TCP/IP. There is even a TDS type 4 JDBC driver. What I don’t understand is how Microsoft can claim that I can make use of all of my existing DB tools, considering the firewall issues. The only way to get at SDS through a firewall is “via HTTP/REST using the ADO.Net Data Services framework”. [Note: REST is not a protocol, it is a set of architecture principles]. Since none of my tools were built on top of ADO.Net Data Services, I really have not gained anything. Is there a client proxy that I can use to connect TOAD through my corporate firewall to SDS?

 

-Chris

[Chris Marino]

What I found very interesting about this post was what seemed both obvious as well as unique among providers...

"But the universal feedback we received from our TAP partners and other early adopters was the need for a relational database delivered as a service"

Seems obvious, right?

But who else is talking about providing this kind of service?

I wouldn't really expect it from the IaaS folks, but then, AWS has something they're calling a DB, but I don't expect to see any relational capabilities in there any time soon.

I'd expect it from the PaaS guys, but Salesforce and GAE aren't anywhere near this either.

What about:

"We are providing an experience where a developer can take an existing application and just change the connection string to point it to the cloud and have it just work"

If that doesn't scare the crap out of the competition, their kidding themselves.

And then there's

"Most importantly for developers, this means symmetric SQL Server functionality and behavior combined with compatibility with the existing tools you are familiar with.

                Tables?...Check

                Stored Procedures?...Check

                Triggers?...Check

                Views?...Check

                Indexes?...Check

                Visual Studio Compatibility?...Check

                ADO.Net Compatibility?...Check

                ODBC Compatibility?...Check"

The full potential of Cloud Computing can only be realized  when existing apps can be moved to the cloud with a simple deploy to cloud 'Easy' button.  Microsoft clearly recognizes this. If only new apps can take advantage of the Cloud (that have to be totally re-engineer for a non-relational DB)  it's going to remain the tinkerers playground.

Khazret's point about pricing is legit, but somewhat orthogonal to the technical and adoption hurdles they are addressing.

CM

[xd...@comcast.net]

I don't think any of the Azure Services from MSFT have announced
pricing yet, and many are just now getting into beta (if I recall my
Jan09 data correctly). I do think that MSFT is poised to come on very
strong for the late summer to fall time frame with their S+S and
related technology releases. Definitely worth looking at IMO.



On Mar 11, 11:23 am, Khazret Sapenov <sape...@gmail.com> wrote:
> Is there any pricing/charge model for SDS yet?The only information, I could

> find is below:
> How will Microsoft charge for the Azure Services Platform?
>
> Subject to certain limits, Azure Services will be available without charge
> during our Community Technology Preview (CTP). Once Microsoft Azure launches
> for commercial use, we will offer a portfolio of services and you will be
> billed according to your actual consumption of each service. Based on
> feedback during the CTP period, pricing offers may be provided based on the
> following parameters.
>

>    - Windows Azure - Compute and Storage services
>    - .NET Services - Access Control, Service Bus, and/or Workflow Services
>    - SQL Services- Database service for LOB applications
>    - SharePoint Services (future) - SharePoint components that developers

>    can utilize and build into their application
>
> How will resource utilization be measured?
>
> Monitoring agents in the Azure platform will measure specific resource
> utilization. However, no specific pricing or consumption models will be
> announced until we have received sufficient input from the user community
> and partners during the CTP period. This will include:
>

>    - CPU time, measured in CPU-hours
>    - Bandwidth for ingress/egress from the data center, measured in GBs
>    - Storage, measured in GBs
>    - Transactions, measured as requests likes Gets & Puts

>
> While technology may be very advanced and novel, I'd like to know how it
> fits my budget.
>
> regards,
> Khazret Sapenov
>
> On Wed, Mar 11, 2009 at 2:15 PM, Christopher Steel <cst...@fortmoon.com>wrote:
>
>
>
> >  Chris,
>
> >                 I agree that this does look like a big deal, thanks for
> > posting the blog link. I am not that familiar with TDS or its performance.
> > Is there a link somewhere that shows some performance benchmarks of TDS
> > versus ODBC and other local connections? What would be the approximate
> > amount of time to upload a 50Gb database?
>
> > -Chris
>

> > *From:* Chris Marino [mailto:christopher.c.mar...@gmail.com]
> > *Sent:* Wednesday, March 11, 2009 1:50 PM
> > *To:* cloud-c...@googlegroups.com
> > *Subject:* [ Cloud Computing ] Something worth discussing...

> >  On Wed, Mar 11, 2009 at 10:06 AM, Jose Gonzalez <jedg...@yahoo.com>

> > wrote:
>
> > Is it me, or is this list becoming more and more like the school yard that
> > is ruled by a couple of bullies whom always have to have the last word?
>
> > This type of comments and behavior is really of a low taste and very
> > unprofessional. The purpose of this forum is for us to bring ideas and talk
> > about them like real professionals. Not to be making jokes at someone else's
> > expense.
>
> > jeg
>

> >  ------------------------------
>
> > *From:* Jan Klincewicz <jan.klincew...@gmail.com>
> > *To:* cloud-c...@googlegroups.com
> > *Sent:* Wednesday, March 11, 2009 7:07:20 AM
> > *Subject:* [ Cloud Computing ] Re: Cloud Computing: How many servers does
> > the world need?
>
> >  On Tue, Mar 10, 2009 at 4:39 PM, John D. Mitchell <jdmitch...@gmail.com>

> > wrote:
>
> >  [and now I really am
> > going to stop participating in this part of this thread -- my doctor
> > says that I need to stop debating with fanatics :-).]
>
> > John
>
> > It's a good step seeking professional help.  Perhaps he can get your meds
> > right..  Being a fanatic can be an asset in this industry.  Being delusional
> > is never a good thing.
>
> > --
> > Cheers,

> > Jan- Hide quoted text -
>
> - Show quoted text -

[Ray Nugent]

Chris, you can do relational DB (MySQL, Oracle, DB2) on most other cloud vendors today (except Google) so I'm not quite sure why this is a big deal? Even though it's announced it's not really usable yet so, again, what's the big deal?

I think the issue is MSFT is two years behind the leaders and not really announcing significant technological differences other than integration with their existing customer base. How do they make significant money shifting that base from one platform to another?

0.02 cents...

Ray

---

From: Chris Marino <christophe...@gmail.com>
To: cloud-c...@googlegroups.com
Sent: Wednesday, March 11, 2009 12:40:57 PM

Subject: [ Cloud Computing ] Re: Something worth discussing...

[Adwait Ullal]

Ray,

 

Your examples of databases run in virtualized instances of an OS, not natively. SSDS provides relational functionality natively (think RDBMS functionality for Simple/Big Table).

 

Hope that helps.

- Adwait
--
Adwait Ullal

w: http://www.adwait.com
p: (408) 898-2581

[Chris Marino]

Ray,

Sure you can run DBs in AWS, that's never been the issue.  That's IaaS.

Getting the PaaS guys to provide an RDBMs is what's new.

I don't want to go down the 'What is Cloud Computing' rabbit hole, but saying they're 2 yrs behind isn't really fair when you consider they're delivering a platform v. infrastructure.

CM

[Ray Nugent]

Chris, thanks for the clarification (and not going down the hole!) My point was I think the lack off comments here about it is simply because AWS is pretty well known by this group and Azure is less so at this point.

Not sure I understand the argument of running on a VM vs OS but I guess that would be important to an OS vendor...

Ray

---

From: Chris Marino <christophe...@gmail.com>
To: cloud-c...@googlegroups.com

Sent: Wednesday, March 11, 2009 2:32:31 PM

[Jeanne Morain]

This is an interesting thread...

 

There are other GAPs that need to be overcome that I have seen some interesting startups try to tackle such as Singularity Technology around on demand capacity based on the application and available hardware from within the app (this would need to include database).

 

MS has not been known to be a thought leader when it comes to management and delivery of applications to the datacenter or desktop but they are masters at commoditizing the market once it starts into the tornado (similar to what they have done with SMS - now SCCM and are trying to do with virtualization). 

 

ISVs do need to jump on board for the cloud and there needs to be away to import legacy applications. Virtualization of the Application - not in a VM but actual application virtualization can be a much stronger avenue to make that happen sooner for legacy apps.  This model has already been proven by many customers in the Citrix and Terminal Services Environment.

 

The real test of the Cloud will be connectivity and being able to "check in and out" of the Cloud as needed based on limited connectivity in many areas still today.  Remember 3G in the US is still nascent and riddled with dropped data, calls, and deadspots.  There are many rural areas that do not have succinct access not only in the US but across developing countries that are building out their infrastructure. 

 

Today's youth will want to take their Apps and Data to go and compliance will mandate stricter controls to tying it down.  Balance can be struck with digital rights management and user based provisioning both within and outside of the cloud but building out the right management paradigm.  This was NOT easy for the physical disconnected world.  Although the datacenter will prove to be much easier because it is pretty locked down - achieving economies of scale, dealing with duplicate data, etc will be a challenge that needs to be overcome to address compliance concerns while meeting demands of a much more technical and mobile workforce.

---

From: Ray Nugent <rnu...@yahoo.com>
To: cloud-c...@googlegroups.com
Sent: Wednesday, March 11, 2009 4:44:51 PM

[Sriram Krishnan]

Khazret Sapenov wrote:
> Is there any pricing/charge model for SDS yet?
> The only information, I could find is below:
> How will Microsoft charge for the Azure Services Platform?
>
>

We haven't announced any pricing information yet. We have committed to
having one in 2009 but I can't be more specific than that.

--
Sriram Krishnan
http://www.sriramkrishnan.com

[kevin...@intel.com]

On Mar 11, 7:40 pm, Chris Marino <christopher.c.mar...@gmail.com>
wrote:

> The full potential of Cloud Computing can only be realized  when existing
> apps can be moved to the cloud with a simple deploy to cloud 'Easy' button.
> Microsoft clearly recognizes this. If only new apps can take advantage of
> the Cloud (that have to be totally re-engineer for a non-relational DB)
> it's going to remain the tinkerers playground.

An 'easy' deploy button really is needed but what I don't see in SDS
is any reason
why I would choose this offering over any other partial relational
solution other
than they are less mature today. I say partial because any app with
significant
scale will still need some non-relational re-engineering work to make
SDS viable,
it's not a silver bullet but more an easier on-ramp because of the
data partitioning
concerns.

My guess is that the re-engineering costs would be significantly
bigger than what
you can save by pressing this "easy" button, but of course its a path
of very little
resistance and that often counts more.

Kev

[Chris Marino]

Not sure I fully understand your point...

If you're saying that if I've got to do non-relational things, then SDS isn't any help, then I would agree. I'll also agree that BigTable, SimpleDB etc. is likely better suited than SDS for these kind of things. Especially if you've got to scale pretty large.

[A separate point but one worth noting is that if you *don't* have to scale, then a simple hash table might be even easier than any of these non-relational stores]

But that's different from taking an existing app that's been designed to:
   1. Serve my internal needs (i.e. not much scaling),
   2. That uses SQL Server,
   3. That's already addressed the non-relational parts with standard programming techniques,
   4. That I want to deploy in the cloud, on a platform (not infrastructure) without re-engineering.

In this case, you've really got to have the things that their putting in SDS.

I realize that those are a lot of 'ifs', but I think that's exactly the market that SDS/Azure is targeting.

CM

[Bernard Golden]

Chris, thanks for bringing this to the group's attention. I was not
aware of it (can't track everything), but for sure this is a big deal.
I got a bit lost in the TDS language, but the money quote is "take
your existing app and run it in the cloud." This is huge and will
significantly reduce the barrier to using the cloud, particularly
among typical MS users.

As to the comment earlier in this thread about "why is this a big
deal, it's just shifting usage from one platform (on-premise
installation) to another (cloud installation)," that's true, as far as
it goes. However, if one (as presumably Microsoft does) believes that
cloud computing is a platform shift akin to client/server (where
Microsoft got rich) and Internet (where Microsoft was late to the
game), it's critical to offer it; otherwise, a platform provider risks
being left at the starting gate in the battle for apps on the new
platform. In this formulation, Microsoft has no choice to offer a
strong cloud capability, even if it does cannibalize its current
business. We haven't seen pricing for Azure, but I'm sure Microsoft
will figure out a way not to go broke with it.

The biggest reservation I had about the earlier version of Azure is
that it imposed the need to use a different architecture and didn't
offer equivalent usage models to its established products; this was
particularly an issue given that a significant proportion of
Microsoft's development base is less technical -- forcing it to adopt
a new usage model imposed adoption friction for Azure.

With this new development, Microsoft has reduced that friction
enormously, and that's a big deal. It gives it a hybrid IaaS/PasS
offering that is pretty compelling. Very impressive.

Chris, thanks again for posting.

On Mar 12, 6:57 am, Chris Marino <christopher.c.mar...@gmail.com>
wrote:

[kevin...@intel.com]

On Mar 12, 1:57 pm, Chris Marino <christopher.c.mar...@gmail.com>
wrote:

> Not sure I fully understand your point...
>
> If you're saying that if I've got to do non-relational things, then SDS
> isn't any help, then I would agree. I'll also agree that BigTable, SimpleDB
> etc. is likely better suited than SDS for these kind of things. Especially
> if you've got to scale pretty large.

Hi Chris,

Sorry, I wasn't so clear, really that to scale you have to step
outside the pure relational model and modify your app. If you are SQL
Server already then SDS is obviously a prime choice but the 'modify
your app' bit that is likely where the larger cost is to the point I
would think it's maybe better to drive the architecture choice from
that rather than DBMS affinity. Comments from the SDS team appear to
indicate they want to try and minimize the point that you have to
start doing non-relational things such as horizontal partitioning and
emploing map-reduce kind of techniques but it's not so clear to me
what kind of techniques you can really employ to achieve that.

>
> But that's different from taking an existing app that's been designed to:
>    1. Serve my internal needs (i.e. not much scaling),
>    2. That uses SQL Server,
>    3. That's already addressed the non-relational parts with standard
> programming techniques,
>    4. That I want to deploy in the cloud, on a platform (not infrastructure)
> without re-engineering.
>
> In this case, you've really got to have the things that their putting in
> SDS.

Ok, so yes to the SDS features. Probably more importantly, when you
push to the cloud you will likely want more scale-out to compensate
for the lack of scale-up options. How much of a problem that is
depends on how scaled-up you are to start with and how much headroom
you want for increased demand, if any. I do like the look of SDS, but
I can see the benefits of easy migration overshadowing in the
marketing the practicalities of actual use. Perhaps this is the market
sweet spot despite that?

I am rather curious about how at the low end & high end we appear to
be seeing a change in RDBMS use patterns driven by FOSS and Scaling
respectively. Maybe that is just a temporary blip while the database
vendors re-align, but if feels a bit more significant to me.

Kev

[Chris Marino]

Kevin, I think we're on the same page here. One minor quibble with
your characterization of apps migrated to cloud is your claim that
they'll likely need to be more scalable. Not sure there is much/any
evidence supporting that, and intuitively, I'd think that the scaling
requirements would remain roughly unchanged internal v. cloud (scale
linearly w/users which I think will be roughly constant or slowly
growing).

As for low and hi ends splitting off from traditional RDBMs, that
seems real to me as well. It seems the primary driver is the need for
transaction processing. Anyone ever see a TPC benchmark for mySQL?
Don't even waste your time looking for one, no one really cares enough
to publish the results. Even more unlikely is one for SimpleDB....

Maybe someone will come up with a better name for these data stores,
or maybe we just need to be pedantic about the difference between DBs
and RDBMSs...

CM

[gaberger]

It seems the realization that our apps rely heavily on key/value pairs and such, there are better architectures for handling scale outside the RDBMS.. The other scaling part for large datasets, heavy transactional load are also benefitting from scale (Greenplum, Vertica, Astor Data, Hadoop, etc..)..  

Attached: Team with Michael Stonebraker published " The End of an Architectural Era (It’s Time for a Complete Rewrite)


Also see http://www.forrester.com/Research/Document/Excerpt/0,7211,53871,00.html


Since we aren’t scaling up anymore (heat, power, etc...) due to the thermodynamics of the chip, we are going wide. The demands for data- reading && writing have grown off the curve so slicing, partitioning, sharding, etc data across many machines has been proven (aka Google) to be a reliable cost-effective way to scale..

-g

--

[John Pugh]

Chris Marino wrote:
> As for low and hi ends splitting off from traditional RDBMs, that
> seems real to me as well. It seems the primary driver is the need for
> transaction processing. Anyone ever see a TPC benchmark for mySQL?
> Don't even waste your time looking for one, no one really cares enough
> to publish the results. Even more unlikely is one for SimpleDB....
>

First off - there are TPC benchmarks for MySQL and people do care.
SimpleDB is not comparable as it's a very different beast. Stick with
Apples & Apples. When you do, MySQL puts MS SQL to shame.

I doubt the change in use patterns can be attributed to FOSS, however it
can be attributed to the need for a more specialized storage mechanism
as SimpleDB was designed to address.

I really don't see how you can put MS SQL into the mix as it really does
not fit into the cloud space aside from the fact that MS requires it's
use for most anything that requires database storage. It's not useful,
is not cross platform, and it really doesn't fit in a "cloud"
environment, IMBO.

[carl...@gmail.com]

Chris,

I completely agree with your point regarding how important it is to
move *existing* applications --- applications already running in the
datacenter --- into the cloud; that it should be as easy as a push of
a "Deploy" button. It's exciting to explore all the possibilities of
new architectures that cloud computing enables, but migrating existing
datacenter apps into the cloud is important if we want to see quick
adoption.

I thought I'd mention Appzero, a product focused on precisely this
challenge. Here's a 6 minute demo video demonstrating movement of an
existing application from datacenter to EC2 to GoGrid:
http://www.appzero.com/content/we-launched-demo-09.

Carl

On Mar 11, 3:40 pm, Chris Marino <christopher.c.mar...@gmail.com>
wrote:

> The full potential of Cloud Computing can only be realized  when existing
> apps can be moved to the cloud with a simple deploy to cloud 'Easy' button.
> Microsoft clearly recognizes this. If only new apps can take advantage of
> the Cloud (that have to be totally re-engineer for a non-relational DB)
> it's going to remain the tinkerers playground.
>

> CM
>
>

[Shannon Prager]

Appistry just made an announcement earlier this week to address the challenge of moving existing applications into a cloud: http://gigaom.com/2009/03/09/appistry-opens-the-cloud-to-almost-all-apps/

Shannon Prager

-----Original Message-----
From: cloud-c...@googlegroups.com [mailto:cloud-c...@googlegroups.com] On Behalf Of carl...@gmail.com
Sent: Thursday, March 12, 2009 2:06 PM
To: Cloud Computing
Subject: [ Cloud Computing ] Re: Something worth discussing...

[Chris Marino]

John, I should have been more precise on my TPC comment. Apologies for
being snarky. As for SimpleDB, yes, it a different beast entirely.
I was being facetious.

Within the context of my comment, I thought is would be clear that was
referring to TPC-C, the popular transaction oriented benchmark in the
suite.

My point about mySQL and TPC-C is based on the published results here:

http://www.tpc.org/tpcc/results/tpcc_results.asp?orderby=dbms

This is the 'Complete TPC-C Results List - Sorted by Database Vendor'

The only place I could find mySQL there was for the TPC-H benchmark on
Kickfire, which is, again, a different context from the one I was
referring to.

As for MS SQL not belonging in this space, I can't speak to that, but
I'm not sure everyone would agree on your criteria for belonging.

CM

[Jim Starkey]

If it's on any use, MySQL uses the open source version of TPC-C called
dbt2. It's the same thing, but without the adult oversight. MySQL
numbers tend to fudge on things like think time. A general problem with
the benchmark, however, is that is a very poor model of an interactive
site, measuring transactions per minute rather the max transactions at
an acceptable latency. It also assumes a flat data distribution, which
eliminates any measure of effective cache usage.

--
Jim Starkey
President, NimbusDB, Inc.
978 526-1376

[Greg Pfister]

On Mar 12, 3:49 pm, Jim Starkey <jstar...@nimbusdb.com> wrote:
> If it's on any use, MySQL uses the open source version of TPC-C called
> dbt2.  It's the same thing, but without the adult oversight.  MySQL
> numbers tend to fudge on things like think time.  A general problem with
> the benchmark, however, is that is a very poor model of an interactive
> site, measuring transactions per minute rather the max transactions at
> an acceptable latency.  

Is this a change? TPC-C as I recall it specified the latency allowed,
and then measured maximum transactions under those conditions.

> It also assumes a flat data distribution, which
> eliminates any measure of effective cache usage.

I guess this depends on your definition of "flat data distribution."

Within one node / input stream, yeah.

But I railed for a whole chapter in "In Search of Clusters" about how
TPC-C is so partitioned that it could scale on clusters with nodes
connected with wet string. >90% of all data is local to the node into
which the request for it came. Makes it useless as a cluster
benchmark.

This was deliberately architected into the benchmark by a certain
company (since swallowed by another) whose name begins with "T", It
was done cleverly; you can't really discover it until you look closely
at the spec's rules for the random number generator. (!)

Attempts to change it were then vilified, saying someone wanted to
make a "cluster buster."

Greg Pfister
http://perilsofparallel.blogspot.com/

[Jeanne Morain]

Carl,

Thanks for the marketing pitch.  You should let people know - as our Microsoft peers did that you work for a vendor - http://www.appzero.com/content/appzero-leadership

My disclaimer - I am one of the original PMs for BSM at Marimba and later BMC, Managed the BMC Configuration Automation Product Line in such evaluated many of the app virt tools (MS, SVS, Trigence/AppZero, Thinstall/VMware), was the VP of Marketing and Alliances at Thinstall (acquired by VMware), Launced ThinApp and was a major contributor to Universal Client strategy at VMware and recently left (December) to join InstallFree as their VP of BD and Corporate Strategy.  www.installfree.com 

For a movement to take off - vendors do need to be a part of ecosystem but these forums should not be a marketing pitch but a venue to educate on the pros, cons, and pittfalls of the technology so the solutions constructed will work for real world scenarios.  Specifically when technology is so integrated into Banking, EchoCardiograms - it literally means life and death.

Realistically - there are many approaches to application virtualization.  Partners and consumers focusing on the cloud need to take into account architectural approach, compliance, manageability and more around the end user experience.

For the group it is important to understand the different types of application virtualization, implications on and off the cloud and ramifications from a security, compliance and architecture perspective.

There are 3 types of approaches to application virtualization (not including the hardware virtualization aspects).  Noting the first two - Agent based and Agentless (embedded Client) were developed prior to the notion of Cloud Computing, Compliance Requirements (SAS 70 Audits) and even BSM in many respects (to deal with Service Desk, Change Control, Asset Mgmt, Risk Management). 

This is still a nascent market and when evolved Application Virtualization will be critical component or approach to enabling cloud computing if the pieces are built out correctly to support broader based use cases both on and off the cloud.  Specifically to deal with Roaming Users and smaller shops struggling to adhere to strict regulations in today's economy with more government oversight to come. 

When thinking about what and how to use these different tools it is important to understand the architectural differences, impact on end user experience, security ramifications and risks, and how they will solve real world issues around lifecycle of the application.

Agent Based - connect to the core OS - redirecting the applications, utilization etc to be managed by the "virtual agent" versus installing with the underlying registry.  Agent based tools often leave the "sequencing" of applications up to IT to determine usage, memory allocation, etc.  Pros - these agents plug directly into the underly OS.  This model works well for Server Based Scenarios - for applications that can only be in the cloud - with always connected users. 

There are pieces though to consider in terms of implementation in that no two users have identical usage patterns so complexity of the application and user experience needs to be considered, negates personal owned PCs where users do not want an agent installed - (such as those used by on call physicians, mortgage brokers, insurance agents, etc), and has limited adaptability off the cloud or for ISVs that have not sanctioned their solution with that tool.

Agentless - These tools embed the "client" component within their "mini-OS" translation layer for each individual application.  Each bubble contains the application and code.  Each piece is managed separately.

Hybrid - Agentless Client.  This approach provides the best of both worlds meaning the client is present but not integrated or dependant on the OS (specifically in cases for server based processing or offline - when the OS may crash or have other issues we are all too familiar with).  The client allows for centralized management of all the virtual bubbles without having to set precedence, complex sequencing etc.  The client itself can be easily updated in a single place on the system versus multiple. 

There are many pros and cons to each approach.  For those really interested in learning more or implementing these types of tools as the Cloud comes in to play (all about Apps and User Experience) - it would be best to visit more objective sites like www.brianmadden.com, - they have a great comparison document they posted that is independant.

My area of interest from this group is the education and proliferation of Compliance, Universal Clients, and Checking Out capability for the Cloud consumers.  Interested in educating and learning more from the group on what needs to happen from a desktop, network, storage etc. is critical - as well as inventory controls for audit and compliance.  Thinking of the cloud in terms of just the server side components would be a disservice to the broadscale adoption and further delay true universal client computing..

----- Original Message ----
From: "carl...@gmail.com" <carl...@gmail.com>
To: Cloud Computing <cloud-c...@googlegroups.com>
Sent: Thursday, March 12, 2009 12:05:57 PM
Subject: [ Cloud Computing ] Re: Something worth discussing...

[Jim Starkey]

Greg Pfister wrote:
> On Mar 12, 3:49 pm, Jim Starkey <jstar...@nimbusdb.com> wrote:
>
>> If it's on any use, MySQL uses the open source version of TPC-C called
>> dbt2. It's the same thing, but without the adult oversight. MySQL
>> numbers tend to fudge on things like think time. A general problem with
>> the benchmark, however, is that is a very poor model of an interactive
>> site, measuring transactions per minute rather the max transactions at
>> an acceptable latency.
>>
>
> Is this a change? TPC-C as I recall it specified the latency allowed,
> and then measured maximum transactions under those conditions.
>

There are latency requirements, but are generally that 90% of
transactions complete in under 5 seconds. Web application usually
strive for less under one second latency measured at the server. A more
meaningful measure would be how many threads could be supported with an
average latency of .5 second and a max of 1 second.

The issue is fairly serious from a usability perspective (five seconds
is a very, very long time to wait for web page). In a nutshell, it is
the difference between on-line and batch.

>
>> It also assumes a flat data distribution, which
>> eliminates any measure of effective cache usage.
>>
>
> I guess this depends on your definition of "flat data distribution."
>
> Within one node / input stream, yeah.
>
> But I railed for a whole chapter in "In Search of Clusters" about how
> TPC-C is so partitioned that it could scale on clusters with nodes
> connected with wet string. >90% of all data is local to the node into
> which the request for it came. Makes it useless as a cluster
> benchmark.
>

Hmmm. I shall have to look that up.

> This was deliberately architected into the benchmark by a certain
> company (since swallowed by another) whose name begins with "T", It
> was done cleverly; you can't really discover it until you look closely
> at the spec's rules for the random number generator. (!)
>

Tell me more, please.

[Ray Nugent]

Amazon must think MSFT is getting close. Pricing just got a great deal better...

http://blogs.zdnet.com/BTL/?p=14420

Ray

[David Kavanagh]

More calculations here...

http://coderslike.us/2009/03/12/you-can-now-by-options-for-amazon-ec2-instances/

[Chris Marino]

There's another name for 'Reserved Instances' and it's been around for a long time.  It's called 'Managed Hosting'

I have to admit that when I saw this news, I thought it was bad news for cloud computing. Or to be more specific, IaaS (as defined by self-serve, on-demand usage-based compute pricing)

Most observers of IssS, and EC2 in particular, know that if you've got to run the instances for any sizable fraction of 7/24 you cross the break-even point of dedicated server at Rackspace/ServerBeach, etc.  Furthermore, it's been argued (by me and others on this group) that that didn't really matter because the premium was based on being able to leverage the 'elastic' part of EC2.  If you couldn't take advantage of that, then you'd be better off at ServerBeach.

So, here Amazon makes the pricing move to look more like Managed Hosting. 

I completely believe Werner when he says:

"some Amazon EC2 customers who have more predictable workloads have asked us for even greater flexibility in the cost model through the ability to reserve capacity."

Which probably also means that those very same customers are saying:

"Hey, I can get a dedicated server at Rackspace for half of what you're charging me, and it's bare metal, with real horsepower, not these poky EC2 Compute Units." 

I know this is true because that's exactly what I said when we decided between EC2 and ServerBeach for some servers we needed.

So, they look more like Managed Hosting, what's wrong with that?. How could this be bad for IaaS?

Because up until now, EC2 had a 'Blue Ocean Strategy' and could reasonably claim it was not Managed Hosting, but "Elastic Computing" something entirely different (see Blue Ocean Strategy for all the gory detail). This pricing move indicates that the elastic part of their value proposition isn't valued as highly by a class of customers they're trying to reach. To me, this is a measure of capitulation and a step forward in the race to the pricing bottom.

The classic Blue Ocean response to those customers would be: Too bad, we're elastic computing, what you're looking for is Managed Hosting.  Sorry.

[Southwest Airlines is the canonical Blue Ocean company that refused to offer meals, assigned seating, etc. in spite of the competition and on going customers requests].

Pure speculation on my part, but the fact that Amazon didn't/couldn't say 'no' to this class of customer tells me that either 1) there aren't enough of the class of customer that does value elasticity, or 2) they're not spending enough, or 3) their not profitable enough, or 4) some combination of 1-3.

And if you believe like I do that Amazon is super smart, is well run and that EC2 is the state of the art in IaaS, then you can only conclude that this is bad news for IaaS. It's going to morph into Managed Hosting.  I don't think that by itself is any great surprise, but what is a bit of a surprise is how fast its happening.

[Greg Pfister]

On Mar 12, 5:35 pm, Jim Starkey <jstar...@nimbusdb.com> wrote:
> Greg Pfister wrote:
[snipped]

> > But I railed for a whole chapter in "In Search of Clusters" about how
> > TPC-C is so partitioned that it could scale on clusters with nodes
> > connected with wet string. >90% of all data is local to the node into
> > which the request for it came. Makes it useless as a cluster
> > benchmark.
>
> Hmmm.  I shall have to look that up.> This was deliberately architected into the benchmark by a certain
> > company (since swallowed by another) whose name begins with "T", It
> > was done cleverly; you can't really discover it until you look closely
> > at the spec's rules for the random number generator. (!)
>
> Tell me more, please.

OK, here goes. This will be a tad fuzzy, since I’m having to dredge it
up out of memory; I analyzed it in grotesque detail for other purposes
(looking at utility of fast cluster communication), but that was 2-3
years ago.

TPC-C is supposed to model orders and payments for inventory in a
warehouse. But when you scale it up, you can have multiple warehouses
– since nobody would have one giant warehouse, right? That gives you
multiple input streams, one for each warehouse.

On a cluster, you’re allowed to partition the warehouses among the
clusters: N warehouses on 1, N on another, and so on. And you send
each warehouse’s input stream to the appropriate node. The most
stressful transaction of the mix is payment; it makes changes to
multiple entries, corresponding to all the items in the order you’re
doing a payment for.

Now, not very many people are going to walk into warehouse A and pay
for something in warehouse B. Some will, so the random number
generator that picks the collection of items you’re paying for is
biased to put most of the items paid for in the warehouse to which you
send the input. Specifically, only 15% of the references made are
uniformly distributed across the nodes; the others are purely local.
(And of that 15%, since it’s uniformly distributed, 1/nth of them end
up on the local node anyway, assuming n nodes.)

Between that and some RO table duplication everybody does, using a
partitioned federated database with two-phase commit across nodes, you
end up you end up with (if I recall correctly) 92.5% of the data
needed by a node being local to that node.

The result of this bias isn't written up explicitly in the benchmark
spec, of course. But everybody who does the benchmark figures it out,
and the result is that TPC-C scales like a banshee on clusters, but
not on single-system multiprocessors, where the partitioning basically
isn't that usable.

Greg Pfister
http://perilsofparallel.blogspot.com/

[mij...@sbcglobal.net]

"So, here Amazon makes the pricing move to look more like Managed Hosting. "

Cloud computiong *is* Managed Hosting + billing + resources elastic on demand...

Any Data Center can be a cloud. Various group contributors mention that, by using the magical formula above, there is a great business opportunity for everyone in IT industry.

Us included. The magic ingredient is billing. Imagine spending more than our salary and begging for money. This is exactly what Enterprise DC today are doing.

A friend said : What? Internal billing? Inside the Enterprise?

Yes! Because of lack of billing, we give the resources for free. We waste them. We have no clue what who uses what, and for how long. We only know if we work for 3 months at spreadsheets. And then it's too late.

Here is where the $42B annual market size for cloud is hidden (IDC estimate)

Miha

Sent from my Verizon Wireless BlackBerry

---

From: Chris Marino
Date: Thu, 12 Mar 2009 21:15:17 -0700
To: <cloud-c...@googlegroups.com>
Subject: [ Cloud Computing ] Re: A big deal

[Jesse L Silver]

Great to read your analysis Chris, but cloud pessimists might think differently if they considered Amazon's move a smart marketing play instead of some kind of "bad news for cloud computing".

EC2 has been at the forefront of cloud pricing and marketing since its inception. Though I find this "reserved instances" messaging a bit confusing, I don't think this is managed hosting. It's a discount program that does two things:

1. Counts on peoples inability to accurately predict their future usage (in other words, many customers over-estimate or under-estimate their future usage) as a way for the Amazon juggernaut to make extra money.

2. More importantly, it in locks customers for predictable periods. If an enterprise signs a 3 year deal w/ Amazon, this reduces Amazon's costs sufficiently that this discount is actually a good deal for both parties in the long run.

Again, perhaps Amazon should have messaged this differently. But this will not be fatal. Quite the opposite. Surely, this discount program will grow the elastic compute business greatly, as overflow traffic will serve their bottom line like "overage minutes" on a cellular phone plan.

[Ray Nugent]

Or maybe their just trying to gain market share...

Ray

---

From: Chris Marino <christophe...@gmail.com>
To: cloud-c...@googlegroups.com

Sent: Thursday, March 12, 2009 9:15:17 PM

Subject: [ Cloud Computing ] Re: A big deal

[Vikram B.Kumar]

On the contrary. Actually this has more to do with IaaS and elasticity than managed hosting.

Any company needs a minimum level of capacity for its load balancers, app servers and databases. Once you are established you need elasticity for your variable demand.

Since the minimum load (1x capacity of your standard traffic) is already predictable and customers want these to be more readily available than non-reserved instances, I do not see anything wrong with this and only a plus when there is no upfront costs of managed hosting and the fixed term contract.

This also shows what kind of companies are becoming Amazon customers. i.,e traditional shops along with successful startups.

[Steve Denegri]

A white paper I wrote last fall predicted this (attached). Amazon won't
be able to compete over the very very long-term (5+ years), mainly
because Microsoft and Google are locking in long-term contracts for
extremely cheap electricity. Unless Amazon starts building mega-sized
data centers all over the globe, they won't come close in their pricing
model from an electricity perspective, because they can't get cost
leverage for one of the most expensive components in providing cloud
computing. The key, un-advertised benefit of renewable energy is that,
if you're a big enough consumer of electricity like Microsoft already is
today, you can lock in your price per kwh through a power purchase
agreement (PPA) for as long as 25 years! Read the report, it's all in
there.

The hidden cost that Amazon and other existing data centers will face is
carbon taxation, once sustainability is officially legislated.
Microsoft and Google won't pay that tax, because the long-term
electricity they are locking in is of the renewable variety.

Bottom line: if you thought all this environmental push in data center
computing was about clean air, trees, and "responsibility", you're
missing the big picture. The largest companies in computing are pushing
"green" technology to fatten their wallets via massive centralization of
IT resources.

-----Original Message-----
From: cloud-c...@googlegroups.com
[mailto:cloud-c...@googlegroups.com] On Behalf Of David Kavanagh
Sent: Thursday, March 12, 2009 7:40 PM
To: cloud-c...@googlegroups.com
Subject: [ Cloud Computing ] Re: A big deal

More calculations here...

http://coderslike.us/2009/03/12/you-can-now-by-options-for-amazon-ec2-in

[malcolm stanley]

An alternative view is that Amazon is actually an insurance company of sorts.
They sell capital efficiency insurance, which we pay for through increased fees for services.

put another way:
traditional hosts are careful not to assume any customer risk;
this especially includes scaling risk: anybody with a six month forward contract on empty rack space in a host knows this..
AMZ has a model where they acquire scaling risk from the customer;
they have built an infrastructure carefully tuned to allow them to do this efficiently.
The risk transfer requires the customer to use this infrastructure and pay the fees associated with it;

in a cash constrained startup environment, this makes a lot of sense.\
Engineers in these environments spend a lot of time trying to mitigate scaling risk;
Amazon provides a seemingly cheap and easy solution which is useful when you are in this mindset.

however, there comes a time when your risk focus changes:
you may actually have achieved or experienced a scaling moment ;
now your focus is ongoing cost control,
because your scaling requirement has gone from indeterminate to a line with a slope on a graph.

In this environment AMZ is not such a great deal:
the risk transfer which justifies AMZ premium pricing no longer makes sense because the scaling risk equation has changed.
A different (insurance) product is required for this customer as the customer risk profile has changed.

seen from this perspective, this is just a customer lifecycle maintenance announcement.
The customers are growing up;
the fast cars are giving way to mini-vans with baby seats.
the initial risk-driven insurance pricing model is like motorcycle insurance, expensive but so necessary;
this newly announced model is like life insurance for customers who never thought they would live past 30....

seen this way I do not think they are violating their blue-sky approach;
I think they are following the customer lifecycle and in doing so tapping as well a new market segment that is much larger than the startup segment, that of mature companies with relatively predictable cyclical scaling requirements.

_________________________________________
malcolm stanley

wireless: 610 772 5216
email: a.malcol...@gmail.com
Read my blog at http://soaringhorse.blogspot.com
_________________________________________

On Fri, Mar 13, 2009 at 12:15 AM, Chris Marino <christophe...@gmail.com> wrote:

[kevin...@intel.com]

On Mar 12, 6:04 pm, Chris Marino <christopher.c.mar...@gmail.com>
wrote:

> Kevin, I think we're on the same page here.  One minor quibble with
> your characterization of apps migrated to cloud is your claim that
> they'll likely need to be more scalable.  Not sure there is much/any
> evidence supporting that, and intuitively, I'd think that the scaling
> requirements would remain roughly unchanged internal v. cloud (scale
> linearly  w/users which I think will be roughly constant or slowly
> growing).
>

Yes, scaling requirements don't change but the hardware is assumed to
be cheaper stuff than what you might buy yourself. To try and put some
data to this, if the DBMS is running on fairly low-end servers, think
I recall $3k/machine was quoted for SDS, then that covers about the
same processing ability as what ~40% of the general server market buy,
assuming of course the people don't spend a significant amount extra
on servers for DBMS's than the average. If you also assumed that an
SDS "container" is really today just an alias for a single-machine
environment, then it's is good enough for that same 40% of the market
without app changes.

On the face of it that's not bad coverage, but if SDS could support
transparent scaling to cover the $3k-$10k servers space, maybe via up
to 4 node clusters or simply larger base machine (although this kind
of goes against the gain a bit) then you get closer to a ~90% coverage
which starts to look very compelling. Guess we will have to wait and
see what size units you can buy in.

[Chris Marino]

Malcom, I agree that this is not so black and white and a natural evolution would mandate this kind of change. Reserved Instances don't abandon the notion of elasticity, they simply flatten the slope.  Everyone fits a simple linear model

Y= mX + b

Where there are different slopes, intercepts, etc.

I think Managed Hosting providers are coming from the other direction and before long, they'll all be indistinguishable. Which is basically my point.  Maybe it's not bad, or good.  Just inevitable.

But if you take this same model and scale it up from the provider side, they'll need a big intercept to cover their fixed costs (i.e. lots of fixed price, long term contracts) so they can offer elastic services at their marginal cost.  This would seem to benefit the larger Managed Hosting providers, provided they can deliver elasticity efficiently.

CM

I guess my point is that

[Ray Nugent]

Chris, I appreciate your view, what I'm not certain about is how a Managed Hosting company can possibly compete with AMZN on a scale basis. AMZN has another business that gives it the ability to scale it's infrastructure and source it less expensively as does MSFT and GOOG. In fact, it would seem the only companies that could provide public cloud would be companies already heavily invested in infrustructure like Telcos, ISPs and the handful of large SaaS vendors that exist. In talking with folks in the VC industry they pretty universally say they would neve invest in a public cloud company becuase of the high cost of doing so (this may say more about the state of the VC biz than the Cloud biz...). So, will Mosso and others may be able to carve out niche businesses in the Gov or DR spaces I think the the Amazons of the world will be hard to catch up to.

Ray

---

From: Chris Marino <christophe...@gmail.com>
To: cloud-c...@googlegroups.com

Sent: Friday, March 13, 2009 7:41:58 AM

Subject: [ Cloud Computing ] Re: A big deal

[Tarry Singh]

"long-term contracts for
extremely cheap electricity"

Oh yeah? Explain

Tarry

--
Kind Regards,

Tarry Singh
______________________________________________________________
Founder, Avastu Blog: Research-Analysis-Ideation
"Do something with your ideas!"
Business Cell: +31630617633
Private Cell: +31629159400
LinkedIn: http://www.linkedin.com/in/tarrysingh
Blogs: http://www.ideationcloud.com

[Gordon Jackson]

When you join a 24-hr gym (e.g. Lifetime Fitness) you pay a monthly fee and you have access to all of the facilities (at your level) 24/7/365. But do you use it that much? Interestingly, the average Lifetime member uses their facility 1.5 days/week. The user gets what they want - the idea that they have a gym whenever they want, and the gym gets to *way* over-subscribe because of the actual usage patterns. The trick is when everyone seems to show up on Monday at 6:30 PM...

The airlines' goals are to have 100% of the seats on every plane leaving the gate occupied. To do this they  oversell the plane. The assumption is that not everyone is going to show up. The trick is getting the math right so that what they end up paying - buying off a customer, or losing one - is less than if the plane were to leave with X empty seats.

The idea is the same here. It is *all* about delivering on the elasticity. The "reserved" notion is a facade - a security blanket for the old schoolers who need to believe that they always have the resources, i.e. static. The trick for AMZ is going to be in how well they manage this. Specifically, what, if any, SLA/guarantee they provide to ensure that the resource is actually going to be there when it is needed, and how they optimize on failing to deliver - e.g. small company reserves 100 and BIG company reserves 100 - in a perfect unlimited resources world they both get what they need, when they need it. In the *real* world, resource contention will be addressed by what is the least to be paid in missed SLA penalites. BIG company pays a bit more, but they receive a higher penalty payback should they not get what they need, when they need it. So, AMZ will optimize resource allocation based upon rules designed to minimize their costs. This is a key to cloud computing - optimizing resource allocation based on rules that ensure that everyone is happy, and if someone needs to be made unhappy, do it in a way that costs the least.

Nominally, both small and BIG companies benefit because there is so much variance in processing needs  that they always *seem* to have their reserved resources. In this way, AMZ can sell "reserved" units over and over again, and bank on not everyone showing up at the same time.

-- 

Gordon Jackson
Technical Director, Data Center Automation
Univa UD

www.univaud.com/reliance

[Tarry Singh]

Oh yeah. It is a big deal, its the constant reconfiguration of the game that MSFT has to play -- they have the robust stack alright to connect to the mid-section of the cloud substrate.

On Wed, Mar 11, 2009 at 6:50 PM, Chris Marino <christophe...@gmail.com> wrote:

I agree w/Jose here.  It's been a while since I've anything on this group worth participating in.  The occasional topic that is interesting quickly become personality driven and IHMO suppresses legitimate criticism and commentary.

Also, it seems that some approaches/providers/alternatives seem to have a Most Favored Vendor status, while other's, inexplicably, can't seem to get a break.

I'm generalizing, of course, but it seems that Amazon can do no wrong, and nobody seems to care about Microsoft. 

What's with that?

Here's an example.  Yesterday Microsoft began fleshing out the details of their SQL Data Services and judging by the interest on this group you'd never know it ever happened.

http://blogs.msdn.com/ssds/archive/2009/03/10/9469228.aspx

IMHO, this is a pretty big freak'n deal. 

I was going to post something here last night, but then waited to see if anyone else would bring it up. Hasn't happened....

Compare that to the chatter we often see posted here about the latest and greatest DB/researchy/shiny thingy that's often not much more than a research project. CouchDB, SimpleDB, Hadoop, Haskell, BigTable, and on and on...

Something happens here and withing a second or two there are threads and replies galore.

Don't get me wrong, I think these topics are interesting and worth discussing, it's just that there's a lot more to Cloud Computing than you'd realize if all you ever read were the threads on this group.

CM

On Wed, Mar 11, 2009 at 10:06 AM, Jose Gonzalez <jed...@yahoo.com> wrote:

Is it me, or is this list becoming more and more like the school yard that is ruled by a couple of bullies whom always have to have the last word?

This type of comments and behavior is really of a low taste and very unprofessional. The purpose of this forum is for us to bring ideas and talk about them like real professionals. Not to be making jokes at someone else's expense.

jeg

---

From: Jan Klincewicz <jan.kli...@gmail.com>
To: cloud-c...@googlegroups.com
Sent: Wednesday, March 11, 2009 7:07:20 AM
Subject: [ Cloud Computing ] Re: Cloud Computing: How many servers does the world need?

On Tue, Mar 10, 2009 at 4:39 PM, John D. Mitchell <jdmit...@gmail.com> wrote:

 [and now I really am
going to stop participating in this part of this thread -- my doctor
says that I need to stop debating with fanatics :-).]

John

It's a good step seeking professional help.  Perhaps he can get your meds right..  Being a fanatic can be an asset in this industry.  Being delusional is never a good thing.


--
Cheers,
Jan

[JL Valente]

All the recent points on the motives and the consequences of Amazon’s additional pricing terms on the industry are valid.

In fact Rackspace’s announcement yesterday vindicates Chris’s point below “..This would seem to benefit the larger Managed Hosting providers, provided they can deliver elasticity efficiently”

 

See “Rackspace Offers Cloud Computing for Cautious Customers”      http://gigaom.com/2009/03/12/rackspace-wants-to-ground-its-cloud/                                                                                                                                                                                                                     as reported by GigaOM

Rackspace Launches CloudServers, a Cloud for Big Companies

Posted: 12 Mar 2009 06:00 AM PDT

Rackspace today is expected to announce its own on-demand computing product, CloudServers. The service is built on the company’s acqusition of Slicehost last year and will offer the same services as Amazon Web Services’ Elastic Compute Cloud. It’s also a cornerstone of Rackspace’s attempts to build out a cloud computing environment that will rival those of Amazon.com, GoGrid and other true cloud computing providers.

A few weeks ago, I spent a couple hours at the Rackspace headquarters in San Antonio chatting with Lew Moorman, chief strategy officer, and Johnathan Bryce, co-founder of Rackspace’s cloud efforts, about the company’s plans for the cloud. Rackspace, which built out its business (and earned $21.7 million last year on sales of $531.9 million) on hosting dedicated servers, has been building a cloud for more than a year. The goal is to offer clients the ability to stay on dedicated servers with the promise of the cloud in case of overflow. Clients can also choose to stick with dedicated servers or migrate entirely to the cloud. This hybrid strategy means Rackspace can offer a fairly unique product, as Amazon, IBM and HP all lack either the dedicated hosting business or the true cloud computing technology. For now, Rackspace may find its closest competitor in ServePath, which has a product called GoGrid Cloud Connect that ties its dedicated hosting business and the GoGrid compute cloud together.

Rackspace is in a similar boat as large software vendors that are trying to build a software as a service, or SaaS, business while protecting their shrinkware. Offering a smooth flow of data between both dedicated and cloud resources gives Rackspace the chance to ride the transition from dedicated computing to cloud computing, while minimizing cannibalization of its hosted business as customers move to the cloud. For many businesses who are worried about issues associated with throwing their data in the cloud, this could be a way to get them familiar with the model.

A hybrid strategy and moving quickly on the cloud trend should help Rackspace stay afloat — and perhaps even speed ahead.

 

From: cloud-c...@googlegroups.com [mailto:cloud-c...@googlegroups.com] On Behalf Of Chris Marino
Sent: Friday, March 13, 2009 7:42 AM
To: cloud-c...@googlegroups.com
Subject: [ Cloud Computing ] Re: A big deal

 

Malcom, I agree that this is not so black and white and a natural evolution would mandate this kind of change. Reserved Instances don't abandon the notion of elasticity, they simply flatten the slope.  Everyone fits a simple linear model

[Chris Marino]

I'd have to dig into RackSpace's 10K/Qs, but I think they're pretty darn big. Big enough to reach the scale that offered the same economics as Amazon.  Not as bit as Google, or Microsoft, for sure, but pretty darn big.

You're right about the cross subsidy that Amazon.com can offer AWS, so maybe they've got their fixed costs covered there.  It'd be interesting to do a more detailed analysis.  I think this is also explains why we're not seeing much/any public cloud IaaS startups (and why VCs are not interested).

CM

[Steve Denegri]

Sorry, short on time, but it’s in the report I wrote.  Renewable energy can be supplied to very large users of power at a fixed cost.  It’s practically that way now, even with fossil fuel based energy, as I showed with CPS Energy and Microsoft in San Antonio.  45% discount for Microsoft relative to other users, that’s a game changer!

[Chris Marino]

Gordon, I like the gym analogy more than the airline, but you've touched on a very interesting part of this whole problem.  If you can sell the same infrastructure to more than one customer, there is the potential to significantly reduce CapEx.

Its also interesting to see that if you could indeed deliver (elastic) instances cheaper than Managed Hosting, you might choose to deliver elasticity AND deliver them at a lower price.  But today, it's premium priced?  In fact, the 'reserved' prices are discounted from the elastic prices, which is counter intuitive if there was, in fact, capacity scarcity.

The current situation seems out of whack since if I use the gym analogy, with Managed Hosting, I can join the guy at a lower price and have my very own stairmaster......

Which, again, leads me to conclude that IaaS and Managed Hosting are rapidly converging.

CM

[Jan Klincewicz]

If you consider DR Services like Sunguard Recovery, they are betting on over-subscription, and that disasters will not befall too many of their customers at one time.  While disaters are, by definition, unpredictable, I'm sure they have calculated the risk to minimize financial catastrophe (then again, we all thought that about the Mortgage industry and foreclosures, eh ?)

It's harder, though, to host, for example, eCommerce sites, where the seasonality IS more predictable (everyone wants to gear up for the holidays) but you still need to accomodate that and make a buck the rest of the year.. while your infrastructure may be quiesced, but is still depreciating in the racks.

Of course, you COULD put it to good use searching for extraterrestrials ....    Maybe the govt. could use the available unused MIPS to search for Bin Laden ot something in the off-season ??

--
Cheers,
Jan

[Daniel Ciruli]

Chris Marino wrote:

>I realize that those are a lot of 'ifs', but I think that's exactly the market that SDS/Azure is targeting.

It is a lot of ifs.

However, one thing to bear in mind is the number of enterprise apps out there that fit exactly your set of "ifs:" It's mind-boggling.

Say what you will about Microsoft, but one of the reasons they have been so successful in the business market is the fact that they have always concentrated on giving relatively low-cost tools to developers: think VBA in Access and Excel, VB 6, etc. Even SQL Server itself succeeded not at the highest end of the market, but by attacking the middle and lower ends and by concentrating on developer experience.

Microsoft is trying to address those same enterprise developers now. It's not necessarily the most glamorous part of the market, but its one they very much want to nail down.


------Original Message------
From: Chris Marino
To: cloud-c...@googlegroups.com
Sent: Mar 12, 2009 6:57 AM
Subject: [ Cloud Computing ] Re: Something worth discussing...

Not sure I fully understand your point...

If you're saying that if I've got to do non-relational things, then SDS isn't any help, then I would agree. I'll also agree that BigTable, SimpleDB etc. is likely better suited than SDS for these kind of things. Especially if you've got to scale pretty large.

[A separate point but one worth noting is that if you *don't* have to scale, then a simple hash table might be even easier than any of these non-relational stores]

But that's different from taking an existing app that's been designed to:
   1. Serve my internal needs (i.e. not much scaling),
   2. That uses SQL Server,
   3. That's already addressed the non-relational parts with standard programming techniques,
   4. That I want to deploy in the cloud, on a platform (not infrastructure) without re-engineering.

In this case, you've really got to have the things that their putting in SDS.

I realize that those are a lot of 'ifs', but I think that's exactly the market that SDS/Azure is targeting.

CM




An 'easy' deploy button really is needed but what I don't see in SDS
is any reason
why I would choose this offering over any other partial relational
solution other
than they are less mature today. I say partial because any app with
significant
scale will still need some non-relational re-engineering work to make
SDS viable,
it's not a silver bullet but more an easier on-ramp because of the
data partitioning
concerns.

My guess is that the re-engineering costs would be significantly
bigger than what
you can save by pressing this "easy" button, but of course its a path
of very little
resistance and that often counts more.

Kev












Sent via BlackBerry by AT&T

[Miha Ahronovitz]

Chris, Gordon,

Who says in order to have a cloud and elasticity, I must buy external resources from Amazon or other providers? Each time a an application access AWS, oh, how nice, this is cloud! It could be a cloud even if never accesses resources outside the compute site

I can have some stand by servers and storage for common increased demand situations, and I can move virtually the hosts from one service in less demand to another in great demand.
UniCloud and UniCluster from Univa (they OEM SGE)  (Gordon knows best) and xVM Ops Center from Sun both can help, in combination to Sun Grid Engine (SGE) , Service Domain Manager, which moves hosts from one service to another.

What is more economic?  (1) Keep some servers iddle and use then only for moderate increased demand? Or (2) buying always cycles from AWS and similar providers when demand increases?

We can use (1)  if we know the cost of keeping some servers on stand-by is far less that accessing AWS.  Then use (2) only a very short peak in demand that we  know in advance it comes.

The pool of idle servers already exists today in every DC as under-utilized hosts. Not everyone has close to 100% utilization.  We can save even more money if we can shut-off power for all unused hosts.
In fact the power consumption of un-used servers may costs more than just have a pool of stanby servers doing nothing.

The nice thing about Amazon , is that apparently, I don't have to pay for the power consumption. But there is no such thing as a free lunch,. The costs are built in the fee structure.

Miha

---

From: Chris Marino <christophe...@gmail.com>
To: cloud-c...@googlegroups.com

Sent: Friday, March 13, 2009 11:38:15 AM
Subject: [ Cloud Computing ] Re: A big deal

[bgi...@gmail.com]

Sent via BlackBerry by AT&T

---

From: Tarry Singh
Date: Fri, 13 Mar 2009 17:16:28 +0100
To: <cloud-c...@googlegroups.com>

Subject: [ Cloud Computing ] Re: Something worth discussing...

[Gordon Jackson]

Hi Miha,

On Fri, Mar 13, 2009 at 2:14 PM, Miha Ahronovitz <mij...@sbcglobal.net> wrote:

Chris, Gordon,

Who says in order to have a cloud and elasticity, I must buy external resources from Amazon or other providers?

I don't *think* that I did :)

In fact, I very much believe, like you, that a cloud can be realized internally - see this post - and that broad adoption at the Enterprise level is most likely going to happen behind the firewall first.

In this thread, I was just pointing out that the notion of "reserved" appeals to folks because it gives them something that is familiar - the idea of "my static resources" - and that AMZ can do this over and over again with the same resources as long as they can effectively handle the potential contention that may arise if everyone wants their resources at the same time.

It's worth noting that this same contention management is applicable/necessary behind the firewall as well - you want to ensure that the revenue-generating application always has enough resources to keep the customers spending, even if that means you need to slow down the accounts payable folks for a little while ;)

[Miha Ahronovitz]

“In this thread, I was just pointing out that the notion of "reserved" appeals to folks because it gives them something that is familiar - the idea of "my static resources" - and that AMZ can do this over and over again with the same resources as long as they can effectively handle the potential contention that may arise if everyone wants their resources at the same time.”

 

Sure. This is like a bank lending money against limited assetts. The classic concept of perfection in clusters was 100% utilization. It meant 100% of the time, the resources are used.

The new cloud concept of perfection is creating the maximum profit through billing, without service deterioration. This concept is in $. This means:

“What do I care if my cluster is 100% utilized or 10% utilized, as long as someone pays for the privilege a hell of a lot of money?

 

Miha

 

BTW, I never meant you or Chris said that renting cycles is a sine-qua-non for qualifying a cloud. Yet most people on the street believe a cloud is something that works with AWS.

[Chris Marino]

I've made no distinction between internal and external clouds, and agree that your scenarios for service migration is an excellent use of excess capacity.

CM

[Nati Shalom]

Miha

 

Theoretically your right at the end of the day if you build your application correctly you can achieve elasticity in different ways. In reality however, there are two main factors that will determine which "pool" to use:

 

1. The size of the pool – this will eventually effect also the cost of maintaing the pool
2. The risk of overfllow

 

Note that the chances that you will reach an overflow in a particular organisation would be higher then a pool that serves different organisations and can pull resouces at that level.

It is also fair to asssme that bigger pool providers will allways have an technology edge as serviing pools becomes core part of their business vs internal organisations which in most cases don't have even the justification to develop the right skillset required to manage such operations.

 

In addition to that there is another challange which is the ability to make the application itself elastic. i.e. how many application are built in away that will enable them to exploit new resources as they arrive without requiring modificatio to configuration and manual intervention?

 

Both are things that I tried to cover in a recent post:

Its time for auto scaling – avoid peak load provisioning for web applications (http://natishalom.typepad.com/nati_shaloms_blog/2009/03/its-time-for-auto-scaling-avoid-peak-load-provisioning.html )

 

 

Nati S

www.gigaspaces.com/cloud

 

 

 

 

 

 

 

---

[Miha Ahronovitz]

Theoretically your right at the end of the day if you build your application correctly you can achieve elasticity in different ways. In reality however, there are two main factors that will determine which "pool" to use:

 

1. The size of the pool – this will eventually effect also the cost of maintaing the pool
2. The risk of overfllow
   
   

As usual from you Nati, you have bright clarifications and focusing on how to achieve elasticity from internal or external pools
I read in your blog about "The Perfect Storm" and "The Digg Effect" which you define:

This effect was known in the financial industry as the “perfect storm,” which basically meant that a chain of events in the market led to a storm of events that brought many systems down. In social networks, this type of viral behavior is also referred to as the Digg effect.

How do we decide on whether to use an internal pool or external, expensive AWS pool to get elasticity? We know how to optimize, in theory, but in practice? In practice we have automatic tools to take the decision for us, so the organizations with private clouds will have the right middleware. The industries you quote (e,.g a Stock Exchange, or a service like Facebook) can't afford to stop. But in private clouds, there may be some tolerance for a lower service level as negotiated in SLA's (Service Level Agreements)

The future application API of the Sun Grid Engine Service Domain Manager, will tell  the policy module: "I need more hosts." Or  "I have two hosts  available" . The policy module will not care what application service is running. All it cares is what hosts are available, where, who needs more hosts and who does have too many hosts, plus how many hosts we have in stand by pool. The policy module also knows what service level agreements apply to each application delivery.

What the policy module does not know today, is when to decide automatically on getting more virtual hosts from an external pool, like AWS. But such a decision engine is not hard to write. It will not be perfect, but it will work and it will save money, based on policies and including prices from external suppliers like AWS as part of the decision making.

how many application are built in away that will enable them to exploit new resources as they arrive without requiring modificatio to configuration and manual intervention?

 

 Look at the blog of Daniel Templeton about the 60,000 cores parallel  facial recognition application.

http://blogs.sun.com/templedf/entry/big_just_got_huge

This mind-blowing application -  it recognizes a face within few seconds with  2 in 10,000 error margin - will not be able to use virtual nodes, e.g. AWS, without slowing down or crashing altogether. Here is a clear instruction for the  future unwritten yet policy module: Don't use external AWS for facial recognition.

2 cents,

miha

---

From: Nati Shalom <natis...@gmail.com>
To: cloud-c...@googlegroups.com
Sent: Friday, March 13, 2009 5:06:18 PM

[Chris Marino]

What the policy module does not know today, is when to decide automatically on getting more virtual hosts from an external pool, like AWS. But such a decision engine is not hard to write. It will not be perfect, but it will work and it will save money, based on policies and including prices from external suppliers like AWS as part of the decision making.

I think you're oversimplifying this problem (regardless of inside v. outside capacity pool).  Sure, the engine might be easy to write, but the engine is not the difficult part.  Its determining (all)  the metrics upon which to make a decision, instrumenting the environment to gather the metrics, specifying a robust policy that actually captures the desired behavior, then actually enforcing that policy without flapping, false positives, etc.

Solutions to this problem has been attempted in one form or another for many years.  At my old company we developed a Service Level Management product and the brutal reality was that the complexity of getting all this to work was beyond the skill of most users, which drove most of them back to the preferred method of addressing this problem: Over provisioning.

Occasionally, complex instrumentation, rules engines, etc. would finally be in place and they would be allowed to do everything right up to enforcement.  The false positives were sufficiently frequent and the remedial actions both costly and error prone that enforcement was left to an admin who would run a script and monitor it's execution.

When it comes to autonomics like this, I think there's tremendous value in the simplistic.  Simple time of day, or day of week gets you pretty far.  6AM-6PM: 10 hosts, 6PM-6AM: 3 hosts.

Which brings us full circle to the 'Reserved Instances' topic which is to address *predictable* demand.

[Miha Ahronovitz]

When it comes to autonomics like this, I think there's tremendous value in the simplistic.  Simple time of day, or day of week gets you pretty far.  6AM-6PM: 10 hosts, 6PM-6AM: 3 hosts.

Chris, Sun Grid Engine's  Service Domain Manager documentation, on how SLA's are handled , is online:

http://wikis.sun.com/display/gridengine62u2/Services

It is actually much less complicated than it sounds. Later this year, we will have the feature of accessing  AWS virtual resource. Our partners BioTeam and UnivaUD  already have this feature. We thrive mainly on HPC applications..

We also have the feature of "advanced reservation". Please read simply as reservations in advance, like a restaurant or a hotel. See here:

http://wikis.sun.com/display/gridengine62u2/Managing+Advance+Reservations

I think this is similar to the 'Reserved Instances' you refer in your post. This may cause a lower utilization - if the project who made the reservation does not use 100%  the resources allocated. There are situations when we can NOT let the autonomics arrange for needed resources (e.g. a solar eclipse that requires massive data to be collected and processed over a few minutes). This is a typical situation when the owners of the cloud maximize the total revenues per unit of time, rather than the utilization

Cheers,

Miha

---

From: Chris Marino <christophe...@gmail.com>
To: cloud-c...@googlegroups.com

Sent: Saturday, March 14, 2009 3:28:45 PM

Subject: [ Cloud Computing ] Re: A big deal

[Nati Shalom]

Chris

 

" Occasionally, complex instrumentation, rules engines, etc. would finally be in place and they would be allowed to do everything right up to enforcement.  The false positives were sufficiently frequent and the remedial actions both costly and error prone that enforcement was left to an admin who would run a script and monitor it's execution."

 

The reason why this is still a complex task is because we tend to think that automation need to "live" outside the application context. But guess what when you try to move different pieces of a given application over the network things start to break. All of sudden you find yourself trying to "teach" this external automative system what's a primary, what's a backup, which pieces has built-in dependency between themselves and need to be treated as a single unit etc.

If in order to do a simple tasks such as start a new instance when one breaks I need to use external monitoring system, add rule engine that will decide based on the monitored event what to do i.e. learn and integrate two products just for that simple task something is broken at the core.

IMO the right way to handle this type of requirements is through an application aware SLA in which the SLA and automation logic will be built-in into the application design and implementation and wouldn’t be treated as an after thought.

 

Nati S.

www.gigaspaces.com/cloud

 

 

 

---

From: cloud-c...@googlegroups.com [mailto:cloud-c...@googlegroups.com] On Behalf Of Chris Marino
Sent: Sunday, March 15, 2009 12:29 AM
To: cloud-c...@googlegroups.com
Subject: [ Cloud Computing ] Re: A big deal

[Jim Starkey]

Miha Ahronovitz wrote:
> When it comes to autonomics like this, I think there's tremendous
> value in the simplistic. Simple time of day, or day of week gets you
> pretty far. 6AM-6PM: 10 hosts, 6PM-6AM: 3 hosts.
>
> Chris, Sun Grid Engine's Service Domain Manager documentation, on how
> SLA's are handled , is online:
>
> http://wikis.sun.com/display/gridengine62u2/Services
>
> It is actually much less complicated than it sounds. Later this year,
> we will have the feature of accessing AWS virtual resource. Our
> partners BioTeam and UnivaUD already have this feature. We thrive
> mainly on HPC applications..
>
> We also have the feature of "advanced reservation". Please read simply
> as reservations in advance, like a restaurant or a hotel. See here:
>
> http://wikis.sun.com/display/gridengine62u2/Managing+Advance+Reservations
>
> I think this is similar to the 'Reserved Instances' you refer in your
> post. This may cause a lower utilization - if the project who made the
> reservation does not use 100% the resources allocated. There are
> situations when we can NOT let the autonomics arrange for needed
> resources (e.g. a solar eclipse that requires massive data to be
> collected and processed over a few minutes). This is a typical
> situation when the owners of the cloud maximize the total revenues per
> unit of time, rather than the utilization
>

I think you need to re-read about "Reserved Instances". They don't
reserve anything. Their only effect is on billing where, in exchange
for a non-refundable annual payment, they give you a per-hour discount.
Actual availability and access to resources is identical to ordinary EC2
instances.

But you gotta wonder why their marketing guys decided to label a
bean-counter gimmick "reserved images"....

[Miha Ahronovitz]

Jim, good point.

miha

-----Original Message-----
From: cloud-c...@googlegroups.com
[mailto:cloud-c...@googlegroups.com] On Behalf Of Jim Starkey
Sent: Sunday, March 15, 2009 8:51 AM
To: cloud-c...@googlegroups.com
Subject: [ Cloud Computing ] Re: A big deal

[Matthew Zito]

Jim,

Where do you see that?  The public documentation is a little unclear, but on the AWS blog, it says:

"Taking these requirements into account, we've created a new EC2 pricing model, which we call Reserved Instances. After you purchase such an instance for a one-time fee, you have the option to launch an EC2 instance of a certain instance type, in a particular availability zone, for a period of either 1 of 3 years. Your launch is guaranteed to succeed; there's no chance of encountering any transient limitations in EC2 capacity. "

I could see how this is important to people - if you're going to use AWS for DR, it would be miserable to find out that at the critical moment, you can only spin up 150 of the 250 servers you need.

Matt

--
Matthew Zito
Chief Scientist
GridApp Systems
P: 646-452-4090
mz...@gridapp.com
http://www.gridapp.com

-----Original Message-----
From: cloud-c...@googlegroups.com on behalf of Jim Starkey
Sent: Sun 3/15/2009 11:51 AM
To: cloud-c...@googlegroups.com
Subject: [ Cloud Computing ] Re: A big deal

[Dan Kearns]

On Sun, Mar 15, 2009 at 8:51 AM, Jim Starkey <jsta...@nimbusdb.com> wrote:

But you gotta wonder why their marketing guys decided to label a
bean-counter gimmick "reserved images"....

It does seem more like a simple price adjustment to remain competitive, which leaves me wondering if they are just opportunistically adding a 30% premium on real elasticity, if that is backed up by some sort of statistical model, or if they just have an overwhelming percentage of static-load users and can't figure out how much of their usage is actually elastic without introducing a financial incentive.

Any predictions on whether the 3-year discount rate will have been more expensive than spot pricing (within ec2) over the same term?

-d

[todd]

Nati Shalom wrote:
>
> Chris
>
> " Occasionally, complex instrumentation, rules engines, etc. would
> finally be in place and they would be allowed to do everything right
> up to enforcement. The false positives were sufficiently frequent and
> the remedial actions both costly and error prone that enforcement was
> left to an admin who would run a script and monitor it's execution."
>
> The reason why this is still a complex task is because we tend to
> think that automation need to "live" outside the application context.
> But guess what when you try to move different pieces of a given
> application over the network things start to break. All of sudden you
> find yourself trying to "teach" this external automative system what's
> a primary, what's a backup, which pieces has built-in dependency
> between themselves and need to be treated as a single unit etc.
>
> If in order to do a simple tasks such as start a new instance when one
> breaks I need to use external monitoring system, add rule engine that
> will decide based on the monitored event what to do i.e. learn and
> integrate two products just for that simple task something is broken
> at the core.
>
> IMO the right way to handle this type of requirements is through an
> application aware SLA in which the SLA and automation logic will be
> built-in into the application design and implementation and wouldn’t
> be treated as an after thought.
>

Nati, isn't the SLA enforced through a global grid container of some
sort? That's really not different than using an external service and is
in many ways not as good because it can only control your system.
Complex systems are made of many parts that must be scaled in harmony.
>
> Nati S.
>
> www.gigaspaces.com/cloud <http://www.gigaspaces.com/cloud>
>
> ------------------------------------------------------------------------
>
> *From:* cloud-c...@googlegroups.com
> [mailto:cloud-c...@googlegroups.com] *On Behalf Of *Chris Marino
> *Sent:* Sunday, March 15, 2009 12:29 AM
> *To:* cloud-c...@googlegroups.com
> *Subject:* [ Cloud Computing ] Re: A big deal
>
>
> What the policy module does not know today, is *when* to decide
> automatically on getting more virtual hosts from an *external*

[Jim Starkey]

From the Reserved Instances FAQ:

Q: How is a Reserved Instance different than an On-Demand Instance?

Functionally, Reserved Instances and On-Demand Instances are exactly the same. They are launched and terminated in the same way, and they function identically once running. This makes it easy for you to seamlessly use both Reserved and On-Demand Instances without making any changes to your code. The only difference is that with a Reserved Instance, you pay a low, one-time payment and receive a lower usage rate to run the instance than with an On-Demand Instance.

See http://aws.amazon.com/ec2/faqs/#What_is_a_Reserved_Instance for the full discussion.

-- 
Jim Starkey
President, NimbusDB, Inc.
978 526-1376

[mij...@sbcglobal.net]

But for either a Reserved Instance or On-Demand Instance does the uiser have a guaranteed access to the resources booked?

This is a must to give "the illusion of infinite resources" that Berkeley U study, inspired by this group , stated as part of the cloud computing definition.

In HPC, the resources booked could be extrelmely large. Does Amazon have the tools to guarantee them?

Miha

Sent from my Verizon Wireless BlackBerry

---

From: Jim Starkey
Date: Sun, 15 Mar 2009 18:33:31 -0400

[Jim Starkey]

mij...@sbcglobal.net wrote:
> But for either a Reserved Instance or On-Demand Instance does the
> uiser have a guaranteed access to the resources booked?
>
> This is a must to give "the illusion of infinite resources" that
> Berkeley U study, inspired by this group , stated as part of the cloud
> computing definition.
>
> In HPC, the resources booked could be extrelmely large. Does Amazon
> have the tools to guarantee them?
>

Amazon doesn't guarantee anything for EC2. If you plunk down for a
"reserved instance", it's non-refundable. Well, not quite. If Amazon
decides to terminate "reserved instances", you get a prorated refund and
revert to the old rates.

Someone will have to explain how a company with any sort of budgeting
process could use "reserved images" since, without notice, the hourly
rate can triple.

But, no, Amazon give no guarantee of access to the resource booked other
than a 10% discount on the next month's bill if nothing happens. Or did
I miss something?

[Nati Shalom]

" Nati, isn't the SLA enforced through a global grid container of some
sort? That's really not different than using an external service and is
in many ways not as good because it can only control your system.
Complex systems are made of many parts that must be scaled in harmony."

Todd the point that I was trying to make is that the application need to
written and architected differently for an SLA driven environment. In our
case we expose that knowlage right at the API and processing-unit level.
This is where users defines the unit of scale and fail-over and the SLA for
that unit.
Once this is done you can start looking at global SLA monitores that can be
used to monitor and take action when needed. The thought that you can take a
given application that was not designed for such scenrio, plugin an external
SLA manager and things will work magically can fit only very simple
scenarios IMO. Try to imagine even a simple web application and imagine what
will happened to that application if I add another machine to share the
load. Most likely nothing!. How many of those applications will know to take
advantage of that machine without the need to manualy change configuration
or in some cases even code? How many of them were designed to do that in
away that wouldn't break in-flight transactions and ensure that nothing get
lost? How many of them were even designed or tested in such a scenario?

Nati S.

[todd]

Got it. Makes sense. It certainly does has to be baked into the
architecture of the application.

[Matthew Zito]

So, I know we were discussing what was different from the regular EC2,
it's clear that there is a very real difference. From an interview at
the reigster:

"When we sell a reserved instance to a customer, we really are
reserving that instance for the customer and we will not commit it to
other purposes when they're not using it," Amazon EC2 general manager
Peter De Santis tells us. "That's an easier capacity-management
problem than predicting future demand. With a reserved instance,
they're giving us the ability to know that that instance will be
needed...

"It's like we're selling a number of seats on an airplane - and we're
not going to overbook the airplane."

http://www.theregister.co.uk/2009/03/20/amazon_on_ec2_disaster_recovery/

Thanks,
Matt

[Jim Starkey]

Matthew Zito wrote:
> So, I know we were discussing what was different from the regular EC2,
> it's clear that there is a very real difference. From an interview at
> the reigster:
>
> "When we sell a reserved instance to a customer, we really are
> reserving that instance for the customer and we will not commit it to
> other purposes when they're not using it," Amazon EC2 general manager
> Peter De Santis tells us. "That's an easier capacity-management
> problem than predicting future demand. With a reserved instance,
> they're giving us the ability to know that that instance will be
> needed...
>
> "It's like we're selling a number of seats on an airplane - and we're
> not going to overbook the airplane."
>
> http://www.theregister.co.uk/2009/03/20/amazon_on_ec2_disaster_recovery/
>
>

Excuse me, but what does it mean to reserve a virtual instance? By
definition, it doesn't exist.

Are they saying that they don't oversubscribe their machines? That they
reserve at least as many cores as the number of reserved instances times
the number of cores per instance? If they're prepared to dedicate
hardware for reserved instances, why do they charge per instance-hour?

[Chris Marino]

Excuse me, but what does it mean to reserve a virtual instance?  By
definition, it doesn't exist.

Are they saying that they don't oversubscribe their machines?  That they
reserve at least as many cores as the number of reserved instances times
the number of cores per instance?  If they're prepared to dedicate
hardware for reserved instances, why do they charge per instance-hour?

Agreed.  This is all very fungible.   Maybe when you run a reserved instance they are assigned to a hardware pool where they maintain a virtual core instance to physical core ratio of no greater than 1:1.   Within that context, it is like reserving a seat on an airplane.

Which begs the question, without the reservation, how likely is it that your instance is competing with other instances on the same physical machine where this ration *is* greater than one? Not sure I'd want to be that poor guy.

I've seen very little discussion w.r.t. performance on EC2. In fact, the only serious analysis I've seen is what came out of the Berkeley analysis.  In fact, that's the only place I've seen 'Performance Unpredictability' even mentioned as an obstacle to cloud computing adoption.

Maybe the analogy ought to be a commuter train/bus.  With a reservation, I'm guaranteed a seat on very the next train. Without it, I might get a seat, maybe not. In fact, I might have to stand, or worse, wait for the next one slowing me down....

CM

[Jim Starkey]

I have to ask the question why none of this is addressed in their SLA.
One would think that people plunking down hard money for a "reserved
instance" would want to know what they're paying for. We know that
they're paying for a discount on the hourly rate, but since that can be
canceled without notice, one can't exactly budget for it.

I don't fault Amazon at all. They are very clear about exactly what
they offer. I do wonder about people finding meaning that their legal
documents don't contain.