Today VMware announced its stand-alone ESXi hypervisor will be
available at no cost. ESXi 3.5 update 2, available today, meets the
criteria for mass cloud deployments: (1) ease of use and (2) maturity
and stability now having been 'battle tested' for six months with
customers. The leading server manufacturers have all embedded VMware
ESXi, including Dell, Fujitsu-Siemens, HP, IBM, and NEC.
This is the first major step for VMware in the cloud computing scene.
Historically VMware has focused on server consolidation and test / dev
and has priced their products accordingly. By making the underlying
hyper visor free they effectivily enable cloud providers with the
ability to utilize ESXi in their massive cloud server deployments
while also helping keep their costs competitive with other cloud
providers such as Amazon who utilize Xen.
You can download ESXI from: http://www.vmware.com/products/esxi/
--
--
Reuven Cohen
Founder & Chief Technologist, Enomaly Inc.
blog > www.elasticvapor.com
-
Get Linked in> http://linkedin.com/pub/0/b72/7b4
We need a standard open source virtual machine specification that can
be implemented by all the major hypervisors to avoid this mess.
Billy Newport
IBM eXtreme Scale | http://www-306.ibm.com/software/webservers/appserv/extremescale/
They have been doing this for a while, now no? This is dangerous in my
humble opinion because basically this is the new BIOS owned by one
company. Virtual images built within this hypervisor will only run on
this hypervisor. Different hypervisors are basically using different
machine definitions and this will cause lock in as you cant move to a
different hypervisor easily.
We need a standard open source virtual machine specification that can
be implemented by all the major hypervisors to avoid this mess.
Billy Newport
IBM eXtreme Scale | http://www-306.ibm.com/software/webservers/appserv/extremescale/
Billy,
No worries :)
There's already such effort, called Project Kensho
quote:
"... Citrix Systems, Inc ...
today announced "Project Kensho," which will deliver Open Virtual Machine Format (OVF) tools that,
for the first time, allow independent software vendors (ISVs) and enterprise IT managers to easily create hypervisor-independent,
portable enterprise application workloads. These tools will allow application workloads to be imported and run across Citrix XenServer™,
Microsoft Windows Server 2008 Hyper-V™ and VMware™ ESX virtual environments. "
source:
http://www.citrix.com/English/NE/news/news.asp?newsID=1679371&ntref=hp_article_headlines_US
Another thing to consider is replacement of Diane Greene with Paul Maritz (worked 14 years for Microsoft and heavily involved in cloud computing later),
which is also a strong indicator of the move, besides their official 5 stage roadmap (Separate, Consolidate, Aggregate, Automate, Liberate) to compute clouds on and off premise.
Also interesting to see VMware's financial results for 2nd quarter 2008
( http://www.vmware.com/company/q208highlights.html), I'll leave its interpretation up to you.
As for whether or not VMware has been involved in cloud computing, I
don't know of any major cloud deployments using VMware.
Reuven
On Mon, Jul 28, 2008 at 12:23 PM, William Newport <bnew...@mac.com> wrote:
>
--
--
www.enomaly.com :: 416 848 6036 x 1
skype: ruv.net // aol: ruv6
-Ian
Don't know how real this will be or if any of them will hold to doing it. I
am sure Xen and VMWare have more to gain and more reliable into following
standards than MS. We have seen MS's standards.
-Ron
When it comes to virtualization and the techniques deployed to achieve (in this case) partitioning one often discusses full virtualization (VMware VI3.x), para-vitualization (Citrix’s Xen), and container or O/S virtualization (a la Parallels - Virtuozzo). Several vendors realize that the Hypervisor is a commodity and are providing management platforms that allow the management of VM's of many "flavors" E.g. BinaryKarma - Fuild VM or DynamicOps. Open Virtual Machine Format (OVF) will help to increase function compatibility. At this point, I see the management platforms working with API’s and/or native management interfaces to allow for a global view and ease of management. However, I don’t suspect that we will have a real push from any of the hypervisor vendors to go beyond that point too fast.
Igor |
The only problem I see, would be their license terms:
VMware grants you a nonexclusive, non-transferable license, without rights
to sublicense, to (i) install or have installed a single instance of the
Software and each Licensed Additional Module on a single Server, unless
permitted to have multiple instances on a single Server or to have multiple
instances on multiple Servers by the payment of applicable license fees
(whether such fees are based on a per Processor, a per Virtual Machine, a
per user or any other VMware approved licensing model); (ii) use the
Software and each Licensed Additional Module solely for information
processing and computing purposes, including the hosting of computer
application-based services from a Virtual Machine and provision of such
services via an internal or external network, provided such services may not
consist of services to a third party that provide primarily computing or
processing power (such as utility computing or grid computing) or any
computer application-based service that is traded, rented, leased or sold on
a Virtual Machine basis; and (iii) use and reproduce the VMware Virtual
Infrastructure Client Software or VMware WebAccess (in object code form
only) for the purposes of installation and operation on an unlimited number
of your own internal computers or terminals solely for the purpose of
accessing the Server on which the Software is installed.;
That's going to restrict what a lot of people are doing/wanting to do (point
ii), because they don't want to loose their service provider licensing
revenue, where it's priced per VM instance per month.
Cheers,
Karl
Reuven
I would be interested in hearing some real world experiences with Live
migration. Mine haven't been very good.
Rackspace has VMWare support ....Don't look at mosso .... Look at Rackspace hosting services ....
"HT"
Dave "HT" Kramer
ht_k...@hotmail.com
972.636.5445
There is no need to add anything to WS-Management to make it RESTful. The basic underlying protocols within WS-Man such as WS-Transfer and WS-Addressing are also clearly RESTful (especially when used in an anonymous reply mode), and if you can believe (as many do) that SNMP is a RESTful architecture even with GET Next state-oriented semantics within the protocol, then WS-Man is equally RESTful. Most practical management protocols (such as SNMPv2 and WS-Enumerate) require some degree of session state to deal with management of large scale resources.
SaaS to really become mainstream needs (a) to separate the management interface from the service interface, and (b) use a widely accepted and simple to implement & use protocol such as WS-Man for that management interface.
Are there any SaaS applications out there that already do this? I've yet to find any but this space is growing rapidly.
I like the 451 definition of a cloud environment (although there are lots of others out there): http://blog.skytap.com/2008/07/451-report-on-cloud-computing/
I don’t believe Rackspace offers the following characteristics on their VMware-based hosting services (but tell me if I’m wrong!):
- Enforce the discipline of a retail model
- An API. If there is no API, then it is not a service
- Virtualizes resources (e.g., CPU, storage) as a service - multi-tenancy is key
- Self-service
- Enables resources to be consolidated
- Flexible
- Available on-demand
- Self-healing
- SLA-driven
-Ian
From:
cloud-c...@googlegroups.com [mailto:cloud-c...@googlegroups.com] On
Behalf Of Khazret Sapenov
Sent: Tuesday, July 29, 2008 7:25 PM
To: cloud-c...@googlegroups.com
Subject: Re: VMware steps into the cloud
On Tue, Jul 29, 2008 at 8:53 PM, Dave Kramer <ht_k...@hotmail.com> wrote:
Following is from Computer World:
Performance - you want the virtualized performance to be approximately equal to what the user is accustomed to. I have talked to a few people who found VMware virtual desktops to be unacceptably slow in their environment, while others have said it's ok. The point is that bandwidth, processor performance, and even the application demands on I/O processing can impact the choice of solutions. The first order of business is to make sure the virtual system performs for your end users. If it doesn't give an acceptable user experience, nothing else will matter.
Rgds,
Moshref
From: cloud-c...@googlegroups.com [mailto:cloud-c...@googlegroups.com] On Behalf Of Ian
Sent: Wednesday, July 30, 2008 9:06 AM
To: cloud-c...@googlegroups.com
(Disclosure: We are a VMware shop)
I can second Ormond's statement. I work for a large financial institution focusing on virtualization and "associated" technologies. One point of clarification is that a) implementation of moving one VM from one host to another differs from vendor to vendor. b) restrictions to the function exist, such as processor family etc.
Another point is that a Virtual Desktop Infrastructure (VDI) consists of a few more components than a "simple" server virtualization environment. I noticed a mentioning/discussion around VDI and thought it might help to point that out. Performance might be impacted by the implementation of a connection broker.
Vmotion could have started as a gimmick for sales - although very speculative statement in nature. The bottom line is that additional functions such as distributing resources fairly without impacting VMs, or in case of VMware 3.5 to develop the function of moving not only the VM but also it's associated data from LUN to LUN, were driven by Vmotion. Just to mention a few.
Igor
|
From: William Newport <bnew...@mac.com> |
Unless of course hiring that student works out cheaper than writing a
management GUI? ;-)
Sassa
2008/7/31 Paul Renaud <ren...@lanigangroup.ca>:
I'd expect the paravirtualisation to happen that way, i.e. expose the
devices via standard API, but I didn't think "the real" virtualisation
worked like that; I thought they'd use protected mode and handle
exceptions when "guest OS" wants to access I/O or execute a privileged
instruction. ...so there'd be no "API" that the guest OS could use:
the guest OS can use the API only when it knows it is
(para)virtualised.
Sassa
2008/7/29 William Newport <bnew...@mac.com>:
On Thu, Jul 31, 2008 at 1:45 AM, Paul Renaud <ren...@lanigangroup.ca> wrote:
> Many folk do believe SNMP to be a Restful architecture. For those that do,
> then WS-Man must be too.
I think Stu's point was that CRUD is different from REST
(http://www.infoq.com/articles/rest-introduction) ... and that with
SNMP you don't get the "HATEOAS" property
(http://www.stucharlton.com/blog/archives/000141.html).
> SNMP became the dominant management protocol of the previous millennium
> because it was SIMPLE -- not based whether it was RESTful or not.
YES!
> Yes there is some unnecessary fluff in WS-MAN (such as mechanisms to
> limiting the amount of info transferred), so there is added-value in
> defining a strict subset profile of WS-Man that can be used to simplify the
> majority of management implementations and promote adoption.
I am sure we would all like to see this in action - can you describe
the subset of WS-Man that you have in mind please and explain for us
non experts (1) how this relates to SNMP, and (2) where, if ever, it
is important that a RESTful principle is used.
That would be great :-)
alexis
Billy Newport
IBM eXtreme Scale | http://www-306.ibm.com/software/webservers/appserv/extremescale/
Why did you bring up BER as a drawback? It is only _encoding_, not the
schema. You aren't seriously suggesting that the enterprise users will
type raw XML for management just like your highschool student does!
Unless of course hiring that student works out cheaper than writing a
management GUI? ;-)
Many folk do believe SNMP to be a Restful architecture. For those that do, then WS-Man must be too. For those that don't, it's certainly easier to see Restful principles at work within WS-Transfer and pretty much all of WS-Addressing. In fact those two protocols (plus SSL) is all you need to use a subset of WS-Man to implement most of the equivalent functionality of SNMP.
Most importantly, to implement the basic and most common management function of polling the value of an attribute on a managed object - WS-Man is amazingly easy to use. I can easily get a high school student, using nothing more than a text editor, an XML schema, and some basic instruction on how to read XML, up and coding valid queries in an afternoon. That is simply not possible with SNMP's BER encoding, MIB tables, etc.
Please do not inflict your proprietary management protocol on us by open sourcing it. Keep it to yourself as I'm sure it does what you need it to. As for your consumers' needs there is no added value in introducing competing open implementations to something that already works, already provides the mechanisms required for managing the cloud, and is already widely accepted.
SNMP became the dominant management protocol of the previous millennium because it was SIMPLE -- not based whether it was RESTful or not. It's simplicity is what motivated thousands of smart people to stop trying to compete with it (it's easy to imagine extensions or improvements on it) and to realize their true objective of driving mass adoption of their product by promoting a common standard.
Having seen a lot of real world solutions I can tell you that your
average Ops person has no desire to read or write XML. SOAP, in this
regard, is just another nasty layer on top of it. Tooling only goes
so far to resolve this issue. In many real world situations, the
dominant monitoring and management tools aren't SNMP-based either.
Take Nagios for example. It's a widely deployed open source tool with
a large number of native plugins, most of which do not use SNMP. As
far as I can tell, SNMP has significant traction only in terms of
network management and monitoring. In terms of systems? Not so
much. Perhaps in commercial land, but no in open source land.
We need new data packaging standards that are user-friendly (JSON or
YAML anyone?) and I, for one, think the centralized monitoring
paradigm is fundamentally broken. If the machine is up and
operational it should monitor itself and the app that runs on it. It
should regularly report back it's status and that of the app. If it's
down, someone should be fixing it. Nothing else scales and nothing
else is as cloud friendly. It's really a distributed computing
problem at the end of the day.
--Randy
Randy Bias, Founder, CloudScale
(877) 636-8589, ran...@cloudscale.net
blog: http://neotactics.com/blog
I'd like to see simple standards like RSS/ATOM/JSON being used to
export metrics etc from entities. This is consumable by anybody with a
browser. I'm always thinking of exposing metrics from IBM eXtreme
Scale as an RSS feed with the last 30 values for example. This would
allow someone to use a company portal to build a management console
and view the news from CNN or indeed this mailing list along side it.
Software like yahoo pipes could be used to mine/build derivative
values from it and all without me having to do anything more complex
than export an RSS feed which is childs play to do.
Billy
Billy Newport
Hi Alexis,
I apologize for this long response, but much of the confusion around cloud
protocol design flows from an inadequate separation of concerns about what
the protocol should be designed to do in the first place. The first such
separation is between the service itself and the management of that service.
Every network service has a service interface that has one or more primary
interfaces that it provides to its consumers on the network. This
interface(s) can and should be very specific to the service that is being
offered. A wide variety of design patterns for network interface design
(such as RESTful) may be useful in defining that interface. But no matter
how well that service interface is designed, something more is required.
Every network service should also have a separate and secure management
interface that is used to manage that service. This interface should NOT be
confused with the service interface (for example it must be available even
when the service is not - e.g. so that maintenance on the service can be
performed remotely).
Unlike the service interface which is best designed to meet the needs of the
service provided, the management interface must be designed to meet the
needs of the remote management application(s) that use it. These management
applications vary greatly and may be autonomic or have users at the steering
wheel; may be old-fashioned single hierarchy manager/agent style managers or
multi-hierarchy or peer-manager style; federated or isolated, etc.
Regardless of how the management side of the application is organized or
controlled, the accepted design pattern for the management interface is:
1. a simple schema describing the resources available for management (e.g.
MIB in SNMP, WS-Addressing compatible XML schema in WS-Man). This maps well
to the "give resources an ID" concept in REST.
2. a mandatory implementation of a synchronous polling mechanism (e.g. GET
in SNMP and anonymous mode WS-Transfer in WS-Man) that enables the retrieval
of the attributes of a single resource within that schema. This also maps
well into the REST pattern of getting the attributes of a URI. The
implementation that responds to the poll may or may not be stateless.
3. optional implementation of an asynchronous eventing mechanism (e.g. Trap
in SNMP and WS-Eventing sink in WS-Man) based on a simple subscription about
the status change on a resource. A subscription is necessary so that a
destination can be supplied for these events. Asynchronous eventing is
optional because although the same management purpose can be accomplished
with synchronous polling, it is far less stress on the network to use
eventing. Asynchronous eventing is non-RESTful.
4. the interface itself must be secure (SNMP has less than ideal community
strings and in v3 user/name password, while WS-Man uses certs). Security is
not prescribed by the REST pattern.
5. optional implementation of lifecycle management for the managed resource
(create, update, delete) subject to the authorization control provided by
the security of the management interface (e.g. SET in SNMP,
Create/Put/Delete in WS-Man). Some folk incorrectly argue that lifecycle
management via CRUD-like operations is non-RESTful but HTTP GET, POST, & PUT
are clearly CRUD-style verbs! (see
http://en.wikipedia.org/wiki/Representational_State_Transfer)
Since most management protocols also use a minimalistic set of commands to
implement the above, and operate in a client/server context, a case can be
made for arguing that they are Restful.
It really doesn't matter whether a management protocol is Restful or not -
the most important thing is that it is both simple & standard so that it
will be widely implemented by both service providers and a wide variety of
management applications. Simplicity gets lost when people confuse the
separation between the service itself and management of the service.
The separation between service and management interfaces has been a
fundamental principle of network systems design since the birth of network
computing. It is fundamental to the object-managed object paradigm and also
shows up in many other places too. A couple of examples of this separation
of concern are ICMP at the network layer and SNMP at the application layer.
For example, in the Internet all network elements implement IP (service
interface) and ICMP (management interface) at the inter-network layer.
Specific network elements such as routers implement a variety of routing
protocols such as EGP, etc.(service interface) and SNMP (management
interface).
This fundamental principle is also well known in non-data networks such as
the public telephone, radio, and TV broadcast networks etc. For example,
the vertical blanking interval in a television network is used to carry
signal synchronization (management interface) that is not displayed as part
of the video scanning stream (service interface).
The separation between service and management exists at the network
application layer too. For example, your website is a network application
that responds to HTTP requests (service interface), but standing up a new
website is not usually performed by that same interface. To create a new
website you would define a management interface with different semantics to
register your website and to PUT the new webpages onto your website. If you
were cunning you might even use HTTP as the transport mechanism for such an
interface. Just because the interfaces are different doesn't mean that the
protocol cannot be shared, but caution must be used when defining a
management protocol based on re-using other protocols.
The need to separate the different concerns between service & management
interface is something that the current band of WS-* protocol experts seem
to have forgotten (hence the protocol bloat within the WS-Man suite). It
pains me to see the W3C committees debating the merits of reliable
messaging, fancy soap optimizations, session-semantics, etc for use in
management protocols. Protocol bloat is what killed off the entire ISO suite
of protocols, including CMIP/CMISE for management.
Defining a strict subset within WS-Man would greatly simplify the situation,
promote adoption, and not break existing implementations.
A first cut at a minimalist subset of WS-Man would be:
1. WS-Transfer with only anonymous Reply-To in response/actions (maps to GET
and PUT in SNMP),
2. WS-Enumerate (maps to GET Next and GET Bulk in SNMP) without filtering,
without expiry, without max time or length, and without Get Status.
3. WS-Addressing as is.
4. Optional WS-Eventing (maps to SNMP Trap) without Subscription End (not
reliable anyway)
5. WS-Security with only X509 signed tokens (possibly with only a single
encryption method)
Cheers,
Paul
I think it is as much a lock-in, as the OS software would be locked-in
after installing it on a physical machine.
However, in a paravirtualised environment this does happen
differently, but then the device drivers are
(para)virtualisation-aware (virtualisation-specific); in fact, it
seems it is the device drivers through which para-virtualisation is
achieved.
That's my understanding of virtualisation.
Sassa
2008/7/31 William Newport <bnew...@mac.com>:
:-) Yes, the XML has advantage only in adoption of the technology by
the developers, as they can play with it without having to bother with
the encoding. If there were ubiquitous libraries for handcrafting BER
out of a declarative text, like ASN.1 definitions with the values
filled in, there wouldn't be any difference. On the other hand, such
libraries would have existed only if BER was adopted by the developers
;-)
Oh, and here's a NSNMP Perl library for you:
http://search.cpan.org/~kragen/NSNMP-0.5/NSNMP.pm
> As an example, why would I want to do a bunch of SOAP stuff to authenticate,
> subscribe, etc. to a log when that same logfile could be simply mapped to a
> URL (something akin to '$ tail -f /var/log/system.log |nc -l 3333', only
> with HTTP and/or SSL auth).
>
>> Unless of course hiring that student works out cheaper than writing a
>> management GUI? ;-)
>
> How do you think [the most useful] GUIs get written in the first place?
Certainly, *not* by handcrafting the XML!
(I don't regard copy-pasting the examples as *handcrafting*, because
in that case you could as well handcraft any BER from the examples.
Without copy-pasting any solution is prone to typos, especially in
long and unreadable URIs (or OIDs for that matter))
Sassa
Sassa
2008/8/1 Randy Bias <ran...@cloudscale.net>:
6. Disaster-proof features: can't rely on distributed systems, because
this interface is for managing them, including lifecycle management
X.509 scales authentication, but this has nothing to do with
authorisation. Try to devise a scalable access control policy and
access control management infrastructure (PMI). Is X.509 PMI part of
WS-Security? And what do you do if the Root CA PKC / PKC repository /
AC repository is not available (the interface should be
disaster-proof)? You fall back to simple unscalable forms of
authentication and authorisation (hence the simplicity of these in
SNMP).
Oh, and both X.509 PKI and X.509 PMI are based on BER-encoded tokens.
Anyone up to handcraft that in XML? ;-)
Sassa
> Cheers,
> Paul
Sassa
2008/8/1 William Newport <bnew...@mac.com>:
-----Original Message-----
From: cloud-c...@googlegroups.com
[mailto:cloud-c...@googlegroups.com] On Behalf Of William Newport
Sent: Thursday, July 31, 2008 7:37 PM
To: cloud-c...@googlegroups.com
Subject: Re: WS-Man (was RE: VMware steps into the cloud)
Are we going to send a million management events per second encoded as
XML within SOAP packets, I don't think so. Network bandwidth is an
issue (especially when you're paying per bit in a cloud...).
While IBM is clearly committed to ws-*, it's not something I work on
directly or worry about at all.
Billy Newport
Paul, as I read your post, I expected you to recommend RESTful interfaces,
period. Not sure I followed why you thought WS-Man is necessary?
BTW - on event notifications, using RSS/ATOM is a perfectly acceptable
RESTful way of doing that.
I'm not known for being shy...
Are we going to send a million management events per second encoded as
XML within SOAP packets, I don't think so. Network bandwidth is an
issue (especially when you're paying per bit in a cloud...).
So, XML is as good on the bandwidth side as BER, if you are sending
mostly text. The BER can save you CPU on the decoding side, as it has
a definite-length form. There probably is no difference on the
encoding side for simple text structures.
So by using BER to consume text messages, you will save a few CPU
cycles on the client end, where they are quite cheap anyway. If you
were to save the bandwidth, you'd probably need to have a Base*46*
encoding.
Sassa
2008/8/1 William Newport <bnew...@mac.com>:
You're confusing management with monitoring. Monitoring is the
instrumentation of systems, which is easily distributable. Management
is the ability to take action (including alerting or response) based
on monitoring data. Monitoring is inherently distributable, but most
historical solutions bind management and monitoring together too
tightly.
Decentralizing management isn't that hard. It's mostly about
federating and focusing on applications, not datacenters. Datacenters
are a paradigm that is soon to be extinct.
(alright, subject to portability of the OS image that does not require
a "cold-reboot" of the image)
Sassa
2008/8/1 Krishna Sankar (ksankar) <ksa...@cisco.com>:
I agree monitoring is scalable, but scaling that is not needed without
scaling management. Wouldn't it be like having a scalable "feed"
without a scalable "feed back"?
I think there are some management operations that don't scale. For
example, federation works, once it is there. But creating a federation
is not scalable, because you can only select the members one by one.
There will be many other bootstrapping functions that won't scale.
Sassa
2008/8/2 Randy Bias <ran...@cloudscale.net>:
It may be an indirect level of awareness but it's coupled and locked in.
> [snip]
> Decentralizing management isn't that hard. It's mostly about
> federating and focusing on applications, not datacenters. Datacenters
> are a paradigm that is soon to be extinct.
I'd like to discuss this assertion some as I've had the same thought
cross my mind. I'm currently thinking that it's not that Datacenters
will become extinct, but our definition of what a datacenter is
will fundamentally change and become more distribute-able. I think
there will still be opportunity and advantages in providing the
infrastructure behind the cloud in some centralized areas -
probably where power and space is cheap. I also think there will
still be a need for investors in new technology within the
infrastructure and they'll want to have a place to 'put it'.
>
>
> --Randy
>
> Randy Bias, Founder, CloudScale
> (877) 636-8589, ran...@cloudscale.net
> blog: http://neotactics.com/blog
>
>
>
Kristi Schultz - Internet: kri...@us.ibm.com
STSM - Blue Cloud Ensemble Management
(507)253-2177 t/l 553-2177 Fax: (507)253-0424
----------------------------------
"Give the customer more than he expects."
"The more you laugh, the better things are."
[Butch Stewart]
>
> I don't think I am confusing them :-) They started talking of
> management, and later monitoring, because management without
> monitoring won't work. Also, monitoring without management is a little
> pointless, unless you want your boss to see the nice dials with the
> metrics (for any particular reason).
>
> I agree monitoring is scalable, but scaling that is not needed without
> scaling management. Wouldn't it be like having a scalable "feed"
> without a scalable "feed back"?
>
> I think there are some management operations that don't scale. For
> example, federation works, once it is there. But creating a federation
> is not scalable, because you can only select the members one by one.
Why do you make this assertion? There are a lot of ways to automatically
federate members based on any number of attributes.
> There will be many other bootstrapping functions that won't scale.
You can scale even the most trivial task using a hierarchy & delegation
model. That's how the human body works - that's how most complex
systems work... Management can be mapped to the same problem domain.
>
>
> Sassa
Billy Newport
On Aug 2, 2008, at 9:55 AM, Kristi Schultz wrote:
Billy Newport
The incremental overhead on eventing is not meaningful since, by definition,
events are asynchronous and do not put a constant load on the network.
Event filtering should be used in subscriptions to limit the possibility of
massive event flows.
Since only an idiot would try to poll a million monitors per sec, or not use
event filters (it's not the job of the protocol to determine how people
choose to use it), the "overhead" argument against using WS-Man for
management just doesn't wash.
When you consider the advantages of having an extensible XML-based
management schema, the flexibility provided by WS-Man makes the case for it
particularly compelling.
The same "overhead" argument against XML was discounted when SOAP was
introduced for RPC over HTTP.
-----Original Message-----
From: cloud-c...@googlegroups.com
[mailto:cloud-c...@googlegroups.com] On Behalf Of Sassa NF
Sent: August 1, 2008 3:14 PM
To: cloud-c...@googlegroups.com
Subject: Re: WS-Man
Since real life is not as black and white as some purists would pretend, I
tried to double click on the design pattern for management to highlight what
aspects of it were consistent with REST and which were not.
I did this not to necessarily recommend REST, but to illustrate that an
assessment of useful protocols is not as trivial as sticking labels on them.
WS-Man is necessary because (a) it already contains what we need for
management of SaaS (b) is well accepted as a standard (c) is easy to
implement and (d) provides the fastest route to critical mass of adoption to
unlock commercial acceptance of SaaS as a mainstream infrastructure.
The history lesson of how SNMP enabled the world that we now enjoy as the
Internet is very relevant and important to the future of SaaS. It was only
when (nearly) everyone at that time decided to standardize a simple and
modest management interface while differentiating, competing, and innovating
on the service interface that the Internet really blossomed. Some
service-interface innovations were standardized and created a platform upon
which other layers of innovation were added. The whole concept of "open
architecture" flowed from those early days.
This did not happen around the competing protocols at the time. DecNet,
SNA, ISO Suite, etc. all had large ecosystems of third parties. But the
lack of effective management prevented these from gaining the critical mass
that we now take for granted as the Internet. These protocols all had
oodles of service capability (e.g. there were lots of superior features in
SNA that are absent in TCP/IP). But the management of DecNet was simple but
proprietary, ISO was complex but standard, and SNA was both complex and
proprietary. Only SNMP was simple and standard.
To quote Marshall Rose (the godfather of SNMP) "The impact of adding network
management to managed nodes must be minimal, reflecting a lowest common
denominator."
History will repeat itself. The question is will the current generation of
SaaS solutions die or flourish before an industry standard for managing them
emerges? History teaches us that they will die unless this industry rallies
quickly around a baseline management standard.
With a baseline management standard widely used, further innovation &
specialization will undoubtedly happen because the world is not black and
white. E.g. the use of RSS for eventing does not have to be seen as an
either/or solution any more than RMON was either/or to SNMP. But we must
rapidly agree on a simple and standard basis to build on.
-----Original Message-----
From: cloud-c...@googlegroups.com
[mailto:cloud-c...@googlegroups.com] On Behalf Of Kristi Schultz
Sent: August 1, 2008 1:31 PM
To: cloud-c...@googlegroups.com
Subject: RE: WS-Man (was RE: VMware steps into the cloud)
>
> No I tried to say that being Restful doesn't matter as much as simple and
> standard when it comes to management protocols.
>
> Since real life is not as black and white as some purists would pretend,
I
> tried to double click on the design pattern for management to highlight
what
> aspects of it were consistent with REST and which were not.
>
> I did this not to necessarily recommend REST, but to illustrate that an
> assessment of useful protocols is not as trivial as sticking labels on
them.
>
>
> WS-Man is necessary because (a) it already contains what we need for
> management of SaaS (b) is well accepted as a standard (c) is easy to
> implement and (d) provides the fastest route to critical mass of adoption
to
> unlock commercial acceptance of SaaS as a mainstream infrastructure.
Ok, let me try this. I am mildly familiar with WS-Man. More familiar with
WSDM. Neither do I find *easy* to understand nor implement. Not to say it
can't be done and not to say that someone who has invested the time to
learn
and understand them to that level can't do it - just that I haven't yet,
so there we are. On the other hand, I have read several articles about
REST.
Probably all inclusive is a couple hours worth of time invested. Now, I'm
making this up since I have no real experience doing this, but if I was
told today, with my current level of knowledge and understanding that a
particular cloud service, was now mine to
manage and it supported a RESTful web services interface, all I should
have to know is the URL to it's management service. Then I would know
that to get information about it, I'd do a HTTP GET requesting XML or
WADL/WSDL? That would tell me what GET/PUT/POST/DELETE URLs are
supported by the management service. The GETs are immutable requests
for information about the resources to manage. PUT/POST is used to
update (e.g. change state) or create new resources. DELETE is used
to remove resources. All resources are accessible via URL and there's
a logical hierarchy to them.
What is it that WS-Man provides over and above this? Maybe I'm being
ignorant or naive, but if so, please educate me.
>
Billy
Billy Newport
Ws-Man does basically what you just described above. That's the beauty of
it. It also makes judicious use of SOAP in the process so that you can
implement it on top of WS-aware web servers.
You are doing yourself a disservice by not reading up on WS-Man. Spend 15
minutes reading WS-Transfer and you'll see what I mean.
What I thought "awareness" was, is when OS knows it is not real
hardware, and does something special to cater for hardware being
virtual.
Sassa
2008/8/2 William Newport <bnew...@mac.com>:
I'd like to meet the customer IT manager that would not want to monitor your
SaaS service using a standard management interface to validate your SLAs.
-----Original Message-----
From: cloud-c...@googlegroups.com
[mailto:cloud-c...@googlegroups.com] On Behalf Of Sassa NF
Sent: August 2, 2008 3:50 AM
To: cloud-c...@googlegroups.com
Billy
http://www.vmware.com/appliances/learn/ovf.html
I get it. Once something has been provisioned, it is not commodity
anymore, as it acquired a purpose.
Sassa
2008/8/3 William Newport <bnew...@mac.com>:
That's right, but you need to give them the attributes first and you
can't do *that* in bulk.
>> There will be many other bootstrapping functions that won't scale.
>
> You can scale even the most trivial task using a hierarchy & delegation
> model. That's how the human body works - that's how most complex
> systems work... Management can be mapped to the same problem domain.
Show me how.
To make this a task that is simpler to visualise, you can explain how
you can create LDAP subtrees of arbitrary complexity in a scalable
way. Let's start with how you are going to name the entries. Naming
already implies some grouping. Then you can explain how you are going
to assign the attributes. It is pointless to assign attributes to all
the entries from a subtree, because otherwise the name of the entry is
equal in its meaning to the attribute that you want to assign. Since
at this stage there are no attributes apart from name, the selection
strategy is not attribute-based, and I cannot see how this task can
scale.
If you delegate the task, the delegates will only be able to build A
Tree, not The Tree (because the tree spec passed from above will have
to be coarse-grained for scalability reasons).
Say, the analogy with a body is good, but it is not fault tolerant: if
a nerve stops working, the body loses the muscle or sense. Management
entities are there to fix problems with "non-management" entities. But
there's nothing to fix problems with the management entities. If you
were to manage the hierarchy of management entities, you would need a
hierarchy of management entities. ;-)
The other point I was making is that the managed entities would either
be managed by a single manager (would you call that
scalable-available-fault-tolerant?), or multiple managers (how do you
build scalable security and conflict resolution here? under constraint
that you cannot use security services that require scalable resources,
because these resources need to be managed, too).
There are things that just *don't* scale. A "leader", for one, doesn't scale.
Sassa
Sassa
2008/8/3 Paul Renaud <ren...@lanigangroup.ca>:
I think scalability is still important, even in self-management world.
Just that in the case of self-management the human input is replaced
with a pattern recognition and an expert system. Say, if this
automated system makes a decision 1000 times faster, then it will
scale just 1000 times more, then stop.
Someone at some point will have to deal with individual servers. The
best scalable thing is when the server can take care of itself, but
sometimes you would still need to cater for or override something
manually, which won't scale.
Management doesn't scale well, that's why when the servers fail in a
shipping container, they are just shut down, and having reached a
critical mass the whole container is simply shipped back to the
manufacturer.
Sassa
2008/8/4 Bob Lozano <bobl...@gmail.com>:
> Couple of thoughts on the issues raised in this thread:
>
> simplicity of use (the general shot against ws-*) will rule the day. If this
> thread is any sampling of what people are going to actually do, ws-man
> doesn't have too much of a future (in practice), and it's relative
> complexity will likely be it's doom.
> the only way mgt (and yes, monitoring as well) of entities at this scale
> will even be tractable is IF large pools are largely self-managing. It is
> simply inconceivable to even care about, much less actively manage
> individual servers.
>
> In particular, each entity (we have called them fabrics - a self-organizing
> group of machines / VMs acting like one thing, in support of a group of
> apps) must be responsible for behaving correctly, on their own.
>
> That way, outside management / monitoring can be reduced to setting new
> objectives. It'll be ok if the mgt services fail, since it's not actually
> needed for ensuring that the apps.
>
> Bob Lozano
> Appistry
>
>
I think scalability is still important, even in self-management world.
Just that in the case of self-management the human input is replaced
with a pattern recognition and an expert system. Say, if this
automated system makes a decision 1000 times faster, then it will
scale just 1000 times more, then stop.
On Fri, Aug 1, 2008 at 7:16 AM, Paul Renaud <ren...@lanigangroup.ca> wrote:
> Hi Alexis,
>
> I apologize for this long response, but much of the confusion around cloud
> protocol design flows from an inadequate separation of concerns about what
> the protocol should be designed to do in the first place. The first such
> separation is between the service itself and the management of that service.
No problem - sorry for my slow reply.
> Every network service should also have a separate and secure management
> interface that is used to manage that service.
> ...
>
> Regardless of how the management side of the application is organized or
> controlled, the accepted design pattern for the management interface is:
>
> 1. a simple schema describing the resources available for management (e.g.
> MIB in SNMP, WS-Addressing compatible XML schema in WS-Man). This maps well
> to the "give resources an ID" concept in REST.
>
> 2. a mandatory implementation of a synchronous polling mechanism (e.g. GET
> in SNMP and anonymous mode WS-Transfer in WS-Man) that enables the retrieval
> of the attributes of a single resource within that schema. This also maps
> well into the REST pattern of getting the attributes of a URI. The
> implementation that responds to the poll may or may not be stateless.
>
> 3. optional implementation of an asynchronous eventing mechanism (e.g. Trap
> in SNMP and WS-Eventing sink in WS-Man) based on a simple subscription about
> the status change on a resource. A subscription is necessary so that a
> destination can be supplied for these events. Asynchronous eventing is
> optional because although the same management purpose can be accomplished
> with synchronous polling, it is far less stress on the network to use
> eventing. Asynchronous eventing is non-RESTful.
>
> 4. the interface itself must be secure (SNMP has less than ideal community
> strings and in v3 user/name password, while WS-Man uses certs). Security is
> not prescribed by the REST pattern.
>
> 5. optional implementation of lifecycle management for the managed resource
> (create, update, delete) subject to the authorization control provided by
> the security of the management interface (e.g. SET in SNMP,
> Create/Put/Delete in WS-Man).
Are all the above resources 'available for management' and accessible
in a uniform way, or are those methods in fact accessing different
sets of resources that are almost-the-same-but-not-quite? Sorry if
this is a stupid question.
> Some folk incorrectly argue that lifecycle
> management via CRUD-like operations is non-RESTful but HTTP GET, POST, & PUT
> are clearly CRUD-style verbs! (see
> http://en.wikipedia.org/wiki/Representational_State_Transfer)
In which case, why is navigating the HTTP based web different from
interacting with a database using SQL?
> It really doesn't matter whether a management protocol is Restful or not -
> the most important thing is that it is both simple & standard so that it
> will be widely implemented by both service providers and a wide variety of
> management applications. Simplicity gets lost when people confuse the
> separation between the service itself and management of the service.
+1
> The separation between service and management exists at the network
> application layer too. For example, your website is a network application
> that responds to HTTP requests (service interface), but standing up a new
> website is not usually performed by that same interface. To create a new
> website you would define a management interface with different semantics to
> register your website and to PUT the new webpages onto your website. If you
> were cunning you might even use HTTP as the transport mechanism for such an
> interface. Just because the interfaces are different doesn't mean that the
> protocol cannot be shared, but caution must be used when defining a
> management protocol based on re-using other protocols.
That sounds worth exploring in more detail if you wish to demonstrate
how to be RESTful, at least with HTTP.
> Defining a strict subset within WS-Man would greatly simplify the situation,
> promote adoption, and not break existing implementations.
>
> A first cut at a minimalist subset of WS-Man would be:
>
> 1. WS-Transfer with only anonymous Reply-To in response/actions (maps to GET
> and PUT in SNMP),
> 2. WS-Enumerate (maps to GET Next and GET Bulk in SNMP) without filtering,
> without expiry, without max time or length, and without Get Status.
> 3. WS-Addressing as is.
> 4. Optional WS-Eventing (maps to SNMP Trap) without Subscription End (not
> reliable anyway)
> 5. WS-Security with only X509 signed tokens (possibly with only a single
> encryption method)
Paul, I have to say Thank-you for providing a first stab. But, ..
forgive me, is the above really simple? Would it be better to set out
a RESTful approach from first principles and then relate it to
existing management systems to show how it worked. Again apologies if
this is ass backwards...
Cheers,
alexis
>
>
> >
>
WS-Man is both simple and standard. I can hand-code a WS-Man request if I need to (yes most of us will use tools to do the dirty work in practice). That is a working definition of simplicity.
Your choice of REST as a starting point for deciding how to approach mgmt of
SaaS is problematic. The starting point should be first deciding what
management capability you wish to have your customers to have and then
choosing a standard way to provide it. It is more important for a public
management interface to be standard, than it is to be RESTful.
The reason why standards are so important is so that your interface will be
consumable by the maximum number of customers possible. Proprietary
interfaces, even elegant ones, are not attractive.
-----Original Message-----
<snip>
> Are all the above resources 'available for management' and accessible in a
uniform way, or are those methods in fact accessing different sets of
resources that are almost-the-same-but-not-quite? Sorry if this is a
stupid question.
<snip>
My point is that SaaS needs management interface standards for use by
customers.
On Tue, Aug 5, 2008 at 4:17 PM, Paul Renaud <ren...@lanigangroup.ca> wrote:
>
> Your choice of REST as a starting point for deciding how to approach mgmt of
> SaaS is problematic.
I did not suggest that. I suggested that it might be easier to
articulate your verbs more RESTfully and then relate them to those you
listed in your WS-Man subset.
> The starting point should be first deciding what
> management capability you wish to have your customers to have and then
> choosing a standard way to provide it. It is more important for a public
> management interface to be standard, than it is to be RESTful.
This is not an 'either or' choice.
You are the one who suggested this could all be RESTful. I am trying
to be a helpful interlocutor :-)
alexis
Yep, but some non-zero amount of management is centralized and won't scale.
> Any other approach is destined to ultimately fail under its own
> weight. The only "centralized" aspect is the goal (or set of goals)
> itself that the system as a whole will work towards.
Hmm... That's probably the confusing bit, that's why it is both
centralised and decentralised.
I meant that it is centralised in the same way as a general monitors
and manages the whole army, and the whole army obeys his orders.
However, his monitoring and decision capabilities *don't* scale; he
can monitor and manage only a small subset of managers, say, colonels.
Now, the system below the general *doesn't* scale, because it cannot
exceed a certain number of colonels, lieutenants, sergeants and
privates. To grow the system further, you need to change its structure
and functionality; say, now the general does not make the decisions
anymore, he obeys and interprets the orders of chief of staff, and the
orders now include requirement to cooperate with the entities outside
the tree under the general (what does this mean on a scale of NATO?).
These changes propagate through the whole tree; say, now every officer
and soldier needs to recognize the soldiers from under the other
generals and be able to interpret the orders they've been given to
build a new strategy to optimally achieve the goals of both. It isn't
a language barrier. It is a problem of the infantry everywhere with
appearance of the airforce. It operates in a completely new way, so it
is hard to tell what you need to do as an individual to enable
efficient operation of that unit.
This problem isn't unique. When a new department is formed, it is
always hard to determine what you *have* to do and what you ought to
double-check with the manager first. Haven't you experienced this?
There will be confusion until a policy is worked out at every level
where integration is required; the larger the new dept (the larger the
new set of functionality), the harder it is to achieve.
Basically, you can still apply the same technique to solve the
management problem when the number of entities changes, i.e. organize
the entities in a new hierarchy, but this only proves scalability of
the technique, and not necessarily scalability of the management
structure alone.
Regards,
Sassa
Just as you said (my emphasis): the rules are *identical*. Managing
homogenous distributed system is a piece of cake and can easily scale.
Is this always the case in a cloud? It wasn't so in the grids (if they
are any different).
> In a certain sense this is more akin to the US-style of military command and
> control (particularly in places like the marines and special forces etc.,
> with operational groups having wide attitude in choosing how to achieve
> their objectives, always operating within a defined set of behavioral
> rules). The analogy is not perfect, but it is relevant.
Now, you talking on the level of a group of commandos. However, what
it really is, is an objective set out on a strategic level. A general
orders invasion of an island and an outline of action (throw an army 4
times the resistance force). Now, every branch of command devises what
this means. Someone agrees airsupport of 20 F19s and 2 B-52s, another
one provides an aircraft carrier, someone else organises troop
training for a sufficient number of units, someone else needs to
provide catering and healthcare. Then a general devises a plan how
this all will work together and sends an order down the chain of
command, which results in a set of tactical missions devised at the
lower levels and debriefing for each squadron of their particular
task. Then each unit coordinates their activity using a variety of
communication strategies.
OK, in a computer world this will read something like "I want to
deploy this app here", which breaks down into providing shared file
space, CPUs and bandwidth. Someone needs to plan and resolve security
implications from passwords to firewalls, someone needs to plan
routing and location, someone needs to provide messaging. I don't
think I need to enumerate all of the systems involved. Whilst each
individual piece of the puzzle scales, the management problem will be
in orchestrating the solutions to these puzzles, and I am not
convinced this can be easily shown to scale in a heterogenous
environment (i.e. multi-purpose multi-tenant multi-app distributed
system).
I respect your opinion and I agree there are solutions for parts of
the puzzle or subset of the problem.
Sassa
> That is, more or less, how an app that has been cloud-enabled with an
> "application fabric" layer like our stuff works in practice. Objectives are
> set at the top, groups are organized and work done from the bottom up.
>
> That's pretty much the same general ideas that Cameron was referring to in
> his comments, I believe.
>
Even a 2-tier management hierarchy can scale to significant size if the
management mechanisms are well thought through. For example, Platform
Computing's ActiveCluster was used successfully in 2001 to create a virtual
production grid of over 120,000 computers using a 2-tier management
hierarchy.
Highly decentralized decision making tend to result in local optimizations
at the expense of a better global solution. Depending on the application,
this can be tolerable, but it is hardly the case that decentralized systems
are always better than centralized systems.
Centralized vs decentralized is a religious issue that will find as many
facts supporting both sides.
-----Original Message-----
From: cloud-c...@googlegroups.com
[mailto:cloud-c...@googlegroups.com] On Behalf Of Sassa NF
Sent: August 5, 2008 3:48 PM
To: cloud-c...@googlegroups.com
Subject: Re: WS-Man (was RE: VMware steps into the cloud)
You are correct that it doesn't have to be an either/or choice. There are
many aspects of most management protocols which can be found to be RESTful.
Perhaps a RESTful assessment of WS-Man might be a good exercise for anyone
who would like to understand it better.
Most technical folk can read the entire collection of WS-Man standards in
under an hour. When I first read them I was struck by the relative
simplicity of the standards selected to be parts of the WS-Management suite.
And although I personally do not care necessarily to ride on the REST
bandwagon, I suspect that those that do will find many examples of REST at
work in those standards.
For example, using WS-Addressing, all attributes on a managed object are
described as resources accessible via a URI. And using WS-Transfer, you can
GET them to examine their values.
My main point however, is that we should be less concerned about whether the
protocol is RESTful and more concerned about ensuring that all cloud
services provide it. Otherwise, the cloud will be more of a fog that few
customers will find their way thru to use.
Cheers, paul
-----Original Message-----
From: cloud-c...@googlegroups.com
[mailto:cloud-c...@googlegroups.com] On Behalf Of Alexis Richardson
Sent: August 5, 2008 11:29 AM
To: cloud-c...@googlegroups.com
Subject: Re: WS-Man (was RE: VMware steps into the cloud)
These are homogenous systems. A solution at any level of a DNS
hierarchy does not differ from another. I am not sure if it is
management, too.
However, managing 50K workforce and a 5 people unit are completely
different tasks in terms of the scope of monitoring and the kind of
decisions you have to make. It is not just 10K times more lines of
code that can be uniformly split between all software development
units in the hierarchy.
> Even a 2-tier management hierarchy can scale to significant size if the
> management mechanisms are well thought through. For example, Platform
> Computing's ActiveCluster was used successfully in 2001 to create a virtual
> production grid of over 120,000 computers using a 2-tier management
> hierarchy.
Cool. This proves it scales to 120K entities, but doesn't prove
arbitrary scaling for arbitrary purpose.
Sassa
> Everything we need to know about cloud computing can be discovered by
> looking at the behavior of real clouds and weather.
Interesting analogy.
> Now the only question left is... who will build the magic carpet to
> transcend all the clouds?
There is a vast eco-system of consultants and applications building up
large skill sets to help companies use the various cloud providers. I
am thinking of companies such as Rightscale and Arjuna here. While I
dont know of a product or service that allows you to swap and change
between them I am sure that one will appear soon.
I wrote a blog post close to this topic a few weeks ago:
http://www.spoutingshite.com/2008/06/30/the-aws-ecosystembarrier-to-innovation/
Ross Cooney
www.emailcloud.com
Why should we not be concerned about this?
You're basically saying that 'architecture doesn't matter, just pick
something', which if the past few years are any indication, is the
road to ruin.
Cheers
Stu