Specification driven provisioning

[Anthony Shortland]

We're great proponents of specification driven provisioning.

Whether the specification is embodied in configuration (like Puppet or Chef) or in a resource model (like AWS Cloud Formation) we're constantly building solutions that start with structured text specifications (e.g. JSON, YAML or XML) as source artifacts relying on an SDLC to build and release to operations.

With that in mind, I'm wondering what the list thinks of the Oasis TOSCA technical committee's work (http://www.oasis-open.org/committees/tosca/) that got going in January?

They've posted an initial specification (TOSCA Specification Version 0.8.2 - http://tinyurl.com/7jc4697) that while obviously incomplete nonetheless points the way to a general abstraction for the specification of cloud applications.

Better than inventing your own each time, surely? Great way to avoid lock-in, perhaps?

Anthony.

[Ernest Mueller]

Hmm.  We did invent our own high level XML system model at National Instruments, so I'm a big believer in the concept.  I don't know if I like this one, though, as it doesn't seem to really get down to the app level (it mentions deployables but has no concept of separate apps on one system) - and it's pretty complex.  Definitely "from big system vendors."

Here's my main frustration with the modelling that CloudFormation, Juju, etc. all provide.  They basically always give up at some level and say "well of course you could put this on top of puppet for provisioning and then on top of capistrano to deploy your apps..."  In my admittedly devops-addled mind, the whole Goddamn point is getting your apps out there.  My interest wanes in a framework that doesn't address that.

Here's a presentation on how we did PIE - http://www.slideshare.net/mxyzplk/pie-101
Basically we considered the overall system and wanted the whole thing to be addressable.  Under the covers it may use different things for implementation of "installing apps" or whatever but the top level model should NOT punt on it.

We devised an XML schema with elements of system, environment, role, image, service, command, and template.  Our definitions were different from TOSCA's - a service was software on board the box, and services can have relationships - so I might define a system of "my SaaS product," an environment of "production," a role of "app tier" that uses an "image" of a specific AMI, and has a "service" that is tomcat and another "service" that is a specific app "contained in" that tomcat, that also "calls" a mySQL service contained in another role.

Therefore in ONE addressing system we could reference clusters, boxes, apps, whatever.  Therefore,

pie restart /mySaaS/prodAWSEast/external-services/2/tomcat

Is a request to restart the tomcat on box #2 in the external-services role in our production AWS east environment.  (we also used zookeeper to know where that is...) Since the service dependencies are defined, we know what apps are affected by that restart and also that there's a mySQL dependency in the case of a larger orchestrated operation.  And when you "pie create /mySaaS/prodAWSWest" all the stuff that comprises a "mySaaS"  down to the apps is provisioned in the new DC.

I think the guys at NI are still working on open sourcing PIE, but at a bare minimum I'd like to see people adopting the philosophy that there's not a system model that is ignorant of apps and then separate app manifests - that is lame.  "DevOps, did I stutter?!?"  Why can't I model my whole system, including where the apps come from (build).  Real answer, because some tools are being built out from the "systems vendors/POV" and others from the "dev vendors/POV".  Quit it!

The revolution will not be televised.

Ernest

[Nathaniel Eliot]

Agreed: switching between deploy systems for different deploy scopes
is a waste of brain-power.

We've got a specification driven process with Ironfan. Because it's
all built in Chef, we can take advantage of a common tooling language
for deploys, with cookbooks for most common components already written
(by vendors or their power-users). We still need to wrestle some with
individual vendor decisions about architecture, but it all lives in a
common code base and uses common tools. Our cluster schemas are in a
Ruby DSL that is intentionally close to the Chef DSL, but I see some
similarities to what you describe in PIE. (I'm also envious of PIE's
deep addressing, and may put something of the sort on the roadmap for
Ironfan.)

I feel a lot less excited by TOSCA, especially since they seem to be
writing the specification first and only then worrying about code
which satisfies it. System deployments have a lot of edge cases, and
if you're not careful you can cut yourself with an ignorant model. As
warty as it can sometimes be, I feel loads better using Chef (and
Ironfan), which I know have extensive practical use informing the
code.

--
Nathaniel Eliot
T9 Productions

[Anthony Shortland]

Your comment "Our definitions were different from TOSCA's - a service was software on board the box" broaches the subject of node/role-centric vs. service-centric modeling.

With ControlTier we always modeled applications from the service perspective (in your sense), but since then - with Rundeck and the current crop of configuration and infrastructure management tools - the world has gone hard over to node/role-centric modeling. Perhaps most people always thought that way (e.g. the "web box" or the "database server"), and we just failed to convince them otherwise!

That said, TOSCA's notion of "node" is more akin to any general resource so I believe there's provision for modeling topology at either level.

Beyond that, while it's the least developed part of the specification, the Plan section opens the door to specifying *what* is required to deploy and manage applications without dictating *how* it is done. Although TOSCA is not a framework in itself, it does provide a missing piece to the loosely-coupled tool-chain framework puzzle.

Anthony.

[Anthony Shortland]

Actually I don't think the TOSCA TC is ever going to worry about "code which satisfies it" -  the point being that the specification is the goal in itself.

That said, I couldn't agree more that their work should be informed by real-world experience. Those edge-cases'll get you every time!

With that in mind, if you are as interested in this as I am, you could consider subscribing to their comment mail group (http://lists.oasis-open.org/archives/tosca-comment) as a way to provide feedback.

Anthony.

Anthony Shortland

[ Developer | ControlTier Open Source Project | mobile: 650.215.3117 aim: anthony....@me.com yahoo: anthony.shortland irc.freenode.net: #controltier skype: anthony.shortland ]

[Lee Hambley]

As the Capistrano maintainer (however, not the original author), I'm a massive proponent of "doing one thing and doing it right", therefore toolkits which provision, and deploy make me disappointed that after years of having the internet we haven't learned these lessons yet. Although I acknowledge in a modern web application stack the line between application and infrastructure is blurry.

Heroku have made an art of this for Ruby (and recently other languages) by trivializing deployment to a few knobs and dials, and pushing code via Git. Their model works exceptionally well, although it is prohibitively expensive.

For me the notion that "app is not infrastucture" boils down to not trusting the tools, as a software-focused jack-of-trades, the ops side still feels like a dark art at times, and the idea that I'd "upgrade" my "application" dependency via Puppet or Chef, the same way as one might upgrade postgresql or something worries me a little, and whilst continuously deploying the app makes sense (unit tests, integration tests) but continuously deploying /infrastructure/ is another level for me, and a level with which I am not personally comfortable.

I hope that these tools continue to specialize, or else people package their applications as .deb, or whatever package the tool is expecting, that's to say, if you want to treat your application as a infrastructure component, engineer and distribute it as such, and use your choice of Chef/Puppet as sugar on top of using apt/emerge/etc to pull the latest copy down onto the servers in your cluster.

Just my not very helpful .2¢

Lee Hambley

[Yaakov Nemoy]

I think you're coming at this from a different angle than what Anthony
is suggesting. At some level in a spec driven system, you will need a
tool like Capistrano, Chef, or Puppet to handle very specific
components of the system. Above that, you need a single source of
truth and a single master for orchestration of your entire network to
keep things in check. Letting Puppet decide when to schedule upgrading
a backend DB server is crazy. Telling puppet to upgrade the backend DB
server and make the backend DB servers consistent right now is
perfectly sane. Using another system that then requires your DB proxy
to restart to find the new servers, which then calls Puppet is also
sane.

Lots of little tools that do one thing well is great, but it's good to
have a top down view of the bigger picture.

My even more worthless 0.02 USD.

-Yaakov