Myopic View of DevOps Misses the Mark

48 views
Skip to first unread message

Guillaume FORTAINE

unread,
Jun 26, 2011, 5:21:10 AM6/26/11
to agile-system-...@googlegroups.com, dev...@googlegroups.com
Hello,

I would greatly appreciate to have the point of view of the DevOps
community about these various questions, if
possible, please :

http://pleasediscuss.com/andimann/20100326/myopic-devops-misses-the-mark/

a) Who handles ongoing support, especially software update for the
unrestrained sprawl of non-standard systems and components.

b) Who ensures each new application doesn’t interfere with existing
and especially legacy systems (and networks, storage, etc.)?

c) Who handles integration with common production systems that cannot
be encapsulated in a VM, like storage arrays (NAS, SAN), networking
fabrics, facilities, etc.

d) Who handles impact analysis, change control and rollback planning
to ensure deployment risk is understood and mitigated?

e) Who is responsible for cost containment and asset rationalization,
when devops keeps rolling out new systems and applications?

f) Who ensures reporting, compliance, data updates, log maintenance,
Db administration, etc. are built into the applications, and
integrated with standard management tools?

g) Who will assure functional isolation, role-based access controls,
change auditing, event management, and configuration control to secure
applications, data, and compliance?

I look forward to your answer,

Best Regards,

Guillaume FORTAINE
gfor...@gfortaine.biz
+33(0)631.092.519

Jon Wood

unread,
Jun 26, 2011, 5:27:44 AM6/26/11
to dev...@googlegroups.com, agile-system-...@googlegroups.com

On Sunday, 26 June 2011 at 10:21, Guillaume FORTAINE wrote:

Hello,

I would greatly appreciate to have the point of view of the DevOps
community about these various questions, if
possible, please :
The answer to all those questions is the same - your sysadmin team. "Devops" is almost by definition not a team in itself, but greater collaboration between teams. Just because your developers and sysadmins have decided to work closer together doesn't mean that everyone stops specialising, and the ops team suddenly doesn't have any time to maintain your infrastructure because they're too busy building autocompletion into your software.

These considerations don't stop being made, but with teams that are working closely they can be made with a better insight into everybody's requirements, rather then the ops team passing down decrees from on high.

Jon

Kief Morris

unread,
Jun 26, 2011, 5:44:38 AM6/26/11
to dev...@googlegroups.com, agile-system-...@googlegroups.com
On Sun, Jun 26, 2011 at 10:27 AM, Jon Wood <j...@blankpad.net> wrote:
> On Sunday, 26 June 2011 at 10:21, Guillaume FORTAINE wrote:
>>
>> Hello,
>>
>> I would greatly appreciate to have the point of view of the DevOps
>> community about these various questions, if
>> possible, please :
>
> The answer to all those questions is the same - your sysadmin team. "Devops"
> is almost by definition not a team in itself, but greater collaboration
> between teams. Just because your developers and sysadmins have decided to
> work closer together doesn't mean that everyone stops specialising, and the
> ops team suddenly doesn't have any time to maintain your infrastructure
> because they're too busy building autocompletion into your software.
> These considerations don't stop being made, but with teams that are working
> closely they can be made with a better insight into everybody's
> requirements, rather then the ops team passing down decrees from on high.
> Jon

I would say the answer is simply "the team". I agree the ops team will
have much of the specialist knowledge and skills around these areas,
but the idea of DevOps is for everyone to have awareness and take
ownership of these concerns, rather than focusing on their own
immediate concerns.

Kief

Kief Morris

unread,
Jun 26, 2011, 5:55:27 AM6/26/11
to dev...@googlegroups.com, agile-system-...@googlegroups.com
On Sun, Jun 26, 2011 at 10:21 AM, Guillaume FORTAINE
<gfor...@gfortaine.biz> wrote:
> I would greatly appreciate to have the point of view of the DevOps
> community about these various questions, if
> possible, please :
>
> http://pleasediscuss.com/andimann/20100326/myopic-devops-misses-the-mark/

I agree with Andi that much of the writing about DevOps has come from
the development perspective, but Ops people who can ditch the attitude
of "[ops] manage what they are given, and keep the business running,
regardless of the mess that gets thrown over the wall at them" will
benefit greatly from DevOps. Andi is dead-on with "apps team have to
pull their weight too, by addressing gaps ..."

DevOps is an opportunity for the ops people to get involved at the
start of the development process and make sure that those various
support and operational concerns are baked in, along with making sure
that infrastructure and support changes are well understood, prepared,
and tested from the start.

This is a vast improvement over having to wait until a "finished"
product is thrown over the wall at them, and then working around the
mess to make it work.

Rather than getting our backs up about DevOps as a threat, we ops
people should seize the opportunity.

Kief

Andrew Clay Shafer

unread,
Jun 26, 2011, 9:13:01 PM6/26/11
to devops

>
> I agree with Andi that much of the writing about DevOps has come from
> the development perspective, but Ops people who can ditch the attitude
> of "[ops] manage what they are given, and keep the business running,
> regardless of the mess that gets thrown over the wall at them" will
> benefit greatly from DevOps. Andi is dead-on with "apps team have to
> pull their weight too, by addressing gaps ..."
>

I would appreciate it if someone could point out the devops writing
that is dev centric.

I feel and hear that the opposite is true.

I responded to Andi when he first posted that.
http://stochasticresonance.wordpress.com/2010/03/26/devops-misnamed/

James Turnbull

unread,
Jun 26, 2011, 9:27:38 PM6/26/11
to dev...@googlegroups.com

Agreed. The DevOpsDays events I've attended the number of dev attendees
were usually 10% or less. One of the key problems I see is that more
engagement, education and exposure is needed in the dev side of the house.

Regards

James Turnbull

--
Author of:
* Pro Puppet (http://tinyurl.com/ppuppet)
* Pro Linux System Administration (http://tinyurl.com/linuxadmin)
* Pro Nagios 2.0 (http://tinyurl.com/pronagios)
* Hardening Linux (http://tinyurl.com/hardeninglinux)

Kit Plummer

unread,
Jun 26, 2011, 10:05:22 PM6/26/11
to dev...@googlegroups.com

On Sunday, June 26, 2011 at 6:27 PM, James Turnbull wrote:

> Andrew Clay Shafer wrote:
> > > I agree with Andi that much of the writing about DevOps has come from
> > > the development perspective, but Ops people who can ditch the attitude
> > > of "[ops] manage what they are given, and keep the business running,
> > > regardless of the mess that gets thrown over the wall at them" will
> > > benefit greatly from DevOps. Andi is dead-on with "apps team have to
> > > pull their weight too, by addressing gaps ..."
> >
> > I would appreciate it if someone could point out the devops writing
> > that is dev centric.
> >
> > I feel and hear that the opposite is true.
> >
> > I responded to Andi when he first posted that.
> > http://stochasticresonance.wordpress.com/2010/03/26/devops-misnamed/
>
> Agreed. The DevOpsDays events I've attended the number of dev attendees
> were usually 10% or less. One of the key problems I see is that more
> engagement, education and exposure is needed in the dev side of the house.
>
> Regards
>
> James Turnbull

Yep. I believe where some of the confusions lies, with regard to the Dev-to-Ops slant, is the recent hububification of "infrastructure as code". To me this just means bringing dev practices (CI, SCM, etc.) to operational tasks - which is really, really good stuff. But, it definitely isn't bringing devs to the operational table for the required collaboration.

Kit

Guillaume FORTAINE

unread,
Jun 27, 2011, 9:38:50 AM6/27/11
to dev...@googlegroups.com
Dear Mister Shafer,

Thank you for your reply.

> I responded to Andi when he first posted that.
> http://stochasticresonance.wordpress.com/2010/03/26/devops-misnamed/

I have read with interest your blog post (and by the way, just bought
the book from James Turnbull "Pro Linux System Administration" that is
quoted in it :) ).

Clearly, I still don't understand what is the value-add of "DevOps"
with the current definition of its proponents and it seems the people
at LISA (Large Installation System Administration) think the same
thing :

http://everythingsysadmin.com/2010/10/devops-aint-nothing-we-havent.html

"Ok, now you know the background for what I really wanted to say. I
was inspired recently because a mailing list I'm on people were
mocking DevOps as "nothing new". The people on this list are all in
the top 10th percentile of system administrators; the cutting edge
that other IT teams hope to emulate."

"So congratulations, DevOps folks! I'm glad that you are popularizing
what some of us have been doing for years."


http://www.usenix.org/event/lisa11/cfp/

"While DevOps is new, it embodies themes long popular at LISA:
automation, performance, scaling, collaboration, and cooperation."


Maybe will I meet some of you at LISA '11 whose theme will be "DevOps:
New Challenges, Proven Values" ?

Best Regards,

Guillaume FORTAINE
gfor...@gfortaine.biz
+33(0)631.092.519

Damon Edwards

unread,
Jun 27, 2011, 10:15:50 AM6/27/11
to dev...@googlegroups.com
Guillame,

I'm not sure which "current definition of [DevOps] proponents" you are referring to, but I think people who sign up for this google group generally feel that there is value in building a community that discusses experiences and shares ideas for identifying and removing bottlenecks, inefficiencies, and conflicts that commonly plague IT organizations (especially those organizations who build and run revenue producing web-based services). 

Do you have a specific question or specific DevOps concept that you don't think is valuable? Quoting other people's old blog posts (especially when both of those people have evolved their DevOps opinions since then), doesn't explain where you are coming from or what you are getting at.


-Damon

John Vincent

unread,
Jun 27, 2011, 10:15:29 AM6/27/11
to dev...@googlegroups.com
> Clearly, I still don't understand what is the value-add of "DevOps"
> with the current definition of its proponents and it seems the people
> at LISA (Large Installation System Administration) think the same
> thing :
>
> http://everythingsysadmin.com/2010/10/devops-aint-nothing-we-havent.html
>

1) I wouldn't call a blog post by one person (even if it *IS* Tom L.)
representative of "the people at LISA". I've been a LISA attendee in
the past and I don't entirely agree with Tom's "old" article.
2) I don't know that everything has to have a value-add. DevOps isn't
something that can be bottled, labeled and sold as magic in a jar.

There are some things about DevOps that are new and some things that
are old. Countless articles and blog posts have been written about
different aspects of DevOps - cultural and technical and even some
that differ from traditional SA or Development work (devops isn't just
about SAs). Automation doesn't buy you immediate agility.
Cross-training teams doesn't buy you cooperation between those teams.

I'm honestly confused as to what value-add there needs to be.

--
John E. Vincent
http://about.me/lusis

gareth rushgrove

unread,
Jun 27, 2011, 10:19:56 AM6/27/11
to dev...@googlegroups.com
On 27 June 2011 14:38, Guillaume FORTAINE <gfor...@gfortaine.biz> wrote:
>
> Clearly, I still don't understand what is the value-add of "DevOps"
> with the current definition of its proponents and it seems the people
> at LISA (Large Installation System Administration) think the same
> thing :
>
> http://everythingsysadmin.com/2010/10/devops-aint-nothing-we-havent.html
>

I think one of the problems here is that the content you're pointing
to is pretty old. Your referenced articles are from March and October
last year. This might seem recent, but the whole coalescence around
the term devops is but 2 years old or so. Both articles referenced
also appear to be written from the point of view of someone who read
about it on the internet as well, rather than someone involved in the
active conversations either online or even better in person. Basically
it's the internets fault :)

The comments on both posts do what I think is a good job of adding to
the conversation the posts start though.

> "Ok, now you know the background for what I really wanted to say. I
> was inspired recently because a mailing list I'm on people were
> mocking DevOps as "nothing new". The people on this list are all in
> the top 10th percentile of system administrators; the cutting edge
> that other IT teams hope to emulate."
>

Here's something important though. A group of systems administrators
talking amongst themselves about how the stuff they are doing is
awesome. But does that list also have the top whatever percentile of
developers? A health scattering of testers? Because if not it's going
to be a one-sided conversation. I'm not saying that's bad or
unhelpful. I'm saying if you're looking for reasons why devops appears
popular it's incredibly inclusive nature is definitely part of it.

> "So congratulations, DevOps folks! I'm glad that you are popularizing
> what some of us have been doing for years."
>

And in any case the top few percent is one thing, popularizing
something to everyone else based on demonstrable benefits is hard
work. Even back in January Damon Edwards was talking of the DevOps
Echo Chamber and how to escape it.

http://www.agileweboperations.com/2011_escaping_devops_echo_chamber

Devops is really just a banner under which start people have been
discussing approaches to common problems and in many cases
popularizing them. Oh, and hacking on code (lots and lots of code. By
my calculation at least half the code on github is written by John
Vincent and Jordan Sissel). And talking to folks outside their
discipline, and in many cases finding common ground.

G

>
> http://www.usenix.org/event/lisa11/cfp/
>

So we move forward 9 months (who knows, maybe thanks to the comments)
from the above referenced blog post and LISA have an entire conference
under the devops banner.

> "While DevOps is new, it embodies themes long popular at LISA:
> automation, performance, scaling, collaboration, and cooperation."
>
>
> Maybe will I meet some of you at LISA '11 whose theme will be "DevOps:
> New Challenges, Proven Values" ?
>
> Best Regards,
>
> Guillaume FORTAINE
> gfor...@gfortaine.biz
> +33(0)631.092.519
>
>
> On Mon, Jun 27, 2011 at 3:13 AM, Andrew Clay Shafer
> <andrew...@gmail.com> wrote:
>>
>>>
>>> I agree with Andi that much of the writing about DevOps has come from
>>> the development perspective, but Ops people who can ditch the attitude
>>> of "[ops] manage what they are given, and keep the business running,
>>> regardless of the mess that gets thrown over the wall at them" will
>>> benefit greatly from DevOps. Andi is dead-on with "apps team have to
>>> pull their weight too, by addressing gaps ..."
>>>
>>
>> I would appreciate it if someone could point out the devops writing
>> that is dev centric.
>>
>> I feel and hear that the opposite is true.
>>
>> I responded to Andi when he first posted that.
>> http://stochasticresonance.wordpress.com/2010/03/26/devops-misnamed/
>>
>

--
Gareth Rushgrove
Web Geek

morethanseven.net
garethrushgrove.com

Ernest Mueller

unread,
Jun 27, 2011, 10:21:26 AM6/27/11
to dev...@googlegroups.com

dev...@googlegroups.com wrote on 06/26/2011 08:27:38 PM:



> From: James Turnbull <ja...@lovedthanlost.net>
> Andrew Clay Shafer wrote:
> >> I agree with Andi that much of the writing about DevOps has come from
> >> the development perspective, but Ops people who can ditch the attitude
> >> of "[ops] manage what they are given, and keep the business running,
> >> regardless of the mess that gets thrown over the wall at them" will
> >> benefit greatly from DevOps. Andi is dead-on with "apps team have to
> >> pull their weight too, by addressing gaps ..."

> > I would appreciate it if someone could point out the devops writing
> > that is dev centric.
> >
> > I feel and hear that the opposite is true.
> >
> > I responded to Andi when he first posted that.
> > http://stochasticresonance.wordpress.com/2010/03/26/devops-misnamed/
>
> Agreed. The DevOpsDays events I've attended the number of dev attendees
> were usually 10% or less. One of the key problems I see is that more
> engagement, education and exposure is needed in the dev side of the house.


Absolutely, this is a big red herring. Lack of dev participation in DevOps is one of the key things we need to work on, IMO. I have been to a lot of DevOps events and they have all been close to too Ops-focused and definitely predominantly attended by ops. Except for the dangerously deluded "NoOps" people, I don't know who is even claiming this.

Ernest
______________________
UN-altered REPRODUCTION and DISSEMINATION of
this IMPORTANT information is ENCOURAGED.

John Vincent

unread,
Jun 27, 2011, 10:28:46 AM6/27/11
to dev...@googlegroups.com
I want to slighty counter this argument about not enough dev
involvement. I've been meaning to bring this up but the amount of
developer involvement is, in my experience, pretty much regional in
nature. I've been to meeting where developers outweighed system admins
(in head count. I never see a developer built like me ;>) and I've
seen the reverse. I see quite a bit of systems code written by
developers to make their life easier.

Also Jordan Sissel has like 20 times more code on github than I do.
We're just from the same school of hate-driven development (his
phrase).

--

gareth rushgrove

unread,
Jun 27, 2011, 10:32:20 AM6/27/11
to dev...@googlegroups.com
On 27 June 2011 15:21, Ernest Mueller <ernest....@ni.com> wrote:
>>
>> Agreed. The DevOpsDays events I've attended the number of dev attendees
>> were usually 10% or less. One of the key problems I see is that more
>> engagement, education and exposure is needed in the dev side of the house.
>
> Absolutely, this is a big red herring. Lack of dev participation in DevOps
> is one of the key things we need to work on, IMO. I have been to a lot of
> DevOps events and they have all been close to too Ops-focused and definitely
> predominantly attended by ops. Except for the dangerously deluded "NoOps"
> people, I don't know who is even claiming this.
>

That's pretty interesting. I'd say most of the London events I've
organised have had a much higher developer attendance. 50:50 at times
even. I don't know if that's because I'm predominantly a developer and
have tended to organise more developer centric events or whether it's
just a London thing. The Hamburg devopsdays event as well certainly
wasn't only 10% developers.

Also given the typical topics up for discussion some developers can
impersonate ops folk pretty well in my experience.

I'd be interested in seeing if this is actually a trend. If anyone
else who's organising or attending events wants to try and gather some
data? I'll start asking at London events.

Gareth

> Ernest
> ______________________
> UN-altered REPRODUCTION and DISSEMINATION of
> this IMPORTANT information is ENCOURAGED.

--

Ernest Mueller

unread,
Jun 27, 2011, 10:33:32 AM6/27/11
to dev...@googlegroups.com

Yeah, my perspective is limited to Austin and California. Last week at DevOpsDays San Jose there was a hand poll taken and IIRC it was more than 2:1 ops to dev...



Ernest

______________________
UN-altered REPRODUCTION and DISSEMINATION of
this IMPORTANT information is ENCOURAGED.


Inactive hide details for gareth rushgrove ---06/27/2011 09:32:46 AM---On 27 June 2011 15:21, Ernest Mueller <ernest.mueller@nigareth rushgrove ---06/27/2011 09:32:46 AM---On 27 June 2011 15:21, Ernest Mueller <ernest....@ni.com> wrote: >>

Andrew Clay Shafer

unread,
Jun 27, 2011, 10:55:09 AM6/27/11
to devops

>
> Clearly, I still don't understand what is the value-add of "DevOps"
> with the current definition of its proponents and it seems the people
> at LISA (Large Installation System Administration) think the same
> thing :
>

First, everything is derivative. devops in general is nothing new per
se, but I see it as the confluence of ideas, technologies and
techniques coming together in specific ways that can be
transformative. IMHO most of the devops community and their ideas
represent an evolution of the ideas championed at LISA for years
flavored with more webops bias and less academic/hpc bias.

Second, Tom's post and the comments capture certain perspectives at a
point in time. That's not the same thing as 'the people at LISA'.
Tom's a reasonable guy with an impressive body of work. I'm sure if we
were all sitting around a table together with Tom now, we'd probably
agree on most points.

Third, I don't know what you are using as a definition of devops, but
I personally don't think these are solved problems. devops is not a
destination, at best it is the beginning of a journey. devops is a
community of practice which is growing the body of knowledge together.
To me devops as a cultural movement is about recognizing that there
are smart people out there who have been faced with similar problems
and learning what I can from them. I would hope people would do that
anyway. You can question the name, you can question the origin of the
practices, but ultimately that is a distraction from making real
progress. Unless you know everything already... :)

Kit Plummer

unread,
Jun 27, 2011, 11:10:53 AM6/27/11
to dev...@googlegroups.com
But John, there is a difference between SysAdmins who write code and developers who implement business functionality.

Kit

--
Kit Plummer :: MaestroDev :: 520.360.4729


On Monday, June 27, 2011 at 7:28 AM, John Vincent wrote:

> I want to slighty counter this argument about not enough dev
> involvement. I've been meaning to bring this up but the amount of
> developer involvement is, in my experience, pretty much regional in
> nature. I've been to meeting where developers outweighed system admins
> (in head count. I never see a developer built like me ;>) and I've
> seen the reverse. I see quite a bit of systems code written by
> developers to make their life easier.
>
> Also Jordan Sissel has like 20 times more code on github than I do.
> We're just from the same school of hate-driven development (his
> phrase).
>

> On Mon, Jun 27, 2011 at 10:21 AM, Ernest Mueller <ernest....@ni.com (mailto:ernest....@ni.com)> wrote:
> > dev...@googlegroups.com (mailto:dev...@googlegroups.com) wrote on 06/26/2011 08:27:38 PM:

James Turnbull

unread,
Jun 27, 2011, 11:21:33 AM6/27/11
to dev...@googlegroups.com
gareth rushgrove wrote:
> That's pretty interesting. I'd say most of the London events I've
> organised have had a much higher developer attendance. 50:50 at times
> even. I don't know if that's because I'm predominantly a developer and
> have tended to organise more developer centric events or whether it's
> just a London thing. The Hamburg devopsdays event as well certainly
> wasn't only 10% developers.
>

So the last two DevOpsDays in Mountain View and the first DevOpsDays
Downunder had pretty small dev turnouts and I know Patrick has commented
on similar at other events.

> Also given the typical topics up for discussion some developers can
> impersonate ops folk pretty well in my experience.

This is something we need to change too - we need to have more topics
(not packaging for example :) ) that are primarily about the Dev in
DevOps not primarily about the Ops in DevOps.

Cheers

James

Kit Plummer

unread,
Jun 27, 2011, 11:35:04 AM6/27/11
to dev...@googlegroups.com

Yep.  One topic (as a software engineer) I'd like to see more discussion about is the notion of "self-service environment" - meaning the ability of a developer/QA to rapidly instantiate a development machine(s) that mimics the operational environment (e.g. OS, VM, framework/library, application configurations across n-machines).

I really love what Patrick has discussed here, using Vagrant:

http://www.jedi.be/blog/2011/03/28/using-vagrant-as-a-team/

I think there is a ton of potential in Vagrant for helping bridge Dev and Ops.  Is there a similar connection that can be made for provisioning dev environments to IaaS providers (mcollective, knife, etc.)?

One thing that I consistently don't see in organizations I talk to is the documented delivery or deployment plan/workflow/pipeline/composition.  I believe in most cases, devs don't have much understanding of what happens to their code once committed.

Lastly, another area where I think there is lots of room for conversation is in code instrumentation - specifically in measuring operational lifecycles/transactions between distributed components/sub-systems (beyond just the front-end).

Great discussion guys!

Kit

Damon Edwards

unread,
Jun 27, 2011, 12:52:52 PM6/27/11
to dev...@googlegroups.com
For about a year now I've been doing the informal "raise your hand" poll (highly scientific). I've been seeing more like 60% "ops" 40% "dev" on average... I'm putting the roles in quotes because there are all kinds of circumstances where the lines are blurred and self-identification is always tricky.

But some interesting caveats...

1. There are a good number of exceptions where it is 90% ops / 10% dev... But I have never seen the reverse.

2. I feel like the majority of DevOps "content" producers (consistent bloggers, active on lists, creating projects, organizing events, etc.) are coming from a ops (or majority ops) background.

All that being said, I do believe that we need to do a better job of attacking DevOps problems from a Dev perspective (as Ben Rockwood recently called the Dev > Ops perspective). Lots of focus has gone into bringing best practices from Dev into Ops... And not nearly as much about how Dev roles/behavior/culture needs to adapt.

patrick.debois

unread,
Jun 27, 2011, 1:53:32 PM6/27/11
to dev...@googlegroups.com
My 2cents on this:

1. Population dev vs ops:

- my observation is that the attendees of a devops meetup/devopsdays
gravitate towards the background of the organizers. It's not strange
because the selection of the topics, friends, mouth-to-mouth advertising
will reflect their interests
- in the past bringing the topic to developer/agile conferences used to
be near to impossible, now I see devops being discussed in Ruby, Php
conferences, Xpdays, Agile 2011, GotoCon, Devoxx.
- this encourages me, I never set out to start a complete separate
circuit of conferences, but somehow it needed to get bootstrapped.
Actually I hope that devops would become a track of all dev/agile
conferences ; just like there are talks for frontend developers, backend
developers, database people, QA people, there is room for devops
oriented people
- A great way to get developers interested is actually to go present at
other conferences and make them aware, much like horse/water
- At Ops is currently at the end of the chain, they feel the pain more.
But the pain is there with Dev team too. But they can get away with
grumbling.

2. It's bringing more dev into ops (and vice versa)

- Making devs aware of operational issues is certainly something devs
can learn from ops
- Ops can clearly learn from devs when it comes to testing,
Agile/Lean/Kanban folks in development

2. Are we all devs , ops, or QA

My current view is that I would be looking for what some people call
T-shaped people . People how a deep technical expertise in either dev,
ops, qa and are *willing* to extend and collaborate. Working with only
generalists will slow down your learning curve and results (YMMV)

And the discussion is not only for Sec, QA, DBA, Windows. Hence I
started the devopsdays Goteborg about devops being 'inclusive'.

3. How to get others more involved:

- metrics, monitoring, logging are all great ways to provide feedback;
but it's not enough to just point them out to devs. They need to be
valuable, f.i. during their testing
- using them for retrospectives (not only inside ops) and sharing the
learning experience to the beginning of the project chain
- the same is for automation: devs care too about reproducibility;
that's why people love things like chef/puppet and vagrant. And from my
experience it takes them less time to learn as they already used to
programming

My guess is that f.i. if we find a way to integrate the use of all these
things in their current workflow/IDE there will a much wider adoption.
In the past the set of tools Ops and Devs had been working on had been
different , now they start to be able to use the same tools. Maybe they
are not managing 1000 servers and you might find their environment
negligible as opposed to your production environment, but I believe this
is where part of the cross over will start to happen.

- why not integrating stuff in their Eclipse/Favorite IDE and make them
accustomed to the tools ops is using
- managing their development desktop with the same tools
- allowing them to deploy in the same way as you would do in production
- give them the same metrics, monitoring for their dev environment
- provide them 'self-servicing' ways to explore new options
- expose logfiles, information they can use for problem solving
- help in getting them do *their work* and they will listen

Tools don't make a difference per s�, but they can change the interest
and behavior , eventually leading to culture change.

Most of all, pair, mingle, eat sushi, or whatever. Talk to them and
don't be set aback when they don't seem interested. Instead of thinking,
them vs us. Think of what *you* can do better in each-other's usual
world , to make things frictionless from both perspectives.

Patrick

Kief Morris

unread,
Jun 27, 2011, 2:51:30 PM6/27/11
to dev...@googlegroups.com, dev...@googlegroups.com
I wonder if part of what's going on here is that many of the people driving devops are a certain flavour of ops person, pulling development oriented practices into their work. This is why there's a perception that devops is dominated by ops people, and at the same time a perception among many ops people that devops is 'developer centric' practices being pushed in their direction.

Anthony Goddard

unread,
Jun 27, 2011, 3:08:25 PM6/27/11
to dev...@googlegroups.com
Hells yeah.

On Jun 27, 2011, at 1:53 PM, patrick.debois wrote:

Tools don't make a difference per sé, but they can change the interest and behavior , eventually leading to culture change.

John Vincent

unread,
Jun 27, 2011, 3:16:21 PM6/27/11
to dev...@googlegroups.com
That's an interesting perspective. Look at things traditionally.
Developers have always "deployed" code (whether locally or to their
own servers on side projects). Tools like capistrano and fabric are
developer initiated, not operations. It's not THAT big of a leap for
them to be brought into the production deploy arena.

Meanwhile Operations people are not typically developers in the bigger
sense of the word. Sure we've been python and perl junkies, shell
script monkies and the occasional tcl folk. Yeah we could write a PHP
frontend to stop having to do things for people ourselves. But agile
never factored into it for the most part. Perl *DOES* have a rich
history of testing behind it but we've always been able to say "I
don't do java" and boot it back to the developers.

I would argue sysadmins/operations teams are having to step out of the
traditional comfort zone a bit more than developers. In that sense
right now, the operations side of the house is undergoing a massive
cultural shift. I would wager in 5 years time (if not sooner)
operations folks will simply HAVE to know more than traditional
scripting languages. Yes, we have to be developers. It might not be
the same problem domain as the official development side of the house
but companies won't tolerate a sysadmin of any sufficient seniority
who isn't willing (or capable of learning) the new model. It's all
about getting the most for your money as a business.

And don't think that developers don't have to step up to the plate
either. Everyone is in the same boat. Comfort zones are changing and
you either keep up or get left behind.

On Mon, Jun 27, 2011 at 2:51 PM, Kief Morris <kiefd...@gmail.com> wrote:
> I wonder if part of what's going on here is that many of the people driving devops are a certain flavour of ops person, pulling development oriented practices into their work. This is why there's a perception that devops is dominated by ops people, and at the same time a perception among many ops people that devops is 'developer centric' practices being pushed in their direction.
>

--

Kit Plummer

unread,
Jun 27, 2011, 3:25:01 PM6/27/11
to dev...@googlegroups.com
Again, I'm feeling like the "kind of development" thang is getting lost.

Ops will likely never be the ones committing to implement a user story, feature, bug fix, or the like. So to say that Ops "have to be developers" is true, but must be caveated with "in the scope of QA/deployment/delivery/operations.

I also think there's a bit of scope required, from a complexity perspective. And...though I don't really want to...it does bring back NoOps to the conversation. There are always going to be "projects" where dev and ops are likely the same thing (read that as NoOps). PaaS environments are somewhat proving this to be true - ala Heroku making it easy to do so. But, in projects of any scale/complexity there is a need to separate roles and responsibilities. This doesn't mean that devs can't help ops write Puppet modules, or that Ops can't help a dev implement an async email notification feature. It just means that at the end of the day one side or the other will be responsible.

Kit

--
Kit Plummer :: MaestroDev :: 520.360.4729

Thom Allen

unread,
Jun 27, 2011, 3:30:00 PM6/27/11
to dev...@googlegroups.com
John,

I agree, and as a developer, I know I need to step up and have  broader understanding of the operations role. I was recently in a situation where I had to configure, setup and manage the dev/stage/production servers for a project. This did a couple of things for me. One, it reminded me how little I knew about the OS. I can't get away with just knowing how to move and copy files. I had no idea about best practices for resource management, security and monitoring. This exercise also convinced me that as a developer, I need to know more about the domain my applications are playing in rather than relying on operations. This is partially why the DevOps movement caught my eye.

--Thom

John Vincent

unread,
Jun 27, 2011, 3:36:31 PM6/27/11
to dev...@googlegroups.com
On Mon, Jun 27, 2011 at 3:25 PM, Kit Plummer <kplu...@maestrodev.com> wrote:
> Again, I'm feeling like the "kind of development" thang is getting lost.
>
> Ops will likely never be the ones committing to implement a user story, feature, bug fix, or the like. So to say that Ops "have to be developers" is true, but must be caveated with "in the scope of QA/deployment/delivery/operations.
>

I'd probably disagree here. If we take the John Willis perspective,
right now operations teams are spending 80% of the time doing bullshit
firefighting muckity muck type work. The other 20% is providing value
to the business. As we start to automate more and the feedback loop
becomes smaller, operations folks will have more time on their hands.
More time to add value to the business. True they might not dive right
in and start implementing new features but they will.

I know one example does not a trend make but the company I was
previously at had almost this exact thing start happening. We had a
senior operations guy who took it upon himself to learn java for a
couple of reasons. One was to make the codebase more understanding of
the environment and all the automation and tools created by
operations. The other was to deal with bugs that never seemed to
register on the radar because there were acceptable work arounds. (Yes
there was a greater problem on the second one).

My point is that, while right now I'm writing glue apps (like the
mentioned email gateway or a sinatra web service that does X), I'm
just as much a part of the development team as anyone else. I just
don't write any java code.

I think we need to be careful that we don't mix title with role.
Titles are typically fixed but roles change based on need. One day I
was a websphere administrator writing jacl scripts for a deploy
console. The next day I was knee deep dealing with DB2 bufferpool hit
ratios. Yet another day I'm standing up a new firewall.

We don't seem to have this roadblock when it comes to operations folks
doing networking or security work. Why then, do we have this mental
roadblock about operations folks doing development work? It's not as
if we're incapable.

P.S. Love you long time, Kit ;)

Thom Allen

unread,
Jun 27, 2011, 3:37:13 PM6/27/11
to dev...@googlegroups.com
Kit,

I don't think I would expect Ops to have the depth of experience a seasoned developer has. But I know from experience, it would be nice if there was more than a casual understanding. And as a developer, I know I need more than a casual understanding of the environment I will be working in. For example, you mention PaaS environments. Over the past few weeks I have failed to deploy applications to a cloud based infrastructure because I didn't understand the basic configuration needs, and I didn't have an expert counterpart to help me. That's a failure on my part to think I didn't need any help.

--Thom

Kit Plummer

unread,
Jun 27, 2011, 3:45:40 PM6/27/11
to dev...@googlegroups.com
No worries John. Don't get me wrong, I want Ops to better understand my environment - and I love working with you guys. ;)

Thom, yep been there...but, that's you and me. I would never expect my junior guys - who are doing the core "business-value-development" stuff to have to worry about that. Back to my scope point. Not everyone on-the-chain should/needs to be collaborating. I suppose some of this is me thinking as a "Systems" guy too...and not just a developer.

Software architecture, which is mainly influenced by more senior dev and ops folks, is where a lot of the DevOps collaboration should happen. Throwing this out there - with a bit of instigation intended.

Kit

--
Kit Plummer :: MaestroDev :: 520.360.4729


On Monday, June 27, 2011 at 12:37 PM, Thom Allen wrote:

> Kit,
>
> I don't think I would expect Ops to have the depth of experience a seasoned developer has. But I know from experience, it would be nice if there was more than a casual understanding. And as a developer, I know I need more than a casual understanding of the environment I will be working in. For example, you mention PaaS environments. Over the past few weeks I have failed to deploy applications to a cloud based infrastructure because I didn't understand the basic configuration needs, and I didn't have an expert counterpart to help me. That's a failure on my part to think I didn't need any help.
>
> --Thom
>
>
> On Mon, Jun 27, 2011 at 1:25 PM, Kit Plummer <kplu...@maestrodev.com (mailto:kplu...@maestrodev.com)> wrote:
> > Again, I'm feeling like the "kind of development" thang is getting lost.
> >
> > Ops will likely never be the ones committing to implement a user story, feature, bug fix, or the like. So to say that Ops "have to be developers" is true, but must be caveated with "in the scope of QA/deployment/delivery/operations.
> >
> > I also think there's a bit of scope required, from a complexity perspective. And...though I don't really want to...it does bring back NoOps to the conversation. There are always going to be "projects" where dev and ops are likely the same thing (read that as NoOps). PaaS environments are somewhat proving this to be true - ala Heroku making it easy to do so. But, in projects of any scale/complexity there is a need to separate roles and responsibilities. This doesn't mean that devs can't help ops write Puppet modules, or that Ops can't help a dev implement an async email notification feature. It just means that at the end of the day one side or the other will be responsible.
> >
> > Kit
> >
> > --

> > Kit Plummer :: MaestroDev :: 520.360.4729 (tel:520.360.4729)

Nathaniel Eliot

unread,
Jun 27, 2011, 5:33:44 PM6/27/11
to dev...@googlegroups.com
At the risk of counter-instigation:

*Everybody* should be trained in collaboration, and at minimum should understand the forces their counterparts deal with. I don't expect programmers to know the ins and outs of Linux, any more than they expect me to be an amazing Ruby program. But understanding what their pain points are ASAP, and understanding how they can help me in my job, has made me a much better op than I otherwise might be. And some of the best developers to work with are those who understand what deployments and being on call mean for an admin (and why we're allergic to "just a small change" to production machines at 4:45PM on a Friday).

Far better to have all your people experienced in collaboration from day one, than to make it a rarefied skill that is only practiced at "the top". Otherwise, where are you going to pull from when the need for expert collaborators arise? How well your company react when the worst happens, and it's just three juniors in different departments who are available when a fire breaks out?

--
Nathaniel Eliot
T9 Productions

Lindsay Holmwood

unread,
Jun 27, 2011, 7:19:29 PM6/27/11
to dev...@googlegroups.com
On 28 June 2011 01:21, James Turnbull <ja...@lovedthanlost.net> wrote:
>
> So the last two DevOpsDays in Mountain View and the first DevOpsDays
> Downunder had pretty small dev turnouts and I know Patrick has commented on
> similar at other events.
>

That's not entirely accurate.

I did a snap poll on the first day of DevOps Down Under and there was
a 50/50 dev/ops split.

I did a lot of work to encourage devs to attend, but the talk content
was skewed towards ops, so there was a lot of attrition and by the
second day it had worn down to ~ 20/80 dev/ops.

It's an interesting hidden bias - we talk a lot about attracting devs,
but sometimes our conversations are too ops focused and end up driving
devs away.

Lindsay


--
w: http://holmwood.id.au/~lindsay/
t: @auxesis

gareth rushgrove

unread,
Jun 27, 2011, 7:33:03 PM6/27/11
to dev...@googlegroups.com
On 27 June 2011 16:21, James Turnbull <ja...@lovedthanlost.net> wrote:
>
>> Also given the typical topics up for discussion some developers can
>> impersonate ops folk pretty well in my experience.
>
> This is something we need to change too - we need to have more topics (not
> packaging for example :) ) that are primarily about the Dev in DevOps not
> primarily about the Ops in DevOps.
>

I'm basically the worst possible example here. I'm a developer who
gets worked up about packaging and ganglia. So what topics do people
think appeal to developers? I'm asking squarely with my organising
london events hat on.

G

> Cheers
>
> James

gareth rushgrove

unread,
Jun 27, 2011, 7:43:57 PM6/27/11
to dev...@googlegroups.com
On 27 June 2011 16:35, Kit Plummer <kplu...@maestrodev.com> wrote:
>
> Yep.  One topic (as a software engineer) I'd like to see more discussion
> about is the notion of "self-service environment" - meaning the ability of a
> developer/QA to rapidly instantiate a development machine(s) that mimics the
> operational environment (e.g. OS, VM, framework/library, application
> configurations across n-machines).
>
> I really love what Patrick has discussed here, using Vagrant:
>
> http://www.jedi.be/blog/2011/03/28/using-vagrant-as-a-team/
>

Does the world eventually become one of:

* developer provisioned vagrant(like) instances and then the same
chef/puppet/whatever recipes/manifests/whatevers use to provision
EC2/OpenStack instances
* local and remote running platform as a service, eg. Cloudfoundry.
Potentially running as above.

?

> One thing that I consistently don't see in organizations I talk to is the
> documented delivery or deployment plan/workflow/pipeline/composition.  I
> believe in most cases, devs don't have much understanding of what happens to
> their code once committed.
>

It's nearly alway documented in code. But that particular part of the
codebase isn't always shared as widely as it should be. From a
developer point of view it's also not always treated like the rest of
the code with tests and the like. Or developers try and take over with
a badly hand rolled version of rsync or the like.

G

> Lastly, another area where I think there is lots of room for conversation is
> in code instrumentation - specifically in measuring operational
> lifecycles/transactions between distributed components/sub-systems (beyond
> just the front-end).
>

> Great discussion guys!
>
> Kit
>
>

--

gareth rushgrove

unread,
Jun 27, 2011, 7:57:31 PM6/27/11
to dev...@googlegroups.com
On 27 June 2011 20:25, Kit Plummer <kplu...@maestrodev.com> wrote:
>
> I also think there's a bit of scope required, from a complexity perspective.
> And...though I don't really want to...it does bring back NoOps to the conversation.
> There are always going to be "projects" where dev and ops are likely the same thing
> (read that as NoOps). PaaS environments are somewhat proving this to be true - ala
> Heroku making it easy to do so.

I always think about Heroku as more outsourcing operations. Running a
PAAS internally is interesting though in that it introduces a hard API
in-between dev and ops, something your average devops folk are
traditionally against. I've never been sure whether NoOps really means
no operations people inside your organisation, or some mythical fairy
land where software and hardware always work 100% of the time.

G

--

Guillaume FORTAINE

unread,
Jun 27, 2011, 9:26:45 PM6/27/11
to dev...@googlegroups.com
Hello,


@ Damon Edwards

> where you are coming from

I come from Lille :) And according to your Linkedin's profile, you
come from San Francisco :) But this question is nonsense given the
Globalization of Information Systems ;)

> what you are getting at.

I am interested in approaching Operational Excellence. Hence, my
interest in DevOps that could be a mean to help me in this is never
ended quest.

> 1. There are a good number of exceptions where it is 90% ops / 10% dev... But I have never seen the reverse.
>
> 2. I feel like the majority of DevOps "content" producers (consistent bloggers, active on lists, creating projects, organizing events, etc.) are coming from a ops (or majority ops) background.
>

I don't know if, we, French people, are contrarian but the last major
DevOps events in France were organized by regional JUGs (Java User
Group). 4 to be more precise (Breizh JUG, Ch'ti JUG, Paris JUG, Mars
JUG) :

http://parisdevops.fr/blog/2011/06/08/devops-debarque-chez-vous.html


@ John Vincent

> 1) I wouldn't call a blog post by one person (even if it *IS* Tom L.)

> representative of "the people at LISA".

One can reasonably call representative the point of view of the
program co-chair of LISA. Though, I agree with Andrew Shafer that, to
quote him, "Tom Limoncelli's post and the comments capture certain


perspectives at a point in time."

> 2) I don't know that everything has to have a value-add.

Everything needs to have value-add. One can't only have as leitmotiv :
"Let's people be happy (but give them a bunch of money)". It doesn't
work like this ;)

According to my understanding, there is a clear value-add in DevOps in
helping to solve the question : How ensure a match between the
Information System and the business objectives ?


@ gareth rushgrove

> So we move forward 9 months (who knows, maybe thanks to the comments)
> from the above referenced blog post and LISA have an entire conference
> under the devops banner.

Maybe will we be able to drink a beer together there to discuss further ? :)


@ Andrew Shafer

> Third, I don't know what you are using as a definition of devops,

"Let's people be happy (but give them a bunch of money)".

> Unless you know everything already... :)

No one can know everything :) But by aggregating the knowledge of
everybody, you could, at least, have a better overview ;) Hence, my
DevOpSpace project :

http://www.linkedin.com/company/devopspace


@ Patrick Debois

Thank you for your feedback. Question : Why didn't you manage to
successfully promote DevOps in Europe ?

I look forward to your answer,

Scott Smith

unread,
Jun 28, 2011, 12:28:36 AM6/28/11
to dev...@googlegroups.com
In my experience, the split is more about personnel than time. In larger groups, anyway. Like it's 80% of the people are doing bullshit firefighting and 20% are doing interesting work. They both work out to be the same, but yeah.

Regardless, I think a lot of this is due to the fact that the "long tail," if you will, of ops people really don't seem to give a shit about the work beyond what's been put in front of their face from 9-6, or whenever they're oncall.

-scott

James Bailey

unread,
Jun 28, 2011, 3:15:38 AM6/28/11
to dev...@googlegroups.com
On 28 June 2011 05:28, Scott Smith <sc...@ohlol.net> wrote:
> In my experience, the split is more about personnel than time. In larger
> groups, anyway. Like it's 80% of the people are doing bullshit firefighting
> and 20% are doing interesting work. They both work out to be the same, but
> yeah.
> Regardless, I think a lot of this is due to the fact that the "long tail,"
> if you will, of ops people really don't seem to give a shit about the work
> beyond what's been put in front of their face from 9-6, or whenever they're
> oncall.

There is coming a time when the long tail won't have a chance to
adapt. 100 years ago we had 1,000s of people working in cotton mills
now those jobs are gone replaced by automated milling machines. 50
years ago we had 1,000 of people building cars those jobs are now now
replaced with robotic assembly lines. Currently we have 1,000s of
people administering systems do we seriously expect those escape the
laws of progress.

Jim :)

Patrick Debois

unread,
Jun 28, 2011, 5:31:03 AM6/28/11
to dev...@googlegroups.com

@ Patrick Debois

Thank you for your feedback. Question : Why didn't you manage to
successfully promote DevOps in Europe ?

LOL! Define successful :)

I'd love to play the perfection game with you/others on this. The perfection game is a simple format to give positive feedback, which is not as simple as it sounds. This “protocol” was devised by Jim & Michele McCarthy in their “Software for your Head” book.
  • Give a score out of 10. The number itself doesn’t have a lot of meaning, we use the number to see if we get closer to perfection (10/10) in each iteration.
  • What we did to earn those points. It’s important to let the proposer know this, so that they keep or reinforce their strong points.
  • What we should do to earn 10/10. This has to be clear, actionable advice. This is the difficult part. When they first start using the perfection game, people usually “sugarcoat” negative feedback to make it look like positive advice. E.g. “Your proposal would be perfect if it wasn’t like this“. What can I do with such feedback? Nothing much.

Looking forward to your feedback,

Patrick


Guillaume FORTAINE

unread,
Jun 28, 2011, 8:32:07 AM6/28/11
to dev...@googlegroups.com
Dear Mister Debois,

Thank you for your reply.

> LOL! Define successful :)

To make a living thanks to this new field that you've introduced :)

> Give a score out of 10. The number itself doesn’t have a lot of meaning, we
> use the number to see if we get closer to perfection (10/10) in each
> iteration.

I would give a score of 5. The middle because good progress has been
done (2 years ago, the devops word didn't exist , now there are nearly
1700 members on the Linkedin group :
http://www.linkedin.com/groups?home=&gid=2825397 ), but one can go
further.

> What we did to earn those points. It’s important to let the proposer know
> this, so that they keep or reinforce their strong points.

To bring in more executive profiles into the party. I would say that
10% of the attendance of a devopsday should include managers from
corporations. So each time this number is reached, a point is earned
:) Without key decisions makers, trying to inject a new Culture inside
an enterprise is nonsense.

> What we should do to earn 10/10. This has to be clear, actionable advice.
> This is the difficult part. When they first start using the perfection game,
> people usually “sugarcoat” negative feedback to make it look like positive
> advice. E.g. “Your proposal would be perfect if it wasn’t like this“. What
> can I do with such feedback? Nothing much.

Given that more than 60% of the enterprises are using ITIL, DevOps
should clearly be lean enough to not fear words like "Agile ITIL" :

http://blogs.rpath.com/wpmu/closing-the-gap/2010/04/08/is-agile-itil-an-oxymoron/

instead of saying :

http://stochasticresonance.wordpress.com/2010/03/26/devops-misnamed/

"I’m not going to really get into what I think about most of ITIL"

So my conclusion is that by making ITIL Agile, DevOps could bring in
more top executives into the party thus significantly expanding to
finally be able to reach a 10/10 :)

Best Regards,

Guillaume FORTAINE
gfor...@gfortaine.biz
+33(0)631.092.519

Ernest Mueller

unread,
Jun 28, 2011, 9:01:23 AM6/28/11
to dev...@googlegroups.com

dev...@googlegroups.com wrote on 06/28/2011 07:32:07 AM:
> Given that more than 60% of the enterprises are using ITIL, DevOps
> should clearly be lean enough to not fear words like "Agile ITIL" :
>
> http://blogs.rpath.com/wpmu/closing-the-gap/2010/04/08/is-agile-
> itil-an-oxymoron/
>
> instead of saying :
>
> http://stochasticresonance.wordpress.com/2010/03/26/devops-misnamed/
>
> "I’m not going to really get into what I think about most of ITIL"
>
> So my conclusion is that by making ITIL Agile, DevOps could bring in
> more top executives into the party thus significantly expanding to
> finally be able to reach a 10/10 :)


That's like making a pig more kissable by putting lipstick on it.

You could make the same argument that agile should have been marketed as "faster waterfall!" My, that would be nice and comforting to the existing power structure.

But what this does is effectively dilute and betray the message. DevOps isn't ITIL. If someone is allowed to think of it as ITIL, they will do stupid things when they implement it.

This is, however,a helpful data point to us, that we need to work on defining a good coherent DevOps message in the service of "escaping the echo chamber," because the longer we don't, the more confused outsiders, analysts, etc. will decide on their own bastardized interpretations of what it is and spread disinformation that will be harmful to the effort in the long run.

Anthony Goddard

unread,
Jun 28, 2011, 9:58:49 AM6/28/11
to dev...@googlegroups.com
Maybe I'm being naive, but I wonder how many folk share my same story..

Ops was fun, but hanging with devs all the time and tinkering with dev made me want to transition from ops to dev. I already used a bunch of "devops" practices but nothing like the well thought out and sensible practices that Devops encouraged. None of my ops colleagues had any interest in writing software to do anything (a few bash / perl scripts maybe) and I was a lonely sysadmin in a world of exciting ruby developers (cue the violin here).

Then devops came along and suddenly I realized I could have my cake and eat it too. I could keep doing ops and keep writing code, they weren't mutually exclusive. To be honest, the most valuable thing in the beginning was being able to search the #devops hashtag. That lead me to other like-minded folk (+ colleagues who did have an interest in writing software) and eventually to things like devopsdays where I met some awesome people, learned some new tricks and became better at what I do. I'm now a very happy ops guy who does a healthy amount of dev and I absolutely have a refreshing & renewed interest in the future of my field.

(cue the dramatic, skeptical, suspenseful music here)
When selling the idea of devops, I've found management at all levels to be supporting and encouraging. All I've seen from 'enterprise' folk - whether they're vendors or senior managers from other organizations has been them trying to misuse the term, sell me more crap I don't need and add "devops" to their ever-expanding list of buzzwords they don't understand.

(mute the soundtrack)
I guess everyone has a different definition and I can only speak from experience - from where I stand, the introduction of "Devops" into our field has been 100% successful. 10/10.


:)
Ant

James Bailey

unread,
Jun 28, 2011, 9:59:34 AM6/28/11
to dev...@googlegroups.com
On 28 June 2011 14:01, Ernest Mueller <ernest....@ni.com> wrote:
> dev...@googlegroups.com wrote on 06/28/2011 07:32:07 AM:
>> Given that more than 60% of the enterprises are using ITIL, DevOps
>> should clearly be lean enough to not fear words like "Agile ITIL" :
>>
>> http://blogs.rpath.com/wpmu/closing-the-gap/2010/04/08/is-agile-
>> itil-an-oxymoron/
>>
>> instead of saying :
>>
>> http://stochasticresonance.wordpress.com/2010/03/26/devops-misnamed/
>>
>> "I’m not going to really get into what I think about most of ITIL"
>>
>> So my conclusion is that by making ITIL Agile, DevOps could bring in
>> more top executives into the party thus significantly expanding to
>> finally be able to reach a 10/10 :)
>
> That's like making a pig more kissable by putting lipstick on it.
>
> You could make the same argument that agile should have been marketed as
> "faster waterfall!" My, that would be nice and comforting to the existing
> power structure.
>
> But what this does is effectively dilute and betray the message. DevOps
> isn't ITIL. If someone is allowed to think of it as ITIL, they will do
> stupid things when they implement it.

I do believe that DevOps can be implemented within an ITIL framework
and I do think that the two can be supportive rather than
antagonistic. The problem is most managers charged with implementing
ITIL see ITIL as the alpha and omega of IT practice and talking DevOps
as a challenge to their authority and methodology. A similar problem
exists within project management using Prince2 when the the PRojects
IN Controlled Environments of the Prince2 framework becomes
inseparable from the Waterfall development process. There lies the
problem the confusion of framework and process.


>
> This is, however,a helpful data point to us, that we need to work on
> defining a good coherent DevOps message in the service of "escaping the echo
> chamber," because the longer we don't, the more confused outsiders,
> analysts, etc. will decide on their own bastardized interpretations of what
> it is and spread disinformation that will be harmful to the effort in the
> long run.

I think DevOps is a limiting term, there is a need to bring the whole
of the business into the process. Limiting this brave new world to the
interaction of Dev and Ops is ultimately going to fail in the
5,000-15,000 employee organisations I contract in, they are not
technology companies, they are not even tech literate, they are not
innovative, they are above all conservative and the power for change
resides in the equally conservative middle manager. For all their
talk of the importance of the employee and "investing in people" they
are deeply mistrustful at best of new ideas coming from the ops/dev
team. Usually they just nod politely to our ideas and ignore them.

ITIL was around for 10-15 years before it achieved mass adoption by
this kind of organisation. DevOps has been around what 2-3, though I
do realise many of us have been quietly and stealthy working this way
for many years.

For me the challenge of Devops is not the practice but finding a way
of conversing with management to make the DevOps process fit within
the ITIL and Project management frameworks.

Someone said that DevOps was about people, processes and tools in that
order. However it seems that most places I work the people that matter
are not ready for that conversation.

Peace Jim :)

John Vincent

unread,
Jun 28, 2011, 10:34:34 AM6/28/11
to dev...@googlegroups.com
Guillame,

I apologize if this comes off with the wrong tone but I think you're
sadly mistaken. No one (that I'm aware of outside of you?) is saying


"Let's people be happy (but give them a bunch of money)".

The money to be made in "selling" DevOps is not institutional (in that
it's a service sold by the likes of IBM or CA). Understand that DevOps
is a movement started by practitioners and not salesmen.

This community of practitioners is (in my personal opinion) one of
meritocracy. We write code, we share ideas and we promote. If one
feels they must assign some sort of tally to the impact of devops in a
business then it's sad to say they'll probably be very frustrated. I
said this in a previous blog post but I'll gladly say it again. The
people you see involved in this community are the current and future
executives that you are saying we should court. Think on that for a
moment. We don't need to be "sold" on DevOps because it's what we know
and do. After 17ish years in this industry, I'll not put up with
things like an inability to automate, lack of meaningful metrics or
backbitting and buck passing between my teams any longer. The business
value is perfectly clear to me. We make more money because we've
stopped pissing around with inconsequential minutia.

I'm very sorry if you expected something else.

--

Sean OMeara

unread,
Jun 28, 2011, 10:47:59 AM6/28/11
to dev...@googlegroups.com
+1 lusis

Will Thames

unread,
Jun 28, 2011, 11:07:45 AM6/28/11
to dev...@googlegroups.com
As with Sean, I agree with John 100% but would also add that there's
plenty of money to be made through devops - by allowing our businesses
to deliver improvements to their products more quickly and efficiently
then the business becomes more efficient with a tighter feedback loop.
As Jez mentioned earlier, continuous delivery and devops should not be
confused, both of these are separate and complementary ways to improve
efficiency - waiting until the end of the development cycle to say
'I'm not releasing this because it doesn't meet this requirement I
forgot to tell you about' is nowhere near as good as 'let's discuss
the user stories we'll deliver in this sprint'.

Plenty of brilliant organisations have been doing devops for years.
The devops focus is allowing the rest of us to find out what the best
organisations are doing and bring some of those best practices into
our own businesses!

Will

Will Thames

unread,
Jun 28, 2011, 12:00:41 PM6/28/11
to dev...@googlegroups.com
Interesting blogpost from appdynamics includes a great quote from John Willis:

"DevOps is about making your business more agile — it’s about going
from ‘Ah-Ha’ to ‘Cha-Ching’ as fast as possible."

http://www.appdynamics.com/blog/2011/06/28/making-the-business-more-agile/

Will

Ernest Mueller

unread,
Jun 28, 2011, 12:54:46 PM6/28/11
to dev...@googlegroups.com, dev...@googlegroups.com
+1, well said.

E

Sent from my iPhone

Damon Edwards

unread,
Jun 28, 2011, 1:01:47 PM6/28/11
to dev...@googlegroups.com
If anyone needs to make the argument to their boss as to why it's worth investing time and money in DevOps problems, I wrote this post about the topic:

It's an older post but I still use the points and the graphics (the "ah-ha" to "ka-ching" graphics are in there) all the time. Making a business more agile (i.e. being able to react quicker to external forces or internal ideas) is a fundamental goal that no MBA is going to disagree with. It's up there with "make more with less" in terms of fool proof arguments. The only trick is that you now have to convince them that 1) you can do it 2) you can do enough of it, quick enough.

It's also got a graphic that shows DevOps in relation to ITIL (building on one of the more common ITIL v3 graphics)

Nikolay Sturm

unread,
Jun 28, 2011, 2:20:02 PM6/28/11
to dev...@googlegroups.com
* Ernest Mueller [2011-06-28]:

> This is, however,a helpful data point to us, that we need to work on
> defining a good coherent DevOps message in the service of "escaping
> the echo chamber," because the longer we don't, the more confused
> outsiders, analysts, etc. will decide on their own bastardized
> interpretations of what it is and spread disinformation that will be
> harmful to the effort in the long run.

A big problem I see whith the current state of devops is that it seems
to be like a shell which people fill with different contents, depending
on their background and needs (infrastructure as code, kanban,
dev-op-whatever cooperation, you name it).

If I compare this with agile software development, I feel agile being
much more like a plant. It has a core (pit?), which consists of a few
principles for software development, and from that grows a large body of
ideas and practices.

DevOpsDays Europe will be about inclusion, I wish for it to also be
about exclusion. Where are the limits of devops? What do we want to
concern ourself with and what is outside the bounds of devops? What is
the essential core?

If devops becomes anything for anyone, it will basically be nothing.

cheers,

Nikolay

--
"It's all part of my Can't-Do approach to life." Wally

Damon Edwards

unread,
Jun 28, 2011, 4:18:16 PM6/28/11
to dev...@googlegroups.com
When you think about DevOps as a problem statement (or a problem
umbrella) it's actually pretty easy to gain consensus.... Something
like "DevOps problems are the bottlenecks, inefficiencies, and
conflicts that plague the development to operations lifecycle and, in
turn, plague the business it supports".

When people don't like that and demand a "DevOps = X" statement... I
just fold in Adam Jacob's statement: "DevOps is a cultural and
professional movement". They say "movement to do what?". And I go
right back to "solve the bottlenecks, inefficiencies, and conflicts
that plague the...."

It's pretty easy to get a general consensus around the idea that:
1) DevOps is a cultural and professional movement and that is it
2) a general statement of the problem types that the movement wants to
solve (there sort of already is this consensus in the loosest way)

I'm not really sure why people get fixated on having DevOps be some
prescriptive methodology to be followed in only one way. I don't see
what the movement gains from that and I think it is going to lead to
endless infighting, naval gazing, and co-option.

Look at how well that worked out for "The Cloud". Countless hours have
been spent fighting about "what is a Cloud?". Who gives a shit. It's
nice we have a lightening rod to bring people together for a bit, now
let's get on with solving real problems. By the way, endless
infighting, naval gazing, and co-option is also pretty descriptive of
the state of the Agile movement.

If there was a general consensus that DevOps is a cultural and
professional movement and then some sort of general problem statement
and we focus our efforts on holding that, then who cares what vendors
or other people do? Anyone can claim they have the answers... and
maybe they do or maybe they don't. But either way they are just part
of the conversation and aren't hurting anything. Everyone learns,
Everyone wins.

If our goal is as altruistic as we say it is, being exclusive or
demanding that there is only One True DevOps Way™ is totally
counterproductive.

Jez Humble

unread,
Jun 28, 2011, 5:45:54 PM6/28/11
to dev...@googlegroups.com, dev...@googlegroups.com
Showing how ITIL and devops can work together is one of my goals for the next year or two. I have an article in cutter in a couple of months discussing this. Also check out Gene Kim's work. Here is one of my blog entries on this topic: http://continuousdelivery.com/2010/11/continuous-delivery-and-itil-change-management/

Sent from my iPhone

Matthew Jones

unread,
Jun 28, 2011, 6:27:23 PM6/28/11
to dev...@googlegroups.com
In my experience, devops to one company is completely different to devops in another. The deciding factor for success... culture.
I've seen it work brilliantly in small and agile software development firms, but not so much in universities and other larger institutions.

Devops works in organisations because the people on the ground want it to work.

Jez Humble

unread,
Jun 28, 2011, 6:44:02 PM6/28/11
to dev...@googlegroups.com, dev...@googlegroups.com
Most of my recent experience is with large orgs. In my experience, it's just not as simple as 

Sent from my iPhone

Jez Humble

unread,
Jun 28, 2011, 6:50:57 PM6/28/11
to Jez Humble, dev...@googlegroups.com
Balls

As I was saying, the people on the ground wanting it to work is necessary but not sufficient. Furthermore, it's possible to win people round (i've seen it happen). Culture can be changed in large orgs. Part of making that happen involves convincing senior people not to block it because of perfectly reasonable concerns around governance and compliance. Since that's the pain I'm experiencing now, it seems reasonable to attack it so that the bottoms up approach can actually work.

Viva bottoms up!

Sent from my iPhone

Guillaume FORTAINE

unread,
Jun 28, 2011, 7:57:20 PM6/28/11
to dev...@googlegroups.com
Hello,

@ Ernest Mueller

> This is, however,a helpful data point to us, that we need to work on
> defining a good coherent DevOps message in the service of "escaping the echo
> chamber," because the longer we don't, the more confused outsiders,
> analysts, etc. will decide on their own bastardized interpretations of what
> it is and spread disinformation that will be harmful to the effort in the
> long run.

+1, well said.


@ John Vincent

> I apologize if this comes off with the wrong tone but I think you're
> sadly mistaken. No one (that I'm aware of outside of you?) is saying

> "Let's people be happy (but give them a bunch of money)".

http://www.agileweboperations.com/2011_escaping_devops_echo_chamber

"All too often, DevOps conversations focus on our personal complaints
rather than actual business problems. Just because we find something a
headache or unpleasant doesn’t necessarily mean that our perceived
problem matters to the business. To put it bluntly, we are paid to
have those headaches. Happiness would be nice, but it’s not much of a
business justification. DevOps needs a different argument if we want
to be heard."


@ James Bailey

> For me the challenge of Devops is not the practice but finding a way
> of conversing with management to make the DevOps process fit within
> the ITIL and Project management frameworks.

Too much widsom in just a single email.


@ Jez Humble

> Showing how ITIL and devops can work together is one of my goals for the next year or two. I have an article in cutter in a couple of months discussing this. Also check out Gene Kim's work. Here is one of my blog entries on this topic: http://continuousdelivery.com/2010/11/continuous-delivery-and-itil-change-management/
>

Thank you for this great post. Especially, I would greatly appreciate
to invite you to a further reading of the one entitled "Devops: It is
Not About ITIL, It is About Proficiency" :

http://theagileexecutive.com/2010/06/29/devops-it-is-not-about-itil-it-is-about-proficiency/

"TIL should really be thought of as a control model and a common
language for process integration.

If your ITIL process are bureaucratic and heavyweight, they’ll
restrict or choke continiuous deployments. If they are lightweight and
modeled for agility, they’ll faciliate the desired control even when
revved to high velocity."


By the way, I have just bought your book "Continuous Delivery".

Otherwise, you are more than welcome to Share on the document that I
have created on DevOpSpace Documents and entitled "Managing a
Planet-Scale Information System, the DevOps way" :

https://docs.google.com/document/d/19iyHH0YSpmvP4lANA9fI8j2Q72CK46VWb8se8TwUqq0/edit?hl=fr&authkey=COHJy9gD


Best Regards,

Guillaume FORTAINE
gfor...@gfortaine.biz
+33(0)631.092.519

Philip J. Hollenback

unread,
Jun 29, 2011, 12:00:47 AM6/29/11
to dev...@googlegroups.com
On Wed, 29 Jun 2011 08:27 +1000, "Matthew Jones" <ma...@geekle.id.au>
wrote:

> In my experience, devops to one company is completely different to
> devops in another. The deciding factor for success... culture. I've
> seen it work brilliantly in small and agile software development
> firms, but not so much in universities and other larger institutions.
>
> Devops works in organisations because the people on the ground want it
> to work.

The issue of implementing devops in a major organization is a tough one.
I wrote a
blog post about this very subject a few month back:

http://www.hollenback.net/index.php/DevOpsBlocker

I work in a very large internet company, and the inertia is simply
astounding. It's extremely hard to implement devops-compatible changes.

I mainly try to work on the communication aspect - I figure if I can
foster more discussion between developers and operations folks, that
will hopefully lead to better understanding (and more compassion) on
both sides. I've also had quite a bit of success getting operations
people to use
development tools (for example, formal code review).

P.
--
Philip J. Hollenback
phi...@pobox.com
www.hollenback.net

Kief Morris

unread,
Jun 29, 2011, 2:39:19 AM6/29/11