How devops is perceived

105 views
Skip to first unread message

Philip J. Hollenback

unread,
Mar 28, 2012, 12:41:42 PM3/28/12
to dev...@googlegroups.com
After the 'manage shell scripts' blowup last week I wrote down a few
thoughts:

http://www.hollenback.net/DevopsPerceptions

I'm worried devops being pigeonholed as 'puppet and chef users'.

P.
--
Philip J. Hollenback
www.hollenback.net
@philiph

Joe McDonagh

unread,
Mar 28, 2012, 12:45:04 PM3/28/12
to dev...@googlegroups.com
If you reversed the Culture and Sharing mentions, the bold letters in
the article would spell out SCAM. Just sayin'.

--
Joe McDonagh

AIM: YoosingYoonickz Google: Joseph.E.McDonagh
IRC: joe-mac Skype: therealjoemac

Kit Plummer

unread,
Mar 28, 2012, 12:49:57 PM3/28/12
to dev...@googlegroups.com
I think we're already there Phil, especially from a "Dever's" perspective.  And speaking to many CIOs at F500, I'm finding that DevOps gets lost as noise in the Continuous Delivery topic.

IMHO DevOps would be best served by a "Practices of an Agile Developer" approach, where there's a detraction from what it is, and a focus on how to do it in easily defined "chapters".  This would enable the natural evolution without everyone opining on what it is, or isn't, or should be.

Kit

-- 
Kit Plummer :: MaestroDev :: 520.360.4729

Matt Ryanczak

unread,
Mar 28, 2012, 4:38:58 PM3/28/12
to dev...@googlegroups.com
On 03/28/2012 12:41 PM, Philip J. Hollenback wrote:
> After the 'manage shell scripts' blowup last week I wrote down a few
> thoughts:
>
> http://www.hollenback.net/DevopsPerceptions

I suspect we (this list) have many different views of what devops is.
The shell scripts episode is one example, but there is also the
"Internet operations ISN'T a part of IT?" thread from a few weeks ago
that also hinted at a difference of opinion regarding the scope of
devops. The latter thread was generally civil and I got a lot out of it,
However, one thing I did not get was any better understanding of what
devops means to most people.

It might be useful to define devops more concisely. This wikipedia
article http://en.wikipedia.org/wiki/Devops tries to do that. Does it
succeed though?

fwiw I think the wikipedia article does an OK job, not great. The venn
diagram is probably the best part :) The article hits on better
collaboration between operations and development but reads too
development centric to me. More should be said about automation, change
and configuration management, repeatable processes, instrumentation
(monitoring) and kiss (always relevant to operations imho). I'm an
operations person though so I see it as devOPS rather than DEVops which
is probably my problem ;)

Perhaps devops means many things (gratis / libre)? devops (the
methodology) is more than chef, puppet, salt or any other tool for sure
though.

~Matt

Spike Morelli

unread,
Mar 31, 2012, 8:34:57 AM3/31/12
to dev...@googlegroups.com
On 28 Mar 2012, at 18:41, Philip J. Hollenback wrote:

I'm worried devops being pigeonholed as 'puppet and chef users'.

sad as it is, this is an unfortunate reality or at least a very widespread phenomenon. Among other sources I get to ask people what they think about devops as part of our hiring process. While it's an open question, carefully crafted to represent no part, the results are pretty much consistent to 'puppet and chef users' with a sprinkle of cloud every now and then.

I've picked up on some of the things you mentioned and tried to offer a perspective of what's needed from this community in order to reverse that trend, but whether or not that will happen it's hard to tell. Unfortunately it feels like only a very small minority is focused on the communication and group dynamic aspects of the movement:


Also, while on the subject, John Allspaw posted this to another thread, but it seems far more appropriate here and worth a lot of visibility so I'm hoping people won't mind:


cheers,

Anthony Shortland

unread,
Mar 31, 2012, 11:32:44 AM3/31/12
to dev...@googlegroups.com
"Part of our hiring process" - funny you should mention this since we're looking for "devops engineers" ourselves. 

I found it quite illuminating to search job postings for the term "devops" on Dice (http://bit.ly/H31bmD - 106 hits) and Monster (http://mnstr.me/Hx1KHe - 66) - also I was told by our Dice sales rep (don't you love them!) that there are more than 80 resumes that contain that keyword. Anyone got broader data on that? We want to know where best to put our recruiting money.

It's a credit to the members of this list that the term has gained such wide coinage, and is still gaining momentum. Makes it a whole lot easier when you talk to contract companies and recruiters and they know what you mean!

That said, it's certainly not clear from scanning these postings that "devops" equates to "Puppet or Chef". In fact you can see calls for a wide-range of skills applicable to service delivery automation in general - both build and deployment and both application and infrastructure provisioning ... that's certainly what we're looking for in a "devops engineer" even if they do specialize in certain tools.

Any help and pointers on how and where to triangulate individuals like that? What works for you?

Anthony.
Anthony Shortland
Professional Services | DTO Solutions, Inc. | mobile: 650.215.3117 aim: anthony....@me.com yahoo: anthony.shortland irc.freenode.net: #controltier, #rundeck skype: anthony.shortland ]

Ernest Mueller

unread,
Mar 31, 2012, 12:38:14 PM3/31/12
to dev...@googlegroups.com
At 07:34 AM 3/31/2012, Spike Morelli wrote:
>On 28 Mar 2012, at 18:41, Philip J. Hollenback wrote:
>>I'm worried devops being pigeonholed as 'puppet and chef users'.
>
>sad as it is, this is an unfortunate reality or at least a very
>widespread phenomenon. Unfortunately it feels like only a very
>small minority is focused on the communication and group dynamic
>aspects of the movement:

Well, sure. Movements like this don't make people any smarter.

Agile. I see a lot of people implement Scrum blindly with no real
understanding of anything here: http://agilemanifesto.org/, and then
religiously defend Scrum's default implementation regardless of its
suitability to their environment. Although that's better than those
that even half ass it more and just say "we've gone to coding sprints
now, we must be agile, woot!" Of course they didn't set up any of the
guard rails, and use it as an exclude to eliminate architecture,
design, and project planning, and are confused when colossal failure results.

DevOps. "You must use chef or puppet that is DevOps now let's spend
the rest of our time fighting over which is better!" That's really
equivalent to the lowest level of sophistication there in
Agile. It's human nature, there are people that can't/don't want to
engage in higher order thought about problems, they want to grab and
go. I kinda wish we could come up with a little bit more of a
playbook, like Scrum is for Agile, where at least someone who doesn't
like to think will have a little more guidance about what a bundle of
best practices *might* look like, at least it gives hints about what
to do outside the world of yum repos.

My own personal stab at "What is DevOps"
(http://theagileadmin.com/what-is-devops/) tries to divide up
principles, methods, and practices and uses agile as the analogy to
show how you have to treat it. Understand the principles, establish
a method, choose practices. If you start by grabbing a single
practice, it might make something better - but it also might make
something worse.

Back at NI, the UNIX group established cfengine management of our Web
boxes. But they didn't want us to use it, that would be work and a
hassle. But then if we installed software that needed, say, an init
script (or really anything outside of /opt) they would freak out
because their lovely configurations were out of sync and yell at us.
Our response was of course "these servers are here to, you know, run
software, not just happily hum along in silence." Automation tools
can make things much worse, not just better.

At this week's Agile Austin DevOps SIG, we had ~30 folks doing
openspaces, and I saw some of this. "I have this problem." "You can
do that in chef or puppet!" "Really?" "Well... I mean, you could
implement it yourself... Kinda using data bags or something..." "So
when you say I can implement it in chef, you're saying that in the
same sense as 'I could implement it in Java?'" "Uh... yeah." "Thanks
kid you've been a big help."

If someone has a systems problem, and you say that the *answer* to
that problem is "chef" or "puppet," you understand neither the
problem nor the tools. It's "when you have a hammer, everything looks
like a nail - and beyond that, every part of a construction job
should be solved by nailing shit together."

We also do need to circle back up and do a better job of defining
DevOps. We backed off that early on, and as a result we have people
as notable as Adrian Cockroft saying "What's DevOps? I see a bunch
of conflicting blog posts, whatever, I'll coin my own *Ops term."
That's on us for not getting our act together. I have yet to see a
good concise DevOps definition that is unique if you remove the word
DevOps and insert something else ("DevOps helps you bring value to
software! DevOps is about delivering a service to a customer!"
s/DevOps/100 other things/.

Hmmm, that was a quality rant, maybe I'll post it on the blog.

Ernest

Christopher Little

unread,
Apr 1, 2012, 5:19:19 PM4/1/12
to dev...@googlegroups.com
We also do need to circle back up and do a better job of defining DevOps.

Can we have a mini-get-together on this in Austin's DevOps Days. So many deep-dive the tooling -- admit it, you know who you are -- without getting the importance of the larger landscape, as to defining DevOps. I mean, getting to "squishy" or not, some sort of general thing, a la CAMS, but a little crisper, that constitutes "what you could say first when you start talking about what DevOps is"

Anybody who knows me knows I pop a couple of bolts when people immediately cough up the tooling hairball answering the DevOps Q . . .  but I also understand that I'm on the far other end of most of the community (non-practitioner, came to this as an corp-manager-type from an old dev background) -- 

there's got to be some middle ground we can find that the larger IT world can get, between, say, me and somebody who says "Puppet/Chef is the DevOps solution" 

(that sound was bolt heads hitting the ground)

Noah Campbell

unread,
Apr 1, 2012, 5:57:58 PM4/1/12
to dev...@googlegroups.com
Perhaps DevOps describes the existing break between Development and Operations.  Continuous Delivery (i.e. developer commits and 10 minutes later it's in prod) and NoOps are radical *process* departures from the traditional COO owns ops and CTO owns engineering and everyone just needs to "get along" ™ 

-Noah

Miles Fidelman

unread,
Apr 1, 2012, 6:16:52 PM4/1/12
to dev...@googlegroups.com
Christopher Little wrote:
> > We also do need to circle back up and do a better job of defining
> DevOps.

For what it's worth, since I'm the guy who kicked off this thread by
commenting (on the lopsa-tech list, as quoted here by Philip Hollenback
) that:

"The devops list is] mostly folks who are users, developers, and fans of
chef, puppet, similar tools. A few folks have a broader sense of
operations. (Call me sensitive, but when I queried about management of
shell scripts, I got jumped on by rabid chef and puppet users.)"

I should emphasize that my comment was about my perception specifically
of this list, and not all the people on this list.

I did not, and do not mean to suggest that my impression of the broader
DevOps community is that it is all about Chef and Puppet. At least in
researching the topic, in places like the devops-toolchain list, in
presentations by folks like Alex Honor, it's always been clear to me
that there is a larger set of perspectives and approaches in the "DevOps
community" focused more on processes as well as tools for continuous
integration.

If there is a broader sense that devops is just about puppet and chef -
as several folks have suggested in this thread (e.g., in reporting
experiences in the hiring process) - that goes beyond the scope of my
specific comment and opinion.

(Note: This is completely separate from the opinions I've been forming
about the degree to which DevOps approaches address the full range of
issues faced by systems administrators, particularly those who are
operating in enterprise environments as distinct from service delivery
environments. For what it's worth, my observation is that DevOps is far
more relevant to those who involved in continuous integration of
software into service provider environments, than those who are managing
slowly changing enterprise environments. But that's me.)

Miles Fidelman

--
In theory, there is no difference between theory and practice.
In practice, there is. .... Yogi Berra


Eric Shamow

unread,
Apr 1, 2012, 6:29:24 PM4/1/12
to dev...@googlegroups.com
For what it's worth, and as someone that joined in the pile-on - it would be a lot more helpful if you posted conciliatory observations about DevOps to the LOPSA list, where most of the actual damage was done.

-Eric

-- 
Eric Shamow
Sent with Sparrow

damone...@gmail.com

unread,
Apr 1, 2012, 7:03:05 PM4/1/12
to dev...@googlegroups.com
Maybe I'm in the minority on this… but I feel like the "but what is and isn't DevOps?" debate is a giant red herring. Specifically, the desire to define DevOps as tangible thing (e.g. "add sugar, flour, water; bake for 35 minutes at 425 degrees; congrats you have hot a fresh DevOps").

In my opinion DevOps is not a thing. DevOps is not an "it". DevOps is a "why". It's something that we've been missing in IT for a long time, a context that inspires us to rethink how we need to fundamentally approach and build the world around us. By definition, there can't be an exact prescription as that context is going to continually evolve and is going to be slightly (or even dramatically) different from company to company. If you get focused on the "it" of DevOps you end up chasing your tail. If you focus on the "why" you get to something meaningful real quick and then can move on to creating value and solving your actual problems.

I've been ranting about this from a "it's not a tool" angle since 2010 (DevOps is not a technology problem, DevOps is a business problem), so I'm not expecting to make any progress on this now. :)

The culture of IT just values technical depth and technical specialization way too much to avoid having any conversation, given a long enough time horizon, not end up in a technology discussion or an outright tools debate. We respect (both in terms of street cred and paycheck size) the person that knows the most about a technology we are interested in…  and just as easily tend to dismiss the value of process and organization specialists (which is where the real DevOps work needs to happen).


On Apr 1, 2012, at 2:19 PM, Christopher Little wrote:

John Allspaw

unread,
Apr 1, 2012, 7:41:31 PM4/1/12
to dev...@googlegroups.com
I am now erasing a reply draft for the hundredth time because Damon said it better than I could.  The community ought to be able handle a non-prescriptive and context-influenced definition that is squishy.

+1

James Turnbull

unread,
Apr 1, 2012, 7:44:08 PM4/1/12
to dev...@googlegroups.com

+2. DevOps is about enabling the business to do business better, faster and cheaper. Who cares how we get there. They don't.

James

Ernest Mueller

unread,
Apr 1, 2012, 7:53:12 PM4/1/12
to dev...@googlegroups.com
Eh, I completely disagree. That's fuzzy across the line to BS.

"What is DevOps?"
"It's better faster cheaper!"
"Uhhh - OK, lots of things things claim to make
business "better". How's it different from anything else?"
"Culture!"
"So... Specifically..."

I think we all believe DevOps is a real thing
that has a distinct specific value that is
different from most existing processes. Us not
wanting to think hard enough about it to quantify it a little is just lazy.

Ernest

At 06:44 PM 4/1/2012, James Turnbull wrote:

>+2. DevOps is about enabling the business to do
>business better, faster and cheaper. Who cares how we get there. They don't.
>
>James
>On Apr 1, 2012 6:41 PM, "John Allspaw"

><<mailto:all...@gmail.com>all...@gmail.com> wrote:
>I am now erasing a reply draft for the hundredth
>time because Damon said it better than I could.

>Â The community ought to be able handle a

>non-prescriptive and context-influenced definition that is squishy.
>
>+1
>
>
>On Apr 1, 2012, at 7:03 PM,

><mailto:damone...@gmail.com>damone...@gmail.com wrote:
>
>>Maybe I'm in the minority on this… but I feel

>>like the but what is and isn't DevOps?" debate

>>is a giant red herring. Specifically, the
>>desire to define DevOps as tangible thing (e.g.
>>"add sugar, flour, water; bake for 35 minutes
>>at 425 degrees; congrats you have hot a fresh DevOps").
>>
>>In my opinion DevOps is not a thing. DevOps is
>>not an "it". DevOps is a "why". It's something
>>that we've been missing in IT for a long time,
>>a context that inspires us to rethink how we
>>need to fundamentally approach and build the
>>world around us. By definition, there can't be
>>an exact prescription as that context is going
>>to continually evolve and is going to be
>>slightly (or even dramatically) different from
>>company to company. If you get focused on the
>>"it" of DevOps you end up chasing your tail. If
>>you focus on the "why" you get to something
>>meaningful real quick and then can move on to
>>creating value and solving your actual problems.
>>
>>I've been ranting about this from a "it's not a
>>tool" angle since 2010

>>(<http://dev2ops.org/blog/2010/11/7/devops-is-not-a-technology-problem-devops-is-a-business-prob.html>DevOps

>>is not a technology problem, DevOps is a
>>business problem), so I'm not expecting to make any progress on this now. :)
>>
>>The culture of IT just values technical depth
>>and technical specialization way too much to
>>avoid having any conversation, given a long
>>enough time horizon, not end up in a technology
>>discussion or an outright tools debate. We
>>respect (both in terms of street cred and
>>paycheck size) the person that knows the most

>>about a technology we are interested in… Â and

>>just as easily tend to dismiss the value of

>>process andd organization specialists (which is

>>where the real DevOps work needs to happen).
>>
>>
>>On Apr 1, 2012, at 2:19 PM, Christopher Little wrote:
>>

>>> >Â We also do need to circle back up and do a

>>> better job of defining DevOps.
>>>
>>>Can we have a mini-get-together on this in
>>>Austin's DevOps Days. So many deep-dive the
>>>tooling -- admit it, you know who you are --
>>>without getting the importance of the larger
>>>landscape, as to defining DevOps. I mean,
>>>getting to "squishy" or not, some sort of
>>>general thing, a la CAMS, but a little
>>>crisper, that constitutes "what you could say
>>>first when you start talking about what DevOps is"
>>>
>>>Anybody who knows me knows I pop a couple of
>>>bolts when people immediately cough up the
>>>tooling hairball answering the DevOps Q . . .

>>>Â but I also understand that I'm on the far

>>>other end of most of the community
>>>(non-practitioner, came to this as an

>>>corp-manager-type from an old dev background) --Â


>>>
>>>there's got to be some middle ground we can
>>>find that the larger IT world can get,

>>>between, say, me and somebody who says "Puppet/Chef is the DevOps solution"Â


>>>
>>>(that sound was bolt heads hitting the ground)
>>>
>>>
>>>
>>>
>>>On Sat, Mar 31, 2012 at 12:38 PM, Ernest
>>>Mueller <<mailto:ernest....@gmail.com>ernest....@gmail.com> wrote:
>>>At 07:34 AM 3/31/2012, Spike Morelli wrote:
>>>On 28 Mar 2012, at 18:41, Philip J. Hollenback wrote:
>>>I'm worried devops being pigeonholed as 'puppet and chef users'.
>>>
>>>
>>>sad as it is, this is an unfortunate reality

>>>or at least a very widespread phenomenon. Â

>>>Unfortunately it feels like only a very small
>>>minority is focused on the communication and
>>>group dynamic aspects of the movement:
>>>
>>>

>>>Well, sure. Â Movements like this don't make people any smarter.
>>>
>>>Agile. Â I see a lot of people implement Scrum

>>>blindly with no real understanding of anything
>>>here:

>>><http://agilemanifesto.org/>http://agilemanifesto.org/,

>>>and then religiously defend Scrum's default
>>>implementation regardless of its suitability

>>>to their environment. Â Although that's better

>>>than those that even half ass it more and just
>>>say "we've gone to coding sprints now, we must
>>>be agile, woot!" Of course they didn't set up
>>>any of the guard rails, and use it as an
>>>exclude to eliminate architecture, design, and
>>>project planning, and are confused when colossal failure results.
>>>

>>>DevOps. Â "You must use chef or puppet that is

>>>DevOps now let's spend the rest of our time
>>>fighting over which is better!" That's really
>>>equivalent to the lowest level of

>>>sophistication there in Agile. Â It's human

>>>nature, there are people that can't/don't want
>>>to engage in higher order thought about

>>>problems, they want to grab and go. Â I kinda

>>>wish we could come up with a little bit more
>>>of a playbook, like Scrum is for Agile, where
>>>at least someone who doesn't like to think
>>>will have a little more guidance about what a
>>>bundle of best practices *might* look like, at
>>>least it gives hints about what to do outside the world of yum repos.
>>>
>>>My own personal stab at "What is DevOps"

>>>(<http://theagileadmin.com/what-is-devops/>http://theagileadmin.com/what-is-devops/)

>>>tries to divide up principles, methods, and
>>>practices and uses agile as the analogy to

>>>show how you have to treat it. Â Understand

>>>the principles, establish a method, choose

>>>practices. Â If you start by grabbing a single

>>>practice, it might make something better - but
>>>it also might make something worse.
>>>
>>>Back at NI, the UNIX group established
>>>cfengine management of our Web boxes. But they
>>>didn't want us to use it, that would be work
>>>and a hassle. But then if we installed
>>>software that needed, say, an init script (or
>>>really anything outside of /opt) they would
>>>freak out because their lovely configurations
>>>were out of sync and yell at us. Our response
>>>was of course "these servers are here to, you
>>>know, run software, not just happily hum along
>>>in silence." Automation tools can make things much worse, not just better.
>>>
>>>At this week's Agile Austin DevOps SIG, we had
>>>~30 folks doing openspaces, and I saw some of

>>>this. Â "I have this problem." Â "You can do
>>>that in chef or puppet!" "Really?" Â "Well...

>>>I mean, you could implement it yourself...
>>>Kinda using data bags or something..." "So
>>>when you say I can implement it in chef,
>>>you're saying that in the same sense as 'I

>>>could implement it in Java?'" Â "Uh... yeah."

>>>"Thanks kid you've been a big help."
>>>
>>>If someone has a systems problem, and you say
>>>that the *answer* to that problem is "chef" or
>>>"puppet," you understand neither the problem
>>>nor the tools. It's "when you have a hammer,
>>>everything looks like a nail - and beyond
>>>that, every part of a construction job should
>>>be solved by nailing shit together."
>>>
>>>We also do need to circle back up and do a

>>>better job of defining DevOps. Â We backed off

>>>that early on, and as a result we have people
>>>as notable as Adrian Cockroft saying "What's

>>>DevOps? Â I see a bunch of conflicting blog

>>>posts, whatever, I'll coin my own *Ops term."
>>>That's on us for not getting our act together.
>>>I have yet to see a good concise DevOps
>>>definition that is unique if you remove the
>>>word DevOps and insert something else ("DevOps

>>>helps you bring value to software! Â DevOps is

James Turnbull

unread,
Apr 1, 2012, 8:02:27 PM4/1/12
to dev...@googlegroups.com

No its not. That's called a summary (and I am on my phone so its a brief summary). I can draw lines between each of those summary statements and real action teams can take both culturally and with tools to achieve that summary state.

For example, faster deployments == faster speed to market. Faster deployments come from ...

James

On Apr 1, 2012 6:53 PM, "Ernest Mueller" <ernest....@gmail.com> wrote:
Eh, I completely disagree.  That's fuzzy across the line to BS.

"What is DevOps?"
"It's better faster cheaper!"
"Uhhh - OK, lots of things things claim to make business "better". How's it different from anything else?"
"Culture!"
"So... Specifically..."

I think we all believe DevOps is a real thing that has a distinct specific value that is different from most existing processes.  Us not wanting to think hard enough about it to quantify it a little is just lazy.

Ernest

At 06:44 PM 4/1/2012, James Turnbull wrote:

+2. DevOps is about enabling the business to do business better, faster and cheaper. Who cares how we get there. They don't.

James

James Turnbull

unread,
Apr 1, 2012, 8:04:27 PM4/1/12
to dev...@googlegroups.com

And to be clear, the "who cares how we get there" is about tool selection. I don't care what tools you select or not as long as they achieve the right outcomes.

James

On Apr 1, 2012 6:53 PM, "Ernest Mueller" <ernest....@gmail.com> wrote:
Eh, I completely disagree.  That's fuzzy across the line to BS.

"What is DevOps?"
"It's better faster cheaper!"
"Uhhh - OK, lots of things things claim to make business "better". How's it different from anything else?"
"Culture!"
"So... Specifically..."

I think we all believe DevOps is a real thing that has a distinct specific value that is different from most existing processes.  Us not wanting to think hard enough about it to quantify it a little is just lazy.

Ernest

At 06:44 PM 4/1/2012, James Turnbull wrote:

+2. DevOps is about enabling the business to do business better, faster and cheaper. Who cares how we get there. They don't.

James

Matt Ryanczak

unread,
Apr 1, 2012, 8:11:57 PM4/1/12
to dev...@googlegroups.com
On 4/1/12 7:53 PM, Ernest Mueller wrote:
> Eh, I completely disagree. That's fuzzy across the line to BS.

I agree with your disagree :) As a community we should be able to come
up with some sort of definition that is suitable for text books,
wikipedia and other forms of documentation. Even if that something is
somewhat vague. How else do the initiated learn what devops is? Perhaps
its more of a mission statement than a definition?

"Devops is an evolving methodology for realizing business goals through
operational best practices and tight integration between software
engineering, quality assurance, technical operations and other business
units."

Just to get the fire started...

~Matt

Charles Henrich

unread,
Apr 1, 2012, 8:16:42 PM4/1/12
to dev...@googlegroups.com, dev...@googlegroups.com

Definitely agree here.  I've seen far far too much focus on tool x or process y as magic bullets instead of focusing holistically on all the tools and processes that for your organization will  improve the business.  For me modern day operations (call it whatever you want) is about enabling the business to get from idea to use with limited friction.   It's up to the teams to decide what the appropriate levers are to do that in their context.  It may be tooling, processes, automation or none of the above and still be the right answer.  

There is no spoon.  

-Crh

(apologies for being terse, sent from an iPhone)

Paul Graydon

unread,
Apr 1, 2012, 8:26:20 PM4/1/12
to dev...@googlegroups.com

I know that it would fall inside possibly a few of those categories, but
I think it's probably worth specifically mentioning Security. It
frequently seems to be ever the bridesmaid and never the bride, but it's
something that should permeate everything. We keep seeing companies
failing and disasters occurring because it isn't. It gets shunted in to
'other business units' at best, an afterthought rather than
forethought. Proof in case the fun over that ruby on rails 'bug' &
github the other week.

Paul

John Allspaw

unread,
Apr 1, 2012, 8:26:58 PM4/1/12
to dev...@googlegroups.com
Ernest -

I might say that to reach for specificity in the term, there be dragons and a frustrating result, maybe 140 characters at a time.

Continuous deployment may help one org realize business gain, and not another. Giving devs root may help further one org realize business gain, but maybe not others. Same with Chef. Same with A/B testing. Same with embedding developers in operations. Or vice versa. And so forth.

What matters is the extent that different domain expertises are given the charter and culture to amplify and compliment each other's abilities and perspectives, as a means to further the business. Everything else is implementation details that can and will change.

I'm not bold enough to assert that devs without root, waterfall dev cycles, or Netsaint and Sybase on AIX can't achieve a devops environment.

I realize this isn't an easy thing to describe, but I simply can't point to a specific tool, org structure, title, methodology, or process that is a recipe to make such an environment.

I feel confident that I can tell the difference between a "programmer" and an "engineer" without employing specific criteria, and I feel the same way about devops. I personally don't use the term, instead I opt to say "developer and operations cooperation, collaboration, and coordination". If the sentence feels weird with that substitution, I then think that the intention didn't match my expectation.

Instead of a specific "what" definition, instead let me suggest that we as practitioners can offer stories of our success and failure, including implementation details, in support of attempting to build such an environment as I described, and take it from there, with an open mind.

We can't 'git pull devops' no matter how hard we (or others) want to.

(from my iphone)

Eric Shamow

unread,
Apr 1, 2012, 8:50:02 PM4/1/12
to dev...@googlegroups.com
On Sunday, April 1, 2012 at 8:26 PM, John Allspaw wrote:
I feel confident that I can tell the difference between a "programmer" and an "engineer" without employing specific criteria, and I feel the same way about devops. I personally don't use the term, instead I opt to say "developer and operations cooperation, collaboration, and coordination". If the sentence feels weird with that substitution, I then think that the intention didn't match my expectation.

Instead of a specific "what" definition, instead let me suggest that we as practitioners can offer stories of our success and failure, including implementation details, in support of attempting to build such an environment as I described, and take it from there, with an open mind.

We can't 'git pull devops' no matter how hard we (or others) want to.

What worries me about the attempts to "harden" the DevOps definition is that it leads us down the same road as ITIL and Agile - poor implementations of what was supposed to be a loose framework or set of guidelines codified into some kind of rigid standard, which then can be applied (or more often, misapplied) - and ends up causing more damage and confusion than it was designed to resolve.

The guidelines to managing and setting processes for a DevOps environment vary wildly in terms of the type of business or organization, the personality and skill sets of the teams involved, and the type of integration that's desired and achievable in the political reality of the org.  At some point somebody may be able to come up with a pithy one-liner to summarize this, but right now I think it's easier to come up with lists of what DevOps is and is not.

DevOps is:
  • Cooperating with other teams in the IT organization to achieve the goals set out by the larger organization as a whole
  • Transparency and data sharing both within and between groups
  • An emphasis on collaborative, multidisciplinary problem-solving over finger-pointing
DevOps is *sometimes*:
  • Using the same toolsets across different parts of the organization, mainly in the attempt to achieve the above.
DevOps is not:
  • An individual toolset or set of toolsets
  • A job description for a developer who is good at operations, or vice versa
  • Agile with scrum taken out and replaced with something else (kanban, etc)
That's not to say that DevOps can't include all the things that it isn't - just that they don't define it.  Kanban, Puppet, Chef, Agile, Pair Programming - these are all things that can be part of the DevOps tool belt, and those of us who have built DevOps-ish organizations will often turn to them, because they can make implementing parts of the transition easy.

But they're tools.  Without the underlying culture, they'll fail just as hard as any other toolset.

-Eric

Miles Fidelman

unread,
Apr 1, 2012, 9:22:56 PM4/1/12
to dev...@googlegroups.com
Eric Shamow wrote:
> For what it's worth, and as someone that joined in the pile-on - it
> would be a lot more helpful if you posted conciliatory observations
> about DevOps to the LOPSA list, where most of the actual damage was done.

Absolutely not. What I said on the lopsa list was:

>/ Anyway, when I asked for tool suggestions on one of the devops lists, where I
/>/ kind of figured was the place to ask about tools, about all I got were
/>/ religious pontifications about puppet or chef being the "one true way" -
/>/ which seems a bit out of line with my experience of real-world operations,
/>/ both in the small, and in the large (been around both, though more as an
/>/ architect for large systems than as an operator).
/>/

And later, when I posted a summary of information I was collecting, that included this statement:

/>/ A couple of folks on the devops list pointed me at this:
/>/ https://bdsm.beginrescueend.com/modules/shell

(thank you, by the way)

Someone asked, rather briefly:

> The devops list?
I replied:

> http://groups.google.com/group/devops
> and
> http://groups.google.com/group/devops-toolchain
>
> Mostly folks who are users, developers, and fans of chef, puppet,


> similar tools. A few folks have a broader sense of operations. (Call
> me sensitive, but when I queried about management of shell scripts, I

> got jumped on by rapid chef and puppet users.)
>

And I stick by that characterization of the opinion I formed based on the responses
that I received to what I thought was a perfectly reasonable question.

Miles Fidelman
/

Ricardo Pascal

unread,
Apr 1, 2012, 9:56:21 PM4/1/12
to dev...@googlegroups.com
I Think that any regular Ops should "Cooperating with other teams in

the IT organization to achieve the goals set out by the larger
organization as a whole"
And have "Transparency and data sharing both within and between groups"

The idea of "An emphasis on collaborative, multidisciplinary
problem-solving over finger-pointing" sounds a little more like to be
a DevOps to me.

But the other concepts like:
* Produce more metrics
* more automation
* with or without chef
* improve deployment process

All of this always existed as Sysadmin duty, in more or less scale at
different companies.

I have the impression that the Dev in DevOps, should reflect (or at
last was supposed to) someone from Ops, doing a little of Dev.

Like searching at code for possible bottlenecks that some function may
cause on the database server. Then test, fix, and tunne it (both the
code and the server). A DevOps should be writing code with developers,
choosing frameworks, libs, drivers

When people talk about DevOps like producing more concise "cookbooks"
for chef/puppet/salt I got a image of a pig with lipstick. Cause
that's what Ruby/DSL look's like for a regular Op who usually does
stuff with shell script.

--
Ricardo Pascal - Sysadmin and Developer
Twitter/Skype: voorloop

Eric Shamow

unread,
Apr 1, 2012, 10:05:55 PM4/1/12
to dev...@googlegroups.com
On Sunday, April 1, 2012 at 9:56 PM, Ricardo Pascal wrote:
But the other concepts like:
* Produce more metrics
* more automation
* with or without chef
* improve deployment process

All of this always existed as Sysadmin duty, in more or less scale at
different companies.

The idea of producing metrics and automation existed, and good Ops teams used them in the right way.  But lots of bad Ops teams produced tons of metrics and a) failed to make them visible, b) failed to talk to Dev about what a meaningful metric indicated, and c) failed to pay attention to them or aggregate them properly.

The idea is to produce more metrics *for the organization* and to assist parts of the organization in the use of tools they have gotten quite good at - Nagios, Cacti, Graphite, etc.  Getting all the reports in one place and visible to everybody is critical.

The idea of automating and improving the development process is not in this case purely to benefit Ops - it is in many ways designed specifically to allow Dev to iterate more quickly and to eliminate miscommunication between Dev and Ops both on application packaging and writing/testing code in environments that were more like production - and enabling getting access to such machines without throwing up roadblocks.
 
I have the impression that the Dev in DevOps, should reflect (or at
last was supposed to) someone from Ops, doing a little of Dev.

Like searching at code for possible bottlenecks that some function may
cause on the database server. Then test, fix, and tunne it (both the
code and the server). A DevOps should be writing code with developers,
choosing frameworks, libs, drivers

Those are all outcomes of good devops - but that's just half the puzzle.  The other half is providing a service environment that is responsive to the needs of development and the organization.  Should Ops have been doing that?  Yes.  Have they been?  Certainly as an industry, no.  Similarly it's about ensuring that Dev cares about application stability just as much as they do about delivering features, and about producing maintainable and repeatable processes.  Should they have been doing it?  Yes.  Were they?  Mostly not.

So DevOps isn't so much about revolutionizing the toolset, it's about tying our stakes together and behaving productively as an industry - whatever that entails.  DevOps has become associated with a set of tools (Puppet, Chef, Graphite, Git) only because at the time in which the movement was created, these are commonly-available, stable frameworks for tooling in such environments.  Had it been 10 years ago we'd be talking about Python and CFengine with SVN, 5 years before that KSH/BASH scripting and pdssh, 5 years before that rlogin.  The tools are the best we have now - the business focus is what's changed.

-Eric



Ramin K

unread,
Apr 1, 2012, 11:36:19 PM4/1/12
to dev...@googlegroups.com
On Wednesday, March 28, 2012 9:41:42 AM UTC-7, Phil Hollenback wrote:
After the 'manage shell scripts' blowup last week I wrote down a few
thoughts:

http://www.hollenback.net/DevopsPerceptions

I'm worried devops being pigeonholed as 'puppet and chef users'.


I think the problem is far more dire on the httpd side. We're all pigeonholed as Apache, Nginx, and IIS users... and there are stats to back that allegation up. :-)

What if the question had been about building a framework around a bash http server that supports logging, easy to use config files, multithread/child procs, modules, etc? Just use Apache seems like a pretty good answer, doesn't it? Hell the question put in this light appears downright insane. Apache vs node, tornado or something in that vein is more complicated. However the first set of questions are going to center around "do you have needs that $x general httpd server can't meet that more specialized solution $y can? Otherwise use $x."

I propose that the reason "Puppet, Chef, cfengine3, pick one" is thrown out so often is not because we care about the toolset, it's because we *don't* care... as long as it meets a minimum set of capabilities (see Apache, Nginx, IIS pick one). As someone above pointed out if we'd had this conversation ten years ago a different set of tools would be tripping off tongues because those were the minimum set of capabilities at the time. Further most of the us have internalized this logic which means we just throw out the conclusion rather than review the steps that got us here. Fixing that may be worthwhile, rehashing the conclusion I'm not so sure about.

Ramin

damone...@gmail.com

unread,
Apr 1, 2012, 7:59:17 PM4/1/12
to dev...@googlegroups.com
+1 for for getting some plus love from the guys at the top of my people I respect list. 

I'm not sure how it will be received, but I'm seriously considering lobbying hard for DevOps Days Mountain View to have no tools talks or panels. People can obviously do what they want in the open space portion of the event… but there are so many great topics (complete with history and academic rigor to boot) to explore without having the 10th retread of a tools panel or another tool talk that you can already find on youtube.

Gildas

unread,
Apr 2, 2012, 3:25:30 AM4/2/12
to dev...@googlegroups.com


Le 2 avr. 2012 08:02, <damone...@gmail.com> a écrit :
> I'm not sure how it will be received, but I'm seriously considering lobbying hard for DevOps Days Mountain View to have no tools talks or panels. People can obviously do what they want in the open space portion of the event… but there are so many great topics (complete with history and academic rigor to boot) to explore without having the 10th retread of a tools panel or another tool talk that you can already find on youtube.
>

No worries, I'll happily second that motion (although I think the time slot during open space for tools-oriented ignite talks worked ok last year).

Gildas

Rob

unread,
Apr 2, 2012, 8:08:54 PM4/2/12
to devops
There are many great and interesting ideas in this thread. Some of
them from the heavyweights in the field. Back to the original concern
of 'DevOps being pigeonholed' I'd agree with that statement. My
experience has led me to be an extremely pragmatic person, you do what
needs to be done, using the best tools available to you, as
responsibly as you can. Sometimes your tool chain may look like shell
scripts, duct tape and a drinking bird. Other times it may look like
ruby, bamboo, jenkins and chef. I'd like to think that the industry
is relying on this emerging segment of specialized generalists that
can bridge the gap between any and all groups to solve real problems.
Typically the problems we face aren't related to features that
endusers enjoy but are directly responsible for allowing that
enjoyment to occur in the first place. I don't think we can ignore
that people from all disciplines can contribute to DevOps, Development
- Operations collaboration, or what I once heard called 'Doing the
right thing'. I speak from experience, my team is comprised of a
business major, two engineers, a computer scientist and a stellar
sysAdmin. It's our collective thinking and approach that makes us
essential to our organization regardless whether it's Dev, QA, Sales,
IT, or even the exec (who loves metrics) who engages us. It's never
about the tools, they are just a means of accomplishing what we've
been asked to do, or what we'd like to do in our commitment to making
the business better and stronger.

So, I'm sad to see DevOps and Chef/Puppet becoming tightly coupled.
What we can and do contribute is much more than the knowledge of some
specific tools.

Rob

On Apr 2, 1:25 am, Gildas <3ntr0...@gmail.com> wrote:
> Le 2 avr. 2012 08:02, <damonedwa...@gmail.com> a écrit :> I'm not sure how it will be received, but I'm seriously considering

Spike Morelli

unread,
Apr 11, 2012, 3:40:39 AM4/11/12
to dev...@googlegroups.com
Hey Anthony,

On 31 Mar 2012, at 17:32, Anthony Shortland wrote:

I found it quite illuminating to search job postings for the term "devops" on Dice (http://bit.ly/H31bmD - 106 hits) and Monster (http://mnstr.me/Hx1KHe - 66) - also I was told by our Dice sales rep (don't you love them!) that there are more than 80 resumes that contain that keyword. Anyone got broader data on that? We want to know where best to put our recruiting money.

I think the job sites are the last place where to put money, but I have a pretty radical position about recruiting. I strongly believe the recruiting industry, at least for tech, is in dire need of reformation (very much like other aspects of engineering, devops being a champion example of that reformation)

It's a credit to the members of this list that the term has gained such wide coinage, and is still gaining momentum. Makes it a whole lot easier when you talk to contract companies and recruiters and they know what you mean!

well, that's what I hope to be contributing to, however I don't really know if we got as far as getting contract companies or recruiters to know what devops means. In fact more times than not they just produce long string of buzzwords and that's exactly what people like Lusis rave against and with reason. It's a hard mission.

That said, it's certainly not clear from scanning these postings that "devops" equates to "Puppet or Chef". In fact you can see calls for a wide-range of skills applicable to service delivery automation in general - both build and deployment and both application and infrastructure provisioning ... that's certainly what we're looking for in a "devops engineer" even if they do specialize in certain tools.

I can't speak for Philip, but the way I approached the "Puppet or Chef" branding argument was 'devops is not supposed to be primarily about tools'. Saying that it's more than puppet or chef and also about deployment and provisioning just seems to reinforce the argument that what we're focusing on is exclusively technology. That's exactly what I was encouraging us to take a distance from.

When you're looking for a devops engineer (operations, development …) I'm sure you don't just look for experience with a specific tech, but you also make sure the candidate has the kind of mentality that we recognise as a building block of devops ( http://bit.ly/HzF6l4 ). So what I was trying to argue is that if we do that, look for that mindset, look for those communication skills, why when we talk about devops we seem to primarily mention tools?

Any help and pointers on how and where to triangulate individuals like that? What works for you?

people on the ground. It will always be the best way to reach out to good folks. We live in an age where if you want you can contribute to software, communities and whatnot and such contributions to me are the primary signal we should look for because it signifies more than just skills, it signals commitment and initiative.

In the startup world there is a saying that everybody does marketing. I'd like to transfer that to this discussion and argue that everybody does recruiting and most importantly that everybody contributes to build the image of your company that will attract good candidates. Yes that only scales so far and every startup that becomes a full fledged company will eventually hire a CMO, but I'd still argue that the role of these individuals is more about co-ordination and strategy than anything else. The folks on the ground, including your customers, are still your greatest resource.

hope that helps and feel free to reach out to me off list if you'd like to talk more in details about recruiting strategies, it's an argument dear to me, but off topic for this list I think.

cheers,

Spike

Mary Carello

unread,
Apr 13, 2012, 7:59:38 PM4/13/12
to devops
Hi Spike, et al.

Question about part of your post: is the crux of the issue that
recruiters you have come across do not fully understand devops or that
they contact about the wrong type of positions (non-devops when you
want devops)?

>I think the job sites are the last place where to put money, but I have a pretty radical position about recruiting. I strongly believe the recruiting industry, at least for tech, is in dire need of reformation (very much like other aspects of engineering, devops being a champion example of that reformation)

Am interested in the example of how devops might be a champion example
of that reformation you mention above. Agree about the job sites.

>about recruiting strategies, it's an argument dear to me, but off topic for this list I think.

We could start another discussion... or rant ;)

Cheers,
M

Spike Morelli

unread,
Apr 14, 2012, 3:14:31 AM4/14/12
to dev...@googlegroups.com
Hi Mary, everybody,

On 14 Apr 2012, at 01:59, Mary Carello wrote:

Question about part of your post: is the crux of the issue that
recruiters you have come across do not fully understand devops or that
they contact about the wrong type of positions (non-devops when you
want devops)?

neither and both :) , I've had that problem, but I find both to be symptoms and not the crux of the issue (see below)

I think the job sites are the last place where to put money, but I have a pretty radical position about recruiting. I strongly believe the recruiting industry, at least for tech, is in dire need of reformation (very much like other aspects of engineering, devops being a champion example of that reformation)

Am interested in the example of how devops might be a champion example
of that reformation you mention above. Agree about the job sites.

because with devops, when you kept hitting problems at release time, you stopped just trying to find a better tool to do your releases and got the people producing the software and releasing the software talking to each other. Dialog , dialog, dialog. I know I'm simplifying a lot, but I hope it helps making a point.

I worked with a lot of recruiters over the years, especially when consulting, and 99% of the times there was little to no dialog between departments and recruiters. And when I got involved more often then not the topics of conversations were around:
1) what tools can I use for outreach?
2) where do I get these people?
3) what keywords can I use?

this to me is the same as an ops group not talking to developers: recruiting is a silo and we toss the ball over the wall to bring people in and then they toss it back when they get a keyword match. Everybody loses.

That said, that's just the tip of the iceberg. Devops is about dialog, but it's also about a bunch of methodologies and tools which themselves have contributed to making that dialog possible. The way engineering departments work has drastically changed. From what I've seen recruiting departments have barely moved an inch. Yes compared to 15 years ago it's possible/easier to reach out to people instead of having to publish a job and then wait for candidates to send their cv in, but that's about it.

In my opinion there is a lot of room for improvement, but that requires to fundamentally rethink the way the whole process works, getting organisations involved early on, empower recruiters with cross training… and that's where I stop because I really feel I'm doing a disservice to the people on this list :)

about recruiting strategies, it's an argument dear to me, but off topic for this list I think.

We could start another discussion... or rant ;)

I try not to rant :), but really, this list is about devops and at this point we're just discussing recruiting so I'd rather stop/take this elsewhere. And mind you, I'd love to, I invite anybody interested to reach out to me, I just don't want to annoy devops folks with my rambling about the recruiting industry.

thank you,

Reply all
Reply to author
Forward
0 new messages