I'm also a big fan of composing tool chains from piece parts - scripts
linked by pipes, mashups of functions accessed via RESTful APIs, etc.
Rundeck looks pretty interesting - given that it can manage shell
scripts and their execution via ssh. I'm sort of wondering what other
kinds of tools people are using for adding a job and configuration
control overlay to shell scripts. And what about orchestration tools
that that take a "mashup maker" approach? Pointers, thoughts, ....
Thanks,
Miles Fidelman
--
In theory, there is no difference between theory and practice.
In practice, there is. .... Yogi Berra
Are you looking for a script organization tool, or a one-to-many
execution tool?
I'm a fan of using Rakefiles to organize a bunch of one- or few- liner
scripts into a namespaced whole. I generally wrap these with small
boilerplate shell script that makes them filesystem location
independent and allows me to pass variables in as required. I keep
these in git, for replication/backup.
You also get the full power of ruby so it's a good gateway drug to
other systems...
- Zack
I'd echo Bryan with regards the worth of looking at the (sometimes
large and potentially intimidating) configuration management tools.
On the other hand I do think there is a place for more adhoc script
execution things, especially in smaller but still dynamic environments
and mainly for asking questions of the systems. I've used Fabric
(http://fabfile.org/) successfully here, partly because it provides
the framework for managing hosts and roles and doing parallel
execution, and the arguments can just be shell scripts. Once it's
setup anyone who's happy with bash can easily jump in, and it's a good
way of collecting useful one-liners. Past a certain number of hosts
something like mcollective or salt are probably more useful, they use
(different) message brokers to avoid problems with lots of ssh
sessions.
G
> - Zack
--
Gareth Rushgrove
Web Geek
Well you've obviously drunk the koolaid :-)
But that's not the problem I'm trying to address. As far as I can tell,
pretty much any configuration management system is only one part of a
toolchain - and right now, I don't see any complete toolchains, many of
the tools seem don't seem to compose very well (e.g., point me at
something that ties together provisioning with IP address management
with DNS record management), and a lot of pieces seem to be distribution
specific (e.g., FIA for debian, kickstarter for redhat-like distros).
Re. configuration tools specifically: I don't find chef, or puppet, or
bcg2, or cfengine recipes/rules all that clearer or more powerful than a
shell script, or perhaps a tcl script with expect statements - while I
find the restrictions associated with picking one of these something I'm
not ready to do.
Hence my interest in something like rundeck, that lets you manage and
compose scripts, AND configurations stored in other systems.
Rake looks pretty interesting. Not sure I want to bit off ruby (doing
most of our development in Erlang and JavaScript at the moment), but it
does seem to do some useful things.
Any thoughts on Rake vs. tcl/tk?
Thanks!
Miles
Is this a joke thread? Still not quite gotten the hang of American sarcasm.
--AJ
Both, but more focused on organization. An ideal tool would:
- provide for editing/managing/versioning scripts (script = anything
that can be invoked at the command line)
- a library of control functions for use within scripts
- invoking scripts, combinations of scripts, pipelines of scripts (in
the Unix sense of pipes) - locally, remotely, across multiple machines
- remote script execution via ssh, rather than some kind of agent
- providing a simple database for keeping track of variables used by
scripts (e.g., IP addresses, DNS records for use by a provisioning
script) - that can be accessed from scripts
- accessing the above via cli, RESTful API, GUI
- cross-platform
- (nice-to-have) minimal environmental requirements (i.e., a step above
the unix shell, say the gnu buildtool suite rather than the JVM + a mass
of libraries, or the ruby ecosystem -- a self-configuring basic
environment would be nice, like perl+cpan)
I'm sure we've each cobbled something together along these lines - be it
through manual processes, standard installs, or something fancier. I'm
kind of wondering what different people have done, what pieces fit
together better than others, what's available that's more integrated but
that doesn't create a monolithic environment.
Rundeck seems to do most (all?) of this, running on Java. It looks
nice, but it's also really new.
tcl/tk provides a lot of this - but without the organizing functions
(tcl/tk + git, or maybe a tcl/tk IDE perhaps?)
I'm basically wondering what else might be out there - perhaps less
visible - newer than tcl/tk, more mature than Rundeck perhaps.
Essentially doing some research before provisioning several new machines
and a bunch of VMs (easy enough to do from manual checklists, but I'm
setting the stage for some future scaling).
Miles
The clojure 'pallet' tool (cm) has some of these principles.
I've seen sloppy admins compose and reuse Bash snippets with Chef all the time. Easy trap for beginners.
--AJ
Why would you think that?
and later, AJ Christensen wrote:
> The clojure 'pallet' tool (cm) has some of these principles.
>
seems interesting, if a little java and cloud centric
> I've seen sloppy admins compose and reuse Bash snippets with Chef all
> the time. Easy trap for beginners.
>
Please elaborate --- are you saying that composing bash snippets is a
trap, or using Chef to do it is a trap? (If the former, who are you
calling a beginner? If the latter, I agree - not what Chef is built
for, and not the tool I'd use for managing shell scripts - hence my query.)
Over the years, I've built up a lot of shell scripts for provisioning
and configuration - which makes converting to something like chef or
puppet a somewhat daunting task (or even something like FAI).I'm also a big fan of composing tool chains from piece parts - scripts
linked by pipes, mashups of functions accessed via RESTful APIs, etc.Rundeck looks pretty interesting - given that it can manage shell
scripts and their execution via ssh. I'm sort of wondering what other
kinds of tools people are using for adding a job and configuration
control overlay to shell scripts. And what about orchestration tools
that that take a "mashup maker" approach? Pointers, thoughts, ....Thanks,
Two people have pointed at this, now, and the description sure sounds
interesting. Anybody know how active the project is - there doesn't
seem to be a mailing list or FAQ, and it seems to have one primary
developer.
Over the years, I've built up a lot of shell scripts for provisioning and configuration - which makes converting to something like chef or puppet a somewhat daunting task (or even something like FAI).
You mean this one?
"None yet, stay tuned I might setup a librelist. "
That's what makes me wonder.
Miles
It's active on GitHub, and you can submit issues and pull requests
there (they accepted one 12 days ago):
https://bdsm.beginrescueend.com/development/github/
I do know Wayne, and he's usually very responsive to input, but I
think he prefers IRC, which is probably why he hasn't set up a mailing
list for this yet (they suck up valuable coding time, right?). Dunno
how far-baked BDSM is or if you should bet the farm on it, but Wayne
wrote RVM, which is a widely used project in the Ruby community
(written in shell script), and has actively maintained that for years
now.
If the lack of a mailing list really bothers you, open a github issue
asking for one ;)
-- Chad
Pardon me for being blunt as well, then. I don't see a particularly
high return on investment for investing time in converting to a tool (be
it puppet, chef, cfengine, or what have you) that addresses only a small
piece of an overall operations toolchain. I'd rather invest time and
effort in getting my existing toolchain more automated, documented, and
under configuration control.
>
> * Every time I've gone into a sysadmin job, or picked up consulting
> work, I've found I've inherited a bunch of unusual bash and perl
> scripts, undocumented, inconsistent and bizarre in methodology (ever
> notice how no two sysadmins write perl / bash scripts alike?). It
> means it takes a stupid amount of time to piece things together and
> understand quite how they interact. Every time something breaks or
> goes wrong I'm left playing guessing games and tracking down whether
> it's a fringe case from some obscure aspect of a script or quite what
> (side note, it's amazing just how many problems I fix by enabling 'use
> strict; use warnings' and clearing up errors reported).
You ever notice that no two systems are alike?
Try actually building your next project in a proper CM tool and come back for your prize!
-s
> You ever notice that no two systems are alike?
Congrats. I figured it would get to this point. You've just described
why you need proper configuration management:
Puppet -
package { "screen": ensure => "installed" }
Chef -
package "screen" do
action :install
end
Guess what. That works the same across all distros. Doesn't matter if
those systems are alike. But here's the real crux of the matter: the
point of configuration management is not to make your special
snowflake systems behave the same. It's to make sure you don't have
special snowflake systems. Systems are cattle...pets. They aren't
artisan crafted, flame broiled, freshly brewed works of art.
I'm very sorry if you haven't gotten a very good welcome to the list.
However, while we may not have a good definition of what devops *IS*,
most people here feel that encouraging a pattern of brittle,
busted-ass inflexible and ultimately unscalable shell scripts is wrong
and downright irresponsible.
Even more telling is the fact that the questions you're asking are the
first sign that you've grown beyond those shell scripts.
SM (nee BDSM) is probably up your alley. Wayne is a solid guy and he
wrote SM mainly out of frustration with a poorly implemented Chef
setup. While I wouldn't suggest it for the reasons I listed above,
Wayne takes his community and his code seriously. Kudos if SM gets you
further along your journey but at some point, you are (and I would
argue already have) reached the limits of your shell addiction.
--
John E. Vincent
http://about.me/lusis
No idea what you mean, but I'm pretty sure it's unnecessarily snarky.
Git/GitHub is a proper CM tool in my book.
I love you, man...
Chad, you beat me to it :-)
Seems to me that rundeck and rerun, and possibly bdsm come pretty
close. Tcl/tk + git as well.
Tcl has been around forever. Rundeck I knew about, rerun and bdsm I
found out in responses to my query.
>> You ever notice that no two systems are alike?
> Congrats. I figured it would get to this point. You've just described
> why you need proper configuration management:
>
> Puppet -
>
> package { "screen": ensure => "installed" }
>
> Chef -
>
> package "screen" do
> action :install
> end
>
No. Configuration is a fairly evolve discipline for defining,
controlling, accounting for, and auditing system configurations.
CVS, Git, Subversion are configuration control systems.
Puppet and Chef are engines for applying configurations - with arguably
cleaner syntax than pure bash scripts.
> I'm very sorry if you haven't gotten a very good welcome to the list.
> However, while we may not have a good definition of what devops *IS*,
> most people here feel that encouraging a pattern of brittle,
> busted-ass inflexible and ultimately unscalable shell scripts is wrong
> and downright irresponsible.
--
For the record, people have been consistently and predictably managing
systems, at very large scale, for a really long time (at least by the
standards of "internet time"). People have been deploying distribution
systems (e.g., electric distribution grids), telecom. networks, large
distributed systems (such as those found in defense environments),
financial trading systems, etc., etc., etc. for decades. (By the way,
my background is large comms. networks, largely supporting distributed
command and control systems.)
I think its Alex Honor who's powerpoint decks that lead with a slide
along the lines of "people -> process -> tools" -- the point being that
tools have to serve the organization and process, not the other way around.
If the goal is to deploy 10,000 hadoop instances, quickly - sure chef is
going to do a great job. If you're managing something more complicated,
process discipline is what counts - you can manage huge huge and
complicated operations with little more than checklists and
whiteboards. The worst thing you want to do is have your tools impose
inappropriate disciplines in environments where operating procedures
have been evolved and tested over time.
I'm confused then. If you're talking about Tcl, why are we still
discussing bash scripts? I don't really see where Tk comes into play.
If you need to slap a gui on something, you'd be better served not
burdening your fellow sysadmins and just sticking it behind a web
interface. I guess AOLServer is still a thing?
>>> You ever notice that no two systems are alike?
>>
>> Congrats. I figured it would get to this point. You've just described
>> why you need proper configuration management:
>>
>> Puppet -
>>
>> package { "screen": ensure => "installed" }
>>
>> Chef -
>>
>> package "screen" do
>> action :install
>> end
>>
> No. Configuration is a fairly evolve discipline for defining, controlling,
> accounting for, and auditing system configurations.
>
> CVS, Git, Subversion are configuration control systems.
>
> Puppet and Chef are engines for applying configurations - with arguably
> cleaner syntax than pure bash scripts.
>
>
Andrew has already addressed this but we seem to have a semantic
confusion here. As he said, in "devops" discussions CM pretty much
always refers to a tool akin to puppet, chef or cfengine. Git, SVN,
CVS, Hg, Bazaar I have NEVER heard referred to as a "configuration
control system". Version control system (VCS) is the term I've always
heard.
Having said that:
"> No. Configuration is a fairly evolve discipline for defining, controlling,
> accounting for, and auditing system configurations."
If that's your definition then pretty much every tool discussed up to
this point accomplishes that task 100%. The nice thing about a CM (my
definition) tool is that's what its DESIGNED to do. I can tell you
that any system, regardless of being managed with Puppet or Chef (I
haven't done cfengine stuff in at least 8 years) I could meet all of
those requirements with my chef or puppet repo. So color me confused
as to what exactly these tools seem to be missing that DOESN'T meet
that need.
Puppet and Chef are engines for applying configurations - with arguably cleaner syntax than pure bash scripts.
Tcl is pretty good at composing command line executables - particularly
when you use expect. (tcl, after all, stands for "tool control
language") Tk lets you put a gui on it if you wish, but that's really
extraneous.
if you think that puppet or chef can support the formal configuration
control requirements of a large distributed system, including hardware
components, software components, configuration files, network devices
and configuration, crypto configuration, ..... - you just haven't dealt
with serious systems -- for that matter, when it comes to definitions,
the best taxonomy of devops that I've seen (of the few that seem to be
around) is the one at
http://code.google.com/p/devops-toolchain/wiki/ArchitectureComponents -
of which puppet/chef addresses only one small piece of an overall tool
chain - NOT including inventory (a key piece of any reasonable
configuration management process)
not to imply that they're not useful tools - but they certainly are only
one piece of provisioning, configuring, monitoring and controlling systems
in any case, my original query was specifically about tools for managing
and composing shell scripts - NOT a philosophical discussion of
competing operations styles and philosophies
if you want some elaboration of what I'm looking for, is a tool that,
ideally, would:
- provide for editing/managing/versioning scripts (script = anything
that can be invoked at the command line)
- a library of control functions for use within scripts
- invoking scripts, combinations of scripts, pipelines of scripts (in
the Unix sense of pipes)
- locally, remotely, across multiple machines
- remote script execution via ssh, rather than some kind of agent
- providing a simple database for keeping track of variables used by
scripts (e.g., IP addresses, DNS records for use by a provisioning
script) - that can be accessed from scripts
- accessing the above via cli, RESTful API, GUI
- cross-platform
- (nice-to-have) minimal environmental requirements (i.e., a step above
the unix shell, say the gnu buildtool suite rather than the JVM + a mass
of libraries, or the ruby ecosystem -- a self-configuring basic
environment would be nice, like perl+cpan)
- tcl has been used this way for a long time, but has its limits (on the
other hand, it has some nice features for interacting with databases -
which makes it useful for, say, linking a virtual-machine provisioning
script to a database of IP address assignments)
- rundeck and rerun are relatively new and get very close
- bdsm also seems to be in the same ballpark
tcl and rundeck I knew about - I have to thank this list and the
devops-toolchain list for pointers to rerun and bdsm, and some
discussion thereof
Ok, if you want to get pedantic about it - they are tools that "coerce"
a system to a specific state (using the technical definition of the word).
This is where I bow out of the conversation. I don't feel the need to
get into a measuring contest over who's managed bigger environments
and the relative merit therein. I'm perfectly happy with the size of
my infrastructures over the years.
> if you want some elaboration of what I'm looking for, is a tool that,
> ideally, would:
>
> - provide for editing/managing/versioning scripts (script = anything that
> can be invoked at the command line)
> - a library of control functions for use within scripts
> - invoking scripts, combinations of scripts, pipelines of scripts (in
> the Unix sense of pipes)
> - locally, remotely, across multiple machines
> - remote script execution via ssh, rather than some kind of agent
> - providing a simple database for keeping track of variables used by
> scripts (e.g., IP addresses, DNS records for use by a provisioning
> script) - that can be accessed from scripts
> - accessing the above via cli, RESTful API, GUI
> - cross-platform
> - (nice-to-have) minimal environmental requirements (i.e., a step above
> the unix shell, say the gnu buildtool suite rather than the JVM + a mass
> of libraries, or the ruby ecosystem -- a self-configuring basic
> environment would be nice, like perl+cpan)
>
Again, congratulations you just just described both Chef and Puppet.
The fact that you've not done enough research (or if you had, you
wouldn't still be using this as your baseline to DISPROVE those as
valid tools), suggests that others are right and you're most likely
trolling at this point.
Good luck with your search.
:-(
This isn't about being pedantic, it is about understanding tools and how you use them. Sure you can take a pair of pliers to remove a nut, but a wrench is a much better tool. All I was trying to say was that Puppet and Chef are often looked at as a way to replace shell scripts directly. They are not. Instead, they solve the problem in a completely different way that causes you to not need the shell scripts to begin with. I get it, you are happy with shell scripts. But just as there are times where you have to use some other language, generally perl in years past, there comes a time where you realize that managing with shell scripts is fundamentally flawed. Your arguments really are akin to telling me how amazing the Oracle installer is. I am sure that it was great compared to what was around when it was first conceived, but I would tell you that is the best example of how not to install software.
-- cwebber
No. I've dealt with seriously large infrastructure that uses LOTS of
different tools, of which scripts are a big piece. So is Nagios, so are
trouble ticket systems.
And no, 18 offices and 100 transactions/second is a trivial sized
operation. 10,000 spread across a university, or a battlefield, starts
to be a large infrastructure. Bank of America, NASDAQ, Fedwire,
MasterCard's transaction network - those are large systems. Now Google
is an example of a truly huge system.
>
> You might have dealt with more serious installations.. but there might
> be a chance you are living in a shell and the shell became your
> reality.. u just cant see or experience the change.
>
No. I'm asking about tools for managing shell scripts. You, on the
other hand, seem to be living in a reality that consists of chef,
puppet, and nothing else. Seems like a case of "if the only tool you
have is a hammer, every problem looks like a nail."
Just to take this argument to the next level of absurdity -- why not scrap your shell scripts then, and just write everything in C?
-Erik
You've made your point. You have descended from heaven to bless us mortals with your wisdom. We reject it because it's just beyond the ability of our tiny brains to comprehend. Etc etc. You can go back to your palace in the sky now please. Come back when you can respect even the "little guys" and the rest of us who are more interested in learning, building, and sharing what we find than throwing our giant egos around.
On Mar 18, 2012, at 10:51 PM, Miles Fidelman wrote:
You've outright written off other tools, apparently without
understanding them given that you're telling people they can't do what
those people are using them for.
On the other hand, many of us here have been down the shell script + SSH
route, and dealt with infrastructure heavily based upon it. We know
what it's like, and many of us probably know what it's like to inherit
such an infrastructure. (For myself, I worked in a good sized ISP who's
complex infrastructure heavily relied on such techniques, last I heard
they were migrating to Puppet.)
You came in and asked a question, people took time out of their day to
give you answers based on their real world experience managing
infrastructures small and large and you have basically spent the entire
thread calling them all idiots and liars.
I'm assuming it'll therefore come as no surprise that there has been a
bit of discussion off-list about whether or not you're trolling. From
what I've seen the prevailing opinion is that you are, and some of those
opinions are coming from people who are managing infrastructures larger
than what you're talking about who haven't bothered participating once
they saw your attitude.
As far as I'm concerned: Take it, or leave it.
You came and asked for advice, you got it. I'm sorry it doesn't appear
to be the advice that you were seeking which seems to be reassurance
that your way of doing this is the one true and holy way if only you
could find ways to make it work better.
Paul
I'm not sure someone's preference for shell scripts requires us to
become assholes to them.
Not everyone has drunk the chef/puppet kool-aid and that's fine. I
think he has some good points about wanting to initially invest his
time in something else and that these tools only form a small part of
the toolchain. That's totally correct. At NI we spent our time
writing PIE instead of uptaking chef or puppet, because provisioning
(and frankly those only do the middle layer of provisioning well,
leaving system spinup and app deployment to other components) wasn't
our most pressing need.
There's also some good points from Andrew about chef/puppet not being
a magical solution. We have a puppet implementation here at BV that
is frustrating to us because in retrospect, it was implemented in a
pretty convoluted and fragile way (in good faith, by following the
random Internet help out there) and isn't all that much better at the
moment (It has promise, yadda yadda) than the old
nest-of-shell-scripts deployment pipeline we have.
Let's not all confuse our preferred approach with the One True Way,
and let's not confuse DevOps with using the shiniest tools.
Ernest
> On the other hand, many of us here have been down the shell script +
> SSH route, and dealt with infrastructure heavily based upon it. We
> know what it's like, and many of us probably know what it's like to
> inherit such an infrastructure. (For myself, I worked in a good sized
> ISP who's complex infrastructure heavily relied on such techniques,
> last I heard they were migrating to Puppet.)
>
> You came in and asked a question, people took time out of their day to
> give you answers based on their real world experience managing
> infrastructures small and large and you have basically spent the
> entire thread calling them all idiots and liars.
No... I asked for suggestions and experiences re. tools for managing
shell scripts. I got precious few answers (a few good ones). If I
asked for help with a Debian boot problem, I wouldn't expect answers
about CentOS boot problems, or why I should be using OS X, or writing my
code in C.
> You came and asked for advice, you got it. I'm sorry it doesn't
> appear to be the advice that you were seeking which seems to be
> reassurance that your way of doing this is the one true and holy way
> if only you could find ways to make it work better.
Opinions and mileage may vary. Obviously some well-respected folks in
the devops world don't find shell scripts and SSH completely obsolete,
or easy to abandon - or we wouldn't have rundeck and rerun floating around.
I asked for input on something specific, and got religious arguments and
suggestions that I'm a troll. Sheesh. (along with a few good
suggestions - for which I thank people).
Thank you!
Miles
--
Joe McDonagh
AIM: YoosingYoonickz Google: Joseph.E.McDonagh
IRC: joe-mac Skype: therealjoemac
No... I asked for suggestions and experiences re. tools for managing
shell scripts. I got precious few answers (a few good ones).
I asked for input on something specific, and got religious arguments and
suggestions that I'm a troll. Sheesh. (along with a few good
suggestions - for which I thank people).
Thank you very much for your measured comments - and for both rerun and
rundeck, one or both of which I expect I might be using in the near future.
I've also become a big fan of the various presentations from you, Alex
Honor, and the crew at DTO Solutions - I've found them very insightful
vis-a-vis real-world approaches to operational systems management.
Particularly DTO's standard slide on process coming before tools (one of
my personal peeves, over the years, are people who design tools that
require massive changes in operations processes - there's a reason that
"groupware" has never really caught on, but that's a discussion for
another time and place).
A couple of questions and comments below, inline:
damone...@gmail.com wrote:
>
> This is where rerun came in. The specific scenario we kept coming
> across was large organizations who had massive investment in shell
> scripts (scripts of all flavors really) both in terms of working code
> and the skills of dozens or even hundreds of people. The individual
> DevOps thought leaders inside these companies saw the benefit of tools
> like Puppet and Chef but the non-technical constraints placed on them
> were holding them back. Someday they knew they could go there, but
> what were they to do in the meantime?
Not just large organizations. I'm currently in a tiny one, wearing
multiple hats. In larger settings, we had the luxury of turning someone
loose to learn and implement new tools. That's a bit harder to do when
everyone is wearing too many hats. Quick and dirty tends to be very
efficient, but not scalable - and that's the situation I find myself
in. Putting scripts, and scraps of paper, and notes on whiteboards, and
spreadsheets containing IP address assignments and DNS under some
measure of configuration control is a good first step toward being able
to replicate/restore systems, and toward sharing knowledge and
responsibilities across a group.
> <snip>
>
> In every situation where rerun went in, if you stand back far enough
> the benefits of the solution look the same as if the heart of the
> toolchain was Puppet or Chef. Sure the benefits might not have been
> nearly as great and upon closer observation there is still lingering
> technical debt that will have to be paid off in the future when they
> do move to Puppet or Chef. But in the meantime their DevOps problems
> did get better in the face of serious constraints and, hopefully, we
> all gained another set of converts to the DevOps way of thinking.
It strikes me that an orchestration framework - be it rundeck, or
capistrano, or mcollective, or what have you - can be a more effective
starting point for automation than something like puppet or chef. I
wonder if you might have an opinion on that?
And... I find that one of the most time consuming operational headaches
is inventory tracking - particularly of things like IP address
assignments, DNS records, user accounts and privileges, and so forth.
It's one thing to use a tool to build and configure a system (say FAI +
chef), but there's a manual step involved in pulling a bunch of IP
addresses off a list and plugging them into the configuration files (or
recipes, or what have you). On the devops-toolchain list, Noah Campbell
reported some shortcoming of CMDB approaches (notably keeping them in
sync with reality), and discussed some of the issues sourcing
information when running stuff under Rundeck control. I'd also be very
interested if you have something to add in this regard.
Thanks Again,
Miles Fidelman
In every situation where rerun went in, if you stand back far enough the benefits of the solution look the same as if the heart of the toolchain was Puppet or Chef. Sure the benefits might not have been nearly as great and upon closer observation there is still lingering technical debt that will have to be paid off in the future when they do move to Puppet or Chef. But in the meantime their DevOps problems did get better in the face of serious constraints and, hopefully, we all gained another set of converts to the DevOps way of thinking.
Should we discourage them if they want to (or are forced to) only use shell scripts to achieve those goals? Or if they don't use or CM(tm) tools of choice? Of course not.At what point do are these activities blessed and called "DevOps"? Who cares...
Sidebar: how did this conversation jump from devops-t...@googlegroups.com, where is started and seemed to belong, to dev...@googlegroups.com?
Actually, no. I asked a very specific question about tools for managing
shell scripts. When things degenerated into religious arguments, I
ventured the observation that complex environments have been around a
lot longer than chef or puppet, and the opinion that I'm not quite sure
that the newest, shiniest tools address various complex environments
I've seen over the years. Perhaps I over-reacted, but then I kind of
have that reaction to those who preach the "one true way."
Miles Fidelman
My fault. Frankly, I didn't know which was a more appropriate list.
Devops seems like the arena where sys admin tools are discussed these
days, but I couldn't quite tell from the archives which list was more
appropriate, so I posted to both. (Now if someone knows a good sys
admin list, rather than a devops list, I'd welcome a pointer. I know
where the network operations folks hang out, but not the sysops.)
NANOG, where even saying "Good morning" results in an argument that
lasts several days.
For straight sysadmin stuff, you could try the LOPSA mailing lists:
https://lopsa.org/mailinglists
- provide for editing/managing/versioning scripts (script = anything
that can be invoked at the command line)
- a library of control functions for use within scripts
- invoking scripts, combinations of scripts, pipelines of scripts (in
the Unix sense of pipes) - locally, remotely, across multiple machines
- remote script execution via ssh, rather than some kind of agent
- providing a simple database for keeping track of variables used by
scripts (e.g., IP addresses, DNS records for use by a provisioning
script) - that can be accessed from scripts
- accessing the above via cli, RESTful API, GUI
- cross-platform
- (nice-to-have) minimal environmental requirements (i.e., a step above
the unix shell, say the gnu buildtool suite rather than the JVM + a mass
of libraries, or the ruby ecosystem -- a self-configuring basic
environment would be nice, like perl+cpan)
A bunch of people on devops-toolchain pointed at rundeck and rerun, both of which look incredibly helpful, and
couple of folks this list pointed me at: https://bdsm.beginrescueend.com/
Essentially it's management framework for running shell scripts, written by Wayne Seguin at Engine Yard,
also the author of RVM (Ruby Version Manager). Information on the web site is just a bit disorganized,
but there's a pretty good manual in pdf, an introductory slideshow, and a comprehensive git repo.
Wayne just spent the morning walking me through it - an incredibly powerful tool, which I'm now going to
go off and use as I rebuild a couple of servers.
I expect that a combination of these tools, along with git and a simple database for tracking things like
IP numbers and DNS records is going to do the job nicely.
Miles Fidelman
Ernest
At 02:31 PM 3/23/2012, Scott Smith wrote:
>Why am I getting this twice on two seperate lists?
>
>On Fri, Mar 23, 2012 at 10:47 AM, Miles Fidelman
><<mailto:mfid...@meetinghouse.net>mfid...@meetinghouse.net> wrote:
>Thanks to all for input. I was looking for a tool that would
>provide for something like this:
>
>- provide for editing/managing/versioning scripts (script = anything
>that can be invoked at the command line)
>- a library of control functions for use within scripts
>- invoking scripts, combinations of scripts, pipelines of scripts (in
>the Unix sense of pipes) - locally, remotely, across multiple machines
>- remote script execution via ssh, rather than some kind of agent
>- providing a simple database for keeping track of variables used by
>scripts (e.g., IP addresses, DNS records for use by a provisioning
>script) - that can be accessed from scripts
>- accessing the above via cli, RESTful API, GUI
>- cross-platform
>- (nice-to-have) minimal environmental requirements (i.e., a step above
>the unix shell, say the gnu buildtool suite rather than the JVM + a mass
>of libraries, or the ruby ecosystem -- a self-configuring basic
>environment would be nice, like perl+cpan)
>
>
>A bunch of people on devops-toolchain pointed at rundeck and rerun,
>both of which look incredibly helpful, and
>couple of folks this list pointed me at:
><https://bdsm.beginrescueend.com/>https://bdsm.beginrescueend.com/
--AJ