MEETING: Tues, Oct. 2nd -- Creating agile infrastructures with AutomateIt
Igal Koshevoy
* Speaker: Igal Koshevoy, AutomateIt's author
* Audience: Technical managers, software engineers, system
administrators
* When: Tuesday, October 2nd, 7pm
* Where: CubeSpace, 622 SE Grand Ave, Portland OR 97214
* Directions: http://www.cubespacepdx.com/directions
Want to speed up your software development process? Improve the
quality of your services? Eliminate many of the risks that threaten
your efforts?
AutomateIt is an open-source tool for automating the setup and
maintenance of servers, applications and their dependencies. It
provides a surprisingly simple, yet powerful, way to manage files,
packages, services, networks, accounts, roles, templates and more.
It's easy to learn and use, yet provides the power tackle the toughest
tasks. It plays well with a variety of different tools, frameworks and
OSes.
For example, you can use AutomateIt to write recipes for simple tasks,
like setting up a PHP-based blog at a shared hosting provider so you
can easily rebuild or migrate it elsewhere. Or use it for more
ambitious tasks, like deploying and maintaining clusters of Ruby on
Rails applications servers running on different OSes with numerous
system-level packages that require unique configurations. Or use it to
refactor the system architecture of your Java EE app and switch to
different components in a safe, previewable manner by effortlessly
applying the same set of changes you validated in development VMs to
staging and eventually production servers. The possibilities are
endless because in addition to AutomateIt's many built-in features,
its extensible architecture makes it easy to adapt to your unique
needs.
For details and downloads, visit http://AutomateIt.org/
The presentation will cover the benefits of an agile infrastructure,
how it's helped real projects, present AutomateIt's features,
demonstrate AutomateIt in action, and briefly discuss why Ruby was a
great choice for implementing AutomateIt.
Speaker bio: Igal Koshevoy has over a decade of experience as a lead
software engineer creating companies' flagship applications, and
usually builds the infrastructure they rely on. He has automated the
setup and maintenance of hundreds of servers running dozens of OSes
for clients like Intel and Oracle.
Thanks: Presentation facilities are generously provided by CubeSpace.
They offer work stations, meeting rooms, and big office amenities to
people who would otherwise be working from their homes, coffee shops,
or wherever they can set up their laptops or use their cell phones.
http://www.cubespacepdx.com/
Duncan Beevers
about the needs that prompted the development of some of these
Managers and seeing how to wield them.
Ben Munat
We've been using puppet to maintain dozens of servers at my work and it's been
one of those "yeah it hurts but it's better than doing it all by hand" situations.
Puppet is a bit complex and can be a bit confounding a time. Very powerful and
flexible, but that can be overwhelming when you start out. Lots of ways to do
things means it's easy to make a mess.
After a cursory examination, AutomateIt looks more like Capistrano with some
extra sysadmin capabilities. Or would you put it closer to Puppet in power?
Ben
Igal Koshevoy
> Yeah, this does look very interesting, wish I could make the meeting.
to hear how folks are solving these sorts of problems.
> We've been using puppet to maintain dozens of servers at my work and it's been
> one of those "yeah it hurts but it's better than doing it all by hand" situations.
>
Great, I'm glad to see more people using automation tools. I tried hard
to create something that would reduce that pain so that more people can
start enjoying the benefits of automation.
> Puppet is a bit complex and can be a bit confounding a time. Very powerful and
> flexible, but that can be overwhelming when you start out. Lots of ways to do
> things means it's easy to make a mess.
>
That's part of why I decided to take my turn at reinventing this wheel. :)
I tried to make AutomateIt easy, familiar and powerful. I got rid of
many of these other tools' hassles and limitations, and created a tool
whose approach and syntax are instantly familiar to anyone that's
written programs on Unix. I hope this will make it easier for people to
learn and use effectively. It provides good built-in solutions with
reasonable defaults for many tasks, but also makes it easy for users to
create their own.
> After a cursory examination, AutomateIt looks more like Capistrano with some
> extra sysadmin capabilities. Or would you put it closer to Puppet in power?
>
AutomateIt is what you'd use to get a system ready before you can use
Capistrano to deploy your app. You'd also use AutomateIt to maintain the
system once your app is running and adapt it as the infrastructure's
needs change.
AutomateIt is arguably more powerful than Puppet and supports most of
its features. It also provides a more comfortable and productive
development environment with an easily extensible plugin architecture,
full-featured programming language, interactive shell, debugger, etc.
However, it doesn't have as many OS-specific drivers/providers, e.g.,
there's no Blastwave package manager support for Solaris yet, but this
can be added by a user in an hour without needing to fork the source or
write any hacks.
A comparison of AutomateIt versus shell, Cfengine and Puppet is
available at http://automateit.org/compare
I would REALLY appreciate it if someone that's used Puppet can email me
a critique of that section so I can provide an honest comparison.
-igal
Thomas Lockney
> A comparison of AutomateIt versus shell, Cfengine and Puppet is
> available at http://automateit.org/compare
>
> I would REALLY appreciate it if someone that's used Puppet can email me
> a critique of that section so I can provide an honest comparison.
One thing that's definitely not clear is how AutomateIt manages
collections of hosts. Both Puppet and CFengine include the idea of
classes of hosts where those classes are based on particular traits of
the systems you have to manage. I saw that your sw includes the tag
concept which appears to be similar, but I didn't see a way to apply
those tags in a declarative way based on the system configurations.
Thomas Lockney
> Puppet is a bit complex and can be a bit confounding a time. Very powerful and
> flexible, but that can be overwhelming when you start out. Lots of ways to do
> things means it's easy to make a mess.
While the constantly changing nature of Puppet and the fact that Luke
Kanies was learning Ruby as he went is a big issue, try using CFengine
sometime if you want some perspective on pain. ;~)
Actually, while I don't use it really currently, Puppet appears to be
getting better quickly as more people with actual Ruby development
experience are getting heavily involved. But that's mostly just my
10000 ft perspective.
> After a cursory examination, AutomateIt looks more like Capistrano with some
> extra sysadmin capabilities. Or would you put it closer to Puppet in power?
Yeah, that was kind of how it looked to me, too.
Igal Koshevoy
> This tool looks really cool Igal! I'm looking forward to hearing
> about the needs that prompted the development of some of these
> Managers and seeing how to wield them.
The automation project I did for your team was a big part of why I
decided that I'd had it with the currently available tools and that I
wanted to see if I could come up with a better solution. I wanted
something that companies like yours could easily learn and simple to
work with, yet force you to compromise on features or trap you into
something that couldn't adapt to your needs.
-igal
Igal Koshevoy
> One thing that's definitely not clear is how AutomateIt manages
> collections of hosts. Both Puppet and CFengine include the idea of
> classes of hosts where those classes are based on particular traits of
> the systems you have to manage.
AutomateIt uses "tags" to generalize what the other tools call "traits",
"classes" and "groups".
Search the tutorial for "config/tags.yml" to find the section that
describes how tags are used:
http://www.automateit.org/documentation/files/TUTORIAL_txt.html
More detailed instructions on automatically included traits, negation
and inclusion are also available:
http://www.automateit.org/documentation/classes/AutomateIt/TagManager.html
Does this answer your questions?
-igal
PS: I really appreciate your feedback and want to make this better.
Igal Koshevoy
While the constantly changing nature of Puppet and the fact that Luke Kanies was learning Ruby as he went is a big issue, try using CFengine sometime if you want some perspective on pain. ;~)
AutomateIt is feature-complete and I'm almost done converting a 3.2 KLOC Cfengine-based setup for a client. AutomateIt has had a comprehensive test suite from the beginning and I don't ship releases unless it passes.
After a cursory examination, AutomateIt looks more like Capistrano with some extra sysadmin capabilities. Or would you put it closer to Puppet in power?Yeah, that was kind of how it looked to me, too.
-igal
Thomas Lockney
> some
> extra sysadmin capabilities. Or would you put it closer to Puppet in power?
>
> Yeah, that was kind of how it looked to me, too.
>
> AutomateIt is a complete replacement for Cfengine and Puppet. From your
> cursory examination, what seemed to be missing? I'm not arguing, blaming or
> rationalizing -- I'd really appreciate it if you or someone else could point
> out what I'm saying that's giving off the wrong impression so I can fix the
> text. Thanks!
Don't get me wrong, I wasn't intending to be critical of it at all. In
fact, I like most of what I see, but I've only given AutomateIt a
quick look at best. As with anything, my statement was based as much
on what I saw in the documentation as anything.
For instance, there's more of a focus on the concept of a "project"
from what I saw. This may be my misunderstand/misinterpretation. But
it seems to take a different approach from what I'm used to in the
CFengine/Puppet-type environment. Where they take a an approach of
trying to bring the system to a known state, your approach appears to
be more about automated remote control and deployment. Granted, this
may not be the case at all, but that's what I see looking at the
tutorial, etc.
I'm certainly quite curious about your system and really hope I can
make it to the demo. Even if it doesn't cover the exact niche as
Puppet or whatever, it's got a lot of interesting capabilities that I
haven't seen directly addressed in other projects (maybe like some odd
blend of Puppet and Capistrano). I also am quite impressed by the
ability to embed it in Java environment -- seeing as I mostly code in
Java by day and am in the systems management field... there are some
possibilities I see here. ;~)
~thomas
--
Thomas Lockney | tloc...@gmail.com | http://opposable-thumbs.net
Igal Koshevoy
> Don't get me wrong, I wasn't intending to be critical of it at all.
really appreciate your comments.
> For instance, there's more of a focus on the concept of a "project"
> from what I saw. This may be my misunderstand/misinterpretation. But
> it seems to take a different approach from what I'm used to in the
> CFengine/Puppet-type environment. Where they take a an approach of
> trying to bring the system to a known state, your approach appears to
> be more about automated remote control and deployment. Granted, this
> may not be the case at all, but that's what I see looking at the
> tutorial, etc.
>
Ah, thanks for explaining that, I think I understand. AutomateIt does
all that and more, but differently.
For example, a hosting provider could use Puppet or Cfengine to manage a
machine. The tool defaults to having a single environment per machine or
site, stores recipes in root-owned paths, and manages these with daemons
running as root. It's difficult or impossible to get multiple instances
of such a tool to co-exist or run as non-root, which limits its utility.
In contrast, anyone can run AutomateIt to execute any recipe. For
example, that hosting provider could create a project in /etc/automateit
to manage their machine. A customer on this machine could then create a
project in ~/automate to manage their Drupal website. These two
AutomateIt instances co-exist, know nothing about each other, run with
different privileges, are managed by different people, and evolve on
their own timetables. AutomateIt doesn't need hard-coded paths or
daemons because it only takes a short command to invoke a recipe from a
shell, cron or via SSH.
I think this approach will make AutomateIt applicable for both simple
tasks and complex projects. Folks can start by writing little,
stand-alone recipes for performing basic tasks, and then combine and
scale these into a larger project that can manage entire data centers.
Does that clarify the situation?
> maybe like some odd blend of Puppet and Capistrano
That's a reasonable analogy.
Here's another: Cfengine and Puppet are like "make" that's been coerced
into treating a limited set of targets as commands. AutomateIt is like
"sh" that's been coerced into treating a limited set of commands as
targets.
Why did I reverse these? Managing systems with purely declarative
languages like Cfengine and Puppet is difficult, unnatural and limiting,
and is thus intended for a much smaller audience. I've only met two
other people that could write non-trivial recipes with these tools and
they cursed a lot when discussing the topic. I believe AutomateIt's
imperative approach is easy, familiar and powerful because it uses the
same conventions you'd use in any program or shell script, so the
potential audience and utility is much greater.
> I also am quite impressed by the ability to embed it in Java environment
Glad to hear it. Companies doing Java development are a big potential
market for such tools because they care a lot about quality,
availability and repeatability, so I definitely want to cater to their
needs.
Thanks again for the comments, I appreciate them!
-igal
Bryan Donovan
together at Lucky Lab or some other place? I'm not sure if I can make
the entire presentation tonight but would like to my Ruby/beer combo
fix. Also, if anyone is interested, they can talk to me about the job
posting I submitted a few days ago (http://groups.google.com/group/
pdxruby/browse_thread/thread/2d90bc4d9f9e80ea).
Thanks,
Bryan
Thomas Lockney
>
> I just want to confirm.. will there be an after-the-meeting get-
> together at Lucky Lab or some other place? I'm not sure if I can make
> the entire presentation tonight but would like to my Ruby/beer combo
> fix.
If so, I imagine it will be somewhere closer to CubeSpace. I'm not
sure what is in that area that would be appropriate, but I doubt
people will want to trek down to the LL.
Jason LaPier
couple more 2-3 blocks away:
http://urbandrinks.com/happyhour/usa/or/portland/southeast/today
- Jason L.
--
My Rails and Linux Blog: http://offtheline.net
Bryan Donovan
them:
Side Door 0.04 miles
Slow Bar 0.04 miles
My Father's Place 0.05 miles
Maiden in the Mist 0.11 miles
La Merde 0.13 miles
Montage 0.13 miles
Morrison Hotel 0.14 miles
Slammer Tavern 0.16 miles
Produce Row Cafe 0.18 miles
Noir 0.21 miles
Gil's Speakeasy 0.24 miles
Holocene 0.27 miles
Rontoms 0.31 miles
Clark Lewis 0.31 miles
B-side Tavern 0.31 miles
On Oct 2, 10:08 am, "Jason LaPier" <jason.lap...@gmail.com> wrote:
> I only see two places real close to SE Grand and Washington, and a
> couple more 2-3 blocks away:http://urbandrinks.com/happyhour/usa/or/portland/southeast/today
>
> - Jason L.
>
Jason LaPier
On 10/2/07, Bryan Donovan <bdo...@gmail.com> wrote:
>
> Here's a few places nearby, although I'm not familiar with most of
> them:
> Montage 0.13 miles
Good food, lousy beer selection - lots of space if it's not busy
> Produce Row Cafe 0.18 miles
Interesting place, kind of small, decent beer
> Rontoms 0.31 miles
This place is cool - very wide open space, lots of room and good beer
- Jason L.
Bryan Donovan
on UrbanDrinks at:
http://urbandrinks.com/happyhour/nearby?address=622+SE+Grand+Ave%2C+Portland%2C+OR+97214&within=10&limit=10
jason kirtland
The Side Door across the street works well for groups up to about
12, bigger if you don't mind spreading out. Plan B (was Acme) is a
little bit of a trek from CubeSpace but accommodates a crowd easily
if it's not too cold out.
-j
Sam Livingston-Gray
feeling *really* lazy, you could even catch the 14 on Morrison and
take it down 7th. (=
The Side Door stinks horribly of cigarette smoke, as I discovered when
I tried to go in last week. I couldn't even make it through the door!
Since any smoking venue will give me a horrible headache, I vote for
nonsmoking.
The few times I've been to Montage, it's been incredibly noisy. (Not
a veto, just an advisory.)
Maiden in the Mist is nice, if you like the kind of place with a
cocktail menu and baked brie. (And lest that be read as a jab, I *do*
like that kind of place.)
-Sam
On 10/2/07, jason kirtland <j...@discorporate.us> wrote: