* Topic: Creating agile infrastructures with AutomateIt * Speaker: Igal Koshevoy, AutomateIt's author * Audience: Technical managers, software engineers, system administrators * When: Tuesday, October 2nd, 7pm * Where: CubeSpace, 622 SE Grand Ave, Portland OR 97214 * Directions: http://www.cubespacepdx.com/directions
Want to speed up your software development process? Improve the quality of your services? Eliminate many of the risks that threaten your efforts?
AutomateIt is an open-source tool for automating the setup and maintenance of servers, applications and their dependencies. It provides a surprisingly simple, yet powerful, way to manage files, packages, services, networks, accounts, roles, templates and more. It's easy to learn and use, yet provides the power tackle the toughest tasks. It plays well with a variety of different tools, frameworks and OSes.
For example, you can use AutomateIt to write recipes for simple tasks, like setting up a PHP-based blog at a shared hosting provider so you can easily rebuild or migrate it elsewhere. Or use it for more ambitious tasks, like deploying and maintaining clusters of Ruby on Rails applications servers running on different OSes with numerous system-level packages that require unique configurations. Or use it to refactor the system architecture of your Java EE app and switch to different components in a safe, previewable manner by effortlessly applying the same set of changes you validated in development VMs to staging and eventually production servers. The possibilities are endless because in addition to AutomateIt's many built-in features, its extensible architecture makes it easy to adapt to your unique needs.
The presentation will cover the benefits of an agile infrastructure, how it's helped real projects, present AutomateIt's features, demonstrate AutomateIt in action, and briefly discuss why Ruby was a great choice for implementing AutomateIt.
Speaker bio: Igal Koshevoy has over a decade of experience as a lead software engineer creating companies' flagship applications, and usually builds the infrastructure they rely on. He has automated the setup and maintenance of hundreds of servers running dozens of OSes for clients like Intel and Oracle.
Thanks: Presentation facilities are generously provided by CubeSpace. They offer work stations, meeting rooms, and big office amenities to people who would otherwise be working from their homes, coffee shops, or wherever they can set up their laptops or use their cell phones. http://www.cubespacepdx.com/
This tool looks really cool Igal! I'm looking forward to hearing about the needs that prompted the development of some of these Managers and seeing how to wield them.
On 9/27/07, Igal Koshevoy <i...@pragmaticraft.com> wrote:
> * Topic: Creating agile infrastructures with AutomateIt > * Speaker: Igal Koshevoy, AutomateIt's author > * Audience: Technical managers, software engineers, system > administrators > * When: Tuesday, October 2nd, 7pm > * Where: CubeSpace, 622 SE Grand Ave, Portland OR 97214 > * Directions: http://www.cubespacepdx.com/directions
> Want to speed up your software development process? Improve the > quality of your services? Eliminate many of the risks that threaten > your efforts?
> AutomateIt is an open-source tool for automating the setup and > maintenance of servers, applications and their dependencies. It > provides a surprisingly simple, yet powerful, way to manage files, > packages, services, networks, accounts, roles, templates and more. > It's easy to learn and use, yet provides the power tackle the toughest > tasks. It plays well with a variety of different tools, frameworks and > OSes.
> For example, you can use AutomateIt to write recipes for simple tasks, > like setting up a PHP-based blog at a shared hosting provider so you > can easily rebuild or migrate it elsewhere. Or use it for more > ambitious tasks, like deploying and maintaining clusters of Ruby on > Rails applications servers running on different OSes with numerous > system-level packages that require unique configurations. Or use it to > refactor the system architecture of your Java EE app and switch to > different components in a safe, previewable manner by effortlessly > applying the same set of changes you validated in development VMs to > staging and eventually production servers. The possibilities are > endless because in addition to AutomateIt's many built-in features, > its extensible architecture makes it easy to adapt to your unique > needs.
> The presentation will cover the benefits of an agile infrastructure, > how it's helped real projects, present AutomateIt's features, > demonstrate AutomateIt in action, and briefly discuss why Ruby was a > great choice for implementing AutomateIt.
> Speaker bio: Igal Koshevoy has over a decade of experience as a lead > software engineer creating companies' flagship applications, and > usually builds the infrastructure they rely on. He has automated the > setup and maintenance of hundreds of servers running dozens of OSes > for clients like Intel and Oracle.
> Thanks: Presentation facilities are generously provided by CubeSpace. > They offer work stations, meeting rooms, and big office amenities to > people who would otherwise be working from their homes, coffee shops, > or wherever they can set up their laptops or use their cell phones. > http://www.cubespacepdx.com/
Yeah, this does look very interesting, wish I could make the meeting.
We've been using puppet to maintain dozens of servers at my work and it's been one of those "yeah it hurts but it's better than doing it all by hand" situations.
Puppet is a bit complex and can be a bit confounding a time. Very powerful and flexible, but that can be overwhelming when you start out. Lots of ways to do things means it's easy to make a mess.
After a cursory examination, AutomateIt looks more like Capistrano with some extra sysadmin capabilities. Or would you put it closer to Puppet in power?
Duncan Beevers wrote: > This tool looks really cool Igal! I'm looking forward to hearing > about the needs that prompted the development of some of these > Managers and seeing how to wield them.
> On 9/27/07, Igal Koshevoy <i...@pragmaticraft.com> wrote: >> * Topic: Creating agile infrastructures with AutomateIt >> * Speaker: Igal Koshevoy, AutomateIt's author >> * Audience: Technical managers, software engineers, system >> administrators >> * When: Tuesday, October 2nd, 7pm >> * Where: CubeSpace, 622 SE Grand Ave, Portland OR 97214 >> * Directions: http://www.cubespacepdx.com/directions
>> Want to speed up your software development process? Improve the >> quality of your services? Eliminate many of the risks that threaten >> your efforts?
>> AutomateIt is an open-source tool for automating the setup and >> maintenance of servers, applications and their dependencies. It >> provides a surprisingly simple, yet powerful, way to manage files, >> packages, services, networks, accounts, roles, templates and more. >> It's easy to learn and use, yet provides the power tackle the toughest >> tasks. It plays well with a variety of different tools, frameworks and >> OSes.
>> For example, you can use AutomateIt to write recipes for simple tasks, >> like setting up a PHP-based blog at a shared hosting provider so you >> can easily rebuild or migrate it elsewhere. Or use it for more >> ambitious tasks, like deploying and maintaining clusters of Ruby on >> Rails applications servers running on different OSes with numerous >> system-level packages that require unique configurations. Or use it to >> refactor the system architecture of your Java EE app and switch to >> different components in a safe, previewable manner by effortlessly >> applying the same set of changes you validated in development VMs to >> staging and eventually production servers. The possibilities are >> endless because in addition to AutomateIt's many built-in features, >> its extensible architecture makes it easy to adapt to your unique >> needs.
>> The presentation will cover the benefits of an agile infrastructure, >> how it's helped real projects, present AutomateIt's features, >> demonstrate AutomateIt in action, and briefly discuss why Ruby was a >> great choice for implementing AutomateIt.
>> Speaker bio: Igal Koshevoy has over a decade of experience as a lead >> software engineer creating companies' flagship applications, and >> usually builds the infrastructure they rely on. He has automated the >> setup and maintenance of hundreds of servers running dozens of OSes >> for clients like Intel and Oracle.
>> Thanks: Presentation facilities are generously provided by CubeSpace. >> They offer work stations, meeting rooms, and big office amenities to >> people who would otherwise be working from their homes, coffee shops, >> or wherever they can set up their laptops or use their cell phones. >> http://www.cubespacepdx.com/
Ben Munat wrote: > Yeah, this does look very interesting, wish I could make the meeting.
Thanks. If you want to meet up sometime, drop me an email. I'm curious to hear how folks are solving these sorts of problems.
> We've been using puppet to maintain dozens of servers at my work and it's been > one of those "yeah it hurts but it's better than doing it all by hand" situations.
Great, I'm glad to see more people using automation tools. I tried hard to create something that would reduce that pain so that more people can start enjoying the benefits of automation.
> Puppet is a bit complex and can be a bit confounding a time. Very powerful and > flexible, but that can be overwhelming when you start out. Lots of ways to do > things means it's easy to make a mess.
That's part of why I decided to take my turn at reinventing this wheel. :)
I tried to make AutomateIt easy, familiar and powerful. I got rid of many of these other tools' hassles and limitations, and created a tool whose approach and syntax are instantly familiar to anyone that's written programs on Unix. I hope this will make it easier for people to learn and use effectively. It provides good built-in solutions with reasonable defaults for many tasks, but also makes it easy for users to create their own.
> After a cursory examination, AutomateIt looks more like Capistrano with some > extra sysadmin capabilities. Or would you put it closer to Puppet in power?
AutomateIt is what you'd use to get a system ready before you can use Capistrano to deploy your app. You'd also use AutomateIt to maintain the system once your app is running and adapt it as the infrastructure's needs change.
AutomateIt is arguably more powerful than Puppet and supports most of its features. It also provides a more comfortable and productive development environment with an easily extensible plugin architecture, full-featured programming language, interactive shell, debugger, etc. However, it doesn't have as many OS-specific drivers/providers, e.g., there's no Blastwave package manager support for Solaris yet, but this can be added by a user in an hour without needing to fork the source or write any hacks.
> I would REALLY appreciate it if someone that's used Puppet can email me > a critique of that section so I can provide an honest comparison.
One thing that's definitely not clear is how AutomateIt manages collections of hosts. Both Puppet and CFengine include the idea of classes of hosts where those classes are based on particular traits of the systems you have to manage. I saw that your sw includes the tag concept which appears to be similar, but I didn't see a way to apply those tags in a declarative way based on the system configurations.
> Puppet is a bit complex and can be a bit confounding a time. Very powerful and > flexible, but that can be overwhelming when you start out. Lots of ways to do > things means it's easy to make a mess.
While the constantly changing nature of Puppet and the fact that Luke Kanies was learning Ruby as he went is a big issue, try using CFengine sometime if you want some perspective on pain. ;~)
Actually, while I don't use it really currently, Puppet appears to be getting better quickly as more people with actual Ruby development experience are getting heavily involved. But that's mostly just my 10000 ft perspective.
> After a cursory examination, AutomateIt looks more like Capistrano with some > extra sysadmin capabilities. Or would you put it closer to Puppet in power?
Duncan Beevers wrote: > This tool looks really cool Igal! I'm looking forward to hearing > about the needs that prompted the development of some of these > Managers and seeing how to wield them.
Thanks, I appreciate it.
The automation project I did for your team was a big part of why I decided that I'd had it with the currently available tools and that I wanted to see if I could come up with a better solution. I wanted something that companies like yours could easily learn and simple to work with, yet force you to compromise on features or trap you into something that couldn't adapt to your needs.
Thomas Lockney wrote: > One thing that's definitely not clear is how AutomateIt manages > collections of hosts. Both Puppet and CFengine include the idea of > classes of hosts where those classes are based on particular traits of > the systems you have to manage.
Thanks for the critique.
AutomateIt uses "tags" to generalize what the other tools call "traits", "classes" and "groups".
Thomas Lockney wrote: > While the constantly changing nature of Puppet and the fact that Luke > Kanies was learning Ruby as he went is a big issue, try using CFengine > sometime if you want some perspective on pain. ;~)
Cfengine always got the job done for my projects, but I've only encountered two other people that could write non-trivial recipes with it because it's so awkward. Puppet has seemed promising for years, but every time that I evaluated it for a project, it was too broken and limited to be usable. I finally gave up waiting and decided to write something myself.
AutomateIt is feature-complete and I'm almost done converting a 3.2 KLOC Cfengine-based setup for a client. AutomateIt has had a comprehensive test suite from the beginning and I don't ship releases unless it passes.
>> After a cursory examination, AutomateIt looks more like Capistrano with some >> extra sysadmin capabilities. Or would you put it closer to Puppet in power?
> Yeah, that was kind of how it looked to me, too.
AutomateIt is a complete replacement for Cfengine and Puppet. From your cursory examination, what seemed to be missing? I'm not arguing, blaming or rationalizing -- I'd really appreciate it if you or someone else could point out what I'm saying that's giving off the wrong impression so I can fix the text. Thanks!
On 9/28/07, Igal Koshevoy <i...@pragmaticraft.com> wrote:
> After a cursory examination, AutomateIt looks more like Capistrano with > some > extra sysadmin capabilities. Or would you put it closer to Puppet in power?
> Yeah, that was kind of how it looked to me, too.
> AutomateIt is a complete replacement for Cfengine and Puppet. From your > cursory examination, what seemed to be missing? I'm not arguing, blaming or > rationalizing -- I'd really appreciate it if you or someone else could point > out what I'm saying that's giving off the wrong impression so I can fix the > text. Thanks!
Don't get me wrong, I wasn't intending to be critical of it at all. In fact, I like most of what I see, but I've only given AutomateIt a quick look at best. As with anything, my statement was based as much on what I saw in the documentation as anything.
For instance, there's more of a focus on the concept of a "project" from what I saw. This may be my misunderstand/misinterpretation. But it seems to take a different approach from what I'm used to in the CFengine/Puppet-type environment. Where they take a an approach of trying to bring the system to a known state, your approach appears to be more about automated remote control and deployment. Granted, this may not be the case at all, but that's what I see looking at the tutorial, etc.
I'm certainly quite curious about your system and really hope I can make it to the demo. Even if it doesn't cover the exact niche as Puppet or whatever, it's got a lot of interesting capabilities that I haven't seen directly addressed in other projects (maybe like some odd blend of Puppet and Capistrano). I also am quite impressed by the ability to embed it in Java environment -- seeing as I mostly code in Java by day and am in the systems management field... there are some possibilities I see here. ;~)
Thomas Lockney wrote: > Don't get me wrong, I wasn't intending to be critical of it at all.
No worries. I asked for feedback so I could make this better, and I really appreciate your comments.
> For instance, there's more of a focus on the concept of a "project" > from what I saw. This may be my misunderstand/misinterpretation. But > it seems to take a different approach from what I'm used to in the > CFengine/Puppet-type environment. Where they take a an approach of > trying to bring the system to a known state, your approach appears to > be more about automated remote control and deployment. Granted, this > may not be the case at all, but that's what I see looking at the > tutorial, etc.
Ah, thanks for explaining that, I think I understand. AutomateIt does all that and more, but differently.
For example, a hosting provider could use Puppet or Cfengine to manage a machine. The tool defaults to having a single environment per machine or site, stores recipes in root-owned paths, and manages these with daemons running as root. It's difficult or impossible to get multiple instances of such a tool to co-exist or run as non-root, which limits its utility.
In contrast, anyone can run AutomateIt to execute any recipe. For example, that hosting provider could create a project in /etc/automateit to manage their machine. A customer on this machine could then create a project in ~/automate to manage their Drupal website. These two AutomateIt instances co-exist, know nothing about each other, run with different privileges, are managed by different people, and evolve on their own timetables. AutomateIt doesn't need hard-coded paths or daemons because it only takes a short command to invoke a recipe from a shell, cron or via SSH.
I think this approach will make AutomateIt applicable for both simple tasks and complex projects. Folks can start by writing little, stand-alone recipes for performing basic tasks, and then combine and scale these into a larger project that can manage entire data centers.
Does that clarify the situation?
> maybe like some odd blend of Puppet and Capistrano
That's a reasonable analogy.
Here's another: Cfengine and Puppet are like "make" that's been coerced into treating a limited set of targets as commands. AutomateIt is like "sh" that's been coerced into treating a limited set of commands as targets.
Why did I reverse these? Managing systems with purely declarative languages like Cfengine and Puppet is difficult, unnatural and limiting, and is thus intended for a much smaller audience. I've only met two other people that could write non-trivial recipes with these tools and they cursed a lot when discussing the topic. I believe AutomateIt's imperative approach is easy, familiar and powerful because it uses the same conventions you'd use in any program or shell script, so the potential audience and utility is much greater.
> I also am quite impressed by the ability to embed it in Java environment
Glad to hear it. Companies doing Java development are a big potential market for such tools because they care a lot about quality, availability and repeatability, so I definitely want to cater to their needs.
I just want to confirm.. will there be an after-the-meeting get- together at Lucky Lab or some other place? I'm not sure if I can make the entire presentation tonight but would like to my Ruby/beer combo fix. Also, if anyone is interested, they can talk to me about the job posting I submitted a few days ago (http://groups.google.com/group/ pdxruby/browse_thread/thread/2d90bc4d9f9e80ea).
On 10/2/07, Bryan Donovan <bdo...@gmail.com> wrote:
> I just want to confirm.. will there be an after-the-meeting get- > together at Lucky Lab or some other place? I'm not sure if I can make > the entire presentation tonight but would like to my Ruby/beer combo > fix.
If so, I imagine it will be somewhere closer to CubeSpace. I'm not sure what is in that area that would be appropriate, but I doubt people will want to trek down to the LL.
On 10/2/07, Thomas Lockney <tlock...@gmail.com> wrote:
> On 10/2/07, Bryan Donovan <bdo...@gmail.com> wrote:
> > I just want to confirm.. will there be an after-the-meeting get- > > together at Lucky Lab or some other place? I'm not sure if I can make > > the entire presentation tonight but would like to my Ruby/beer combo > > fix.
> If so, I imagine it will be somewhere closer to CubeSpace. I'm not > sure what is in that area that would be appropriate, but I doubt > people will want to trek down to the LL.
Here's a few places nearby, although I'm not familiar with most of them: Side Door 0.04 miles Slow Bar 0.04 miles My Father's Place 0.05 miles Maiden in the Mist 0.11 miles La Merde 0.13 miles Montage 0.13 miles Morrison Hotel 0.14 miles Slammer Tavern 0.16 miles Produce Row Cafe 0.18 miles Noir 0.21 miles Gil's Speakeasy 0.24 miles Holocene 0.27 miles Rontoms 0.31 miles Clark Lewis 0.31 miles B-side Tavern 0.31 miles
On Oct 2, 10:08 am, "Jason LaPier" <jason.lap...@gmail.com> wrote:
> On 10/2/07, Thomas Lockney <tlock...@gmail.com> wrote:
> > On 10/2/07, Bryan Donovan <bdo...@gmail.com> wrote:
> > > I just want to confirm.. will there be an after-the-meeting get- > > > together at Lucky Lab or some other place? I'm not sure if I can make > > > the entire presentation tonight but would like to my Ruby/beer combo > > > fix.
> > If so, I imagine it will be somewhere closer to CubeSpace. I'm not > > sure what is in that area that would be appropriate, but I doubt > > people will want to trek down to the LL.
> Here's a few places nearby, although I'm not familiar with most of > them: > Side Door 0.04 miles > Slow Bar 0.04 miles > My Father's Place 0.05 miles > Maiden in the Mist 0.11 miles > La Merde 0.13 miles > Montage 0.13 miles > Morrison Hotel 0.14 miles > Slammer Tavern 0.16 miles > Produce Row Cafe 0.18 miles > Noir 0.21 miles > Gil's Speakeasy 0.24 miles > Holocene 0.27 miles > Rontoms 0.31 miles > Clark Lewis 0.31 miles > B-side Tavern 0.31 miles
> On Oct 2, 10:08 am, "Jason LaPier" <jason.lap...@gmail.com> wrote:
> > On 10/2/07, Thomas Lockney <tlock...@gmail.com> wrote:
> > > On 10/2/07, Bryan Donovan <bdo...@gmail.com> wrote:
> > > > I just want to confirm.. will there be an after-the-meeting get- > > > > together at Lucky Lab or some other place? I'm not sure if I can make > > > > the entire presentation tonight but would like to my Ruby/beer combo > > > > fix.
> > > If so, I imagine it will be somewhere closer to CubeSpace. I'm not > > > sure what is in that area that would be appropriate, but I doubt > > > people will want to trek down to the LL.
> On 10/2/07, Bryan Donovan <bdo...@gmail.com> wrote:
>> I just want to confirm.. will there be an after-the-meeting get- >> together at Lucky Lab or some other place? I'm not sure if I >> can make the entire presentation tonight but would like to my >> Ruby/beer combo fix.
> If so, I imagine it will be somewhere closer to CubeSpace. I'm not > sure what is in that area that would be appropriate, but I doubt > people will want to trek down to the LL.
The Side Door across the street works well for groups up to about 12, bigger if you don't mind spreading out. Plan B (was Acme) is a little bit of a trek from CubeSpace but accommodates a crowd easily if it's not too cold out.
"Trek?" Lucky Lab is only 0.7 miles from CubeSpace. If you're feeling *really* lazy, you could even catch the 14 on Morrison and take it down 7th. (=
The Side Door stinks horribly of cigarette smoke, as I discovered when I tried to go in last week. I couldn't even make it through the door! Since any smoking venue will give me a horrible headache, I vote for nonsmoking.
The few times I've been to Montage, it's been incredibly noisy. (Not a veto, just an advisory.)
Maiden in the Mist is nice, if you like the kind of place with a cocktail menu and baked brie. (And lest that be read as a jab, I *do* like that kind of place.)
-Sam
On 10/2/07, jason kirtland <j...@discorporate.us> wrote:
> > On 10/2/07, Bryan Donovan <bdo...@gmail.com> wrote:
> >> I just want to confirm.. will there be an after-the-meeting get- > >> together at Lucky Lab or some other place? I'm not sure if I > >> can make the entire presentation tonight but would like to my > >> Ruby/beer combo fix.
> > If so, I imagine it will be somewhere closer to CubeSpace. I'm not > > sure what is in that area that would be appropriate, but I doubt > > people will want to trek down to the LL.
> The Side Door across the street works well for groups up to about > 12, bigger if you don't mind spreading out. Plan B (was Acme) is a > little bit of a trek from CubeSpace but accommodates a crowd easily > if it's not too cold out.
|
