Need some general advice about Cap/rbenv

[Roy Miller]

I spent the last two days trying to figure out how to make a my deploy to a Vagrant box run faster. It takes roughly 30 minutes. Not unexpected considering that I'm trying to create a box almost from bare metal (i.e., it has the OS and pretty much nothing else), but it's too slow for what I need.

Part of the process is using knife-solo to provision the box. I wrote a rake task for it, called with a "before" hook in Cap. It works just great, and Cap manages the entire deploy process, which is nice. The slowdown comes when I install Ruby. I'm installing directly from source. It works, but man, it's dog slow, even on a beefier AWS box. That one recipe takes 15-20 minutes to run for a fresh box.

So I experimented with getting rbenv working. It seems to take much less time to install Ruby. I have no idea why, but the time drops to about 5 minutes. Much better. Getting it to work with Cap was a little challenging, believe it or not, but I got it working -- until I hit a snag.

Part of my provisioning process is to set up the deploy user in an automated fashion. Cap doesn't complain when I don't use the capistrano-rbenv gem. As soon as I plug that in, however, the initial rbenv:validate check fails because ... the deploy user isn't there yet, of course, and rbenv says it can't authenticate.

So I'm stuck. If I don't use rbenv with Cap, the Ruby install takes forever. If I use it, I can't deploy until the deploy user is there, and it's not there until after I provision the box. Catch-22.

Does anyone have a suggestion about how to escape the conundrum?

[Lee Hambley]

I spent the last two days trying to figure out how to make a my deploy to a Vagrant box run faster. It takes roughly 30 minutes. Not unexpected considering that I'm trying to create a box almost from bare metal (i.e., it has the OS and pretty much nothing else), but it's too slow for what I need.

​The short answer: Don't.

The longer, and more helpful one: If you start from a naked Ubuntu (or similar) base box, you're going to waste a lot of time, all the time setting the box up. The Vagrant author also produces a tool called Packer (http://packer.io/), packer (example manifest and etc here: https://github.com/capistrano/packer) allows you to easily build a base box for Vagrant (amongst other things)​

​The linked Packer template won't install Ruby (check `./scripts/`), but you have a script for that already

 

Part of the process is using knife-solo to provision the box. I wrote a rake task for it, called with a "before" hook in Cap. It works just great, and Cap manages the entire deploy process, which is nice. The slowdown comes when I install Ruby. I'm installing directly from source. It works, but man, it's dog slow, even on a beefier AWS box. That one recipe takes 15-20 minutes to run for a fresh box.

​You can also use knife solo to provision the box with Packer.

 

So I experimented with getting rbenv working. It seems to take much less time to install Ruby. I have no idea why, but the time drops to about 5 minutes. Much better. Getting it to work with Cap was a little challenging, believe it or not, but I got it working -- until I hit a snag.

​The time drops, because those tools will install a binary packaged managed by their communities, if one is found. They also almost certainly install less extensions to Ruby than the script I gave you (OpenSSL.) Also, you trade 10 minutes of installation time, once with 10 minutes of debugging every time you try and deploy/automate anything.

 

Part of my provisioning process is to set up the deploy user in an automated fashion. Cap doesn't complain when I don't use the capistrano-rbenv gem. As soon as I plug that in, however, the initial rbenv:validate check fails because ... the deploy user isn't there yet, of course, and rbenv says it can't authenticate.

​Right, that's why we discourage the use of Capistrano for *provisioning*, Cap excels at short, rapid fire processes. Provisioning is anything but.

 

So I'm stuck. If I don't use rbenv with Cap, the Ruby install takes forever. If I use it, I can't deploy until the deploy user is there, and it's not there until after I provision the box. Catch-22.

tl;dr: Use Packer.​​

Hope that helps Roy.

[Roy Miller]

I get your point, Lee. I don't see what I'm doing as making Capistrano responsible for provisioning (knife-solo does that), it's just kicking the process off as a prerequisite to doing the actual deploy. But I understand how one could see "Capistrano drives it" as making Cap do too much.

It certainly does take a lot of time building up from a bare box. I've already used Packer, so I'm familiar with it and I already know how to provision a box like what I want, Ruby and all. So I'll probably switch over to using the Packer AWS builder and chef-solo provisioner. That should accomplish the same goal. As you mention, the debugging time should drop, too.

Going that route also will let me avoid installing rbenv and using the Cap/rbenv integration. It works fine, no complaints, but it'll be unnecessary for what I'm trying to accomplish. If I need to switch Rubies (the primary purpose of rbenv), I'll re-provision the box. They're supposed to be Phoenix servers anyway.

Thanks for the advice.

[Lee Hambley]

It's always a fine balance, definitely there's something our industry has to fine it's way with. Years ago the unit of deployment was "bare metal boxes", recently it's become "diffs from a source control mechanism"… but the less we deploy, the more we assume about what came before, and I so I expect our devops movement towards more atomic deployments of assets. Probably that will be binary-identical virtual machines. (but then of course, we have to improve the way we handle configuration and consensus) 

We're not there yet, but for now, Packer provisioning my own nearly immobile components, and relying on a combination of nfs and etcd are giving me the flexibility that I need.

- Cheers

Lee Hambley

--

http://lee.hambley.name/

+49 (0) 170 298 5667

--
You received this message because you are subscribed to the Google Groups "Capistrano" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capistrano+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/capistrano/2475811c-dd4e-4bd0-93d7-784206111163%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Roy Miller]

Yep, makes sense to me.

The work I've done to figure out the approach I was using definitely isn't wasted. I learned a ton about Cap, knife-solo, and rake specfically, and provisioning and deployment in general. Now I'll shift to figuring out how to reuse my recipes via Packer instead.

One great thing about knife-solo is that it integrates with Berkshelf to resolve cookbook dependencies when you "knife solo cook". Very nice feature. I won't be doing that anymore, but I'll see if I can figure out how to have my Packer provisioning hook into that somehow to save headache. It might be as easy as using a script provisioner to fire "berks install" and prepare the cookbooks. I'll find out.

Thanks again for the advice. And from now on I'll post all Cap-related questions here instead of potentially putting non-issues on a particular tool's issues list at GitHub (e.g., capistrano/rbenv).

[Lee Hambley]

One great thing about knife-solo is that it integrates with Berkshelf to resolve cookbook dependencies when you "knife solo cook". Very nice feature. I won't be doing that anymore, but I'll see if I can figure out how to have my Packer provisioning hook into that somehow to save headache. It might be as easy as using a script provisioner to fire berks install and prepare the cookbooks. I'll find out.

I have a Makefile for my Packer build (make, make clean, etc) which I use to kick off things like Berkshelf and friends when I'm using them.

Lee Hambley

--

http://lee.hambley.name/

+49 (0) 170 298 5667

To view this discussion on the web, visit https://groups.google.com/d/msgid/capistrano/2b0d26b7-8214-4735-a8c1-d08164098190%40googlegroups.com.

[Roy Miller]

Sounds like a good idea. I don't know anything about Makefiles, so I'll have to research that and how it integrates with Packer. If you can point me to any good resources, feel free to share :)

[Lee Hambley]

Here's the makefile from Harrow.io:

vagrant: harrow-Debian_64.box

-vagrant box remove $(basename $<)

vagrant box add $(basename $<) $<

files/supervisord.tar.gz: files/supervisord/*

tar -cvf $@ $(wildcard $<)

harrow-Debian_64.box: harrow-Debian_64.json files/supervisord.tar.gz $(wildcard script/*.sh) http/preseed.cfg

packer build $<

clean:

-rm harrow-Debian_64.box

.PHONY: clean

​Losely speaking the names on the left, before the first : are "targets" (tasks) the things after that semi colon are other targets, or files that are relied upon to build this target, and the indented lines (e.g. tar)​

http://lee.hambley.name/2013/01/07/gnu-make-still-great-for-workflow-control.html

​ find more here.​

Lee Hambley

--

http://lee.hambley.name/

+49 (0) 170 298 5667

To view this discussion on the web, visit https://groups.google.com/d/msgid/capistrano/90b1d950-b89b-473c-b81c-63066a3f8da6%40googlegroups.com.