Re: [capistrano] slow `bundle:install` command during `deploy:finalize_update` task

[Lee Hambley]

Hi Tom,

The Bundler team manage the Capistrano integration, and this problem comes up time and time again, it's done this way so that your installed "RELEASE_DIRECTORY" is a completely isolated copy of your app.

You'd better raise the issue with the Bundler team, or at least check their faq/docs to see if what you are trying to do it already possible.

As for "reasons against" - just that you are going against the grain, and trying to "break" a reliability feature for speed, but reliable, fast, cheap, pick two.

- Lee

Lee Hambley

--

http://lee.hambley.name/

+49 (0) 170 298 5667

On 12 May 2013 03:02, TomPave <scritto...@gmail.com> wrote:

Hello,

I'm trying to improve the speed of my deploy script.

As it is now it works perfectly, but `bundle:install` is incredibly slow: it takes between 2 and 4 minutes every time.

I noticed that Capistrano doesn't use the default directory for the user who executes the rack app server: ~/.rvm/....

Rather, it installs the gems in the vendor folder of each release. for example:

/var/www/wonderingmachine/releases/20130512001352/vendor/bundle/ruby/2.0.0/gems/unicorn-4.6.2

After Capistrano has slowly run `bundle:install`, I can run it manually again and as expected it only takes few seconds (it doesn't need to install any new gem).

I have a Gemfile and a Gemfile.lock in my app, and they are both checked in the git repository. The Gemfile.lock doesn't change that often and I don't see why Capistrano should download and install all the gems each time.

I imagine that using the default gem directory would speed thing up, as Capistrano will need to install new gems only when I want it to update something (that is, when I commit an updated Gemfile.lock)

Is there any reason why I shouldn't do it?

Anyway, I see there is a variable names "bundle_dir".

Could you help me set it up?

    # other ENV variables are set as well

    #

    default_environment["GEM_HOME"]     = "/home/runner/.rvm/gems/ruby-2.0.0-p0"

    default_environment["GEM_PATH"]     = "/home/runner/.rvm/gems/ruby-2.0.0-p0:/home/runner/.rvm/gems/ruby-2.0.0-p0@global"

    set :bundle_dir, "....?"

Thanks a lot

--
--
* You received this message because you are subscribed to the Google Groups "Capistrano" group.
* To post to this group, send email to capis...@googlegroups.com
* To unsubscribe from this group, send email to capistrano+...@googlegroups.com For more options, visit this group at http://groups.google.com/group/capistrano?hl=en
---
You received this message because you are subscribed to the Google Groups "Capistrano" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capistrano+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Donovan Bray]

I wrote my own bundler recipes that essentially caches the last successful bundle.  That cache is then copied to the new timestamp release, and rebundled for the new release. As long as no deploy failure happens then the rebundled directory is copied back to the cache.

This also means a rollback is clean, because each timestamp release has the bundle that matches it's Gemfile.lock.

See these links from my cap-recipes project:

https://github.com/donnoman/cap-recipes/blob/master/lib/cap_recipes/tasks/bundler/install.rb

https://github.com/donnoman/cap-recipes/blob/master/lib/cap_recipes/tasks/bundler/hooks.rb

[Lee Hambley]

I wrote my own bundler recipes that essentially caches the last successful bundle.  That cache is then copied to the new timestamp release, and rebundled for the new release. As long as no deploy failure happens then the rebundled directory is copied back to the cache.

That sounds like something you should ask Bundler to adopt Donovan?

Lee Hambley

--

http://lee.hambley.name/

+49 (0) 170 298 5667

[TomPave]

Thank you for the suggestions, guys.

I see how trading reliability for speed might be an issue, but since I'm not working on a cluster of servers with some critical commercial app, I think I'll risk :-).

Also, thanks for the idea about caching the bundled gems directory, but that seems overkilling.

Still, it convinced me that using the default user directory (~/.rvm) is not such a great idea.

I eventually used:

set :bundle_dir, File.join(fetch(:shared_path), 'bundle')

...it installs everything in a shared folder that is used by any "current" release.

I just tried to deploy the app three times (with minor modifications to a private view), and after the first execution of `bundle:install` the others are very fast (6-10 seconds).

[Jonathan Rochkind]

On 5/11/2013 9:02 PM, TomPave wrote:
> I noticed that Capistrano doesn't use the default directory for the user
> who executes the rack app server: ~/.rvm/....
> Rather, it installs the gems in the vendor folder of each release. for
> example:
> /var/www/wonderingmachine/releases/20130512001352/vendor/bundle/ruby/2.0.0/gems/unicorn-4.6.2

This is not unique to Capistrano, but bundler-recommended setup for any
deployment install, with OR without Capistrano. It's one of the effects
of using "bundle --deployment", which is what bundler recommends in
deployment copies of an app.

I would recommend asking questions in a Bundler-related venue about why
bundler makes this decision/recommendation about deployment -- before
deciding to make a different decision yourself, understanding the
intention.

If you do want to make a different decision, it can quite possibly be
done simply with different or additional bundler arguments than
"--deployment" -- you probably don't need to do a lot of hacking, just
write a capistrano task that executes bundler with different arguments
that the bundler-recommended "--deployment".

But it probably doesn't make sense for Capistrano to ship with a task
that does anything other than what bundler recommends for deployment. If
you think what bundler recommends for deployment is incorrect, best take
that up with bundler.

Jonathan

[Jonathan Rochkind]

Or perhaps release as your own gem with an alternate bundler cap recipe,
if you're interested? It does sound like a good approach.

On 5/12/2013 11:15 AM, Lee Hambley wrote:
> I wrote my own bundler recipes that essentially caches the last
> successful bundle. That cache is then copied to the new timestamp
> release, and rebundled for the new release. As long as no deploy
> failure happens then the rebundled directory is copied back to the
> cache.
>
>
> That sounds like something you should ask Bundler to adopt Donovan?
>
> Lee Hambley
> --
> http://lee.hambley.name/
> +49 (0) 170 298 5667
>
>
> On 12 May 2013 15:36, Donovan Bray <donn...@gmail.com

> +49 (0) 170 298 5667 <tel:%2B49%20%280%29%20170%20298%205667>

>
>
> On 12 May 2013 03:02, TomPave <scritto...@gmail.com

> capis...@googlegroups.com <mailto:capis...@googlegroups.com>

> * To unsubscribe from this group, send email to
> capistrano+...@googlegroups.com

> <mailto:capistrano%2Bunsu...@googlegroups.com> For more

> options, visit this group at
> http://groups.google.com/group/capistrano?hl=en
> ---
> You received this message because you are subscribed to the
> Google Groups "Capistrano" group.
> To unsubscribe from this group and stop receiving emails
> from it, send an email to
> capistrano+...@googlegroups.com

> <mailto:capistrano%2Bunsu...@googlegroups.com>.

> For more options, visit
> https://groups.google.com/groups/opt_out.
>
>
>
> --
> --
> * You received this message because you are subscribed to the
> Google Groups "Capistrano" group.
> * To post to this group, send email to

> capis...@googlegroups.com <mailto:capis...@googlegroups.com>

> * To unsubscribe from this group, send email to
> capistrano+...@googlegroups.com

> <mailto:capistrano%2Bunsu...@googlegroups.com> For more

> options, visit this group at
> http://groups.google.com/group/capistrano?hl=en
> ---
> You received this message because you are subscribed to the
> Google Groups "Capistrano" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to capistrano+...@googlegroups.com

> <mailto:capistrano%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
> --
> --
> * You received this message because you are subscribed to the Google
> Groups "Capistrano" group.
> * To post to this group, send email to capis...@googlegroups.com

> <mailto:capis...@googlegroups.com>

> * To unsubscribe from this group, send email to
> capistrano+...@googlegroups.com

> <mailto:capistrano%2Bunsu...@googlegroups.com> For more options,

> visit this group at http://groups.google.com/group/capistrano?hl=en
> ---
> You received this message because you are subscribed to the Google
> Groups "Capistrano" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to capistrano+...@googlegroups.com

> <mailto:capistrano%2Bunsu...@googlegroups.com>.