cap deploy failing sshkit::Runner no such file or directory - archive.tar.gz

383 views
Skip to first unread message

deadb...@gmail.com

unread,
Feb 18, 2015, 12:39:31 PM2/18/15
to capis...@googlegroups.com
Versions:
  • Ruby 1.9.3
  • Capistrano 3.2.1
  • Rake / Rails / etc
Platform:
  • Working on.... centos 6
  • Deploying to... 

I'm new to Capistrano, though very familiar with unix scripting, ssh, etc..  I don't understand why cap deploy is failing, see output below.  I've got cap deploy:pull working, and part of cap deploy works, it uploads the right files to the releases/<release>/ directory.  Then it goes on to do something (I've no clue what it is trying to do) and it fails.

$ cat config/deploy.rb


# config valid only for Capistrano 3.1

lock '3.2.1'

require "capistrano/copy.rb"


set :log_level, :debug

set :servicename, "XYZ Service"

all_servers =  %w{10.11.11.11}

set :app_name,     'nrpe'

set :deploy_to,    '/tmp/blah/' + fetch(:app_name)


set :business_name, 'our.testing'  

set :deploy_from,  '/srv/props/' + fetch(:business_name) + '/' + fetch(:app_name) + '/app'

# This is where we rsync the code/config from

set :repo_base,    '/var/tmp'

set :repo_url,     fetch(:repo_base) + '/' + fetch(:app_name) + "/"

set :copy_cache, false

set :copy_compression, :zip


# from https://github.com/wercker/capistrano-scm-copy

# Default value for :scm is :git

set :scm, :copy



namespace :deploy do


  desc "Show us the uname"

  task :uname do

    on all_servers do |host|

      puts capture(:'uname -a')

    end

  end




  desc "Go get our code"

  task :pull do


    run_locally do

      execute "rsync -avH '#{repo_url}' '#{fetch(:deploy_from)}'"

    end



  end


end




[cstewart@da1shal012new nrpe]$ cap production deploy

INFO[ca2c8c3d] Running /usr/bin/env mkdir -pv /tmp/blah/nrpe/shared /tmp/blah/nrpe/releases on 10.11.11.11

DEBUG[ca2c8c3d] Command: /usr/bin/env mkdir -pv /tmp/blah/nrpe/shared /tmp/blah/nrpe/releases

INFO[ca2c8c3d] Finished in 0.183 seconds with exit status 0 (successful).

DEBUG[969505ec] Running /usr/bin/env [ -f /tmp/blah/nrpe/current/REVISION ] on 10.11.11.11

DEBUG[969505ec] Command: [ -f /tmp/blah/nrpe/current/REVISION ]

DEBUG[969505ec] Finished in 0.007 seconds with exit status 1 (failed).

tar -cvzf archive.tar.gz Capfile app config lib

Capfile

app/

app/CHANGES

app/Capfile

app/LICENSE

app/GPL

app/mkmf.log

app/OPENSSL

app/bin/

app/bin/dir_fresh.sh

app/bin/nrpe.cfg

app/bin/nrpe.cfg~

app/bin/ssleay32.dll

app/bin/nrpe_nt.log

app/bin/fraudtest.xml

app/bin/libeay32.dll

app/bin/NRPE_NT.exe

app/bin/test.cmd

app/bin/diskspace_check.sh

app/bin/dir_size.sh

app/bin/dbnstest.sh

app/INSTALL

config/

config/deploy/

config/deploy/staging.rb

config/deploy/production.rb

config/deploy.rb.1

config/deploy.rb

lib/

lib/capistrano/

lib/capistrano/tasks/

tar -cvzf archive.tar.gz Capfile app config lib

Capfile

app/

app/CHANGES

app/Capfile

app/LICENSE

app/GPL

app/mkmf.log

app/OPENSSL

app/bin/

app/bin/dir_fresh.sh

app/bin/nrpe.cfg

app/bin/nrpe.cfg~

app/bin/ssleay32.dll

app/bin/nrpe_nt.log

app/bin/fraudtest.xml

app/bin/libeay32.dll

app/bin/NRPE_NT.exe

app/bin/test.cmd

app/bin/diskspace_check.sh

app/bin/dir_size.sh

app/bin/dbnstest.sh

app/INSTALL

config/

config/deploy/

config/deploy/staging.rb

config/deploy/production.rb

config/deploy.rb.1

config/deploy.rb

lib/

lib/capistrano/

lib/capistrano/tasks/

==> release_path: /tmp/blah/nrpe/releases/20150218171024 is created on all roles <==

INFO[b90a47db] Running /usr/bin/env mkdir -p /tmp/blah/nrpe/releases/20150218171024 on 10.11.11.11

DEBUG[b90a47db] Command: /usr/bin/env mkdir -p /tmp/blah/nrpe/releases/20150218171024

INFO[b90a47db] Finished in 0.008 seconds with exit status 0 (successful).

DEBUG[28d3e164] Running /usr/bin/env mktemp on 10.11.11.11

DEBUG[28d3e164] Command: /usr/bin/env mktemp

DEBUG[28d3e164] /tmp/tmp.bAgr4auZbV

DEBUG[28d3e164] Finished in 0.018 seconds with exit status 0 (successful).

DEBUGUploading archive.tar.gz 0.0%

DEBUGUploading archive.tar.gz 2.69%

DEBUGUploading archive.tar.gz 5.37%

DEBUGUploading archive.tar.gz 8.06%

DEBUGUploading archive.tar.gz 10.74%

DEBUGUploading archive.tar.gz 13.43%

DEBUGUploading archive.tar.gz 16.11%

DEBUGUploading archive.tar.gz 18.8%

DEBUGUploading archive.tar.gz 21.49%

DEBUGUploading archive.tar.gz 24.17%

DEBUGUploading archive.tar.gz 26.86%

INFOUploading archive.tar.gz 29.54%

DEBUGUploading archive.tar.gz 32.23%

DEBUGUploading archive.tar.gz 34.91%

DEBUGUploading archive.tar.gz 37.6%

INFOUploading archive.tar.gz 40.28%

DEBUGUploading archive.tar.gz 42.97%

DEBUGUploading archive.tar.gz 45.66%

DEBUGUploading archive.tar.gz 48.34%

DEBUGUploading archive.tar.gz 51.03%

DEBUGUploading archive.tar.gz 53.71%

DEBUGUploading archive.tar.gz 56.4%

DEBUGUploading archive.tar.gz 59.08%

DEBUGUploading archive.tar.gz 61.77%

DEBUGUploading archive.tar.gz 64.46%

DEBUGUploading archive.tar.gz 67.14%

INFOUploading archive.tar.gz 69.83%

DEBUGUploading archive.tar.gz 72.51%

DEBUGUploading archive.tar.gz 75.2%

DEBUGUploading archive.tar.gz 77.88%

DEBUGUploading archive.tar.gz 80.57%

DEBUGUploading archive.tar.gz 83.25%

DEBUGUploading archive.tar.gz 85.94%

DEBUGUploading archive.tar.gz 88.63%

DEBUGUploading archive.tar.gz 91.31%

DEBUGUploading archive.tar.gz 94.0%

DEBUGUploading archive.tar.gz 96.68%

DEBUGUploading archive.tar.gz 99.37%

INFOUploading archive.tar.gz 100.0%

INFO[943be656] Running /usr/bin/env tar -xzf /tmp/tmp.bAgr4auZbV -C /tmp/blah/nrpe/releases/20150218171024 on 10.11.11.11

DEBUG[943be656] Command: /usr/bin/env tar -xzf /tmp/tmp.bAgr4auZbV -C /tmp/blah/nrpe/releases/20150218171024

INFO[943be656] Finished in 0.027 seconds with exit status 0 (successful).

INFO[cacf55f0] Running /usr/bin/env rm /tmp/tmp.bAgr4auZbV on 10.223.132.247

DEBUG[cacf55f0] Command: /usr/bin/env rm /tmp/tmp.bAgr4auZbV

INFO[cacf55f0] Finished in 0.006 seconds with exit status 0 (successful).

==> release_path: /tmp/blah/nrpe/releases/20150218171024 is created on all roles <==

INFO[a5717de9] Running /usr/bin/env mkdir -p /tmp/blah/nrpe/releases/20150218171024 on 10.11.11.11

DEBUG[a5717de9] Command: /usr/bin/env mkdir -p /tmp/blah/nrpe/releases/20150218171024

INFO[a5717de9] Finished in 0.007 seconds with exit status 0 (successful).

DEBUG[69ffbd93] Running /usr/bin/env mktemp on 10.11.11.11

DEBUG[69ffbd93] Command: /usr/bin/env mktemp

DEBUG[69ffbd93] /tmp/tmp.hvY0EB454U

DEBUG[69ffbd93] Finished in 0.006 seconds with exit status 0 (successful).

cap aborted!

SSHKit::Runner::ExecuteError: Exception while executing on host 10.11.11.11: No such file or directory - archive.tar.gz


Errno::ENOENT: No such file or directory - archive.tar.gz


Tasks: TOP => copy:create_release => copy:deploy

(See full trace by running task with --trace)

The deploy has failed with an error: #<SSHKit::Runner::ExecuteError: Exception while executing on host 10.11.11.11  No such file or directory - archive.tar.gz>



-deadbeef


Lee Hambley

unread,
Feb 18, 2015, 12:49:37 PM2/18/15
to Capistrano
archive.tar.gz must be part of your own construction, I guess @deadbeef? Obviously one of your servers doesn't have the file, I might suggest that you have a different role set on the task that uploads the file, and the task that expects it to be there to extract it.

--
You received this message because you are subscribed to the Google Groups "Capistrano" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capistrano+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/capistrano/6b998dca-c3b9-4928-a3b3-2ba1d821d37d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

deadb...@gmail.com

unread,
Feb 18, 2015, 2:00:45 PM2/18/15
to capis...@googlegroups.com
Lee,

your comment of "must be part of your own construction" at first had me thinking but wait I've posted the entire contents of my deploy.rb and I'm not up to speed enough with capistrano (and have never done anything with Ruby either) so it wasn't something I did.  Turns out that file name is coming from the plugin I found https://github.com/wercker/capistrano-scm-copy  as I need to duplicate the behavior of cap2 and not use git/svn.  Just need cap to upload the contents of a local-directory to the remote servers, instead of pulling from a repo, and do the normal symlink stuff, rollback, etc...  Damn readme of the plugin doesn't talk about this archive file at all.  Thus my confusion.  Am I right that cap 3 won't just upload a local directory structure?

yeah deadbeaf via - wikipedia - DEADBEEF was originally used to mark newly allocated areas of memory that had not yet been initialized—when scanning a memory dump, it is easy to see the DEADBEEF  

-deadbeef

Lee Hambley

unread,
Feb 19, 2015, 7:31:45 AM2/19/15
to Capistrano
Everyone always overengineers this, look at this blog post I wrote which does exactly what you need: http://lee.hambley.name/2013/06/11/using-capistrano-v3-with-chef.html Fewer tools, less code, easier to follow. And 100% less plugins and magic.

deadb...@gmail.com

unread,
Feb 19, 2015, 12:01:44 PM2/19/15
to capis...@googlegroups.com
I've read your posting and it is making some sense.  If using the file/FileList task, what value should :scm be set to such that a 'cap deploy'  does not try to run git?

-deadbeef

Lee Hambley

unread,
Feb 19, 2015, 12:22:57 PM2/19/15
to Capistrano
The post I linked replaces Cap, since if you're not using an SCM you lose most of the benefit of Cap. If you need a list of files throwing into a tarball and uploading, on more than one machine then you need to look at Rake+SSHKit which are two thirds of Capistrano, but Capistrano specifically adds rollbacks, plugins for Ruby, Rails, Migrations, changelogs/etc, it's all wasted on you if you're just doing tarball uploads though.

--
You received this message because you are subscribed to the Google Groups "Capistrano" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capistrano+...@googlegroups.com.

Jonathan Rochkind

unread,
Feb 19, 2015, 12:24:27 PM2/19/15
to capis...@googlegroups.com
Even if you're doing tarball uploads, mightn't you still want rollbacks,
migrations, changelogs, etc? I understand that Cap might not support
that, but it's not obvious to me that it would be 'wasted' if it did!

On 2/19/15 12:22 PM, Lee Hambley wrote:
> The post I linked replaces Cap, since if you're not using an SCM you
> lose most of the benefit of Cap. If you need a list of files throwing
> into a tarball and uploading, on more than one machine then you need to
> look at Rake+SSHKit which are two thirds of Capistrano, but Capistrano
> specifically adds rollbacks, plugins for Ruby, Rails, Migrations,
> changelogs/etc, it's all wasted on you if you're just doing tarball
> uploads though.
>
> Lee Hambley
> http://lee.hambley.name/
> +49 (0) 170 298 5667
>
> On 19 February 2015 at 18:01, <deadb...@gmail.com
> <mailto:deadb...@gmail.com>> wrote:
>
> I've read your posting and it is making some sense. If using the
> file/FileList task, what value should :scm be set to such that a
> 'cap deploy' does not try to run git?
>
> -deadbeef
>
> On Thursday, February 19, 2015 at 6:31:45 AM UTC-6, Lee Hambley wrote:
>
> Everyone always overengineers this, look at this blog post I
> wrote which does exactly what you need:
> http://lee.hambley.name/__2013/06/11/using-capistrano-__v3-with-chef.html
> <http://lee.hambley.name/2013/06/11/using-capistrano-v3-with-chef.html>
> Fewer tools, less code, easier to follow. And 100% less plugins
> and magic.
>
> Lee Hambley
> http://lee.hambley.name/
> +49 (0) 170 298 5667 <tel:%2B49%20%280%29%20170%20298%205667>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Capistrano" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to capistrano+...@googlegroups.com
> <mailto:capistrano+...@googlegroups.com>.
> <https://groups.google.com/d/msgid/capistrano/647a1381-7cef-439a-8685-1ea1e71589c4%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Capistrano" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to capistrano+...@googlegroups.com
> <mailto:capistrano+...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/capistrano/CAN_%2BVLUyQskkLRO9FRkwo0iYEiERREoCL6EEqwAjk2D17thH7g%40mail.gmail.com
> <https://groups.google.com/d/msgid/capistrano/CAN_%2BVLUyQskkLRO9FRkwo0iYEiERREoCL6EEqwAjk2D17thH7g%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Lee Hambley

unread,
Feb 19, 2015, 12:27:57 PM2/19/15
to Capistrano
Well, that might be true, but in this case I think it's a justified statement :) As I understood it, he's trying to use a plugin I'm not familiar with to remove functionality he doesn't need from Cap, usually "cap does more than I want" means I ought to drop down and use Rake+SSHKit. (Although, admittedly, until we get around to *actually* making the SCM optional, there may be valid, mature workflows which mandate the lack of an SCM, and require rollbacks, bundler, rvm, etc) I don't think that's the case here though. And, if it is, I can't help him since I don't know what the no-scm plugin is doing wrong in this case (seems like not even uploading the tarball from what I can see)
--
You received this message because you are subscribed to the Google
Groups "Capistrano" group.
To unsubscribe from this group and stop receiving emails from it, send
--
You received this message because you are subscribed to the Google Groups "Capistrano" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capistrano+unsubscribe@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/capistrano/54E61C49.3020703%40jhu.edu.

deadb...@gmail.com

unread,
Feb 19, 2015, 1:43:57 PM2/19/15
to capis...@googlegroups.com
The system that is in place now uses cap (v2) and uses rollbacks and simply copies the contents of . to |host|.  I found when I tried that same setup with cap (v3), it broke as cap now requires the use of a SCM which was never used in the current setup.

I tried that plugin to add functionality to cap, that is the ability to simply upload <something> to |host| instead of using git.  I was hoping that cap would then continue on with creating releases, symlinks, rollbacks, etc.. When I read the blog post you wrote, I removed the plugin and tried again, using native cap without extras and ran into the problem of cap requiring SCM.

If I understand what you wrote correctly below, cap must have an SCM. Which I believe is different from cap 2, correct?

The plugin does upload the archive.tar.gz file, but it appears to be doing it twice.  The file is uploaded and expanded in .../releases/2015.../ as it should be. (See log on previous post.)  In its .rake file I see

on roles(tar_roles) do

      # Make sure the release directory exists

      puts "==> release_path: #{release_path} is created on #{tar_roles} roles <=="


The last lines output is shown twice when I run a simple cap deploy, which I don't understand. In this simple test setup, there is only a single target so not sure why 2x.  I don't know if the problem lies with the plugin or cap or my config/deploy.rb file.

I maybe off but it looks like the copy action is being called twice.  Which I suspect has something to do with tasks and dependencies, but since I don't understand cap much I could be off (and I know even less about ruby, other than I seriously dislike its syntax as it is not like anything I'm used to using. :) )   

The apps being deployed are not ruby apps.  The |host| targets do not have access to the repo (which isn't git/svn/hg anyway). The machine where cap is run from does have access to the "repo" and copies the files locally, and then the files can be uploaded to the targets.

Given the scenario I'm dealing with, what do the capistrano experts advise?  I could even live with a modified version of rollbacks where backup tarballs are kept on the host where cap runs from. Thinking that if I have to use Rake+SSHKit, then maybe the way to go would be create a task that creates a local tarball as a backup first.

Thank you,
-deadbeef
--
You received this message because you are subscribed to the Google
Groups "Capistrano" group.
To unsubscribe from this group and stop receiving emails from it, send
--
You received this message because you are subscribed to the Google Groups "Capistrano" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capistrano+...@googlegroups.com.

Lee Hambley

unread,
Feb 19, 2015, 3:11:23 PM2/19/15
to Capistrano
Alright, so sounds like you need Capistrano after all. (Welcome back ;-)) so the obvious next step would be for you to share (ideally a Github gist, or pastie, or something *not* inline in the email) your configuration. We'll need your Capfile, deploy.rb, and your stage (./config/deploy/________.rb) file to figure out what's not working. I understand your need for privacy, but if you butcher too much of the host config you might butcher out the detail that we need to diagnose the issue. If you prefer feel free to mail that stuff to Jonathan and I off-list, and whatever the solution, we can post a follow up to the list maintaining your privacy once we diagnose the issue.

You will need the no-scm plugin, we just have to look a little harder to find out why it's not working.

deadb...@gmail.com

unread,
Feb 19, 2015, 3:29:56 PM2/19/15
to capis...@googlegroups.com
Thank you Lee.  See http://pastie.org/9963632

I've included the copy.rake file too, just in case you need it.

Lee Hambley

unread,
Feb 19, 2015, 3:53:06 PM2/19/15
to Capistrano
Alright, so I looked into that - I'd suggest a couple of thins, first of all take Capistrano and SSHKit from Master, if you don't have a Gemfile, you might need to read this, http://bundler.io/v1.3/gemfile.html if you do have a Gemfile, change the `gem "capistrano"` line to something like this:

gem "capistrano", github: "capistrano/capistrano"
gem "sshkit", github: "capistrano/sshkit"

This will get the latest bleeding edge pre-release versions, which are scheduled for release soon (read: when I get half a day to write the changelogs, and do some final testing)

Otherwise, since you only *have* one server that I can see defined here, I can't imagine what is going wrong.

roles(:all) is a special case which should match any server defined with either role() or server(), on #294 you can see that the copy.rake is checking to see if you specified a special role name for `tar_roles` (you didn't), the 2nd argument to fetch is the default, :all…  so you should be in good hands so far.

I'd suggest the following, add and run this, and let me see the output:

task :debug_tar_roles do
  on roles(fetch(:tar_roles, :all)) do |h|
    puts "Host Obj: #{h}"
    info "#{capture(:hostname)}: #{capture(:uptime)}"
  end
end

This should distil the issue here, which is `roles()` all doesn't seem to be doing what we think… beyond that, come back, share the log, and in the meantime i'll pick more thoroughly through your pastie/original log. (knowing now that you only have one server)

deadb...@gmail.com

unread,
Feb 20, 2015, 10:53:29 AM2/20/15
to capis...@googlegroups.com

$ cap production debug_tar_roles

Host Obj: 10.223.132.247

DEBUG[aabe7db0] Running /usr/bin/env hostname on 10.223.132.247

DEBUG[aabe7db0] Command: /usr/bin/env hostname

DEBUG[aabe7db0] localhost.localdomain

DEBUG[aabe7db0] Finished in 0.184 seconds with exit status 0 (successful).

DEBUG[4d67cb3d] Running /usr/bin/env uptime on 10.223.132.247

DEBUG[4d67cb3d] Command: /usr/bin/env uptime

DEBUG[4d67cb3d 09:44:43 up 140 days,  8:34,  2 users,  load average: 0.00, 0.01, 0.05

DEBUG[4d67cb3d] Finished in 0.009 seconds with exit status 0 (successful).

INFOlocalhost.localdomain: 09:44:43 up 140 days,  8:34,  2 users,  load average: 0.00, 0.01, 0.05

deadb...@gmail.com

unread,
Feb 20, 2015, 11:03:53 AM2/20/15
to capis...@googlegroups.com
I'm still reading about Gemfile and how I use it, etc., like I said no ruby experience what so ever and the app is not a ruby app. I got the Gemfile created and put in the lines you suggested, but trying to figure out where to go from there.

Lee Hambley

unread,
Feb 20, 2015, 11:38:22 AM2/20/15
to Capistrano
Alright, from the output you can see that for some reason (and I don't see why in your code that you posted) that the command is being run *twice*. This means somewhere you have the same IP address being setup twice, maybe in Capfile, and your production.rb, or something? Search for it, and make sure it only happens once.

If you have the Gemfile with those contentns:

$ gem install bundler
$ bundle install
$ bundle exec cap whatever:task:you:want:to:execute

This allows you to use pre-release versions of gems, etc, it will also generate a Gemfile.lock, which you may or may not to choose to commit into your SCM and then encourage your colleagues to use Bundler, then they don't get bitten if we release a new (incompatible) version of Cap in the future, since bundler will always take the version in the .lock file.

deadb...@gmail.com

unread,
Feb 20, 2015, 2:59:34 PM2/20/15
to capis...@googlegroups.com
Lee,

turns out the plugin requires tar_roles to be defined, else I see this weirdness of what looks like trying to run twice.  I'd prefer to define the list of servers once. 

I tried the following but tar_roles does not get defined. Is there a way have a variable have the same contents as the value of a defined role?  Apparently a role is different than a variable.

role :OurServers, %w{10.223.132.247}

set :tar_roles, fetch(:OurServers)



Thank you

Lee Hambley

unread,
Feb 20, 2015, 4:10:35 PM2/20/15
to Capistrano

I think you’re confusing variables with roles.

Roles are groups of servers (anything Enumerable which looks like a hostname, or a server DSN, etc), variables are set and fetched with set() and fetch().

The plugin expects to fetch() a variable with the name of :tar_roles which contains a role name, this role name defaults to the special case :all.

Here the plugin looks up the variable :tar_roles.

Thus, if you have:

role :our_custom_role, %w{hostone hosttwo hostthree}
set :tar_roles, :our_custom_role

You’re telling the plugin what it seems to need to know, if you do what you are doing:

role :our_custom_role, %w{hostone hosttwo hostthree}
set :tar_roles, fetch(:our_custom_role)

You’re essentially setting :tar_roles to nothing, since fetch(:a_role_name) will find nothing.

--
You received this message because you are subscribed to the Google Groups "Capistrano" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capistrano+...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages