Slow when running several cap at the same time

[Fajar Maulana Firdaus]

Hi,

I am working on a mini site that will run capistrano with different configuration. So I use combination of sidekiq and cap. Inside the sidekiq worker I call system "cap production deploy". But before that, I want to stress test the code with a simple rake task (see below). The task is just printing the application variable.

When I run once 

(1..1).each {|i| HardWorker.perform_async('test',i)}

the output is great because it's finished in 0.709 sec for each worker to finish
When I run five times like so:

(1..5).each {|i| HardWorker.perform_async('test',i)}

the output took double the time 1.55 sec for each worker to finish
And it gets worse when I run 15 times like so: 

(1..15).each {|i| HardWorker.perform_async('test',i)}

the output become 4.72 sec for each worker

So is there anything in particular that cause capistrano to run slower when invoked several time at the same time?

Thank you,

Versions:

• Ruby 2.0
• Capistrano 3.1.0
• Rake / Rails / etc 4.0

Platform:

• OSX 10.9

Logs:

• http://pastie.org/8566562
  

Files:

• Sidekiq worker:

# app/workers/hard_worker.rb
class HardWorker
  include Sidekiq::Worker


  def perform(name, count)
    puts "Doing hard work #{name} #{count}"
    system "cap stress stress:test"
  end
end

• Custom rake task:

namespace :stress do
  desc "Spit out variable application"
  task :test do
    puts "application variable => #{fetch(:application)}"
  end
end

• Capfile

set :user, "ubuntu"
set :group, "www-data"
#set :use_sudo, false


# Load DSL and Setup Up Stages
require 'capistrano/setup'


# Includes default deployment tasks
require 'capistrano/deploy'


# require rvm on remote server to run rake tasks
#require 'capistrano/rvm'


#require helpers
import 'helpers/checks.rb'


# Loads custom tasks from `lib/capistrano/tasks' if you have any defined.
Dir.glob('tasks/*.rake').each { |r| import r }

• deploy.rb

set :custom_variables, "SOMEVARIABLES"
set :format, :pretty
# set :log_level, :info
set :pty, true
set :keep_releases, 3

• Stage files stress.rb

set :stage, :stress
role :app, ["#{fetch(:user)}@jkt1.myserver.com"]
role :web, ["#{fetch(:user)}@jkt1.myserver.com"]
role :db, ["#{fetch(:user)}@jkt1.myserver.com"]

[Cynthia Kiser]

> So is there anything in particular that cause capistrano to run slower when invoked several time at the same time?

I could easily imagine you getting network or disk IO contention. What do your system level stats indicate?

--
Cynthia Kiser
cynthi...@gmail.com

[Fajar Maulana Firdaus]

Could you explain more on contention? How do I measure my system level stats ?