random error when parallel bundle install

[expxiaoli]

I use jenkins's master/slave to parallel run testcase written with rspec. I setup 8 executors in each slave, and each executor will run such command in the same time: gem install bundler & bundle install & rake rspec ... All executors share the same ruby gems

Random errors happens when it executes "bundle install", and all of them is about "cannot load such file --- <%file_name%>". This random error frequency is low (happens in 2% bundle install) and the <%file_name%> in errors is quite different: this time is "bundler/index", and that time can be another, for example, "bundler/vendor".

I am not sure whether it is caused by parallel running "bundle install". Does anyone meet similar problems?

Here is my environment:

The OS of each slave: CentOS release 6.3 (Final)

Ruby version: ruby 1.9.3p125 (2012-02-16 revision 34643)

Bundler version: 1.3.5

Rspec verison: 2.12.2

RVM version: 1.18.20 (stable)

Gem version: 1.8.25

Below is one of these random error:

/home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/source/rubygems.rb:165:in fetch_specs': cannot load such file -- bundler/index (LoadError) from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/source/rubygems.rb:67:inspecs' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/lazy_specification.rb:52:in__materialize__' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/spec_set.rb:88:inblock in materialize' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/spec_set.rb:85:in map!' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/spec_set.rb:85:inmaterialize' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/definition.rb:140:in missing_specs' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/installer.rb:72:inblock in run' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/ui.rb:96:in silence' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/installer.rb:69:inrun' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/installer.rb:14:in install' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/cli.rb:247:ininstall' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/vendor/thor/task.rb:27:in run' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/vendor/thor/invocation.rb:120:ininvoke_task' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/vendor/thor.rb:344:in dispatch' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/vendor/thor/base.rb:434:instart' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/bin/bundle:20:in block in <top (required)>' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/lib/bundler/friendly_errors.rb:3:inwith_friendly_errors' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/gems/bundler-1.3.5/bin/bundle:20:in <top (required)>' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/bin/bundle:19:inload' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/bin/bundle:19:in <main>' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/bin/ruby_noexec_wrapper:14:ineval' from /home/CORP/jenkins/.rvm/gems/ruby-1.9.3-p125@recommendation-test/bin/ruby_noexec_wrapper:14:in `'

[xavie...@rhnh.net]

On Wednesday, August 7, 2013 12:22:05 AM UTC-7, expxiaoli wrote:

I use jenkins's master/slave to parallel run testcase written with rspec. I setup 8 executors in each slave, and each executor will run such command in the same time: gem install bundler & bundle install & rake rspec ... All executors share the same ruby gems 

Random errors happens when it executes "bundle install", and all of them is about "cannot load such file --- <%file_name%>". This random error frequency is low (happens in 2% bundle install) and the <%file_name%> in errors is quite different: this time is "bundler/index", and that time can be another, for example, "bundler/vendor".

I'm speculating, but I'm not surprised that running `bundle install` in parallel against the same gem location causes problems. There is no "global" lock in bundler, and it isn't designed to be run like this.

I would recommend:

1. Running `bundle install` in a single process before spawning workers, or
2. Using a different gemset for each process.