Issues using serverspec with test-kitchen and vagrant.

[Scott Carter]

Hi Chef Testing Gurus!

I am working on implementing some of the cool new techniques that I learned at chefconf last week and I am running into a few roadblocks.  I am hoping that some of you on here have seen similar issues before and help me get un-stuck.  I have been trying to get this simple example working for the better part of a day now and the errors I am getting from the serverspec run simply don't mean much to me.  I have tried using google to find answers but all the suggested fixes I have come across have led to different errors.

I am using test-kitchen with the Vagrant driver and vagrant is using the vmware_workstation plugin to deploy VMs.  I am able to successfully spin up new VMs using kitchen create <suite-name>-sles11sp3 and converge the node using kitchen converge <suite-name>-sles11sp3, and I am even able to run the setup task to create the busser and setup the serverspec plugin on the target host.  All my issues come into play with the verify step of the process.  It seems as though the error changes based on what I have in my spec_helper.rb.  I started with the contents that we used in the training and that produced the following error.

spec_helper.rb

require 'serverspec'


include Serverspec::Helper::Exec

connector_spec.rb

require 'spec_helper'


describe 'connector' do
  it 'should have created a log4j.properties file' do
    expect(file '/usr/local/horizon/conf/log4j.properties').to be_installed
  end


  it 'should have enabled the horizon-workspace service on runlevel 3' do
    expect(service 'horizon-workspace').to be_enabled.with_level(3)
  end


  it 'should have installed the c2-rpm package' do
    expect(package 'c2-rpm').to be_installed
  end
end

Output from Kitchen Verify

I then went on to try some other configurations in my spec_helper.rb.  Seeing as though I am using Vagrant I thought maybe the exec helper was not the right thing.  I change my spec_helper to use Serverspec::Helper::Ssh however this yielded the same error message when I tried to run it.  I was then reading the serverspec documentation where I saw another example of how I should be setting up my spec_helper.rb...

spec_helper.rb (from serverspec docs)

require 'serverspec'
require 'pathname'
require 'net/ssh'

include Serverspec::Helper::Ssh
include Serverspec::Helper::RedHat

Output from verify (different than the last error)

The last configuration I tried I got from reading the documentation on the vagrant-serverspec github page.  This configuration also did not work.

spec_helper.rb

require 'serverspec'
require 'pathname'
require 'net/ssh'

include SpecInfra::Helper::Ssh
include SpecInfra::Helper::DetectOS

Output from verify

Please help me guys!  I have tried everything I can think of and I feel like I am running in circles.  I must be missing something here but what?

Thanks,

Scott

[Mischa Taylor]

Serverspec doesn't currently support SLES/suse (as far as I know).  There's an open ticket for serverspec on this here: https://github.com/serverspec/serverspec/issues/337  You might want to ask Miyashita what the plans for adding SLES/suse support to serverspec and/or help contribute to the effort.

(Also this might be a case where BATS might be a better choice in which to write your tests, given that serverspec doesn't current handle the environment in which you wish to run tests.  t think it supports SLES/suse.  There's a good intro to BATS in Stephen Nelson-Smith's "Test-Driven Infrastructure with Chef, Second Edition".)

[Scott Carter]

Mischa,

I saw that github page yesterday which prompted me to try and get my stuff to work on the built in centos-64 platform that test-kitchen provides.  The errors I am getting when running on that platform are identical so I am still stumped here.

Do you have any examples handy of using BATS with test-kitchen?

Thanks,

Scott

[Scott Carter]

Failing on centos-64 platform as well.

spec_helper.rb

require 'serverspec'

require 'pathname'

require 'net/ssh'

include Serverspec::Helper::Ssh

Output

spec_helper.rb

require 'serverspec'

require 'pathname'

require 'net/ssh'

include SpecInfra::Helper::Ssh

include SpecInfra::Helper::DetectOS

Output

[Scott Carter]

Just tested on Ubuntu 13.10.  It fails with the same errors there as well.

[Mischa Taylor]

I need to see your full cookbook source.  Would you please put it up on Github?  I tried all those platforms this morning using a Windows host (CentOS 6.4, Ubuntu 13.10) when I was writing the instructions on setting up Windows with Test Kitchen to answer Ryan Hass's question.  I had no issues, so there's either something in your setup or cookbook that's probably misconfigured.  But it's hard for me to tell just from Test Kitchen log output you posted.

[Scott Carter]

There might be company policy that prevents me from uploading everything to github.  Let me check with my manager on that one.  That being said.  I have been able to get the tests to at least run now.  It seems the issue is that test-kitchen/serverspec does not know how to connect the node that has been spun up with vagrant.  After changing my spec_helper.rb to the following I was able to get the tests to run and pass with kitchen verify...

spec_helper.rb

require 'serverspec'

require 'pathname'

require 'net/ssh'

include SpecInfra::Helper::Ssh

include SpecInfra::Helper::RedHat

RSpec.configure do |c|

  # Add SSH before hook in case you use the SSH backend

  # (not required for the Exec backend)

  c.before do

    host  = File.basename(Pathname.new(example.metadata[:location]).dirname)

    if c.host != host

      c.ssh.close if c.ssh

      c.host  = '192.168.21.146'

      options = Net::SSH::Config.for(c.host)

      user    = 'vagrant' || Etc.getlogin

      c.ssh   = Net::SSH.start(c.host, user, :password => 'vagrant')

    end

  end

end

In the rspec configuration you see above I had to hardcode the IP of the vagrant VM, the username, and the password in order to get this to work.  Obviously this is not ideal for productivity if I am tearing down and re-creating VMs constantly as part of my dev/test workflow.  I guess my question now becomes how can I get serverspec the information it needs without having to manually edit this file and enter the proper IP address each time I destroy and re-create a VM with test kitchen?  I was looking at a command that is available 'vagrant ssh-config' but I am not sure how and if that should be used to gather this information in an automated fashion.  There must be an easy way to do this that I just don't know about.  Any suggestions?

[Mischa Taylor]

It would help to at least send me a directory listing with "tree" or "ls -lar".  I suspect that you might not be putting your tests in the correct directory structure that we covered in the class.  If they are in "test/integration/default/serverspec" or "test/integration/default/serverspec/localhost" serverspec should be automatically picking up the proper IP address of nodes Test Kitchen is spinning up.

But I can only guess without seeing your configuration file harness around the spec_helper.rb to see how you are spinning up the nodes (like your .kitchen.yml) and a few more details like the directory structure.

[Mischa Taylor]

If posting the actual cookbook source is a concern, create a pared-down example cookbook with just enough to reproduce the issue.  I'm sure the issue can be reproduced with just a small example reflecting your cookbook design & structure without revealing any proprietary details.

[Scott Carter]

For now attached is the output of "tree" so you can see the structure and I am also attaching the .kitchen.yml, spec_helper.rb, and connector_spec.rb so you can see whats going on in there.  I have now been able to get all the tests that I have written so far to pass as long as I populate the spec_helper.rb with the proper IP address after converging a node spun up with Vagrant.  Even better is they are all passing on a SLES 11 SP3 VM that I built.  I will also work on pairing down what we have or getting the ok to post to github so you can see the whole thing.  Thanks!

-Scott

[Mischa Taylor]

I think I have the answer, you were close.  Change your spec_helper.rb to resemble the following:

spec_helper.rb

require 'serverspec'

include Serverspec::Helper::Exec

include Serverspec::Helper::DetectOS

This should fix everything but the first resource with the be_installed matcher.

For that first file resource - it does not support the "be_installed" matcher.  Use "be_file" instead (notice the file resource has no "be_installed" matcher in the serverspec docs).  Change connector_spec.rb accordingly:

connector_spec.rb

require 'spec_helper'


describe 'connector' do
  it 'should have created a log4j.properties file' do


    expect(file '/usr/local/horizon/conf/log4j.properties').to be_file


  end


  it 'should have enabled the horizon-workspace service on runlevel 3' do
    expect(service 'horizon-workspace').to be_enabled.with_level(3)
  end


  it 'should have installed the c2-rpm package' do
    expect(package 'c2-rpm').to be_installed
  end
end




My apologies for just trying to eyeball your source instead of running it - the issues were clear when I gave it a spin.

[Mischa Taylor]

I'm also adding another slide or two to the deck explaining DetectOS - I should have mentioned that in the class.  I wrote about it in my blog: http://misheska.com/blog/2013/08/06/getting-started-writing-chef-cookbooks-the-berkshelf-way-part3/ But forgot to add that essential bit to the class deck...

[Mischa Taylor]

OK, I just added a couple of new slides explaining DetectOS to the decks:

Docker Version: https://opscode.box.com/s/1aw069lr0822keg7wtzo

Vagrant Version: https://opscode.box.com/s/dq6nkbjeqmel4v0sue1j

[Scott Carter]

This was the Key!  I am up and running now.  Thanks a ton for your help. :-)

[Mischa Taylor]

Awesome!  And I guess it also works with SLES which is good to know

--
You received this message because you are subscribed to the Google Groups "Testing with Chef" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testing-with-c...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.