run beaker tests on a live system

[Thomas Müller]

hi 

I'm trying to get started with beaker testing. we will introduce the role/profile pattern and want to enforce the acceptance criterias per profile class. Operations would like to reuse the puppet tests on live systems to verify the state. 

This leads to the requirement to be able to disable the vagrant part, skip applying of manifests and  just check files/services/ports/... . 

I discovered that rspec is able to have tags on describe/context/it and filter it with maybe "rspec --tag liveacceptance:true" to only include the relevant tests.

Is it somehow possible to do this with beaker-rspec?

- Thomas

  

[Hunter Haugen]

On Tue, May 5, 2015 at 6:31 AM, Thomas Müller <tho...@chaschperli.ch> wrote:

hi 

I'm trying to get started with beaker testing. we will introduce the role/profile pattern and want to enforce the acceptance criterias per profile class. Operations would like to reuse the puppet tests on live systems to verify the state. 

This leads to the requirement to be able to disable the vagrant part, skip applying of manifests and  just check files/services/ports/... . 

For skipping vagrant, I believe you can make a nodeset with "none" as the hypervisor. Beaker will try and ssh to the host by name, assuming it is already up.

 

I discovered that rspec is able to have tags on describe/context/it and filter it with maybe "rspec --tag liveacceptance:true" to only include the relevant tests.

You would definitely want to be careful on how to structure your tests. Any tests with `apply_manifest` or the like would need to be separate from any assertions. Perhaps even in separate test files, as `rspec` can take a list of *_spec.rb files to run, and you could run something like `rspec spec/acceptance/1_puppet_apply/ spec/acceptance/2_assert/` under normal testing circumstances, and `rspec spec/acceptance/2_assert/` when you only want to run your assertions.

 

Is it somehow possible to do this with beaker-rspec?

Also, I would like to point out that your use case is EXACTLY what http://serverspec.org/ is created to do: assertions without modifying the system. Serverspec is included with beaker-rspec, but perhaps it would be better to just skip the beaker part altogether on the live systems, but use it to configure the test systems as described above with the 1_* 2_* directories?

 

- Thomas

  

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/3991f661-8891-4adf-8b7e-a7d01b92bbbe%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thomas Müller]

 

 

I discovered that rspec is able to have tags on describe/context/it and filter it with maybe "rspec --tag liveacceptance:true" to only include the relevant tests.

You would definitely want to be careful on how to structure your tests. Any tests with `apply_manifest` or the like would need to be separate from any assertions. Perhaps even in separate test files, as `rspec` can take a list of *_spec.rb files to run, and you could run something like `rspec spec/acceptance/1_puppet_apply/ spec/acceptance/2_assert/` under normal testing circumstances, and `rspec spec/acceptance/2_assert/` when you only want to run your assertions.

Splitting into 2 files would mean not having all dependent tests in 1 file.  I'm not sure if this is the way to go.

 

 

Is it somehow possible to do this with beaker-rspec?

Also, I would like to point out that your use case is EXACTLY what http://serverspec.org/ is created to do: assertions without modifying the system. Serverspec is included with beaker-rspec, but perhaps it would be better to just skip the beaker part altogether on the live systems, but use it to configure the test systems as described above with the 1_* 2_* directories?

For now i've started checking for a env variable LIVESYSTEM=true inside the test_spec.rb and then include the serverspec spec_helper.rb instead of the puppetlabs spec_helper_acceptance.rb. Inside the test_spec.rb i've flagged the beaker related tests with beakeronly. Then I can run the tests without the beaker specific tests: "LIVESYSTEM=true rspec --tag ~beakeronly spec/acceptance/test_spec.rb" . 

we then want to distribute the test files to every client with the profile applied to have the ability to run the tests as needed by the operations people.

- Thomas