Code coverage for puppet resources

[Gareth Rushgrove]

For anyone else who likes writing tests for their puppet manifests,
I've just added basic code coverage to rspec-puppet:

A blog post here about how to use it:

http://www.morethanseven.net/2014/01/25/code-coverage-for-puppet-modules/

I'd be interested in any feedback on how to improve or add to this.

Cheers

Gareth

--
Gareth Rushgrove
@garethr

devopsweekly.com
morethanseven.net
garethrushgrove.com

[Nikola Petrov]

Bah!!

I am waiting for this from months. Thanks so much!

Now I will be able to actually measure my coverage and not blindly rely
on proper TDD.

--
Nikola

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAFi_6yLzEYJ4vbMh28PwoHoYwtGGTnfpra16TnoGrFuGqyGuSg%40mail.gmail.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[gh]

On 1/26/14 7:50 AM, Gareth Rushgrove wrote:
> For anyone else who likes writing tests for their puppet manifests,
> I've just added basic code coverage to rspec-puppet:
>
> A blog post here about how to use it:
>
> http://www.morethanseven.net/2014/01/25/code-coverage-for-puppet-modules/
>
> I'd be interested in any feedback on how to improve or add to this.
>
> Cheers
>
> Gareth
>

Gareth,

Great blog post. I tried this on a module with 100% coverage and notice
that it reports back 50% coverage. After some poking around, found that
it is checking code in spec/fixtures/, the stdlib module actually. There
seems to be an open issue[1] regarding the inability to exclude things.
Curious if you found a work around for testing your code and not the
modules pulled in from .fixtures.yml.

[1] - https://github.com/lemurheavy/coveralls-public/issues/184

BR,
-g

[Gavin Williams]

Gareth/Garret

Not sure if it helps, but I've managed to exclude stuff from Coveralls by using simplecov filters...

E.g.: https://github.com/fatmcgav/puppet-glassfish/commit/5f7d40c2257f469a297edd04a7dbac068306b82b

Gav

[Gareth Rushgrove]

Currently the rspec-puppet coverage stuff does check coverage across
all resources, including those from dependent modules. For example
this module depends on the puppetlabs/apt module -
https://travis-ci.org/garethr/garethr-nginx/jobs/18133670#L113

I purposefully didn't add too much configuration to the first pass of
the code, but it would be simple to add some options to ignore
specific modules (I think). I thought I'd wait to see if anyone found
it useful first.

Gareth

> [1] - https://github.com/lemurheavy/coveralls-public/issues/184
>
> BR,
> -g
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/52F44A2E.3070903%40garretthoneycutt.com.

> For more options, visit https://groups.google.com/groups/opt_out.

[Martin Alfke]

Great add-on for rspec-puppet.
I really missed that functionality.
Adding exclude options would be great.

Many thanks,

Martin

>
>> [1] - https://github.com/lemurheavy/coveralls-public/issues/184
>>
>> BR,
>> -g
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/52F44A2E.3070903%40garretthoneycutt.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
> --
> Gareth Rushgrove
> @garethr
>
> devopsweekly.com
> morethanseven.net
> garethrushgrove.com
>
> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAFi_6y%2B-pZuGexUYciyXDkENL4EQQSJqTBHzaB5NfW9eujt%3D3g%40mail.gmail.com.

[Chris Denneen]

Gareth,

I know this thread is old but do you have a way in the Rakefile to exclude the dependent modules?

For example dependent modules that include different package resources based on OSfamily and version are something that module controls and tests I just need to make sure that the module is included in my catalog and any defined types used in my module get tested. (dependentmodule::resource['foo'])

I've seen some other posts hacking rspec coverage.rb which probably isn't good (https://github.com/rodjek/rspec-puppet/issues/364).

I've noticed the filtered?(resource) in rspec-puppet just documented of how to implement.

Hopefully you can help here.

Thanks

[Gareth Rushgrove]

On 17 June 2016 at 15:24, Chris Denneen <cden...@gmail.com> wrote:
> Gareth,
>
> I know this thread is old but do you have a way in the Rakefile to exclude
> the dependent modules?
> For example dependent modules that include different package resources based
> on OSfamily and version are something that module controls and tests I just
> need to make sure that the module is included in my catalog and any defined
> types used in my module get tested. (dependentmodule::resource['foo'])
> I've seen some other posts hacking rspec coverage.rb which probably isn't
> good (https://github.com/rodjek/rspec-puppet/issues/364).
> I've noticed the filtered?(resource) in rspec-puppet just documented of how
> to implement.
> Hopefully you can help here.
>

I _think_ the coverage support now does this by default?

Anything from fixtures should now be ignored.
https://github.com/rodjek/rspec-puppet/pull/258

Other folks have made a whole bunch of improvements to the coverage
support in rspec-puppet since I added it.

Gareth

> https://groups.google.com/d/msgid/puppet-users/2d63d692-f837-42da-a44e-c5b73dbfa015%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Chris Denneen]

Well for example if i use your puppet-module-skeleton... create a module that uses the rspec-puppet-facts for centos 5,6,7.

include ::selinux (jfryman/selinux)

in coverage report it shows the following missing:

 Package[policycoreutils-devel]

 Package[policycoreutils-python]

 Package[policycoreutils]

first is for 7, second for 6, and third for 5 major releases.

While I can test for these in my _spec test I'd rather only be testing that class_selinux is included and any parameters I might be using against it. or maybe even the selinux::module or selinux::boolean resources that I might be setting... 

should hopefully be a pretty simple test to reproduce but it's definitely not excluding those resources if that's what that merge was intended for.

[Chris Denneen]

Looks like there are few issues open like these:

https://github.com/rodjek/rspec-puppet/issues/285

https://github.com/rodjek/rspec-puppet/issues/158

If anyone knows a way in Rakefile to exclude from coverage I'd be greatful.