Help unit testing profile with dependency on ntp 5.0 module (module data)

[Stephen Nesbitt]

All:

I'm struggling to unit test a very simple profile with a dependency on the ntp 5.0.0 module - the ntp version implementing module data. The problem is that none of the default values for ntp are visible/available to the unit test as indicated by the failure:

  1) profile::ntp::client with default values for all parameters profile::ntp::client should compile into a catalogue without dependency cycles

     Failure/Error: it { is_expected.to compile.with_all_deps }

     

      error during compilation: Evaluation Error: Error while evaluating a Function Call, Class[Ntp]:
         expects a value for parameter 'autoupdate'
         expects a value for parameter 'broadcastclient'
         expects a value for parameter 'config'
         ...

The profile::ntp::client class is very simple:

class profile::ntp::client {
 include ::ntp
}

My spec helper is:

require 'puppetlabs_spec_helper/module_spec_helper'

RSpec.configure do |c|
 c.after(:suite) do
 RSpec::Puppet::Coverage.report!(95)
 end
end

My .fixtures.yml

fixtures:
 forge_modules:
 ntp: 'puppetlabs/ntp'
 stdlib: 'puppetlabs/stdlib'
 symlinks:
 profile: "#{source_dir}/../profile"

My unit test:
require 'spec_helper'

describe 'profile::ntp::client' do
  context 'with default values for all parameters' do
    describe 'profile::ntp::client' do
      it { is_expected.to compile.with_all_deps }
      # it { is_expected.to contain_class('profile::ntp::client') }
      # it { is_expected.to contain_class('::ntp') }

      end
  end
end

Puppet version is 4.7.0. Host OS is ubuntu 16.04

Any help in resolving this would be much appreciated.

-steve

[David Schmitt]

Hi Steve,

I can reproduce this locally, and it looks like some kind of setup issue around how (rspec-)puppet is loading lookup data.

I'll look into it, and keep you posted.

Regards, David

[David Schmitt]

Hi again,

The issue you are seeing is connected to the test not having facts available to properly populate the hierarchy used by data-in-modules in the NTP module.

I've added

  let :facts do
    {
      os: {
        name: 'Debian',
        family: 'Debian',
        release: { major: 'stretch/sid', full: 'stretch/sid' }
      }
    }
  end

to the outer-most describe, and with that the test passed. Before you say it, yes the error messaging around this is atrocious and needs improvement. I've created https://tickets.puppetlabs.com/browse/PUP-6856 to track this.


Thanks for reporting this!
David

[Stephen Nesbitt]

Thanks David! I really appreciate it!!

[Garrett Honeycutt]

On 11/1/16 6:12 AM, David Schmitt wrote:
> Hi Steve,
>
> I can reproduce this locally, and it looks like some kind of setup issue
> around how (rspec-)puppet is loading lookup data.
>
> I'll look into it, and keep you posted.
>
> Regards, David
>
> On Monday, October 31, 2016 at 2:39:54 AM UTC, Stephen Nesbitt wrote:
>
> All:
>
> I'm struggling to unit test a very simple profile with a dependency
> on the ntp 5.0.0 module - the ntp version implementing module data.
> The problem is that none of the default values for ntp are
> visible/available to the unit test as indicated by the failure:
> |
> 1) profile::ntp::client with default values for all parameters
> profile::ntp::client should compile into a catalogue without
> dependency cycles

> Failure/Error: it { is_expected.to <http://is_expected.to>
> compile.with_all_deps }
>
> error during compilation:EvaluationError:Errorwhileevaluating
> a FunctionCall,Class[Ntp]:
> expects a value forparameter 'autoupdate'
> expects a value forparameter 'broadcastclient'
> expects a value forparameter 'config'

> ...
> |
>
>
> The profile::ntp::client class is very simple:
>
> |

> classprofile::ntp::client {

> include ::ntp
> }
> |
>
> My spec helper is:
>
> |
> require'puppetlabs_spec_helper/module_spec_helper'
>
> RSpec.configure do|c|
> c.after(:suite)do
> RSpec::Puppet::Coverage.report!(95)
> end
> end
> |
>
> My .fixtures.yml
>
> |
>
> fixtures:
> forge_modules:
> ntp:'puppetlabs/ntp'
> stdlib:'puppetlabs/stdlib'
> symlinks:
> profile:"#{source_dir}/../profile"
> |
>
>
> My unit test:
>
> |
>
> require 'spec_helper'
>
> describe 'profile::ntp::client' do
> context 'with default values for all parameters' do
> describe 'profile::ntp::client' do

> it { is_expected.to <http://is_expected.to> compile.with_all_deps }
> # it { is_expected.to <http://is_expected.to>

> contain_class('profile::ntp::client') }

> # it { is_expected.to <http://is_expected.to> contain_class('::ntp') }

>
> end
> end
> end
>
> |
>
> Puppet version is 4.7.0. Host OS is ubuntu 16.04
>
>
> Any help in resolving this would be much appreciated.
>
>
> -steve
>

Hi,

I noticed that your .fixtures.yml do not include versions. This means
that they will always test against the latest version. You probably want
to change this to use the version you actually use.

Best regards,
-g


--
Garrett Honeycutt
@learnpuppet
Puppet Training with LearnPuppet.com
Mobile: +1.206.414.8658

[David Schmitt]

On Tuesday, November 1, 2016 at 7:25:01 PM UTC, Garrett Honeycutt wrote:

I noticed that your .fixtures.yml do not include versions. This means
that they will always test against the latest version. You probably want
to change this to use the version you actually use.

That really depends on the amount of time and the cadence people are willing to invest in staying up-to-date. Shops with strict version requirements and a waterfall-y workflow really should do that. Others with a more fluid/agile/pipeline-oriented workflow are much better suited to address upcomping changes when they happen, instead of batching it up.

Cheers, D.
 

[Stephen Nesbitt]

Garrett - Thanks for the heads up!

David - thanks for the comment!

I will note that I am now maintaining dependency information in 3 locations - Puppetfile, metdata.json and .fixtures, Guess what is going to happen at some point ;-)

Maybe I'm not aware of something, but there really should be a single canonical source of dependency requirements.

-steve

[David Schmitt]

https://github.com/puppetlabs/puppetlabs_spec_helper/pull/107 and https://github.com/puppetlabs/puppetlabs_spec_helper/pull/109 have partial solutions, but never moved forward yet. https://github.com/rnelson0/puppet-generate-puppetfile is another partial solution.

d.

--
You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/ph6_kdzr0Ec/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/8df9baef-ea18-4b1f-80f1-184a98566e49%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Gavin Williams]

Stephen

If you're already using R10k or Librarian-Puppet, as indicated by the Puppetfile, then it should be possible to simplify that... 

For example, my .fixtures.yml looks like:

--

fixtures:

  symlinks:

    [module_name]: "#{source_dir}"

Puppetfile:

#!/usr/bin/env ruby

forge 'http://forge.puppetlabs.com'

metadata

# mod 'puppet-archive',

#   :git => 'https://github.com/fatmcgav/puppet-archive.git',

#   :ref => 'support-1.8.7-puppet4'

The Puppetfile above pulls a list of required modules from your metadata.json file, and Librarian-Puppet takes care of installing and resolving deps... 

Rakefile addition:

# use librarian-puppet to manage fixtures instead of .fixtures.yml

# offers more possibilities like explicit version management, forge downloads,...

task :librarian_spec_prep do

  sh "librarian-puppet install --path=spec/fixtures/modules/"

end

task :spec_prep => :librarian_spec_prep

task :librarian_clean do

  sh "librarian-puppet clean"

end

HTH

Gav

[David Schmitt]

On 2 November 2016 at 18:10, Gavin Williams <fatm...@gmail.com> wrote:

Stephen

If you're already using R10k or Librarian-Puppet, as indicated by the Puppetfile, then it should be possible to simplify that... 

For example, my .fixtures.yml looks like:

--

fixtures:

  symlinks:

    [module_name]: "#{source_dir}"

With the latest puppetlabs_spec_helper versions you can even leave that out. See the first example at https://github.com/puppetlabs/puppetlabs_spec_helper#fixtures-examples

 

Puppetfile:

#!/usr/bin/env ruby

forge 'http://forge.puppetlabs.com'

metadata

# mod 'puppet-archive',

#   :git => 'https://github.com/fatmcgav/puppet-archive.git',

#   :ref => 'support-1.8.7-puppet4'

The Puppetfile above pulls a list of required modules from your metadata.json file, and Librarian-Puppet takes care of installing and resolving deps... 

Rakefile addition:

# use librarian-puppet to manage fixtures instead of .fixtures.yml

# offers more possibilities like explicit version management, forge downloads,...

task :librarian_spec_prep do

  sh "librarian-puppet install --path=spec/fixtures/modules/"

end

task :spec_prep => :librarian_spec_prep

task :librarian_clean do

  sh "librarian-puppet clean"

end

Nifty! I guess the same can be used in the acceptance tests if necessary. the one gap (that is also not addressed by any other current tooling) is if you have different testing requirements for different environments (e.g. validate production, vs. smoke-testing upstream versions pre-upgrade.

Thanks a lot for sharing!

d.

HTH

Gav


On Wednesday, 2 November 2016 16:41:03 UTC, Stephen Nesbitt wrote:

Garrett - Thanks for the heads up!

David - thanks for the comment!

I will note that I am now maintaining dependency information in 3 locations - Puppetfile, metdata.json and .fixtures, Guess what is going to happen at some point ;-)

Maybe I'm not aware of something, but there really should be a single canonical source of dependency requirements.

-steve

On Wednesday, November 2, 2016 at 6:55:05 AM UTC-7, David Schmitt wrote:

On Tuesday, November 1, 2016 at 7:25:01 PM UTC, Garrett Honeycutt wrote:

I noticed that your .fixtures.yml do not include versions. This means
that they will always test against the latest version. You probably want
to change this to use the version you actually use.

That really depends on the amount of time and the cadence people are willing to invest in staying up-to-date. Shops with strict version requirements and a waterfall-y workflow really should do that. Others with a more fluid/agile/pipeline-oriented workflow are much better suited to address upcomping changes when they happen, instead of batching it up.

Cheers, D.
 

--

You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/ph6_kdzr0Ec/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/e546756d-b24d-417c-9d78-0d8162210edc%40googlegroups.com.

[Rob Nelson]

I recently found onceover which has a rake task to generate the fixtures, https://github.com/dylanratcliffe/onceover#generate_fixtures, which has some advantages over my tool currently. I think I will reach out to Dylan to see if that can be abstracted out, as onceover may be a little heavyweight if you only want that one function. (It's possible you could add the gen to your Gemfile and use the rake task without conflicting with your existing setup, but I haven't tried that yet.)

You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CALF7fHY64QLggrRh%3DcHm-6kfFZSYoEQE%2BdyeK-XUucQs9zgYzQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Nelson
rnel...@gmail.com