rspec-puppet issues under Ruby 2

Trevor Vaughan

unread,

Nov 25, 2014, 8:25:20 PM11/25/14

to puppe...@googlegroups.com

Hi All,

I've got a couple of issues running rspec-puppet tests under Ruby 2 that I was hoping someone could shed some light on.

First, under Ruby 2, any validation functions in the code appear to be getting parsed *after* the inline templates. I like to stick my validation at the bottom of the file to prevent users from wading through a sea of garbage to get to the meat of the code so this isn't thrilling.

Second, even though I have code properly stashed under Puppetx::Me in various lib directories, I don't seem to be able to access them across modules in Ruby 2.

Under 1.8.7 both of these work just fine.

Note: I have *not* seen any issues in production relating to this at this time.

Thanks,

Trevor

--

Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

Dan Bode

unread,

Nov 28, 2014, 11:30:19 AM11/28/14

to puppe...@googlegroups.com

On Tue, Nov 25, 2014 at 5:25 PM, Trevor Vaughan <tvau...@onyxpoint.com> wrote:

Hi All,

I've got a couple of issues running rspec-puppet tests under Ruby 2 that I was hoping someone could shed some light on.

First, under Ruby 2, any validation functions in the code appear to be getting parsed *after* the inline templates. I like to stick my validation at the bottom of the file to prevent users from wading through a sea of garbage to get to the meat of the code so this isn't thrilling.

Second, even though I have code properly stashed under Puppetx::Me in various lib directories, I don't seem to be able to access them across modules in Ruby 2.

I have also seen this issue in Ruby 1.9.3. It's actually on my list of things to root cause. I was able to track it down to the following:

rspec-puppet dynamically adds all lib directories to puppet's libdir (https://github.com/rodjek/rspec-puppet/blob/master/lib/rspec-puppet/support.rb#L140) (not sure exactly why it is not possible to rely on Puppet to do the right thing). I printed $LOAD_PATH from the top of files that were trying to load external dependencies and noticed that the part of the LOAD_PATH translated from Puppet[:libdir] is of the form:

[

'somenormal_path1', 'somenormalpath_1', 'libdir_part1:libdir_part2:...'

]

I added some code to translate those ':' delimited paths translated from libdir into array elements and loading external libraries from fixtures started working.

As an FYI, I did attempt to modify that code from rspec-puppet to set libdir as an array instead of a ':' delim string, and it led to failures from Puppet

Under 1.8.7 both of these work just fine.

Note: I have *not* seen any issues in production relating to this at this time.

Thanks,

Trevor

--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoXBGgij%2BWcnMcR-BTz7EjRRCOoLuFp162nwFh7YAeREJA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Trevor Vaughan

unread,

Nov 28, 2014, 5:49:16 PM11/28/14

to puppe...@googlegroups.com

How bizarre. This does seem like a bit of a show stopper to me in terms of testing in multiple environments.

Hopefully, we can figure it out at some point.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CA%2B0t2Lz06Oct%3DvTTnB_YGxZ%3DCn0HBMHw%3DyiD42m56H5dP_64iw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Kylo Ginsberg

unread,

Nov 28, 2014, 6:08:18 PM11/28/14

to puppe...@googlegroups.com

On Fri, Nov 28, 2014 at 8:30 AM, Dan Bode <bod...@gmail.com> wrote:

On Tue, Nov 25, 2014 at 5:25 PM, Trevor Vaughan <tvau...@onyxpoint.com> wrote:
Hi All,

I've got a couple of issues running rspec-puppet tests under Ruby 2 that I was hoping someone could shed some light on.

First, under Ruby 2, any validation functions in the code appear to be getting parsed *after* the inline templates. I like to stick my validation at the bottom of the file to prevent users from wading through a sea of garbage to get to the meat of the code so this isn't thrilling.

Second, even though I have code properly stashed under Puppetx::Me in various lib directories, I don't seem to be able to access them across modules in Ruby 2.

I have also seen this issue in Ruby 1.9.3. It's actually on my list of things to root cause. I was able to track it down to the following:

rspec-puppet dynamically adds all lib directories to puppet's libdir (https://github.com/rodjek/rspec-puppet/blob/master/lib/rspec-puppet/support.rb#L140) (not sure exactly why it is not possible to rely on Puppet to do the right thing). I printed $LOAD_PATH from the top of files that were trying to load external dependencies and noticed that the part of the LOAD_PATH translated from Puppet[:libdir] is of the form:

[
'somenormal_path1', 'somenormalpath_1', 'libdir_part1:libdir_part2:...'
]

I added some code to translate those ':' delimited paths translated from libdir into array elements and loading external libraries from fixtures started working.

As an FYI, I did attempt to modify that code from rspec-puppet to set libdir as an array instead of a ':' delim string, and it led to failures from Puppet

There's been some work/conversation around libdir being split/not-split in a puppet PR: https://github.com/puppetlabs/puppet/pull/3114

Kylo

Under 1.8.7 both of these work just fine.

Note: I have *not* seen any issues in production relating to this at this time.

Thanks,

Trevor

--
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoXBGgij%2BWcnMcR-BTz7EjRRCOoLuFp162nwFh7YAeREJA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CA%2B0t2Lz06Oct%3DvTTnB_YGxZ%3DCn0HBMHw%3DyiD42m56H5dP_64iw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Kylo Ginsberg | ky...@puppetlabs.com | irc: kylo | twitter: @kylog

Join us at PuppetConf 2015, October 5-9 in Portland, OR - http://2015.puppetconf.com.

Register early to save 40%!

Dan Bode

unread,

Nov 28, 2014, 6:14:37 PM11/28/14

to puppe...@googlegroups.com

On Fri, Nov 28, 2014 at 3:08 PM, Kylo Ginsberg <ky...@puppetlabs.com> wrote:

On Fri, Nov 28, 2014 at 8:30 AM, Dan Bode <bod...@gmail.com> wrote:
On Tue, Nov 25, 2014 at 5:25 PM, Trevor Vaughan <tvau...@onyxpoint.com> wrote:
Hi All,

I've got a couple of issues running rspec-puppet tests under Ruby 2 that I was hoping someone could shed some light on.

First, under Ruby 2, any validation functions in the code appear to be getting parsed *after* the inline templates. I like to stick my validation at the bottom of the file to prevent users from wading through a sea of garbage to get to the meat of the code so this isn't thrilling.

Second, even though I have code properly stashed under Puppetx::Me in various lib directories, I don't seem to be able to access them across modules in Ruby 2.

I have also seen this issue in Ruby 1.9.3. It's actually on my list of things to root cause. I was able to track it down to the following:

rspec-puppet dynamically adds all lib directories to puppet's libdir (https://github.com/rodjek/rspec-puppet/blob/master/lib/rspec-puppet/support.rb#L140) (not sure exactly why it is not possible to rely on Puppet to do the right thing). I printed $LOAD_PATH from the top of files that were trying to load external dependencies and noticed that the part of the LOAD_PATH translated from Puppet[:libdir] is of the form:

[
'somenormal_path1', 'somenormalpath_1', 'libdir_part1:libdir_part2:...'
]

I added some code to translate those ':' delimited paths translated from libdir into array elements and loading external libraries from fixtures started working.

As an FYI, I did attempt to modify that code from rspec-puppet to set libdir as an array instead of a ':' delim string, and it led to failures from Puppet

There's been some work/conversation around libdir being split/not-split in a puppet PR: https://github.com/puppetlabs/puppet/pull/3114

after looking at that conversation, it seems to be attempting to address the same issue observed in this email thread.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CALsUZFFESu%2BA1Fj9u2Z%3Do-i-ZfBqCvf4dUf9iuVGC%2BEMJWoxLg%40mail.gmail.com.

Gavin Williams

unread,

Nov 29, 2014, 5:28:11 AM11/29/14

to puppe...@googlegroups.com

Yeh, I hit the same issue with the LOAD_PATH which lead to that PR and raising PUP-3663.

I've worked around it by modifying the LOAD_PATH in spec_helper.rb.
Can post it when in work on Monday if that helps.

Cheers
Gav

Trevor Vaughan

unread,

Nov 29, 2014, 6:50:23 PM11/29/14

to puppe...@googlegroups.com

It would indeed and I appreciate the info.

Trevor

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/ca8b4133-e015-49db-ab52-c5caaf0921b6%40googlegroups.com.

Gavin Williams

unread,

Dec 1, 2014, 4:20:33 AM12/1/14

to puppe...@googlegroups.com

The code I've added to work-around the Puppet issue is here[1].

HTH.

Gav

[1]https://gist.github.com/fatmcgav/aa7295798502d660c6a3

Trevor Vaughan

unread,

Dec 1, 2014, 2:44:23 PM12/1/14

to puppe...@googlegroups.com

Well that makes a ridiculous amount of sense.

Thanks for posting, I'll check it out tomorrow.

Trevor

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/cd38651c-0e73-4914-a0a9-0c919b2d98b4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Josh Cooper

unread,

Dec 2, 2014, 2:13:39 AM12/2/14

to puppe...@googlegroups.com

On Mon, Dec 1, 2014 at 11:44 AM, Trevor Vaughan <tvau...@onyxpoint.com> wrote:

Well that makes a ridiculous amount of sense.

Thanks for posting, I'll check it out tomorrow.

Trevor

On Mon, Dec 1, 2014 at 4:20 AM, Gavin Williams <fatm...@gmail.com> wrote:
The code I've added to work-around the Puppet issue is here[1].

HTH.

Gav

[1]https://gist.github.com/fatmcgav/aa7295798502d660c6a3

Thanks Gavin for posting that, and for the ticket and PR. I think your "work-around" is really the right thing to be doing.

Puppet's autoloader has a fatal flaw that causes a host of issues where code cannot be loaded. See the linked issues in https://projects.puppetlabs.com/issues/7316

A good example is the case where you have a module defining a new type in lib/puppet/type/foo.rb:

require 'puppet/type/bar'

Puppet::Type.newtype(:foo) do

end

And the `foo` type requires helper code `bar` from within the same (or different) module. The problem with puppet's autoloader is that it only participates in the loading of `foo`, but is "out of the loop" when `foo` requires `bar`. As a result we have the autoloader loading `foo`, but Ruby loading `bar`. What could possibly go wrong?

1. Well, the autoloader searches the current environment's modulepath to load `foo`, but Ruby only knows about $LOAD_PATH. As a result, modules often fail to load their helper code[1], or the module works around it by using the absolute path to the helper code[2], or it tries to rescue LoadError and do something "smart"[3]. This is a terrible pattern.

2. The autoloader uses `Kernel.load` to load `foo`[4], but Ruby requires `bar`. If the class has already been required, the former will reload the class, whereas the latter will not. In long running processes, e.g. master, it is possible for the autoloader to reload a new version of a function, but never reload the helper code that the function relies on[5]. So you have a new version of `foo` running with an old version of `bar`, leading to nearly impossible to debug scenarios, e.g. NoMethodErrors that magically disappear when you restart the master[6].

3. Historically, the Puppet[:libdir] setting has allowed multiple directories to be added to the autoloader's search path, but it has no effect on Ruby's $LOAD_PATH. While this "works" in some cases, e.g when using puppet as a library or rspec tests[7], it would fail hard if you ever tried to run `puppet apply` with that setting[8].

4. To solve https://projects.puppetlabs.com/issues/7316, Andy and I took the approach that puppet (the application) should be responsible for setting up the ruby $LOAD_PATH[9], and puppet (the library) should just use the current $LOAD_PATH.

I think we should do something similar for the rspec-puppet case, where the thing that runs the tests is responsible for updating the $LOAD_PATH to include module specific paths. But this raises a few questions:

Does this belong in rspec-puppet, puppetlabs_spec_helper, or puppet's own testing methods (lib/puppet/test/helper.rb)?

What environment/modulepath should be used, especially once legacy environment support is removed[10]?

Josh

[1] https://tickets.puppetlabs.com/browse/PUP-3336

[2] https://github.com/puppetlabs/puppetlabs-mysql/blob/master/lib/puppet/provider/mysql_database/mysql.rb#L1

[3] https://github.com/puppetlabs/puppetlabs-registry/blob/master/lib/puppet/provider/registry_key/registry.rb#L2-L12

[4] https://github.com/puppetlabs/puppet/blob/3.7.3/lib/puppet/util/autoload.rb#L62

[5] https://tickets.puppetlabs.com/browse/PUP-1033

[6] https://tickets.puppetlabs.com/browse/PUP-2380

[7] https://github.com/rodjek/rspec-puppet/blob/master/lib/rspec-puppet/support.rb#L140

[8] https://tickets.puppetlabs.com/browse/PUP-3336?focusedCommentId=114681&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-114681

[9] https://github.com/puppetlabs/puppet/blob/3.7.3/lib/puppet/util/command_line.rb#L135-L141

[10] https://tickets.puppetlabs.com/browse/PUP-3268

--

Josh Cooper

Developer, Puppet Labs

Trevor Vaughan

unread,

Dec 2, 2014, 9:04:11 AM12/2/14

to puppe...@googlegroups.com

I think that this should go in rspec-puppet since it is core to custom functions and types functioning in many cases. Also, users may, or may not, be using the other testing components.

Thanks,

Trevor

--

You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CA%2Bu97u%3DaUST95KmRF-E4wx43q3wMfsK8SNvXTEY4U9aHsYe_EA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Jeff McCune

unread,

Dec 2, 2014, 12:28:16 PM12/2/14

to puppe...@googlegroups.com

On Fri, Nov 28, 2014 at 3:08 PM, Kylo Ginsberg <ky...@puppetlabs.com> wrote:

As an FYI, I did attempt to modify that code from rspec-puppet to set libdir as an array instead of a ':' delim string, and it led to failures from Puppet

There's been some work/conversation around libdir being split/not-split in a puppet PR: https://github.com/puppetlabs/puppet/pull/3114

I plan to release rspec-puppet 1.1.0 today as per the discussion at https://github.com/rodjek/rspec-puppet/issues/213

It's been a long time since the last release, so if anyone has been working with the rspec-puppet "master" branch, please let me know if you've run into backwards-compatibility issues in master that would motivate a 2.0.0 release, or pulling out the incompatible changes and doing two releases, 1.1.0 and 2.0.0.

If we do end up doing a major version, the usual recommendation to lock to the major version stands:

gem 'rspec-puppet', '~> 1.0'

-Jeff

Joshua Hoblitt

unread,

Dec 2, 2014, 2:10:08 PM12/2/14

to puppe...@googlegroups.com

On 12/02/2014 10:28 AM, Jeff McCune wrote:
> It's been a long time since the last release, so if anyone has been working
> with the rspec-puppet "master" branch, please let me know if you've run
> into backwards-compatibility issues in master that would motivate a 2.0.0
> release, or pulling out the incompatible changes and doing two releases,
> 1.1.0 and 2.0.0.

I commented on the GH issue with the breakage I've observed and suggest
a major version bump.

-Josh

--

Jeff McCune

unread,

Dec 2, 2014, 3:15:10 PM12/2/14

to puppe...@googlegroups.com

On Tue, Dec 2, 2014 at 9:28 AM, Jeff McCune <je...@puppetlabs.com> wrote:

On Fri, Nov 28, 2014 at 3:08 PM, Kylo Ginsberg <ky...@puppetlabs.com> wrote:
As an FYI, I did attempt to modify that code from rspec-puppet to set libdir as an array instead of a ':' delim string, and it led to failures from Puppet

There's been some work/conversation around libdir being split/not-split in a puppet PR: https://github.com/puppetlabs/puppet/pull/3114

I plan to release rspec-puppet 1.1.0 today as per the discussion at https://github.com/rodjek/rspec-puppet/issues/213

Scratch that, looks like Tim is the only person who can push new releases to rubygems.org. I've tagged the release and got everything ready in the master branch but I can't proceed with publication.

-Jeff

Gavin Williams

unread,

Dec 2, 2014, 5:10:30 PM12/2/14

to puppe...@googlegroups.com

Is it possible to work in a change to rspec-puppet for PUP-3336?

Maybe as simple as modifying the LOAD_PATH as shown in my example above?

Cheers
Gav

Wil Cooley

unread,

Dec 4, 2014, 12:49:13 AM12/4/14

to puppet-dev group

On Tue, Dec 2, 2014 at 12:15 PM, Jeff McCune <je...@puppetlabs.com> wrote:

I plan to release rspec-puppet 1.1.0 today as per the discussion at https://github.com/rodjek/rspec-puppet/issues/213

Scratch that, looks like Tim is the only person who can push new releases to rubygems.org. I've tagged the release and got everything ready in the master branch but I can't proceed with publication.

Kudos to you for taking this up Jeff! I will be especially happy to not have to rewrite function tests when they fail so I can actually see the cause of the failure (PR#142).

Wil

Reply all

Reply to author

Forward