Jira (HI-578) Symbolized facts causes "ignore bad definition" error in hierarchy

5 views
Skip to first unread message

Dan Dunckel (JIRA)

unread,
Aug 24, 2017, 12:29:03 PM8/24/17
to puppe...@googlegroups.com
Dan Dunckel created an issue
 
Hiera / Bug HI-578
Symbolized facts causes "ignore bad definition" error in hierarchy
Issue Type: Bug Bug
Affects Versions: HI 3.4.0
Assignee: Unassigned
Created: 2017/08/24 9:28 AM
Environment:
    1. Environment

```
Bundler 1.14.6
Rubygems 2.5.1
Ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
GEM_HOME /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing
GEM_PATH /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing:/home/dunk/.rvm/gems/ruby-2.3.1@global
RVM 1.29.1 (latest)
Git 1.9.1
Platform x86_64-linux
rubygems-bundler (1.4.4)
```

      1. Gemfile

```ruby
source "https://rubygems.org"

gem 'puppet', '4.7.0'

group :test do
gem 'puppet-lint', '2.3.0'
gem 'puppetlabs_spec_helper', '2.2.0'
gem 'rspec-puppet', '2.6.8'
gem 'parallel_tests', '2.14.2'
gem 'rspec-puppet-facts', '1.8.0', :require => false
gem 'metadata-json-lint', '2.0.2'
gem 'semantic_puppet', '1.0.1' # this gem is needed (for metadata-json-lint) until we move to Puppet 4.9.x
end

group :development, :test do
gem 'beaker-rspec', '6.1.0', :require => false
gem 'pry', '0.10.4'
end
```

      1. Gemfile.lock

```
GEM
remote: https://rubygems.org/
specs:
CFPropertyList (2.3.5)
addressable (2.5.1)
public_suffix (~> 2.0, >= 2.0.2)
aws-sdk-v1 (1.67.0)
json (~> 1.4)
nokogiri (~> 1)
beaker (3.20.0)
aws-sdk-v1 (~> 1.57)
beaker-hiera (~> 0.0)
beaker-hostgenerator
beaker-puppet (~> 0.0)
beaker-vmpooler (~> 0.1)
docker-api
fission (~> 0.4)
fog (~> 1.38)
google-api-client (~> 0.9)
hocon (~> 1.0)
in-parallel (~> 0.1)
inifile (~> 3.0)
minitar (~> 0.5.4)
minitest (~> 5.4)
net-scp (~> 1.2)
net-ssh (~> 4.0)
open_uri_redirections (~> 0.2.1)
rsync (~> 1.0.9)
stringify-hash (~> 0.0)
thor (~> 0.19)
unf (~> 0.1)
beaker-hiera (0.1.1)
stringify-hash (~> 0.0.0)
beaker-hostgenerator (0.10.1)
deep_merge (~> 1.0)
stringify-hash (~> 0.0.0)
beaker-puppet (0.3.0)
in-parallel (~> 0.1)
stringify-hash (~> 0.0.0)
beaker-rspec (6.1.0)
beaker (~> 3.0)
rspec (~> 3.0)
serverspec (~> 2)
specinfra (~> 2)
beaker-vmpooler (0.1.0)
rbvmomi (~> 1.9)
stringify-hash (~> 0.0.0)
builder (3.2.3)
coderay (1.1.1)
declarative (0.0.9)
declarative-option (0.1.0)
deep_merge (1.1.1)
diff-lcs (1.3)
docker-api (1.33.6)
excon (>= 0.38.0)
json
excon (0.57.1)
facter (2.4.6)
facterdb (0.3.11)
facter
jgrep
faraday (0.12.1)
multipart-post (>= 1.2, < 3)
fast_gettext (1.1.0)
fission (0.5.0)
CFPropertyList (~> 2.2)
fog (1.40.0)
fog-aliyun (>= 0.1.0)
fog-atmos
fog-aws (>= 0.6.0)
fog-brightbox (~> 0.4)
fog-cloudatcost (~> 0.1.0)
fog-core (~> 1.43)
fog-digitalocean (>= 0.3.0)
fog-dnsimple (~> 1.0)
fog-dynect (~> 0.0.2)
fog-ecloud (~> 0.1)
fog-google (<= 0.1.0)
fog-json
fog-local
fog-openstack
fog-powerdns (>= 0.1.1)
fog-profitbricks
fog-rackspace
fog-radosgw (>= 0.0.2)
fog-riakcs
fog-sakuracloud (>= 0.0.4)
fog-serverlove
fog-softlayer
fog-storm_on_demand
fog-terremark
fog-vmfusion
fog-voxel
fog-vsphere (>= 0.4.0)
fog-xenserver
fog-xml (~> 0.1.1)
ipaddress (~> 0.5)
json (>= 1.8, < 2.0)
fog-aliyun (0.1.0)
fog-core (~> 1.27)
fog-json (~> 1.0)
ipaddress (~> 0.8)
xml-simple (~> 1.1)
fog-atmos (0.1.0)
fog-core
fog-xml
fog-aws (1.4.0)
fog-core (~> 1.38)
fog-json (~> 1.0)
fog-xml (~> 0.1)
ipaddress (~> 0.8)
fog-brightbox (0.11.0)
fog-core (~> 1.22)
fog-json
inflecto (~> 0.0.2)
fog-cloudatcost (0.1.2)
fog-core (~> 1.36)
fog-json (~> 1.0)
fog-xml (~> 0.1)
ipaddress (~> 0.8)
fog-core (1.44.3)
builder
excon (~> 0.49)
formatador (~> 0.2)
fog-digitalocean (0.3.0)
fog-core (~> 1.42)
fog-json (>= 1.0)
fog-xml (>= 0.1)
ipaddress (>= 0.5)
fog-dnsimple (1.0.0)
fog-core (~> 1.38)
fog-json (~> 1.0)
fog-dynect (0.0.3)
fog-core
fog-json
fog-xml
fog-ecloud (0.3.0)
fog-core
fog-xml
fog-google (0.1.0)
fog-core
fog-json
fog-xml
fog-json (1.0.2)
fog-core (~> 1.0)
multi_json (~> 1.10)
fog-local (0.3.1)
fog-core (~> 1.27)
fog-openstack (0.1.21)
fog-core (>= 1.40)
fog-json (>= 1.0)
ipaddress (>= 0.8)
fog-powerdns (0.1.1)
fog-core (~> 1.27)
fog-json (~> 1.0)
fog-xml (~> 0.1)
fog-profitbricks (3.0.0)
fog-core (~> 1.42)
fog-json (~> 1.0)
fog-rackspace (0.1.5)
fog-core (>= 1.35)
fog-json (>= 1.0)
fog-xml (>= 0.1)
ipaddress (>= 0.8)
fog-radosgw (0.0.5)
fog-core (>= 1.21.0)
fog-json
fog-xml (>= 0.0.1)
fog-riakcs (0.1.0)
fog-core
fog-json
fog-xml
fog-sakuracloud (1.7.5)
fog-core
fog-json
fog-serverlove (0.1.2)
fog-core
fog-json
fog-softlayer (1.1.4)
fog-core
fog-json
fog-storm_on_demand (0.1.1)
fog-core
fog-json
fog-terremark (0.1.0)
fog-core
fog-xml
fog-vmfusion (0.1.0)
fission
fog-core
fog-voxel (0.1.0)
fog-core
fog-xml
fog-vsphere (1.11.2)
fog-core
rbvmomi (~> 1.9)
fog-xenserver (0.3.0)
fog-core
fog-xml
fog-xml (0.1.3)
fog-core
nokogiri (>= 1.5.11, < 2.0.0)
formatador (0.2.5)
gettext (3.2.4)
locale (>= 2.0.5)
text (>= 1.3.0)
gettext-setup (0.26)
fast_gettext (~> 1.1.0)
gettext (>= 3.0.2)
locale
google-api-client (0.13.1)
addressable (~> 2.5, >= 2.5.1)
googleauth (~> 0.5)
httpclient (>= 2.8.1, < 3.0)
mime-types (~> 3.0)
representable (~> 3.0)
retriable (>= 2.0, < 4.0)
googleauth (0.5.1)
faraday (~> 0.9)
jwt (~> 1.4)
logging (~> 2.0)
memoist (~> 0.12)
multi_json (~> 1.11)
os (~> 0.9)
signet (~> 0.7)
hiera (3.4.0)
hocon (1.2.5)
httpclient (2.8.3)
in-parallel (0.1.17)
inflecto (0.0.2)
inifile (3.0.0)
ipaddress (0.8.3)
jgrep (1.4.1)
json
json (1.8.6)
json-schema (2.8.0)
addressable (>= 2.4)
json_pure (1.8.6)
jwt (1.5.6)
little-plugger (1.1.4)
locale (2.1.2)
logging (2.2.2)
little-plugger (~> 1.1)
multi_json (~> 1.10)
mcollective-client (2.11.0)
json
stomp
systemu
memoist (0.16.0)
metaclass (0.0.4)
metadata-json-lint (2.0.2)
json-schema (~> 2.8)
spdx-licenses (~> 1.0)
method_source (0.8.2)
mime-types (3.1)
mime-types-data (~> 3.2015)
mime-types-data (3.2016.0521)
mini_portile2 (2.2.0)
minitar (0.5.4)
minitest (5.10.2)
mocha (1.2.1)
metaclass (~> 0.0.1)
multi_json (1.12.1)
multipart-post (2.0.0)
net-scp (1.2.1)
net-ssh (>= 2.6.5)
net-ssh (4.1.0)
net-telnet (0.1.1)
nokogiri (1.8.0)
mini_portile2 (~> 2.2.0)
open_uri_redirections (0.2.1)
os (0.9.6)
parallel (1.11.2)
parallel_tests (2.14.2)
parallel
pry (0.10.4)
coderay (~> 1.1.0)
method_source (~> 0.8.1)
slop (~> 3.4)
public_suffix (2.0.5)
puppet (4.7.0)
facter (> 2.0, < 4)
hiera (>= 2.0, < 4)
json_pure (~> 1.8)
puppet-lint (2.3.0)
puppet-syntax (2.4.1)
rake
puppetlabs_spec_helper (2.2.0)
mocha (~> 1.0)
puppet-lint (~> 2.0)
puppet-syntax (~> 2.0)
rspec-puppet (~> 2.0)
rake (12.0.0)
rbvmomi (1.11.3)
builder (~> 3.0)
json (>= 1.8)
nokogiri (~> 1.5)
trollop (~> 2.1)
representable (3.0.4)
declarative (< 0.1.0)
declarative-option (< 0.2.0)
uber (< 0.2.0)
retriable (3.0.2)
rspec (3.6.0)
rspec-core (~> 3.6.0)
rspec-expectations (~> 3.6.0)
rspec-mocks (~> 3.6.0)
rspec-core (3.6.0)
rspec-support (~> 3.6.0)
rspec-expectations (3.6.0)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.6.0)
rspec-its (1.2.0)
rspec-core (>= 3.0.0)
rspec-expectations (>= 3.0.0)
rspec-mocks (3.6.0)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.6.0)
rspec-puppet (2.6.8)
rspec
rspec-puppet-facts (1.8.0)
facter
facterdb (>= 0.3.0)
json
mcollective-client
puppet
rspec-support (3.6.0)
rsync (1.0.9)
semantic_puppet (1.0.1)
gettext-setup (>= 0.3)
serverspec (2.39.1)
multi_json
rspec (~> 3.0)
rspec-its
specinfra (~> 2.68)
sfl (2.3)
signet (0.7.3)
addressable (~> 2.3)
faraday (~> 0.9)
jwt (~> 1.5)
multi_json (~> 1.10)
slop (3.6.0)
spdx-licenses (1.1.0)
specinfra (2.69.0)
net-scp
net-ssh (>= 2.7, < 5.0)
net-telnet
sfl
stomp (1.4.4)
stringify-hash (0.0.2)
systemu (2.6.5)
text (1.3.1)
thor (0.19.4)
trollop (2.1.2)
uber (0.1.0)
unf (0.1.4)
unf_ext
unf_ext (0.0.7.4)
xml-simple (1.1.5)

PLATFORMS
ruby

DEPENDENCIES
beaker-rspec (= 6.1.0)
metadata-json-lint (= 2.0.2)
parallel_tests (= 2.14.2)
pry (= 0.10.4)
puppet (= 4.7.0)
puppet-lint (= 2.3.0)
puppetlabs_spec_helper (= 2.2.0)
rspec-puppet (= 2.6.8)
rspec-puppet-facts (= 1.8.0)
semantic_puppet (= 1.0.1)

BUNDLED WITH
1.14.6
```

Labels: github
Priority: Normal Normal
Reporter: Dan Dunckel
  1. Summary
    When I try to perform a `hiera.lookup` request in a spec, the hierarchy doesn't find the `kernel` fact. Because of that, Hiera shows this message:

Ignoring bad definition in :hierarchy: 'kernel/'

And as a result, the data I need is not found.

    1. spec/spec_helper.rb
      ```
      require "puppetlabs_spec_helper/module_spec_helper"
      require "rspec-puppet-facts"
      include RspecPuppetFacts
  1. needed to perform lookups inside specs
    require "hiera"

RSpec.configure do |c|
c.default_facts =

{ "hostname" => `hostname -f`.strip, "ipaddress" => `hostname -i`.strip, "puppetversion" => "4.6.1", }

c.fail_fast = true if ENV['FAIL_FAST'] == "true"

  1. needed to evaluate hiera lookups in the puppet code
    c.hiera_config = "spec/fixtures/hiera/hiera.yaml"

if ENV['COVERAGE'] == "true"
c.after(:suite) do
RSpec::Puppet::Coverage.report!
end
end
end

  1. Default to "no" as our specs fail on default facts (i.e. "$::networking")
    ENV['STRICT_VARIABLES'] = "no" unless ENV['STRICT_VARIABLES']
    ```
    1. spec/fixtures/hiera/hiera.yaml
      ```
      :backends:
  • yaml
    :yaml:
    :datadir: spec/fixtures/hiera
    :hierarchy:
  • "fqdn/% {fqdn}

    "

  • "environment/% {environment}

    "

  • "hostgroup/% {hostgroup}

    "

  • "os/% {operatingsystem}/%{operatingsystemrelease}"
    - "os/%{operatingsystem}

    "

  • "kernel/% {kernel}

    "

  • "common"
    ```
    1. spec/fixtures/hiera/os/Ubuntu.yaml
      ```
      cron_service: 'cron'
      ```
    1. spec/fixtures/hiera/kernel/Linux.yaml
      ```
      cron_service: 'crond'
      ```
    1. manifests/linux.pp
      ```
      class ta_base::linux() {
      $cron_service = hiera('cron_service')

service

{ $cron_service: enable => true, ensure => true, hasrestart => true, hasstatus => true, }

}
```

    1. spec/classes/linux_spec.rb
      ```
      require "spec_helper"

hiera = Hiera.new(:config => "spec/fixtures/hiera/hiera.yaml")

describe "ta_base::linux" do
on_supported_os.each do |os, facts|
context "on #

{os}

" do
let(:facts)

{ facts }

cron_service = hiera.lookup("cron_service", nil, facts)

it

{ is_expected.to compile.with_all_deps }

it

{ is_expected.to contain_service(cron_service) .with( "enable" => true, "ensure" => true ) }

end
end
end
```

    1. Example run
      ```
      $ SPEC=spec/classes/linux_spec.rb FAIL_FAST=true rake spec_standalone
      /home/dunk/.rvm/rubies/ruby-2.3.1/bin/ruby -I/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-core-3.6.0/lib:/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-support-3.6.0/lib/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-core-3.6.0/exe/rspec spec/classes/linux_spec.rb --color
      DEBUG: 2017-08-23 17:38:05 -0600: Hiera YAML backend starting
      DEBUG: 2017-08-23 17:38:05 -0600: Looking up cron_service in YAML backend
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'fqdn/'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'environment/'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'hostgroup/'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'os//'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'os/'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'kernel/'
      DEBUG: 2017-08-23 17:38:05 -0600: Looking for data source common
      DEBUG: 2017-08-23 17:38:05 -0600: Looking up cron_service in YAML backend
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'fqdn/'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'environment/'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'hostgroup/'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'os//'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'os/'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'kernel/'
      DEBUG: 2017-08-23 17:38:05 -0600: Looking for data source common
      DEBUG: 2017-08-23 17:38:05 -0600: Looking up cron_service in YAML backend
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'fqdn/'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'environment/'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'hostgroup/'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'os//'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'os/'
      DEBUG: 2017-08-23 17:38:05 -0600: Ignoring bad definition in :hierarchy: 'kernel/'
      DEBUG: 2017-08-23 17:38:05 -0600: Looking for data source common

ta_base::linux
on centos-6-x86_64
should compile into a catalogue without dependency cycles
should contain Service[] with enable => true and ensure => true (FAILED - 1)

Failures:

1) ta_base::linux on centos-6-x86_64 should contain Service[] with enable => true and ensure => true
Failure/Error:
it

{ is_expected.to contain_service(cron_service) .with( "enable" => true, "ensure" => true ) }

ArgumentError:
No title provided and "Service" is not a valid resource reference

  1. /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/puppet-4.7.0/lib/puppet/resource.rb:572:in `extract_type_and_title'
  2. /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/puppet-4.7.0/lib/puppet/resource.rb:557:in `type_and_title'
  3. /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/puppet-4.7.0/lib/puppet/resource/catalog.rb:355:in `resource'
  4. /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-puppet-2.6.8/lib/rspec-puppet/matchers/create_generic.rb:85:in `matches?'
  5. ./spec/classes/linux_spec.rb:17:in `block (4 levels) in <top (required)>'

Finished in 3.93 seconds (files took 4.26 seconds to load)
2 examples, 1 failure

Failed examples:

rspec ./spec/classes/linux_spec.rb[1:1:2] # ta_base::linux on centos-6-x86_64 should contain Service[] with enable => true and ensure => true

/home/dunk/.rvm/rubies/ruby-2.3.1/bin/ruby -I/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-core-3.6.0/lib:/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-support-3.6.0/lib/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-core-3.6.0/exe/rspec spec/classes/linux_spec.rb --color failed
```

From what I can see, this comes down to the `facts` hash having `kernel` as a `Symbol` instead of a `String`, because it properly loads `spec/fixtures/hiera/kernel/Linux.yaml` when I do this:

```

  1. manifest
    facts['kernel'] = "Linux"
    cron_service = hiera.lookup("cron_service", nil, facts)
  1. log output from running test
    DEBUG: 2017-08-23 17:39:14 -0600: Looking up cron_service in YAML backend
    DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'fqdn/'
    DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'environment/'
    DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'hostgroup/'
    DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'os//'
    DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'os/'
    DEBUG: 2017-08-23 17:39:14 -0600: Looking for data source kernel/Linux
    DEBUG: 2017-08-23 17:39:14 -0600: Found cron_service in kernel/Linux
    ```
  • The last 2 lines being the key difference:

Looking for data source kernel/Linux
Found cron_service in kernel/Linux

So....

  • Am I configuring something incorrectly?
  • Should I post an issue to https://tickets.puppetlabs.com/ for Hiera?
  • Should I just eat dinner?
  • Yes. Yes I should.
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
Atlassian logo

Dan Dunckel (JIRA)

unread,
Aug 24, 2017, 12:30:04 PM8/24/17
to puppe...@googlegroups.com
Dan Dunckel updated an issue
Change By: Dan Dunckel
Environment: ## Environment


```
Bundler   1.14.6
Rubygems  2.5.1
Ruby      2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
GEM_HOME  /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing
GEM_PATH  /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing:/home/dunk/.rvm/gems/ruby-2.3.1@global
RVM       1.29.1 (latest)
Git       1.9.1
Platform  x86_64-linux
rubygems-bundler (1.4.4)
```

### Gemfile


```ruby
source "https://rubygems.org"

gem 'puppet', '4.7.0'

group :test do
  gem 'puppet-lint', '2.3.0'
  gem 'puppetlabs_spec_helper', '2.2.0'
  gem 'rspec-puppet', '2.6.8'
  gem 'parallel_tests', '2.14.2'
  gem 'rspec-puppet-facts', '1.8.0', :require => false
  gem 'metadata-json-lint', '2.0.2'
  gem 'semantic_puppet', '1.0.1' # this gem is needed (for metadata-json-lint) until we move to Puppet 4.9.x
end

group :development, :test do
  gem 'beaker-rspec', '6.1.0', :require => false
  gem 'pry', '0.10.4'
end
```

### Gemfile.lock


```
GEM
  remote: https://rubygems.org/
  specs:
    CFPropertyList (2.3.5)
    addressable (2.5.1)
      public_suffix (~> 2.0, >= 2.0.2)
    aws-sdk-v1 (1.67.0)
      json (~> 1.4)
      nokogiri (~> 1)
    beaker (3.20.0)
      aws-sdk-v1 (~> 1.57)
      beaker-hiera (~> 0.0)
      beaker-hostgenerator
      beaker-puppet (~> 0.0)
      beaker-vmpooler (~> 0.1)
      docker-api
      fission (~> 0.4)
      fog (~> 1.38)
      google-api-client (~> 0.9)
      hocon (~> 1.0)
See environment section       in -parallel (~> 0.1)
```  issue description

Dan Dunckel (JIRA)

unread,
Aug 24, 2017, 12:33:03 PM8/24/17
to puppe...@googlegroups.com
Dan Dunckel updated an issue
# h1.  Summary

When I try to perform a `hiera.lookup` request in a spec, the hierarchy doesn't find the `kernel` fact.  Because of that, Hiera shows this message:

    Ignoring bad definition in :hierarchy: 'kernel/'

And as a result, the data I need is not found.

## h2.  spec/spec_helper.rb
``` {code}
require "puppetlabs_spec_helper/module_spec_helper"
require "rspec-puppet-facts"
include RspecPuppetFacts

# needed to perform lookups inside specs

require "hiera"

RSpec.configure do |c|
  c.default_facts = {
    "hostname"      => `hostname -f`.strip,
    "ipaddress"     => `hostname -i`.strip,
    "puppetversion" => "4.6.1",
  }

  c.fail_fast = true if ENV['FAIL_FAST'] == "true"

  # needed to evaluate hiera lookups in the puppet code

  c.hiera_config = "spec/fixtures/hiera/hiera.yaml"

  if ENV['COVERAGE'] == "true"
    c.after(:suite) do
      RSpec::Puppet::Coverage.report!
    end
  end
end

# Default to "no" as our specs fail on default facts (i.e. "$::networking")

ENV['STRICT_VARIABLES'] = "no" unless ENV['STRICT_VARIABLES']
``` {code}

## h2.  spec/fixtures/hiera/hiera.yaml
``` {code}
:backends:
  - yaml
:yaml:
  :datadir: spec/fixtures/hiera
:hierarchy:
  - "fqdn/%{fqdn}"
  - "environment/%{environment}"
  - "hostgroup/%{hostgroup}"
  - "os/%{operatingsystem}/%{operatingsystemrelease}"
  - "os/%{operatingsystem}"
  - "kernel/%{kernel}"
  - "common"
``` {code}

## h2.  spec/fixtures/hiera/os/Ubuntu.yaml
``` {code}
cron_service: 'cron'
``` {code}

## h2.  spec/fixtures/hiera/kernel/Linux.yaml
``` {code}
cron_service: 'crond'
``` {code}

## h2.  manifests/linux.pp
``` {code}
class ta_base::linux() {
  $cron_service = hiera('cron_service')

  service { $cron_service:
    enable     => true,
    ensure     => true,
    hasrestart => true,
    hasstatus  => true,
  }
}
``` {code}

## h2.  spec/classes/linux_spec.rb
``` {code}
require "spec_helper"

hiera = Hiera.new(:config => "spec/fixtures/hiera/hiera.yaml")

describe "ta_base::linux" do
  on_supported_os.each do |os, facts|
    context "on #{os}" do
      let(:facts) { facts }

      cron_service = hiera.lookup("cron_service", nil, facts)

      it { is_expected.to compile.with_all_deps }

      it { is_expected.to contain_service(cron_service)
        .with(
          "enable"  => true,
          "ensure"  => true
        )
      }
    end
  end
end
``` {code}

## h2.  Example run
``` {code}
     # /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/puppet-4.7.0/lib/puppet/resource.rb:572:in `extract_type_and_title'
     # /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/puppet-4.7.0/lib/puppet/resource.rb:557:in `type_and_title'
     # /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/puppet-4.7.0/lib/puppet/resource/catalog.rb:355:in `resource'
     # /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-puppet-2.6.8/lib/rspec-puppet/matchers/create_generic.rb:85:in `matches?'
     # ./spec/classes/linux_spec.rb:17:in `block (4 levels) in <top (required)>'


Finished in 3.93 seconds (files took 4.26 seconds to load)
2 examples, 1 failure

Failed examples:

rspec ./spec/classes/linux_spec.rb[1:1:2] # ta_base::linux on centos-6-x86_64 should contain Service[] with enable => true and ensure => true

/home/dunk/.rvm/rubies/ruby-2.3.1/bin/ruby -I/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-core-3.6.0/lib:/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-support-3.6.0/lib/home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing/gems/rspec-core-3.6.0/exe/rspec spec/classes/linux_spec.rb --color failed
``` {code}

From what I can see, this comes down to the `facts` hash having `kernel` as a `Symbol` instead of a `String`, because it properly loads `spec/fixtures/hiera/kernel/Linux.yaml` when I do this:

``` {code}
# manifest

facts['kernel'] = "Linux"
cron_service = hiera.lookup("cron_service", nil, facts)

# log output from running test

DEBUG: 2017-08-23 17:39:14 -0600: Looking up cron_service in YAML backend
DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'fqdn/'
DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'environment/'
DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'hostgroup/'
DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'os//'
DEBUG: 2017-08-23 17:39:14 -0600: Ignoring bad definition in :hierarchy: 'os/'
DEBUG: 2017-08-23 17:39:14 -0600: Looking for data source kernel/Linux
DEBUG: 2017-08-23 17:39:14 -0600: Found cron_service in kernel/Linux
``` {code}
- The last 2 lines being the key difference:


      Looking for data source kernel/Linux
      Found cron_service in kernel/Linux

So....
- Am I configuring something incorrectly?
- Should I post an issue to https://tickets.puppetlabs.com/ for Hiera?
- Should I just eat dinner?
  - Yes.  Yes I should.


h2. Environment
{code}

Bundler   1.14.6
Rubygems  2.5.1
Ruby      2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
GEM_HOME  /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing
GEM_PATH  /home/dunk/.rvm/gems/ruby-2.3.1@puppet-testing:/home/dunk/.rvm/gems/ruby-2.3.1@global
RVM       1.29.1 (latest)
Git       1.9.1
Platform  x86_64-linux
rubygems-bundler (1.4.4)
{code}

h3. Gemfile

{code}

source "https://rubygems.org"

gem 'puppet', '4.7.0'

group :test do
  gem 'puppet-lint', '2.3.0'
  gem 'puppetlabs_spec_helper', '2.2.0'
  gem 'rspec-puppet', '2.6.8'
  gem 'parallel_tests', '2.14.2'
  gem 'rspec-puppet-facts', '1.8.0', :require => false
  gem 'metadata-json-lint', '2.0.2'
  gem 'semantic_puppet', '1.0.1' # this gem is needed (for metadata-json-lint) until we move to Puppet 4.9.x
end

group :development, :test do
  gem 'beaker-rspec', '6.1.0', :require => false
  gem 'pry', '0.10.4'
end
{code}

h3. Gemfile.lock
{code}

GEM
  remote: https://rubygems.org/
  specs:
    CFPropertyList (2.3.5)
    addressable (2.5.1)
      public_suffix (~> 2.0, >= 2.0.2)
    aws-sdk-v1 (1.67.0)
      json (~> 1.4)
      nokogiri (~> 1)
    beaker (3.20.0)
      aws-sdk-v1 (~> 1.57)
      beaker-hiera (~> 0.0)
      beaker-hostgenerator
      beaker-puppet (~> 0.0)
      beaker-vmpooler (~> 0.1)
      docker-api
      fission (~> 0.4)
      fog (~> 1.38)
      google-api-client (~> 0.9)
      hocon (~> 1.0)
      in-parallel (~> 0.1)
{code}

Dan Dunckel (JIRA)

unread,
Aug 24, 2017, 12:34:04 PM8/24/17
to puppe...@googlegroups.com
Dan Dunckel updated an issue
h1. Summary
When I try to perform a
 `  {code} hiera.lookup ` {code}  request in a spec, the hierarchy doesn't find the `kernel` fact.  Because of that, Hiera shows this message:


    Ignoring bad definition in :hierarchy: 'kernel/'

And as a result, the data I need is not found.

h2. spec/spec_helper.rb
{code}
require "puppetlabs_spec_helper/module_spec_helper"
require "rspec-puppet-facts"
include RspecPuppetFacts

# needed to perform lookups inside specs
require "hiera"

RSpec.configure do |c|
  c.default_facts = {
    "hostname"      => `hostname -f`.strip,
    "ipaddress"     => `hostname -i`.strip,
    "puppetversion" => "4.6.1",
  }

  c.fail_fast = true if ENV['FAIL_FAST'] == "true"

  # needed to evaluate hiera lookups in the puppet code
  c.hiera_config = "spec/fixtures/hiera/hiera.yaml"

  if ENV['COVERAGE'] == "true"
    c.after(:suite) do
      RSpec::Puppet::Coverage.report!
    end
  end
end

# Default to "no" as our specs fail on default facts (i.e. "$::networking")
ENV['STRICT_VARIABLES'] = "no" unless ENV['STRICT_VARIABLES']
{code}

h2. spec/fixtures/hiera/hiera.yaml
{code}
:backends:
  - yaml
:yaml:
  :datadir: spec/fixtures/hiera
:hierarchy:
  - "fqdn/%{fqdn}"
  - "environment/%{environment}"
  - "hostgroup/%{hostgroup}"
  - "os/%{operatingsystem}/%{operatingsystemrelease}"
  - "os/%{operatingsystem}"
  - "kernel/%{kernel}"
  - "common"
{code}

h2. spec/fixtures/hiera/os/Ubuntu.yaml
{code}
cron_service: 'cron'
{code}

h2. spec/fixtures/hiera/kernel/Linux.yaml
{code}
cron_service: 'crond'
{code}

h2. manifests/linux.pp
{code}
class ta_base::linux() {
  $cron_service = hiera('cron_service')

  service { $cron_service:
    enable     => true,
    ensure     => true,
    hasrestart => true,
    hasstatus  => true,
  }
}
{code}

h2. spec/classes/linux_spec.rb
{code}
require "spec_helper"

hiera = Hiera.new(:config => "spec/fixtures/hiera/hiera.yaml")

describe "ta_base::linux" do
  on_supported_os.each do |os, facts|
    context "on #{os}" do
      let(:facts) { facts }

      cron_service = hiera.lookup("cron_service", nil, facts)

      it { is_expected.to compile.with_all_deps }

      it { is_expected.to contain_service(cron_service)
        .with(
          "enable"  => true,
          "ensure"  => true
        )
      }
    end
  end
end
{code}

Dan Dunckel (JIRA)

unread,
Aug 24, 2017, 12:35:03 PM8/24/17
to puppe...@googlegroups.com
Dan Dunckel updated an issue
h1. Summary
When I try to perform a {
code noformat }hiera.lookup{ code noformat } request in a spec, the hierarchy doesn't find the `kernel` fact.  Because of that, Hiera shows this message:

Dan Dunckel (JIRA)

unread,
Aug 24, 2017, 12:35:03 PM8/24/17
to puppe...@googlegroups.com
Dan Dunckel updated an issue
h1. Summary
When I try to perform a
 {noformat}  * hiera.lookup {noformat} *  request in a spec, the hierarchy doesn't find the  `  * kernel ` *  fact.  Because of that, Hiera shows this message:

{code}     Ignoring bad definition in :hierarchy: 'kernel/' {code}

Dan Dunckel (JIRA)

unread,
Aug 24, 2017, 12:36:02 PM8/24/17
to puppe...@googlegroups.com
Dan Dunckel updated an issue
h1. Summary
When I try to perform a *hiera.lookup* request in a spec, the hierarchy doesn't find the *kernel* fact.  Because of that, Hiera shows this message:
- *  The last 2 lines being the key difference:
{code}
      Looking for data source kernel/Linux
      Found cron_service in kernel/Linux
{code}
So....
- Am So, am  I configuring something incorrectly?

Dan Dunckel (JIRA)

unread,
Aug 24, 2017, 12:38:11 PM8/24/17
to puppe...@googlegroups.com
Dan Dunckel commented on Bug HI-578
 
Re: Symbolized facts causes "ignore bad definition" error in hierarchy

Stepping thru the code brought me to here: https://github.com/puppetlabs/hiera/blob/3.4.0/lib/hiera/backend.rb#L332

where Hiera checks if fqdn is a key of the facts hash via value.include?(segment)

  • value is the facts hash
  • segment is the variable to be interpolated, in this case: fqdn

Because the hash has fqdn as a Symbol and not a String, the check fails, resulting in the Ignoring bad definition in :hierarchy: 'fqdn/' error.

Dan Dunckel (JIRA)

unread,
Aug 24, 2017, 12:39:02 PM8/24/17
to puppe...@googlegroups.com

Moses Mendoza (JIRA)

unread,
Aug 28, 2017, 6:37:03 PM8/28/17
to puppe...@googlegroups.com

Moses Mendoza (JIRA)

unread,
Aug 28, 2017, 6:37:03 PM8/28/17
to puppe...@googlegroups.com

Henrik Lindberg (JIRA)

unread,
Dec 6, 2017, 4:25:03 AM12/6/17
to puppe...@googlegroups.com
Henrik Lindberg commented on Bug HI-578
 
Re: Symbolized facts causes "ignore bad definition" error in hierarchy

Dan Dunckel What I want to know is how facts ended up being encoded with symbols as keys - they are not supposed to be in that form. I therefore think that the problem is in how tests are set up. That is - something brought in by these:

require "puppetlabs_spec_helper/module_spec_helper"
require "rspec-puppet-facts"
include RspecPuppetFacts

We do not want $facts to be populated with symbol keys. While that may have worked in the past because there was string/symbol monkey patching in place that equated symbols with strings - that monkeypatch is gone in puppet 5 because of all the problems it created. We do not want that back and the correct thing to do here is to ensure that tests feed the right data into the rest of the machinery.

The proposed solution, to search again with the key turned into a symbol has several other problems:

  • it makes all searches slower since for every key that is not found in a particular backend, there would be an extra lookup.
  • hiera 3 is deprecated in favor of hiera 5, and the change does not carry forward

I think the real solution is to fix the testing framework.

This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db)
Atlassian logo

Henrik Lindberg (JIRA)

unread,
Dec 8, 2017, 4:27:03 AM12/8/17
to puppe...@googlegroups.com
Henrik Lindberg assigned an issue to Dan Dunckel
 
Change By: Henrik Lindberg
Assignee: Henrik Lindberg Dan Dunckel

Michael Watters (JIRA)

unread,
Aug 15, 2018, 9:16:03 AM8/15/18
to puppe...@googlegroups.com
Michael Watters commented on Bug HI-578
 
Re: Symbolized facts causes "ignore bad definition" error in hierarchy

Has anybody found a solution for this issue?  I have rspec tests failing because facts are not resolving properly.  When rspec runs I see the same error as originally reported.

DEBUG: 2018-08-15 09:08:15 -0400: Hiera YAML backend starting
DEBUG: 2018-08-15 09:08:15 -0400: Looking up chrony::packages in YAML backend
DEBUG: 2018-08-15 09:08:15 -0400: Ignoring bad definition in :hierarchy: ''

hiera.yaml looks like this. 

---
:backends:
  - yaml
:hierarchy:
  - "%{operatingsystem}"
  - common
:yaml:
  :datadir: 'data'

 

 While I can hack around this I'd rather have tests for each operating system run in the same code block.

Here are the gems that I have installed.

puppet (5.5.3, 4.2.3)
puppet-blacksmith (3.4.0)
puppet-debugger (0.9.0, 0.8.1)
puppet-lint (2.3.3, 2.3.0)
puppet-module-posix-default-r2.3 (0.1.1)
puppet-module-posix-dev-r2.3 (0.0.1)
puppet-module-posix-system-r2.3 (0.1.1)
puppet-retrospec (1.4.1)
puppet-syntax (2.4.1)
puppet_facts (0.2.1)
puppetlabs_spec_helper (2.4.0, 2.3.2
 
rspec (3.7.0)
rspec-core (3.7.1, 3.7.0)
rspec-expectations (3.7.0, 3.6.0)
rspec-its (1.2.0)
rspec-mocks (3.7.0)
rspec-puppet (2.6.14, 2.6.9)
rspec-puppet-facts (1.8.0)
rspec-puppet-utils (3.4.0)
rspec-support (3.7.0, 3.6.0)

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Josh Cooper (JIRA)

unread,
Jan 22, 2020, 10:53:05 PM1/22/20
to puppe...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages