error testing puppet 3.x upgrade: You need rubygems to use Hiera

[Tim Mooney]

All-

I've been struggling with this all afternoon, so it's time to ask for
some help.

The TL;DR version:

On a 2.7.14 puppet client promoted to be a test 3.4.2 master, whenever a
client connects I get:

Warning: Puppet.features.rubygems? is deprecated. Require rubygems in your
application's entry point if you need it.
(at /usr/lib/ruby/site_ruby/1.8/puppet/util/feature.rb:17:in `add')
Error: You need rubygems to use Hiera at /etc/puppet/manifests/users.pp:243 on
node rh6client.example.com
Error: You need rubygems to use Hiera at /etc/puppet/manifests/users.pp:243 on
node rh6client.example.com
Error: You need rubygems to use Hiera at /etc/puppet/manifests/users.pp:243 on
node rh6client.example.com

I have the RHEL-provided rubygems package installed (it gets installed
automatically as a dependency of puppet 3.4.2 from Puppet Labs), so why
the error?


The full version:

We're currently using puppet 2.7.14 with facter 1.5.9, both on master and
on our clients. All of our clients are currently RHEL 5.x or RHEL 6.x,
though we'll likely be adding other OSes once we're on Puppet 3.x.

I'm beginning our testing of 3.4.x by following the documentation here:

http://docs.puppetlabs.com/guides/upgrading.html

I'm following the "Option 1" route, promoting a client to be the master.
The client I'm promoting is a fresh install of RHEL 6.5 with our puppet 2.7.14,
and I have done a puppet apply on it to get all local config in place for
a basic puppet client in our environment.

I then copied all of /etc/puppet and /var/lib/puppet from our
current master over to the client I wish to promote.

To promote the client to be the test master, I uninstall our locally-built
RPMs for facter & puppet, then install the PuppetLabs EL6 rpms for

facter-1.7.4-1.el6
hiera-1.3.0-1.el6
puppet-3.4.2-1.el6
puppet-server-3.4.2-1.el6
rubygem-json-1.5.5-1.el6
ruby-rgen-0.6.5-1.el6

That also auto-installed, for dependencies, Red Hat's packages for

ruby-rdoc-1.8.7.352-13.el6.x86_64
ruby-irb-1.8.7.352-13.el6.x86_64
rubygems-1.3.7-5.el6.noarch

I have updated the auth.conf to have "allow_ip" stanzas for the one custom
file serving location we use.

I start up the temporary puppet master:

# puppet master --no-daemonize --verbose
Notice: Starting Puppet master version 3.4.2

I then pick an existing client, rh6client.example.com, uninstall
facter & puppet, install the Puppet Labs facter & puppet, which also
pull in Puppet Labs' hiera & rubygem-json, as well as the Red Hat ruby-rdoc,
ruby-irb, and rubygems.

I point the client at the temporary master and receive (on the client)
the error:

# puppet agent --server pm-tmp.example.com --test --noop
Info: Retrieving plugin
Info: Loading facts in /var/lib/puppet/lib/facter/printers.rb
Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
Info: Loading facts in /var/lib/puppet/lib/facter/net_location.rb
Info: Loading facts in /var/lib/puppet/lib/facter/biosversion.rb
Info: Loading facts in /var/lib/puppet/lib/facter/net_info.rb
Info: Loading facts in /var/lib/puppet/lib/facter/ipmi_product.rb
Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /var/lib/puppet/lib/facter/pacemaker.rb
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: You
need rubygems to use Hiera at /etc/puppet/manifests/users.pp:243 on node rh6client.example.com
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run


On the temporary server, it has output a bunch of "Info" statements about
processing for auth.conf, but follows that with:

Info: Caching node for rh6client.example.com
Info: Caching node for rh6client.example.com
Warning: Puppet.features.rubygems? is deprecated. Require rubygems in your
application's entry point if you need it.
(at /usr/lib/ruby/site_ruby/1.8/puppet/util/feature.rb:17:in `add')
Error: You need rubygems to use Hiera at /etc/puppet/manifests/users.pp:243 on
node rh6client.example.com
Error: You need rubygems to use Hiera at /etc/puppet/manifests/users.pp:243 on
node rh6client.example.com
Error: You need rubygems to use Hiera at /etc/puppet/manifests/users.pp:243 on
node rh6client.example.com


That part of the manifest looks like this:


# used on the netflow server
@user { 'ncsprime':
home => '/var/netflow/ncsprime',
gid => 'ncsprime',
comment => 'Cisco NCS Prime backup file owner',
membership => inclusive,
uid => '709844',
password => hiera('ncsprime_password', '!!'),
}

Our /etc/puppet/hiera.yaml that I copied from our existing 2.7.14 master
to the promoted 3.4.2 temporary master is

---
:backends: - yaml

:hierarchy: - secure/fqdn/%{clientcert}
- fqdn/%{clientcert}
- secure/location/%{location}
- location/%{location}
- secure/common
- common

:yaml:
:datadir: /etc/puppet/hiera-data




I don't understand what I'm missing; I have the rubygems package installed
on both the promoted master and the client I'm trying to test with. I know
that hiera moved into the core at 3.x and that on my real master I will need
to uninstall the 'rubygem-hiera-puppet' package, but the client I promoted
to be the temporary master has never had that installed.

Google searches for this error just turn up the "install rubygems" response.
Brent Clark posted to this list in December with the same problem, but
there were no follow-up responses.

Any suggestions as to what the problem really is, because it's *not* that
I'm missing rubygems.

Thanks,

Tim
--
Tim Mooney Tim.M...@ndsu.edu
Enterprise Computing & Infrastructure 701-231-1076 (Voice)
Room 242-J6, Quentin Burdick Building 701-231-8541 (Fax)
North Dakota State University, Fargo, ND 58105-5164

[Ramin K]

Any chance you have a symlink to
/usr/lib/ruby/site_ruby/1.8/hiera/backend/ (or something similar)
floating around in your /var/lib/puppet? This was a common way to get
Hiera to work in Puppet 2.7 before the necessary glue was built into
3.x. If you do, remove it and restart.

Ramin

[Tim Mooney]

In regard to: Re: [Puppet Users] error testing puppet 3.x upgrade: You need...:

> Any chance you have a symlink to
> /usr/lib/ruby/site_ruby/1.8/hiera/backend/ (or something similar) floating
> around in your /var/lib/puppet? This was a common way to get Hiera to work in
> Puppet 2.7 before the necessary glue was built into 3.x. If you do, remove it
> and restart.

I checked, and there are no symlinks anywhere in /var/lib/puppet on
the client that was promoted to be the test server:

# cd /var/lib/puppet
# find . -type l -print
# find . -name 'site_ruby' -print
# find . -name 'hiera' -print
# find . -name 'backend' -print
# find . -name '*.rb' -print
#

After I sent my message to the list last night, I also decided to try
deleting everything (except the ssl directory) under /var/lib/puppet
on the promoted client and then re-installing the puppet & puppet-server
packages, and that still didn't make any difference.

I definitely appreciate the response and the suggestions for stuff
to look for, but unfortunately it doesn't seem to be what's causing
the issue.

Tim

[Tim Mooney]

In regard to: [Puppet Users] error testing puppet 3.x upgrade: You need...:

All-

Does anyone have any thoughts on what's causing this issue?

Since I posted the original question, I've also tried starting with
a fresh RHEL 6.5 system that's never had puppet on it, and installed 3.4.2
from Puppet Labs. I still get exactly the same error when a client
connects to the test master: "You need rubygems to use Hiera".

I can't imagine what the problem really is, but it definitely isn't that
rubygems is missing; it gets installed as part of the package dependency
for puppet 3.4.2.

Thanks,

Tim

[Moses Mendoza]

Tim, this is a very odd message that you're getting.

The reason I say so is that code is from the old hiera-puppet, and it
was removed from hiera-puppet before the 1.0.0 release (here's the
pull request: https://github.com/puppetlabs/hiera-puppet/pull/11), and
as such was never part of hiera-puppet that was merged into puppet
"proper" in 3.0.0.

Doesn't seem like it is, but do you possibly have an old hiera-puppet
somewhere that's still getting loaded by your master?

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/alpine.SOC.2.11.1401171651430.29424%40dogbert.cc.ndsu.NoDak.edu.
>
> For more options, visit https://groups.google.com/groups/opt_out.



--
Moses Mendoza
Puppet Labs

Join us at PuppetConf 2014, September 23-24 in San Francisco