Problems with new PE 2016.4 install

86 views
Skip to first unread message

Jonathan Gazeley

unread,
Nov 3, 2016, 9:52:35 AM11/3/16
to Puppet Users
Hi folks,

I've been running open-source Puppet 3.x for years but this week I'm
dabbling with a new installation of PE 2016.4. I installed from the
pointy-clicky installer and so far the PE server only has itself in the
inventory, but is failing to do a puppet run. It bails with this error:

Could not retrieve catalog from remote server: Error 500 on SERVER:
Server Error: Evaluation Error: Error while evaluating a Resource
Statement, Evaluation Error: Error while evaluating a Method call, 'dig'
parameter 'data' expects a Collection value, got String at
/opt/puppetlabs/puppet/modules/puppet_enterprise/manifests/master/puppetserver.pp:673:42
on node puppet4-prod.resnet.bris.ac.uk

I think that manifest is something that came with PE and not something
I've installed, so I've no idea where to start. Any ideas?

Thanks,
Jonathan

--
Jonathan Gazeley
Senior Systems Administrator
IT Services
University of Bristol

Justin Stoller

unread,
Nov 3, 2016, 5:30:13 PM11/3/16
to puppet...@googlegroups.com
On Thu, Nov 3, 2016 at 2:52 AM, Jonathan Gazeley <Jonathan...@bristol.ac.uk> wrote:
Hi folks,

I've been running open-source Puppet 3.x for years but this week I'm dabbling with a new installation of PE 2016.4. I installed from the pointy-clicky installer and so far the PE server only has itself in the inventory, but is failing to do a puppet run. It bails with this error:

Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Method call, 'dig' parameter 'data' expects a Collection value, got String at /opt/puppetlabs/puppet/modules/puppet_enterprise/manifests/master/puppetserver.pp:673:42 on node puppet4-prod.resnet.bris.ac.uk

I think that manifest is something that came with PE and not something I've installed, so I've no idea where to start. Any ideas?

Looking at that line in the puppet configuration it seems that we're digging into the mountpoints:

$tmp_mount_options = $::mountpoints.dig( '/tmp', 'options' )

Dig expects to be called on a collection[1], and reading the error I would assume that the fact `mountpoints` is returning a string or that the "/tmp" key within the fact is returning a string. Which is not how Facter should be behaving[2].

Have you changed anything relating to Facter or its configuration?


 - Justin



Thanks,
Jonathan


--
Jonathan Gazeley
Senior Systems Administrator
IT Services
University of Bristol

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/a6b299e2-7ff0-edaa-6f38-4747bb64d758%40bristol.ac.uk.
For more options, visit https://groups.google.com/d/optout.

Peter Huene

unread,
Nov 3, 2016, 5:41:25 PM11/3/16
to puppet...@googlegroups.com
On Thu, Nov 3, 2016 at 10:30 AM, Justin Stoller <jus...@puppet.com> wrote:


On Thu, Nov 3, 2016 at 2:52 AM, Jonathan Gazeley <Jonathan...@bristol.ac.uk> wrote:
Hi folks,

I've been running open-source Puppet 3.x for years but this week I'm dabbling with a new installation of PE 2016.4. I installed from the pointy-clicky installer and so far the PE server only has itself in the inventory, but is failing to do a puppet run. It bails with this error:

Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Method call, 'dig' parameter 'data' expects a Collection value, got String at /opt/puppetlabs/puppet/modules/puppet_enterprise/manifests/master/puppetserver.pp:673:42 on node puppet4-prod.resnet.bris.ac.uk

I think that manifest is something that came with PE and not something I've installed, so I've no idea where to start. Any ideas?

Looking at that line in the puppet configuration it seems that we're digging into the mountpoints:

$tmp_mount_options = $::mountpoints.dig( '/tmp', 'options' )

Dig expects to be called on a collection[1], and reading the error I would assume that the fact `mountpoints` is returning a string or that the "/tmp" key within the fact is returning a string. Which is not how Facter should be behaving[2].

Perhaps we're picking up a (deprecated for forever now) `stringify_facts = true` in Puppet's settings, perhaps from a previous installation?

My guess is that Facter is operating correctly since this fact is tested against the schema definition to verify it is always a map.
 

Have you changed anything relating to Facter or its configuration?


 - Justin



Thanks,
Jonathan

--
Jonathan Gazeley
Senior Systems Administrator
IT Services
University of Bristol

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/a6b299e2-7ff0-edaa-6f38-4747bb64d758%40bristol.ac.uk.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
--
Peter Huene - Senior Software Engineer

peter...@puppet.com | @peterhuene

AJ Johnson

unread,
Nov 3, 2016, 7:05:12 PM11/3/16
to Puppet Users
Hi Jonathan -

Can you run facter -p (interested in what mountpoints is) and `mount`, and paste the output.

Also can you run:

puppet apply -e 'notify { "${::mountpoints}": }; notify { "${::mountpoints["/"]}": }'


Thanks,


-aj

Jonathan Gazeley

unread,
Nov 4, 2016, 10:04:40 AM11/4/16
to puppet...@googlegroups.com
Thanks for your response. Output of those commands follows:

[jg4461@puppet4-prod ~]$ facter -p mountpoints
/,/dev,/sys/kernel/security,/sys/fs/cgroup/systemd,/sys/fs/pstore,/sys/kernel/config,/sys/fs/selinux,/dev/mqueue,/dev/hugepages,/sys/kernel/debug,/boot

[jg4461@puppet4-prod ~]$ mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs
(rw,nosuid,seclabel,size=3996484k,nr_inodes=999121,mode=755)
securityfs on /sys/kernel/security type securityfs
(rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
devpts on /dev/pts type devpts
(rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs
(ro,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup
(rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/blkio type cgroup
(rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup
(rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/devices type cgroup
(rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/net_cls type cgroup
(rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup on /sys/fs/cgroup/perf_event type cgroup
(rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/freezer type cgroup
(rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/hugetlb type cgroup
(rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/memory type cgroup
(rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/cpuset type cgroup
(rw,nosuid,nodev,noexec,relatime,cpuset)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/sda3 on / type ext4 (rw,relatime,seclabel,data=ordered)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs
(rw,relatime,fd=34,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
/dev/sda1 on /boot type ext4 (rw,relatime,seclabel,data=ordered)
tmpfs on /run/user/991 type tmpfs
(rw,nosuid,nodev,relatime,seclabel,size=801120k,mode=700,uid=991,gid=460)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
tmpfs on /run/user/56933 type tmpfs
(rw,nosuid,nodev,relatime,seclabel,size=801120k,mode=700,uid=56933,gid=464)

[jg4461@puppet4-prod ~]$ puppet apply -e 'notify { "${::mountpoints}":
}; notify { "${::mountpoints["/"]}": }'
Error: Evaluation Error: The value '/' cannot be converted to Numeric.
at line 1:59 on node puppet4-prod.resnet.bris.ac.uk

Thanks,
Jonathan

On 03/11/16 17:26, AJ Johnson wrote:
> Hi Jonathan -
>
> Can you run facter -p (interested in what mountpoints is) and `mount`,
> and paste the output.
>
> Also can you run:
>
> puppet apply -e 'notify { "${::mountpoints}": }; notify {
> "${::mountpoints["/"]}": }'
>
>
> Thanks,
>
>
> -aj
>
>
> On Thursday, November 3, 2016 at 2:52:35 AM UTC-7, Jonathan Gazeley wrote:
>
> Hi folks,
>
> I've been running open-source Puppet 3.x for years but this week I'm
> dabbling with a new installation of PE 2016.4. I installed from the
> pointy-clicky installer and so far the PE server only has itself in the
> inventory, but is failing to do a puppet run. It bails with this error:
>
> Could not retrieve catalog from remote server: Error 500 on SERVER:
> Server Error: Evaluation Error: Error while evaluating a Resource
> Statement, Evaluation Error: Error while evaluating a Method call,
> 'dig'
> parameter 'data' expects a Collection value, got String at
> /opt/puppetlabs/puppet/modules/puppet_enterprise/manifests/master/puppetserver.pp:673:42
>
> on node puppet4-prod.resnet.bris.ac.uk
> <http://puppet4-prod.resnet.bris.ac.uk>
>
> I think that manifest is something that came with PE and not something
> I've installed, so I've no idea where to start. Any ideas?
>
> Thanks,
> Jonathan
>
> --
> Jonathan Gazeley
> Senior Systems Administrator
> IT Services
> University of Bristol
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/ea15755d-5e67-4333-96f4-870d239287a7%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/ea15755d-5e67-4333-96f4-870d239287a7%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.


Jonathan Gazeley

unread,
Nov 4, 2016, 10:07:07 AM11/4/16
to puppet...@googlegroups.com
No, I haven't knowingly changed anything about Facter's configuration.
In fact the config file I'm using is migrated from a previous Puppet 3
install and explicitly has stringify_facts = false.

Thanks,
Jonathan

On 03/11/16 17:41, Peter Huene wrote:
> On Thu, Nov 3, 2016 at 10:30 AM, Justin Stoller <jus...@puppet.com
> <mailto:jus...@puppet.com>> wrote:
>
>
>
> On Thu, Nov 3, 2016 at 2:52 AM, Jonathan Gazeley
> <Jonathan...@bristol.ac.uk
> <mailto:Jonathan...@bristol.ac.uk>> wrote:
>
> Hi folks,
>
> I've been running open-source Puppet 3.x for years but this week
> I'm dabbling with a new installation of PE 2016.4. I installed
> from the pointy-clicky installer and so far the PE server only
> has itself in the inventory, but is failing to do a puppet run.
> It bails with this error:
>
> Could not retrieve catalog from remote server: Error 500 on
> SERVER: Server Error: Evaluation Error: Error while evaluating a
> Resource Statement, Evaluation Error: Error while evaluating a
> Method call, 'dig' parameter 'data' expects a Collection value,
> got String at
> /opt/puppetlabs/puppet/modules/puppet_enterprise/manifests/master/puppetserver.pp:673:42
> on node puppet4-prod.resnet.bris.ac.uk
> <http://puppet4-prod.resnet.bris.ac.uk>
> it, send an email to puppet-users...@googlegroups.com
> <mailto:puppet-users%2Bunsu...@googlegroups.com>.
> <https://groups.google.com/d/msgid/puppet-users/a6b299e2-7ff0-edaa-6f38-4747bb64d758%40bristol.ac.uk>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/CA%2B%3DBEqU%3D1rQq%2Bws%3DjpybcxJWWfeKsu-WKJXx%2BSpnr_ebhuCE2Q%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-users/CA%2B%3DBEqU%3D1rQq%2Bws%3DjpybcxJWWfeKsu-WKJXx%2BSpnr_ebhuCE2Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
>
>
>
> --
> --
> Peter Huene - Senior Software Engineer
> peter...@puppet.com <mailto:peter...@puppet.com> | @peterhuene
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/CACZQQfMDTCxkzq1zGXxO2t%2BOnsJUyRB0X88%2BBWa%2BKZZY_vE3Yw%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-users/CACZQQfMDTCxkzq1zGXxO2t%2BOnsJUyRB0X88%2BBWa%2BKZZY_vE3Yw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.


--

John McCabe

unread,
Nov 4, 2016, 11:27:49 AM11/4/16
to Puppet Users
Hi Jonathan,
Could you also share the facter version/path and the OS that you've encountered this on.

facter --version
which facter

I ask because the `facter -p mountpoints` output you're getting doesn't match what I see, which shows output for physical mounts only, for example:

root@masterlocal:~/puppet-enterprise-2016.4.0-ubuntu-16.04-amd64# facter -p mountpoints
{
  / => {
    available => "15.62 GiB",
    available_bytes => 16771174400,
    capacity => "16.00%",
    device => "/dev/mapper/localhost--vg-root",
    filesystem => "ext4",
    options => [
      "rw",
      "relatime",
      "errors=remount-ro",
      "data=ordered"
    ],
    size => "18.60 GiB",
    size_bytes => 19966849024,
    used => "2.98 GiB",
    used_bytes => 3195674624
  },
  /boot => {
    available => "419.11 MiB",
    available_bytes => 439472128,
    capacity => "11.13%",
    device => "/dev/sda1",
    filesystem => "ext2",
    options => [
      "rw",
      "relatime",
      "block_validity",
      "barrier",
      "user_xattr",
      "acl"
    ],
    size => "471.60 MiB",
    size_bytes => 494512128,
    used => "52.49 MiB",
    used_bytes => 55040000
  }
}

Thanks,
John
>     To view this discussion on the web visit
>     https://groups.google.com/d/msgid/puppet-users/CA%2B%3DBEqU%3D1rQq%2Bws%3DjpybcxJWWfeKsu-WKJXx%2BSpnr_ebhuCE2Q%40mail.gmail.com
>     <https://groups.google.com/d/msgid/puppet-users/CA%2B%3DBEqU%3D1rQq%2Bws%3DjpybcxJWWfeKsu-WKJXx%2BSpnr_ebhuCE2Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
>     For more options, visit https://groups.google.com/d/optout
>     <https://groups.google.com/d/optout>.
>
>
>
>
> --
> --
> Peter Huene - Senior Software Engineer
> peter...@puppet.com <mailto:peter...@puppet.com> | @peterhuene
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com

Jonathan Gazeley

unread,
Nov 4, 2016, 2:14:16 PM11/4/16
to puppet...@googlegroups.com
Thanks John. Output follows:

[jg4461@puppet4-prod ~]$ facter --version
3.4.1 (commit d0f32490f41ca12722c13e4c48d4572ae4f9bfdc)

[jg4461@puppet4-prod ~]$ which facter
/usr/local/bin/facter

Cheers,
Jonathan
> <javascript:>
> > <mailto:jus...@puppet.com <javascript:>>> wrote:
> >
> >
> >
> > On Thu, Nov 3, 2016 at 2:52 AM, Jonathan Gazeley
> > <Jonathan...@bristol.ac.uk <javascript:>
> > <mailto:Jonathan...@bristol.ac.uk <javascript:>>> wrote:
> >
> > Hi folks,
> >
> > I've been running open-source Puppet 3.x for years but
> this week
> > I'm dabbling with a new installation of PE 2016.4. I
> installed
> > from the pointy-clicky installer and so far the PE server
> only
> > has itself in the inventory, but is failing to do a puppet
> run.
> > It bails with this error:
> >
> > Could not retrieve catalog from remote server: Error 500 on
> > SERVER: Server Error: Evaluation Error: Error while
> evaluating a
> > Resource Statement, Evaluation Error: Error while
> evaluating a
> > Method call, 'dig' parameter 'data' expects a Collection
> value,
> > got String at
> >
> /opt/puppetlabs/puppet/modules/puppet_enterprise/manifests/master/puppetserver.pp:673:42
>
> > on node puppet4-prod.resnet.bris.ac.uk
> <http://puppet4-prod.resnet.bris.ac.uk>
> > <http://puppet4-prod.resnet.bris.ac.uk
> <javascript:>
> > <mailto:puppet-users%2Bunsu...@googlegroups.com
> <javascript:>>.
> > send an email to puppet-users...@googlegroups.com <javascript:>
> > <mailto:puppet-users...@googlegroups.com <javascript:>>.
> <https://groups.google.com/d/msgid/puppet-users/CA%2B%3DBEqU%3D1rQq%2Bws%3DjpybcxJWWfeKsu-WKJXx%2BSpnr_ebhuCE2Q%40mail.gmail.com?utm_medium=email&utm_source=footer
> > peter...@puppet.com <javascript:> <mailto:peter...@puppet.com
> <javascript:>> | @peterhuene
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Puppet Users" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to puppet-users...@googlegroups.com <javascript:>
> > <mailto:puppet-users...@googlegroups.com <javascript:>>.
> <https://groups.google.com/d/msgid/puppet-users/CACZQQfMDTCxkzq1zGXxO2t%2BOnsJUyRB0X88%2BBWa%2BKZZY_vE3Yw%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/puppet-users/CACZQQfMDTCxkzq1zGXxO2t%2BOnsJUyRB0X88%2BBWa%2BKZZY_vE3Yw%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
> Jonathan Gazeley
> Senior Systems Administrator
> IT Services
> University of Bristol
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/63bea08f-dffa-4e27-b5f3-3903e78b94ff%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/63bea08f-dffa-4e27-b5f3-3903e78b94ff%40googlegroups.com?utm_medium=email&utm_source=footer>.

Peter Huene

unread,
Nov 4, 2016, 2:32:41 PM11/4/16
to puppet...@googlegroups.com

On Nov 4, 2016 3:04 AM, "Jonathan Gazeley" <Jonathan...@bristol.ac.uk> wrote:
>
> Thanks for your response. Output of those commands follows:
>
> [jg4461@puppet4-prod ~]$ facter -p mountpoints
> /,/dev,/sys/kernel/security,/sys/fs/cgroup/systemd,/sys/fs/pstore,/sys/kernel/config,/sys/fs/selinux,/dev/mqueue,/dev/hugepages,/sys/kernel/debug,/boot
>

This looks like it may be coming from a custom fact that is overriding the expected built-in.

Could you run 'facter -d -p mountpoints' and see if the output mentions discovery of a mountpoints.rb or a change in fact value?

Thanks!

> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/060971bb-91aa-1849-7673-cc79dd82fd4a%40bristol.ac.uk.

Jonathan Gazeley

unread,
Nov 7, 2016, 2:46:51 PM11/7/16
to puppet...@googlegroups.com
On 04/11/16 14:32, Peter Huene wrote:
> This looks like it may be coming from a custom fact that is overriding
> the expected built-in.
>
> Could you run 'facter -d -p mountpoints' and see if the output mentions
> discovery of a mountpoints.rb or a change in fact value?
>
> Thanks!
>

You're exactly right. There was a custom mountpoints.rb in an old module
I'm using. Nothing seems to be referencing it and I have no idea why it
was ever needed - it's gone now and it works.

Thanks!
Reply all
Reply to author
Forward
0 new messages