Jira (PDB-4326) PuppetDB rejects catalogs with tags added automatically by Puppet

2 views
Skip to first unread message

Olav Morken (JIRA)

unread,
Apr 2, 2019, 8:54:03 AM4/2/19
to puppe...@googlegroups.com
Olav Morken created an issue
 
PuppetDB / Bug PDB-4326
PuppetDB rejects catalogs with tags added automatically by Puppet
Issue Type: Bug Bug
Affects Versions: PDB 6.3.0
Assignee: Unassigned
Attachments: puppetdb.log
Components: PuppetDB
Created: 2019/04/02 5:53 AM
Environment:
  • PuppetDB 6.3.0 on Debian Stretch: 6.3.0-1stretch
  • Puppet server 6.3.0 on Debian Stretch: 6.3.0-1stretch
  • Puppet agent 6.4.0 on Debian Stretch: 6.4.0-1stretch

 

Priority: Normal Normal
Reporter: Olav Morken

Puppet automatically tags resources with the resource title, if it considers the title to be a valid tag name:

https://github.com/puppetlabs/puppet/blob/6.4.0/lib/puppet/resource.rb#L325

To determine if it is a valid tag name, it uses the following regex:

  ValidTagRegex = /\A[[:alnum:]][[:alnum:]:.-]*\Z/u

(https://github.com/puppetlabs/puppet/blob/6.4.0/lib/puppet/util/tagging.rb#L4)

Since PUP-7579, this regex allows unicode characters. The result is that when you have a resource with a non-ascii characters in its title, it will still add a tag with that title. This can be triggered by for example:

{{  notify { 'norwegian_characters_æøå': }}}

This is legal according to the documentation:

Resource titles can contain any characters whatsoever. They are case-sensitive.

(https://puppet.com/docs/puppet/6.4/lang_reserved.html#resources)

The problem is that PuppetDB has not been updated to allow unicode characters in tags. Thus there is a mismatch between what the Puppet server considers to be a valid tag name and what PuppetDB considers to be a valid tag name.

This in turn causes PuppetDB to fail to store the catalog for the node. The Puppet server does not log anything except the normal "Puppet 'replace_catalog' command for <hostname> submitted to PuppetDB with UUID <...>", however, looking at the logs in PuppetDB, we find:

2019-04-02T14:40:14.185+02:00 ERROR [p.p.command] [81,969] [replace catalog] Fatal error on attempt 0 for vltrd077.cn.uninett.no
clojure.lang.ExceptionInfo: throw+: {:fatal true, :cause #error {
:cause "Resource '{:type \"Notify\", :title \"norwegian_characters_?\"}' has an invalid tag 'norwegian_characters_?'. Tags must match the pattern /\\A[a-z0-9_][a-z0-9_:\\-.]*
Z/."

(I have attached the full stack trace as puppetdb.log.)

Since PuppetDB does not store the new catalog for the node, we cannot query PuppetDB for resources from that node.

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Olav Morken (JIRA)

unread,
Apr 2, 2019, 8:57:02 AM4/2/19
to puppe...@googlegroups.com
Olav Morken updated an issue
Change By: Olav Morken
Puppet automatically tags resources with the resource title, if it considers the title to be a valid tag name:



To determine if it is a valid tag name, it uses the following regex:

{{  ValidTagRegex = /\A[[:alnum:]_][[:alnum:]_:.-]*\Z/u}}

([https://github.com/puppetlabs/puppet/blob/6.4.0/lib/puppet/util/tagging.rb#L4)]


Since PUP-7579, this regex allows unicode characters. The result is that when you have a resource with a non-ascii characters in its title, it will still add a tag with that title. This can be triggered by for example:

{{  notify \{ 'norwegian_characters_æøå': }}}

This is legal according to the documentation:
{quote}Resource titles can contain any characters whatsoever. They are case-sensitive.
{quote}
([https://puppet.com/docs/puppet/6.4/lang_reserved.html#resources)]


The problem is that PuppetDB has not been updated to allow unicode characters in tags. Thus there is a mismatch between what the Puppet server considers to be a valid tag name and what PuppetDB considers to be a valid tag name.

This in turn causes PuppetDB to fail to store the catalog for the node. The Puppet server does not log anything except the normal "Puppet 'replace_catalog' command for <hostname> submitted to PuppetDB with UUID <...>", however, looking at the logs in PuppetDB, we find:
{quote}2019-04-02T14:40:14.185+02:00 ERROR [p.p.command] [81,969] [replace catalog] Fatal error on attempt 0 for vltrd077.cn.uninett.no

clojure.lang.ExceptionInfo: throw+: {:fatal true, :cause #error {
:cause "Resource '\{:type \"Notify\", :title \"norwegian_characters_???\"}' has an invalid tag 'norwegian_characters_???'. Tags must match the pattern /\\A[a-z0-9_][a-z0-9_:\\-.]*\\Z/."
{quote}

(I have attached the full stack trace as puppetdb.log.)

Since PuppetDB does not store the new catalog for the node, we cannot query PuppetDB for resources from that node.

Olav Morken (JIRA)

unread,
Apr 2, 2019, 9:03:05 AM4/2/19
to puppe...@googlegroups.com

Olav Morken (JIRA)

unread,
Apr 2, 2019, 9:03:05 AM4/2/19
to puppe...@googlegroups.com

Austin Blatt (JIRA)

unread,
Apr 2, 2019, 1:48:03 PM4/2/19
to puppe...@googlegroups.com

Austin Blatt (JIRA)

unread,
Apr 2, 2019, 1:52:02 PM4/2/19
to puppe...@googlegroups.com
Austin Blatt commented on Bug PDB-4326
 
Re: PuppetDB rejects catalogs with tags added automatically by Puppet

Thanks Olav Morken for the detailed bug report, it helped a lot in tracking down the problem quickly. I've got a PR up to fix this [here|https://github.com/puppetlabs/puppetdb/pull/2882].

Austin Blatt (JIRA)

unread,
Apr 10, 2019, 5:28:03 PM4/10/19
to puppe...@googlegroups.com

Austin Blatt (JIRA)

unread,
Apr 11, 2019, 3:04:04 PM4/11/19
to puppe...@googlegroups.com

Heston Hoffman (JIRA)

unread,
Apr 15, 2019, 5:30:04 PM4/15/19
to puppe...@googlegroups.com

Austin Blatt (JIRA)

unread,
Jun 25, 2019, 6:23:04 PM6/25/19
to puppe...@googlegroups.com
Austin Blatt updated an issue
Change By: Austin Blatt
Fix Version/s: PDB 6.0.4
Fix Version/s: PDB 5.2.9
Reply all
Reply to author
Forward
0 new messages