Jira (PUP-8394) cache/locales/config.yaml not stable across runs

2 views
Skip to first unread message

Daniel Parks (JIRA)

unread,
Jan 25, 2018, 2:36:03 AM1/25/18
to puppe...@googlegroups.com
Daniel Parks created an issue
 
Puppet / Bug PUP-8394
cache/locales/config.yaml not stable across runs
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2018/01/24 11:35 PM
Priority: Normal Normal
Reporter: Daniel Parks

Puppet Version: 5.3.4
Puppet Server Version: 2018.1.0-rc8
OS Name/Version: various

/opt/puppetlabs/puppet/cache/locales/config.yaml often changes despite no code changes being made. It appears to be random.

The only changes are the names of the project and package, and that swaps between powershell and mongodb.

This has been going on for a while; I think it was present in Hoyt.

Desired Behavior:

No changes to locales.

Actual Behavior:

Info: Retrieving locales
 
Debug: Dynamically-bound server lookup failed, falling back to server setting
 
Debug: Dynamically-bound port lookup failed; falling back to masterport setting
 
Debug: Failed to load library 'msgpack' for feature 'msgpack': cannot load such file -- msgpack
 
Debug: Puppet::Network::Format[msgpack]: feature msgpack is missing
 
Debug: file_metadata supports formats: json pson yaml
 
Debug: Using cached connection for https://puppet.ops.puppetlabs.net:8140
 
Debug: Caching connection for https://puppet.ops.puppetlabs.net:8140
 
Debug: Using cached connection for https://puppet.ops.puppetlabs.net:8140
 
Debug: Caching connection for https://puppet.ops.puppetlabs.net:8140
 
Debug: Executing: 'diff -u /opt/puppetlabs/puppet/cache/locales/config.yaml /tmp/puppet-file20180124-8206-1t5hn79'
 
Notice: /File[/opt/puppetlabs/puppet/cache/locales/config.yaml]/content: 
--- /opt/puppetlabs/puppet/cache/locales/config.yaml	2018-01-24 23:18:17.000000000 -0800
 
+++ /tmp/puppet-file20180124-8206-1t5hn79	2018-01-24 23:24:12.000000000 -0800
 
@@ -4,11 +4,11 @@
 
 gettext:
 
   # This is used for the name of the .pot and .po files; they will be
 
   # called <project_name>.pot?
 
-  project_name: puppetlabs-mongodb
 
+  project_name: puppetlabs-powershell
 
   # This is used in comments in the .pot and .po files to indicate what
 
   # project the files belong to and should bea little more desctiptive than
 
   # <project_name>
 
-  package_name: puppetlabs-mongodb
 
+  package_name: puppetlabs-powershell
 
   # The locale that the default messages in the .pot file are in
 
   default_locale: en
 
   # The email used for sending bug reports.
 
 
 
Debug: Using cached connection for https://puppet.ops.puppetlabs.net:8140
 
Debug: Caching connection for https://puppet.ops.puppetlabs.net:8140
 
Notice: /File[/opt/puppetlabs/puppet/cache/locales/config.yaml]/content: content changed '{md5}de7cdcbf7623f11daca5fa8cd11dc546' to '{md5}9560ee58ab338acc8caea792e44df8c9'
 
Debug: /File[/opt/puppetlabs/puppet/cache/locales/config.yaml]: The container /opt/puppetlabs/puppet/cache/locales will propagate my refresh event
 
Debug: Finishing transaction 48072140
 
Debug: Evicting cache entry for environment 'production'
 
Debug: Caching environment 'production' (ttl = 0 sec)
 
Debug: Loading external facts from /opt/puppetlabs/puppet/cache/facts.d
 
Debug: Facter: loading external fact directories from config file

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db)
Atlassian logo

Josh Cooper (JIRA)

unread,
Jan 26, 2018, 1:29:03 PM1/26/18
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8394
 
Re: cache/locales/config.yaml not stable across runs

I think this is due to flattening when downloading locale info, similar to how facts with the same name in different modules can collide, and it's random which one wins. This is big problem. Each module's config.yaml needs to be namespaced so it doesn't collide. /cc Maggie Dreyer, Michael Smith, Eric Putnam

Josh Cooper (JIRA)

unread,
Jan 26, 2018, 1:34:05 PM1/26/18
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8394

Confirmed with Maggie Dreyer that config.yaml does not need to be downloaded to the agent. So the locales mount should exclude it from its find and search methods.

Josh Cooper (JIRA)

unread,
Jan 26, 2018, 1:35:04 PM1/26/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
 
Change By: Josh Cooper
Sprint: Platform Core Hopper

Josh Cooper (JIRA)

unread,
Jan 26, 2018, 1:35:04 PM1/26/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Team: Platform Core

Josh Cooper (JIRA)

unread,
Jan 26, 2018, 1:35:04 PM1/26/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sub-team: Coremunity

Josh Cooper (JIRA)

unread,
Jan 26, 2018, 1:35:05 PM1/26/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 5.3.5

Josh Cooper (JIRA)

unread,
Jan 26, 2018, 1:37:02 PM1/26/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Affects Version/s: PUP 5.3.4

Josh Cooper (JIRA)

unread,
Jan 26, 2018, 2:35:02 PM1/26/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 5.4.0

Josh Cooper (JIRA)

unread,
Jan 26, 2018, 2:36:03 PM1/26/18
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8394
 
Re: cache/locales/config.yaml not stable across runs

It looks like the issue is made worse because locales/config.yaml is included in modules as part of modulesync, even if the modules don't yet have any translations like mongodb and powershell.

Josh Cooper (JIRA)

unread,
Jan 26, 2018, 2:52:03 PM1/26/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Acceptance Criteria: If a module with a {{locales/config.yaml}} is installed on the master, then running {{puppet agent --test}} should not pluginsync the file to {{$vardir/locales/config.yaml}}.
If multiple modules with {{locales/config.yaml}} are installed on the master, then running {{puppet agent --test}} should not display a diff, since the file should be downloaded or present on the agent.

Josh Cooper (JIRA)

unread,
Jan 29, 2018, 1:56:02 PM1/29/18
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8394
 
Re: cache/locales/config.yaml not stable across runs

Maggie Dreyer will module translations always be of the form *.po? I'm thinking we should whitelist based on that instead of excluding config.yaml.

Maggie Dreyer (JIRA)

unread,
Jan 29, 2018, 2:01:02 PM1/29/18
to puppe...@googlegroups.com
Maggie Dreyer commented on Bug PUP-8394

Yes. If that should change in the future we can update the whitelist.

Josh Cooper (JIRA)

unread,
Feb 5, 2018, 12:24:02 PM2/5/18
to puppe...@googlegroups.com
Josh Cooper assigned an issue to Josh Cooper
 
Change By: Josh Cooper
Assignee: Josh Cooper
This message was sent by Atlassian JIRA (v7.5.1#75006-sha1:7df2574)
Atlassian logo

Josh Cooper (JIRA)

unread,
Feb 5, 2018, 12:30:03 PM2/5/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sprint: Platform Core Hopper KANBAN

Josh Cooper (JIRA)

unread,
Feb 7, 2018, 2:13:02 AM2/7/18
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8394
 
Re: cache/locales/config.yaml not stable across runs

It looks like puppet already excludes config.yaml when making the locales request. While not the perfect solution, I don't think this specific issue is present in the released version of 5.3.4.

That said, I think we need to remove *.pot from the default pluginsignore setting, since that applies to all plugin related mounts, and instead add that to the locales specific mount.

Josh Cooper (JIRA)

unread,
Feb 7, 2018, 3:03:02 AM2/7/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Flagged: Impediment

Josh Cooper (JIRA)

unread,
Feb 7, 2018, 2:32:03 PM2/7/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Affects Version/s: PUP 5.3.4

Josh Cooper (JIRA)

unread,
Feb 7, 2018, 2:32:04 PM2/7/18
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8394
 
Re: cache/locales/config.yaml not stable across runs

Puppet 5.3.3 didn't make a locales request. Puppet 5.3.4 does make a locales request but excludes *.pot and config.yaml

Info: Retrieving locales
 
<- "GET /puppet/v3/file_metadatas/locales?environment=production&links=follow&recurse=true&source_permissions=ignore&ignore=.svn&ignore=CVS&ignore=.git&ignore=.hg&ignore=%2A.pot&ignore=config.yaml&checksum_type=md5 HTTP/1.1\r\nAccept: application/json, application/x-msgpack, text/pson\r\nX-Puppet-Version: 5.3.4\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nUser-Agent: Puppet/5.3.4 Ruby/2.1.9-p490 (x86_64-darwin15.0)\r\nHost: n57nuqwfsaql7hq.delivery.puppetlabs.net:8140\r\n\r\n"
 
-> "HTTP/1.1 200 OK \r\n"
 
-> "X-Puppet-Version: 5.3.4\r\n"
 
-> "Content-Type: application/json; charset=utf-8\r\n"
 
-> "Server: WEBrick/1.3.1 (Ruby/2.4.3/2017-12-14) OpenSSL/1.0.2n\r\n"
 
-> "Date: Wed, 07 Feb 2018 19:27:28 GMT\r\n"
 
-> "Content-Length: 1090\r\n"
 
-> "Connection: Keep-Alive\r\n"
 
-> "\r\n"
 
reading 1090 bytes...

so even though I have different versions of config.yaml in my modules:

# find /etc/puppetlabs/code/environments/production/modules -name config.yaml -exec md5sum {} \; -print
 
841206ffdcf2451b3c6f45ddf754c0fe /etc/puppetlabs/code/environments/production/modules/concat/locales/config.yaml
 
/etc/puppetlabs/code/environments/production/modules/concat/locales/config.yaml
 
5097a0a36a08ac68835001a96b376b46 /etc/puppetlabs/code/environments/production/modules/apache/locales/config.yaml
 
/etc/puppetlabs/code/environments/production/modules/apache/locales/config.yaml
 
d3795b3334faf13674a5d1e3759064ac /etc/puppetlabs/code/environments/production/modules/stdlib/locales/config.yaml
 
/etc/puppetlabs/code/environments/production/modules/stdlib/locales/config.yaml

They are not downloaded to the agent:

$ find ~/.puppetlabs/opt/puppet/cache/locales
 
/Users/josh/.puppetlabs/opt/puppet/cache/locales
 
/Users/josh/.puppetlabs/opt/puppet/cache/locales/ja
 
/Users/josh/.puppetlabs/opt/puppet/cache/locales/ja/puppetlabs-apache.po
 
/Users/josh/.puppetlabs/opt/puppet/cache/locales/ja/puppetlabs-stdlib.po
 
/Users/josh/.puppetlabs/opt/puppet/cache/locales/puppetlabs-stdlib.pot

I suspect this issue occurred in a prerelease version of puppet in the lead up to the 5.3.4 release. I'm going to close this cannot reproduce.

John Duarte (JIRA)

unread,
Oct 21, 2019, 10:58:06 AM10/21/19
to puppe...@googlegroups.com
John Duarte updated an issue
 
Change By: John Duarte
QA Risk Assessment: Needs Assessment No Action
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages