Jira (PUP-9560) Class references in Hiera data: parsing error?

[Stephen Kenny (JIRA)]

Stephen Kenny created an issue

Puppet / PUP-9560 Class references in Hiera data: parsing error? Issue Type: Bug Affects Versions: PUP 6.0.5 Assignee: Thomas Hallgren Components: Hiera & Lookup Created: 2019/03/14 5:21 AM Priority: Normal Reporter: Stephen Kenny Puppet Version: 6.0.5-1stretch Puppet Server Version: 6.2.0-1stretch  OS Name/Version: Debian 9 I'm using the lookup function to create a file resource from Hiera data: '/primary':   ensure: 'directory'   mode: '0755'   owner: 'root'   group: 'root'   before: 'Class[nfs]' This results in the following error when the agent runs on the node: "Server Error: Invalid relationship:  File[/primary] { before => Class[nfs] }, because Class[nfs] doesn't seem to be in the catalog". But here's the thing: if the first letter of the class name is capitalized - to be clear, as in Class[Nfs] - the agent run proceeds smoothly.   This is at odds with what happens if I create the file resource explicitly in a manifest.  In that case, it is not necessary to capitalize the class name in the "before =>" attribute. So on the face of it, it seems something is amiss in the Hiera lookup, or parsing, or ... something ... (Incidentally, this happens no matter which class I'm referencing - it's not restricted to the "nfs" (derdanne-nfs module) class.) Stephen Kenny   Add Comment

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)

[Thomas Hallgren (JIRA)]

Thomas Hallgren commented on PUP-9560

Re: Class references in Hiera data: parsing error? A stack trace from the agent would tell us the origin of the error and the execution path that led up to it. Would it be possible to attach that to this ticket?

Add Comment

[Thomas Hallgren (JIRA)]

Thomas Hallgren assigned an issue to Unassigned

Puppet / PUP-9560

Class references in Hiera data: parsing error?

Change By: Thomas Hallgren Assignee: Thomas Hallgren

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-9560

Re: Class references in Hiera data: parsing error? It is probably the relationship validation logic that does this differently than at apply time. (The "invalid relationship" message indicates this - don't think that comes from anywhere else besides the relationship validator).

Add Comment

[Jorie Tappa (JIRA)]

Jorie Tappa updated an issue

Puppet / PUP-9560

Class references in Hiera data: parsing error?

Change By: Jorie Tappa Team: Server

Add Comment

[Jorie Tappa (JIRA)]

Jorie Tappa updated an issue

Puppet / PUP-9560 Class references in Hiera data: parsing error?

Change By: Jorie Tappa Sub-team: Language

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-9560

Re: Class references in Hiera data: parsing error? Stephen Kenny I tried to reproduce this on latest Puppet 6 with: class test { } notify { "goes first": before => 'Class[test]' } include test And it works with both lower and upper case 'test'. Does this example work for you?

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg assigned an issue to Stephen Kenny

Puppet / PUP-9560

Class references in Hiera data: parsing error?

Change By: Henrik Lindberg Assignee: Stephen Kenny

Add Comment

[Stephen Kenny (JIRA)]

Stephen Kenny commented on PUP-9560

Re: Class references in Hiera data: parsing error? Sorry I let the trail go cold on this - I'll do the testing and get back on here a.s.a.p. - might not be 'til tomorrow morning.

Add Comment

[Stephen Kenny (JIRA)]

Stephen Kenny updated an issue

Puppet / PUP-9560

Class references in Hiera data: parsing error?

Change By: Stephen Kenny Attachment: agent-run.debug

Add Comment

[Stephen Kenny (JIRA)]

Stephen Kenny commented on PUP-9560

Re: Class references in Hiera data: parsing error? Finally getting back to this - big apologies for taking so long.   First of all, I have to contradict an assertion I made in my original post.  I said then that the problem occurred only when I had the file resource data specified in my Hiera data, and that everything was ok if the resource was explicitly defined in the node's manifest (i.e., that in the latter case all was good and it didn't matter if the class name was capitalized or not in the "before =>" attribute). Now, for whatever reason (server & agent updates, or I was simply wrong earlier), I find this is not a correct assertion.  I now see  behaviour that exactly mirrors my Hiera scenario.  That is, in the main manifest, "before => Class[Nfs]" works, but "before => Class[nfs]" fails, with the same error as before:

Invalid relationship: File[/primary] { before => Class[nfs] }, because Class[nfs] doesn't seem to be in the catalog

I am seeing exactly the same behaviour with your test example: "before => 'Class[Test]" results in a successful agent  run; "before => 'Class[test]" fails (with the same kind of error message).   My Puppet server and agent versions are 6.3.0 and 6.4.2, respectively.  The OS for both server and agent is Debian 9.9.  Here are the (failing) code blocks from the node's manifest: file { '/primary':     ensure => 'directory',     owner => 'root',     group => 'root',     mode => '0644',     before => 'Class[nfs]', } include nfs

notify { "goes first":     before => 'Class[test]' } include test

Thomas Hallgren mentioned producing a stack trace.  Attached is the latter part of the output of "puppet agent --test --debug" - if this isn't enough, let me know what would be better  -  agent-run.debug

Add Comment

[Stephen Kenny (JIRA)]

Stephen Kenny commented on PUP-9560

Re: Class references in Hiera data: parsing error?

Sorry, didn't quote the code snippets properly ...

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-9560

Re: Class references in Hiera data: parsing error?

Stephen Kenny To be sure - do you see the problem with my example when running it with puppet apply ? I want the code I showed to be the only code you execute, not embedded into something else. This to assert you have a puppet version that does this fundamental thing correctly. Secondly, if a puppet apply does NOT show the error, then put my example as the only logic executed for a node asking for a catalog. Did that work or not? This to assert if there is a difference between apply (which is all Ruby code, and skips one step compared to a server based compilation), and a request to the server (multiple projects, a slightly different code path).

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-9560

Re: Class references in Hiera data: parsing error?

Stephen Kenny The attached information in your post is not a "stack trace" - you get the stack trace from for example: puppet apply -e "notice(doesnotexist())" --debug With that you would get a stack trace for the error that there is no function "doesnotexist()". What is helpful to us for figuring out what could be wrong is to see such a stacktrace of where the Ruby code was when it encountered the problem. Testing as I suggested with my example - if you get an error from that, then I would want to see that stacktrace.

Add Comment

[Stephen Kenny (JIRA)]

Stephen Kenny commented on PUP-9560

Re: Class references in Hiera data: parsing error?

Your code snippet works fine for me when I execute it via puppet apply Next, the second test:  I created a node manifest with the snippet as the only content.   The agent doesn't like the class definition in the node manifest -  it exits with the error message "Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Could not parse for environment production: Classes, definitions, and nodes may only appear at toplevel or inside other classes". So I removed that first line ("class test { }") from the node's manifest, and created a simple module in my modules directory called "test", with an init.pp that contains only that line. As before, a run of the agent on the node fails when the node manifest says  before => 'Class[test]' but succeeds when it says before => 'Class[Test]' When it fails, the error message from the agent is as before: "Error 500 on SERVER: Server Error: Invalid relationship: Notify[goes first] { before => Class[test] }, because Class[test] doesn't seem to be in the catalog". (I'm not clear on how to generate a stack trace when the server compiles the catalog ...)

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-9560

Re: Class references in Hiera data: parsing error?

Super, thanks Stephen Kenny. The stack trace on the server side will be in the server side logs. To be clear (and to rule out upper/lower case issues) - did you put the test class in the init.pp of a "test" module - and all directory names/file names are in lower case? Now we have a very simple way to reproduce the problem.

Add Comment