Jira (PUP-10139) Deferring template rendering is awkward

25 views
Skip to first unread message

Ben Ford (JIRA)

unread,
Nov 13, 2019, 2:38:05 PM11/13/19
to puppe...@googlegroups.com
Ben Ford created an issue
 
Puppet / Bug PUP-10139
Deferring template rendering is awkward
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2019/11/13 11:37 AM
Priority: Normal Normal
Reporter: Ben Ford

Currently users have to jump through awkward hoops to render secrets agent side:

$variables = {
  'password' => Deferred('vault_lookup::lookup',
                  ["secret/test", 'https://vault.docker:8200']),
}
 
# compile the template source into the catalog
file { '/etc/secrets.conf':
  ensure  => file,
  content => Deferred('inline_epp',
               [file('mymodule/secrets.conf.epp'), $variables]),
}

The problem with the solution is that the template file has to be stored in the files directory and that's super unintuitive.

After some conversation in Slack, I'm proposing a find_template() function that would make this make a lot more sense.

$variables = {
  'password' => Deferred('vault_lookup::lookup',
                  ["secret/test", 'https://vault.docker:8200']),
}
 
# compile the template source into the catalog
file { '/etc/secrets.conf':
  ensure  => file,
  content => Deferred('inline_epp',
               [find_template('mymodule/secrets.conf.epp').file, $variables]),
}

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Ben Ford (JIRA)

unread,
Nov 13, 2019, 2:41:07 PM11/13/19
to puppe...@googlegroups.com

Josh Cooper (JIRA)

unread,
Nov 13, 2019, 7:43:10 PM11/13/19
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-10139

Does the template need to be in the files directory so that the agent can download it (metadata and content)? It's confusing because we've always said templates need to be in the templates directory: https://puppet.com/docs/puppet/latest/modules_fundamentals.html#templates-in-modules. Maybe we should distinguish between agent vs server side templates so the distinction is clearer?

Henrik Lindberg (JIRA)

unread,
Nov 14, 2019, 5:00:04 AM11/14/19
to puppe...@googlegroups.com

To render on the agent side you need to be using inline_epp since templates in files are not available from the server (as I understand it).
The issue is that the file() function only reads from files and that there is no way to "find a template" so you can give the file() a full path
to a template. By adding that you get the desired functionality - i.e. find_template("mything/foo.epp").file() - and you can now use that with a deferred
call to inline_epp.

The other alternative is to make templates available on the agent side the same way files are (but in a separate namespace). That way, it would be possible to use a deferred call to `epp`.

Ben Ford (JIRA)

unread,
Nov 14, 2019, 12:20:05 PM11/14/19
to puppe...@googlegroups.com
Ben Ford commented on Bug PUP-10139

Josh Cooper currently yes, the template file needs to be in the files directory. This PR adds the ability to get the source of a file in the templates directory by using find_template('name.epp").file)

Ben Ford (JIRA)

unread,
Nov 14, 2019, 12:21:05 PM11/14/19
to puppe...@googlegroups.com
Ben Ford updated an issue
 
Change By: Ben Ford
Currently users have to jump through awkward hoops to render secrets agent side:

{code}

$variables = {
  'password' => Deferred('vault_lookup::lookup',
                  ["secret/test", 'https://vault.docker:8200']),
}

# compile the template source into the catalog
file { '/etc/secrets.conf':
  ensure  => file,
  content => Deferred('inline_epp',
               [file('mymodule/secrets.conf.epp'), $variables]),
}
{code}

The problem with the solution is that the *template* file has to be stored in the *files* directory and that's super unintuitive.

After some conversation in Slack, I'm proposing a {{find_template()}} function that would make this make a lot more sense
in that the template source is still stored in the traditional {{templates}} directory .

{code}

$variables = {
  'password' => Deferred('vault_lookup::lookup',
                  ["secret/test", 'https://vault.docker:8200']),
}

# compile the template source into the catalog
file { '/etc/secrets.conf':
  ensure  => file,
  content => Deferred('inline_epp',
               [find_template('mymodule/secrets.conf.epp').file, $variables]),
}
{code}

Henrik Lindberg (JIRA)

unread,
Nov 14, 2019, 12:58:04 PM11/14/19
to puppe...@googlegroups.com
Henrik Lindberg commented on Bug PUP-10139
 
Re: Deferring template rendering is awkward

I am fine with adding a find_template, it was something I wanted to get done a long time ago but forgot about.

Rob Braden (JIRA)

unread,
Nov 18, 2019, 12:56:03 PM11/18/19
to puppe...@googlegroups.com
Rob Braden updated an issue
 
Change By: Rob Braden
Sprint: Release Engineering Kanban

Rob Braden (JIRA)

unread,
Nov 18, 2019, 12:56:03 PM11/18/19
to puppe...@googlegroups.com
Rob Braden updated an issue
Change By: Rob Braden
Sprint: Release Engineering Kanban Community PRs

Rob Braden (JIRA)

unread,
Nov 18, 2019, 12:56:04 PM11/18/19
to puppe...@googlegroups.com
Rob Braden updated an issue
Change By: Rob Braden
Team: Coremunity

Josh Cooper (JIRA)

unread,
Dec 2, 2019, 5:50:04 PM12/2/19
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 6.12.0

Josh Cooper (JIRA)

unread,
Dec 5, 2019, 2:51:04 PM12/5/19
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sprint: Community PRs Platform Core KANBAN

Melissa Stone (JIRA)

unread,
Dec 6, 2019, 1:22:04 PM12/6/19
to puppe...@googlegroups.com
Melissa Stone commented on Bug PUP-10139
 
Re: Deferring template rendering is awkward

This has passed ci as a part of puppet-agent 6.11.1.111.ga4de1e8

Melissa Stone (JIRA)

unread,
Dec 6, 2019, 1:23:04 PM12/6/19
to puppe...@googlegroups.com

Ben Ford before we can resolve this ticket, can you add release notes?

Melissa Stone (JIRA)

unread,
Dec 11, 2019, 4:45:03 PM12/11/19
to puppe...@googlegroups.com
Melissa Stone updated an issue
 
Change By: Melissa Stone
Release Notes Summary: This makes it easier users to render templates on the agent, meaning that
they can use secret values resolved by deferred functions directly with
e.g. Vault. That means that the master no longer needs keys to the kingdom.
Release Notes: Bug Fix

Melissa Stone (JIRA)

unread,
Dec 11, 2019, 4:46:04 PM12/11/19
to puppe...@googlegroups.com
 
Re: Deferring template rendering is awkward

I just copied the body from the commit message for the release notes. It looked more than sufficient.

Reply all
Reply to author
Forward
0 new messages