Signal boost on HI-490 - moving hiera.yaml out of codedir

56 views
Skip to first unread message

Eric Sorenson

unread,
Mar 14, 2016, 8:43:55 PM3/14/16
to Puppet Developers
As a result of some introspection around r10k workflows, I came to agree with the statement in the title of HI-490: "the location of hiera.yaml in puppet-agent is a mistake." The root of the problem is that the current hiera.yaml is a mixture of global configuration (datadir location, merge behaviour, the backend configuration) and "code" like settings, namely the hierarchy itself. We chose to put it in $codedir but this has caused problems when people try to manage the file with puppet modules because it then conflicts with the control repo/r10k deploy workflow. (The PE-13367 ticket I mention in the description there is about the file sync service, but more generally r10k+webook management runs into the same problem.)

There was some conversation that spun off into a google doc and seemed to coalesce around the following proposal:

1. puppet-agent packaging would be updated to install a default hiera.yaml at $confdir/hiera.yaml
2. both puppet and hiera would check in the old location, $codedir/hiera.yaml, and fall back to the new location $confdir/hiera.yaml 
3. we would document the new location and encourage users to move their hiera.yaml

This then raises the question of when we yank support for the old location, $codedir/hiera.yaml. Here the suggestion is:
1. for puppet-agent this happens in a major release of puppet/hiera/puppet-agent
2. for Puppet Enterprise additionally, we check if there is a $codedir/hiera.yaml and block the upgrade if it exists

I wanted to raise visibility on this and see what the wider puppet-dev audience thought. Please feel free to chime in here or on the ticket and I'll summarize before taking any action.


--eric0

Eli Young

unread,
Mar 14, 2016, 8:49:09 PM3/14/16
to Puppet Developers

One side effect of this change would appear to be the inability to use/test a new hiera, uh, hierarchy or backend in a feature branch without affecting the master branch.

 

Sent from my Windows 10 phone

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/55526912-dd49-4fca-8ec6-2f59da7eca84%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

 

Eric Sorenson

unread,
Mar 14, 2016, 8:54:12 PM3/14/16
to puppe...@googlegroups.com

On Mar 14, 2016, at 5:48 PM, Eli Young <elys...@gmail.com> wrote:

One side effect of this change would appear to be the inability to use/test a new hiera, uh, hierarchy or backend in a feature branch without affecting the master branch.

But ... you already can't do that because the hierarchy's global. Unless I greatly misunderstand something about your workflow.

Enabling per-environment hierarchy changes is a primary goal of environment data: http://docs.puppetlabs.com/puppet/4.3/reference/lookup_quick.html

Eric Sorenson - eric.s...@puppetlabs.com - freenode #puppet: eric0
puppet platform // coffee // techno // bicycles

Eli Young

unread,
Mar 14, 2016, 8:56:22 PM3/14/16
to puppe...@googlegroups.com

You’re right! I was confusing something else. Ignore my previous email.

 

Sent from my Windows 10 phone

 

--

You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

Rob Nelson

unread,
Mar 14, 2016, 9:13:59 PM3/14/16
to puppe...@googlegroups.com
I've not seen a conflict with r10k, can you elaborate on that? Curious if I'm hitting it and not knowing it!

However, I have seen it cause great confusion with modules like hunner/hiera or jlambert121/puppet that want to manage it, because there's an ugly set of possible locations depending on oss vs enterprise and then various versions. There have been a LOT of changes in the location and adding another possible location to everyone's module matrix seems like it may make the problem worse. So when it comes to timing, where it's at now seems like a reasonable location until such time as per environment hiera configs are available, IMHO. 
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


--

R.I.Pienaar

unread,
Mar 15, 2016, 2:23:24 AM3/15/16
to puppe...@googlegroups.com


On 15 Mar 2016, at 02:13, Rob Nelson <rnel...@gmail.com> wrote:

I've not seen a conflict with r10k, can you elaborate on that? Curious if I'm hitting it and not knowing it!

However, I have seen it cause great confusion with modules like hunner/hiera or jlambert121/puppet that want to manage it, because there's an ugly set of possible locations depending on oss vs enterprise and then various versions. There have been a LOT of changes in the location and adding another possible location to everyone's module matrix seems like it may make the problem worse. So when it comes to timing, where it's at now seems like a reasonable location until such time as per environment hiera configs are available, IMHO. 

Per environment configs have been around for a bit in 4. Some bug fixes should land in next and then they should be totally usable I think



On Monday, March 14, 2016, Eric Sorenson <eric.s...@puppetlabs.com> wrote:
As a result of some introspection around r10k workflows, I came to agree with the statement in the title of HI-490: "the location of hiera.yaml in puppet-agent is a mistake." The root of the problem is that the current hiera.yaml is a mixture of global configuration (datadir location, merge behaviour, the backend configuration) and "code" like settings, namely the hierarchy itself. We chose to put it in $codedir but this has caused problems when people try to manage the file with puppet modules because it then conflicts with the control repo/r10k deploy workflow. (The PE-13367 ticket I mention in the description there is about the file sync service, but more generally r10k+webook management runs into the same problem.)

There was some conversation that spun off into a google doc and seemed to coalesce around the following proposal:

1. puppet-agent packaging would be updated to install a default hiera.yaml at $confdir/hiera.yaml
2. both puppet and hiera would check in the old location, $codedir/hiera.yaml, and fall back to the new location $confdir/hiera.yaml 
3. we would document the new location and encourage users to move their hiera.yaml

This then raises the question of when we yank support for the old location, $codedir/hiera.yaml. Here the suggestion is:
1. for puppet-agent this happens in a major release of puppet/hiera/puppet-agent
2. for Puppet Enterprise additionally, we check if there is a $codedir/hiera.yaml and block the upgrade if it exists

I wanted to raise visibility on this and see what the wider puppet-dev audience thought. Please feel free to chime in here or on the ticket and I'll summarize before taking any action.


--eric0

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/55526912-dd49-4fca-8ec6-2f59da7eca84%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

Rob Nelson

unread,
Mar 15, 2016, 4:46:44 AM3/15/16
to puppe...@googlegroups.com
Ah, I misunderstood Eric's response to Eli then. Well as long as the location is the same regardless of edition and only varies by version (I.e. $::puppetversion >= 4.5.0 or similar) it shouldn't muck things up too much. 
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/5C2ED28A-B0F0-4D55-886E-BE22B128FDF5%40devco.net.

For more options, visit https://groups.google.com/d/optout.


--

Reply all
Reply to author
Forward
0 new messages