arrays, loops, etc
seph
their existence" I thought I'd write up the problem I'm running into. I
was chatting on irc about it, I don't think puppet has a clean solution.
Like the other poster, I'm defining an object that takes an array. In
my case, I'm defining gpg keystore, which can contain a number of
keys. (actually part of a larger svn repository object) It would be
called something like:
gpg::keystore{ "/svn/repo/conf/pubring.gpg":
keys => ["XXXXXX", "YYYYYYY"],
}
The obvious way to deal with that array, is to use a require, or to have
the definition directly call the key function.
gpg::addkey{ $keys:
store => $keystore, #as passed in as $name
}
But, this requires that the resources be named with the keyid. Which
fails when I have multiple keystores -- they can't both define
gpg::addkey["XXXXX"].
It's hard to see a nice solution to this. If puppet supported
for loops, I could do something. Or if I could pass some kind of
multidimensional bit in the $name array expansion.
I can get some of it, by inverting the logic. So instead of defining
keystores with key attributes, I instead define keys and the locations
they should get added to. But I find that much harder to maintain, and
it scales differently.
Depending on the details, using inline_template can get part way. But
it's a lot of extra complexity, and I don't think it solves all
problems.
I think I'll probably just rethink my setup, so I only have 1 keystore
per machine, but I'm not very pleased with that. Anyone have any
better suggestion? Any chance at getting better puppet support for
this sort of array handling?
seph
Paul Lathrop
You are making a couple of classic mistakes here. For one, thinking of
definitions as "functions" can only end in tears. Definitions are just
ways of abstracting a collection of resources into a single resource.
The second mistake you make is in thinking procedurally. Puppet's
model is declarative, not procedural. You don't loop through an array
and add each item to a keystore. You declare a keystore. You declare
keys, with parameters saying which stores they should be part of.
That's the way you should model this problem to avoid fighting the
tool.
Since I have a hard time understanding precisely what you are trying
to accomplish, I can't be more specific. But I can advise you take a
step back, remind yourself that definitions are not functions (you
don't "call" a gpg::keystore, you "declare" it, and the terminology is
very important in understanding the model), and then try to rethink
the problem in terms of resources.
--Paul
"My pants growl with the hunger of a thousand bubblebees. And it feels
like a Koala crapped a rainbow in my brain!" -MasterShakezula
seph
I am aware of the whole declarative vs. procedural thing. I do still
have some trouble with it, though I think fewer problems than my first
email implied.
In my case, I declare a keystore. It has parameters like owner, group,
and mode. I'd like to declare which keys are in the keystore as part of
the keystore declaration. But I see no way to accomplish this.
Instead, I need to declare the keys, and which keystores their part
of. This is contrary to my model of declaring the keystore. I want to
say:
define keystore(keys) {
require Keys[$keys]
}
define key() {
# an exec to ensure the key is part of the keystore
}
keystore{/tmp/k1:
keys => Key[xxxxx],
}
keystore{/tmp/k2:
keys => Key[xxxxx, yyyyy],
}
This feels clean and declarative. But, it fails -- both things depend on
Key[xxxxx]. All of the ways I can think to accomplish it, are very
clunky and much more procedural.
Declaring a keystore, and declaring which keys are in which keystores
feels very repetitive, and much more functional. It means specifying
lots of steps on the way, instead of just the end state.
I think it would be really elegant, if $name could be
multidimensional:
define keystore(keys) {
require Keys[$name::$keys]
}
define key() {
$store = $1
$key = $2
# an exec to ensure the key is part of the keystore
}
keystore{/tmp/k1:
keys => Key[xxxxx],
}
keystore{/tmp/k2:
keys => Key[xxxxx, yyyyy],
}
seph
Paul Lathrop
> I am aware of the whole declarative vs. procedural thing. I do still
> have some trouble with it, though I think fewer problems than my first
> email implied.
>
> In my case, I declare a keystore. It has parameters like owner, group,
> and mode. I'd like to declare which keys are in the keystore as part of
> the keystore declaration. But I see no way to accomplish this.
>
> Instead, I need to declare the keys, and which keystores their part
> of. This is contrary to my model of declaring the keystore. I want to
> say:
>
> define keystore(keys) {
> require Keys[$keys]
> }
>
> define key() {
> # an exec to ensure the key is part of the keystore
> }
>
> keystore{/tmp/k1:
> keys => Key[xxxxx],
> }
> keystore{/tmp/k2:
> keys => Key[xxxxx, yyyyy],
> }
>
> This feels clean and declarative. But, it fails -- both things depend on
> Key[xxxxx]. All of the ways I can think to accomplish it, are very
> clunky and much more procedural.
While it may seem clean to you, it doesn't map very well into Puppet.
You can't just have a require hanging out like that in the middle of a
define, just for example. Here's what I'd do (again, not really
understanding the "key" and "keystore" here):
define key($key_arg1, $key_arg2, $keystore) {
# resources to create the key and add it to each keystore.
}
define keystore($keystore_arg1, $path) {
# an exec to create the (empty) keystore
}
key {
"key1":
key_arg1 => value,
key_arg2 => value,
keystore => ["default"],
require => Keystore["default"];
"key2":
key_arg1 => value,
key_arg2 => value,
keystore => ["default", "secure"],
require => Keystore["default", "secure"];
}
keystore {
"default":
keystore_arg1 => value,
path => "/tmp/ks/default";
"secure":
keystore_arg1 => value,
path => "/root/ks/secure";
}
Does that make sense?
--Paul
Thomas Bellman
> While it may seem clean to you, it doesn't map very well into Puppet.
> You can't just have a require hanging out like that in the middle of a
> define, just for example. Here's what I'd do (again, not really
> understanding the "key" and "keystore" here):
[...]
> key {
> "key1":
> key_arg1 => value,
> key_arg2 => value,
> keystore => ["default"],
> require => Keystore["default"];
> "key2":
> key_arg1 => value,
> key_arg2 => value,
> keystore => ["default", "secure"],
> require => Keystore["default", "secure"];
> }
Haven't you just shifted around the problem? You still have an array
that you need to process each element of, only it is now a list of
keystores per key, instead of a list of keys per keystore.
In order to not have any array parameters, you would need to do it
something like this:
keystore { "default": ...; "secure": ...; }
key {
"key1-in-default": key=>"key1", store=>"default", ...;
"key2-in-default": key=>"key2", store=>"default", ...;
"key1-in-secure": key=>"key1", store=>"secure", ...;
"key2-in-secure": key=>"key2", store=>"secure", ...;
}
However, this soon gets pretty tiring, especially if you have many
keys and keystores...
/Bellman
David Schmitt
> Paul Lathrop wrote:
>
>> While it may seem clean to you, it doesn't map very well into Puppet.
>> You can't just have a require hanging out like that in the middle of a
>> define, just for example. Here's what I'd do (again, not really
>> understanding the "key" and "keystore" here):
> [...]
>> key {
>> "key1":
>> key_arg1 => value,
>> key_arg2 => value,
>> keystore => ["default"],
>> require => Keystore["default"];
>> "key2":
>> key_arg1 => value,
>> key_arg2 => value,
>> keystore => ["default", "secure"],
>> require => Keystore["default", "secure"];
>> }
>
> Haven't you just shifted around the problem? You still have an array
> that you need to process each element of, only it is now a list of
> keystores per key, instead of a list of keys per keystore.
The actual execution is always procedural (either within a custom type
or exec). There is no problem to iterate over all keystores _there_.
> In order to not have any array parameters, you would need to do it
> something like this:
>
> keystore { "default": ...; "secure": ...; }
> key {
> "key1-in-default": key=>"key1", store=>"default", ...;
> "key2-in-default": key=>"key2", store=>"default", ...;
> "key1-in-secure": key=>"key1", store=>"secure", ...;
> "key2-in-secure": key=>"key2", store=>"secure", ...;
> }
>
> However, this soon gets pretty tiring, especially if you have many
> keys and keystores...
I would recommend such a notation only where you want different
parameters for the same key in different keystores. Then I'd go for
something like this:
key {
"/path/to/keystores/storefile1/keyname1": param => foo;
"/path/to/keystores/storefile1/keyname2": param => foo;
"/path/to/keystores/storefile1/keyname3": param => foo;
"/path/to/keystores/storefile2/keyname1": param => bar;
"/path/to/keystores/storefile2/keyname2": param => bar;
"/path/to/keystores/storefile2/keyname3": param => bar;
}
Where Key["/foo/bar"] denotes the key "bar" in the keystore located at
"/foo", with an automatic dependency from Key["/foo/bar"] to
Keystore["/foo"].
Regards, DavidS
Thomas Bellman
> Thomas Bellman wrote:
>> Haven't you just shifted around the problem? You still have an array
>> that you need to process each element of, only it is now a list of
>> keystores per key, instead of a list of keys per keystore.
>
> The actual execution is always procedural (either within a custom type
> or exec). There is no problem to iterate over all keystores _there_.
Yes, but Paul seemed to argue that having array parameters and looping over
them was "un-Puppet-like" and bad design. And then he suggested a variant
that still used array parameters. I happen to disagree with that sentiment.
If you want to use an exec to process the elements of the array, you need
to be able to pass that array to exec someway. The only way I can think
of in standard Puppet is to use inline_template() to create the command;
not exactly pretty...
Custom types are nice, but I think most Puppet users don't want to write
them, and would rather stay within the Puppet language.
That said, there is a way to loop over arrays within Puppet, and that
is by passing it as the name to a helper definition. The OP would do
something like this:
define keystore($keys, ...)
{
keystore_helper {
$keys: ...;
}
...
}
define keystore_helper(...)
{
# Each element of $keys will be available as $name here
# Do something with it here
...
}
This is not unproblematic at the moment, though. Since the name must
be unique for each keystore_helper declaration, that makes it impossible
to have more than one keystore declaration with the same keys parameter.
One would have to preprocess the $keys parameter to prepend the keystore
name to each element, and there's no way to do that in stock 0.24.8.
(I think you will be able to do it in 0.25, by abusing inline_template()
and split(), but that's rather ugly.)
/Bellman
seph
> Yes, but Paul seemed to argue that having array parameters and looping over
> them was "un-Puppet-like" and bad design. And then he suggested a variant
> that still used array parameters. I happen to disagree with that sentiment.
I think this is about where I'm coming from. I don't think I can define
a custom type, and the native puppet support isn't quite enough rope. I
don't think it's a declarative vs. procedural issue.
> That said, there is a way to loop over arrays within Puppet, and that
> is by passing it as the name to a helper definition. The OP would do
> something like this:
>
> define keystore($keys, ...)
> {
> keystore_helper {
> $keys: ...;
> }
> ...
> }
>
> define keystore_helper(...)
> {
> # Each element of $keys will be available as $name here
> # Do something with it here
> ...
> }
>
> This is not unproblematic at the moment, though. Since the name must
> be unique for each keystore_helper declaration, that makes it impossible
> to have more than one keystore declaration with the same keys parameter.
> One would have to preprocess the $keys parameter to prepend the keystore
> name to each element, and there's no way to do that in stock 0.24.8.
> (I think you will be able to do it in 0.25, by abusing inline_template()
> and split(), but that's rather ugly.)
Exactly. I'd love for names to be multidimensional.
seph