Allowing duplicate namevars in custom types

[jwil...@gmail.com]

Hi all!

I'm working on a custom type that applies xpaths to files.  I expect the interface to look something like:

xpath { "some update":
      xpath => "/some/xpath",
      command => "replace",
      value => "newvalue",
      file => "/path/to/file.xml"
}

I'm having a little trouble picking the namevar.  It seems logical that the xpath attribute should be the namevar, but I'm a little hung up on the fact that it has to be unique among all xpath resources.  In theory, I might want to apply the same xpath to many different files but with different values.  I ran across a mention in the documentation that the exec type allows duplicate namevars, but going through the implementation, it isn't clear to me how that is done.  Is it just a matter of each exec declaration having a unique title but the same command?

Thanks for the help!

[Raphink]

If more than one parameter make sense for a namevar, then you can use composite namevars, by using `isnamevar` for sevaral parameters and adding a `self.title_patterns` method to define how to parse the title and feed the namevar parameters.

See https://github.com/hercules-team/augeasproviders_postgresql/blob/master/lib/puppet/type/pg_hba.rb for an example.

[jwil...@gmail.com]

I did try to do that for another type I wrote and it was a mess.  I ended up having a single namevar that required a particular format that my type then processed to get the parameters I needed.  I'll take a look at your example; I'm sure I'll learn something from it.  Maybe it's worth another stab.

In the meantime, I based my new type on the regular augeas type and the exec type and will see if I hit a problem with duplicate namevars.

[jcbollinger]

Multiple Exec resources may have the same command and/or title.  This is a hard-coded special case, and other types cannot emulate it.

No physical resource can be identified by an XPath expression alone.  Only in the context of an XML file does an XPath expression represent anything concrete, and the natural identifier of that thing (or things) is the combination of file and XPath expression.  Your type absolutely begs for a compound namevar.

With that said, I can't say I care for the rest of what I see about your type.  In particular, the presence of a 'command' attribute suggests that you are trying to model an action on some resource, instead of modeling the resource itself.  You probably can make that work, but it runs against Puppet's design and mode of operation, which may end up presenting practical problems.


John

[jwil...@gmail.com]

Well, I was thinking of doing a type similar to what xmlstarlet does, including being able to add and remove nodes.  I don't really need the added behavior of adding and removing nodes from the file right now, so I left command as a future expansion parameter, but it basically only accepts 'replace' and is set to default to that.

 I'm only doing this type because Augeas doesn't like the particular XML file that has the value I need to change, but REXML parses it just fine.  I really just want to do what Augeas can do to files, just using XPaths instead.  I understand your point about the Puppet way of doing things.  This type should really be saying "this XML document should have a path with this value" and, as my example is written, it looks more imperative than that.  In practice, I think will basically be declarative since the only operation the type can do is retrieving an existing XPath, comparing it to the desired value, and then setting it if it doesn't match.

I haven't had a lot of success getting compound namevars to work.  I've added title_patterns to a different custom type before and done all of the basically undocumented steps for making it happen and hit a snag in the Puppet support code that meant my provider didn't get the values.  I have this type doing the XPath operations properly now, so I'll try to go back and tweak it according to the postgres example that Raphink linked and see if I can get compound namevars to work.  Hopefully you or the group will be able to get me on the right path if I can't get them working. :-)

[jcbollinger]

On Thursday, February 12, 2015 at 1:53:25 PM UTC-6, jwil...@gmail.com wrote:

Well, I was thinking of doing a type similar to what xmlstarlet does, including being able to add and remove nodes.  I don't really need the added behavior of adding and removing nodes from the file right now, so I left command as a future expansion parameter, but it basically only accepts 'replace' and is set to default to that.

You ran off the rails at "what xmlstarlet does" (emphsis added), and went rumbling across the countryside with talk about "need[ing ...] added behavior" (emphsis mine).  You are trying to model action rather than state, and that tends to fit poorly into Puppet's scheme of things.

It appears that the resource you want to model is a node (or maybe nodes) in a user-specified XML file, as identified by an XPath expression.  That's fine.  But what details of the state of such a resource can you / should you model?  Here are some possibilities:

• presence / absence
  
• node value (== content for element nodes)
• attribute presence / absence / value when the resource is an element node

In the event that you specify an instance absent, you could perhaps rely on an optional parameter to support matched nodes being replaced with an alternative node instead of being.

In no case does a 'command' have anything to do with the state you are (should be) modeling.  That's something that the type's provider should determine internally in the event that a resource instance is out of sync.
 
Thus, you might support any or all of these, and perhaps more:

# Matching nodes must not appear in the target file;
# the mechanism for removing any that are present is to delete them
xpath { 'Some node':
      file => '/path/to/file.xml',
      xpath => '/some/xpath',
      ensure => 'absent'
}

or

# Matching nodes must not appear in the target file;
# the mechanism for removing any that are present is to replace them
# with the specified alternative.
xpath { 'Some node':
      file => '/path/to/file.xml',
      xpath => '/some/xpath',
      ensure => 'absent',
      replacement => '<different_node/>',
}

or

# The target file must contain (one / all possible) matching element
# nodes, their content must be as specified, and they must have
# (at least / exactly) the specified attributes.
xpath { 'Some node':
  file => '/path/to/file.xml',
  xpath => '/some/xpath',
  ensure => 'present',
  value => 'element content',
  attributes => { 'a' => 'val1', 'b' => 'val2' }  
}

Types should focus on identifying resources and describing their properties.  How to get from here to there (and how to determine whether you are already there) is the domain of your type's provider(s), and should be exposed via the type as little as possible.

I'm only doing this type because Augeas doesn't like the particular XML file that has the value I need to change, but REXML parses it just fine.  I really just want to do what Augeas can do to files, just using XPaths instead.  I understand your point about the Puppet way of doing things.  This type should really be saying "this XML document should have a path with this value" and, as my example is written, it looks more imperative than that.  In practice, I think will basically be declarative since the only operation the type can do is retrieving an existing XPath, comparing it to the desired value, and then setting it if it doesn't match.

Then you don't need a 'command' parameter at all.  It serves no useful purpose, as the needed action, if any, is always determined by the current state and the specified target state.  "[L]ooks more imperative than that" is not an excuse, it's a red flag.

 

I haven't had a lot of success getting compound namevars to work.  I've added title_patterns to a different custom type before and done all of the basically undocumented steps for making it happen and hit a snag in the Puppet support code that meant my provider didn't get the values.  I have this type doing the XPath operations properly now, so I'll try to go back and tweak it according to the postgres example that Raphink linked and see if I can get compound namevars to work.  Hopefully you or the group will be able to get me on the right path if I can't get them working. :-)

Compound namevars are a feature that hasn't seen a lot of love, so I'm not surprised to hear that it's uncomfortable to work with.  That doesn't change that fact that it's by far the best match for what you describe.

I'm glad you have your type functioning adequately.  You are welcome to return for more advice later, but don't be surprised if you hear the same thing when you do.


John

[jwil...@gmail.com]

There's a lot of great design advice in here. Thank you for taking the time to write it up; I'll definitely do a deeper dive on this when I've solved my immediate problem and check this advice against some of the other rules I've written.  I often find Puppet to be as much of a mind-bender as writing Lisp if you are used to a straight procedural mindset.  Especially hard has been the lack of iteration.  I've avoided the temptation of turning on the future parser to get maps and lambdas since I think it would lead down the road of more straight scripting, as you've warned me, instead of staying in a declarative mindset.

Going through the Puppet Types and Providers book, it became obvious that 'ensure' was what I really wanted instead of 'command'.  In the provider, ensure=>present would mean 'create',  ensure=>absent would mean 'delete' and your thoughts on doing some sort of replacement are interesting.  I might simplify it in my provider so the combination of ensure=>present and an is/should mismatch would be an implicit replace.  Maybe a value parameter for doing simple string replacements and a tree parameter  for adding an arbitrary path would be a nice addition.

For the moment, I just need to replace some existing values in an existing XML file, so I'm following YAGNI and probably won't implement the rest.  This type is also hidden behind a type that models a java.util.Preference key and value, so the higher level type is much more Puppet-y.  The plan is ultimately to replace the xpath type with a simple Augeas call once its parsing bug is fixed. Augeas is a lot easier to deal with than straight XPath and doing a full featured XPath is way beyond the scope of what I'm doing.

I also did get the compound namevar working.  This time through it was much easier.  My first custom type was learning a lot of framework stuff and adding that undocumented bit was just too much.  My other type also implements prefetch and instances, which I think interacted badly with compound namevars in some fashion.  Now that I have a good working example, I might go back and play with this some more.

Thanks again for your help!