PuppetDB curl and puppetdbquery tentative to filter nodes on an array type parameter including a value

[Baptiste Grenier]

Hello puppeters,

I have check_mk::host resource exported by my nodes, and this resources
contain a host_tags parameter that is an array.

I am trying to query PuppetDB to filer my nodes to retrieve all the nodes
exporting this Check_mk::Host resources containing a specific value (nat) in the
host_tags array.

My ultime goal is to be able to write something like this in my manifests, to
retrieve all the check_mk hosts that are behind NAT:

----------8<------------------------------------------------------
$nat_nodes = query_nodes('@@Check_mk::Host{host_tags = "nat"}')
----------8<------------------------------------------------------

With the = meaning that the nat tag is included in the host_tags array. (I
would love to have a construction like the in operator of the puppet language [1]...)

So I started playing with the puppet query module [2], the following works:
----------8<------------------------------------------------------
puppet query nodes "@@Check_mk::Host['node1.domain.tld']"
----------8<------------------------------------------------------

But not the following:
----------8<------------------------------------------------------
puppet query nodes '@@Check_mk::Host{host_tags = "nat"}'
----------8<------------------------------------------------------

So I started looking at the raw PuppetDB API using curl to see if I was able to
find the correct syntaxe.

I was able to filter the nodes on the complete host_tags parameter value:
----------8<------------------------------------------------------
curl --cacert /var/lib/puppet/ssl/certs/ca.pem \
--cert /var/lib/puppet/ssl/certs/puppet.domain.tld.pem \
--key /var/lib/puppet/ssl/private_keys/puppet.domain.tld.pem \
'https://puppet.domain.tld:8081/v4/resources' \
-X GET \
--data-urlencode 'query=["and",
["and",
["=", "type", "Check_mk::Host"],
["=", "exported", true]],
["=", ["parameter", "host_tags" ], [ "wan", "nat", "RedHat" ]]]'
----------8<------------------------------------------------------

The output is something like this:

----------8<------------------------------------------------------
[ {
"tags" : [ "profile::check_mk", "check_mk::host", "baseclass", "agent",
"host", "class", "profile", "check_mk::agent", "node1.domain.tld",
"check_mk", "node" ],
"file" : "/etc/puppet/environments/production/modules/check_mk/manifests/agent.pp",
"type" : "Check_mk::Host",
"title" : "node1.domain.tld",
"line" : 28,
"resource" : "0405e0d6caa542a1fc1f348f42524f1f9f9e4875c",
"environment" : "production",
"certname" : "node1.domain.tld",
"parameters" : {
"host_tags" : [ "wan", "nat", "RedHat" ]
},
"exported" : true
},
(...)
]
----------8<------------------------------------------------------

Then I saw in the v4 API documentation [3] that the equality operator on arrays
should math if any one of their elements match, so I tried something like this:

----------8<------------------------------------------------------
curl --cacert /var/lib/puppet/ssl/certs/ca.pem \
--cert /var/lib/puppet/ssl/certs/puppet.domain.tld.pem \
--key /var/lib/puppet/ssl/private_keys/puppet.domain.tld.pem \
'https://puppet.domain.tld:8081/v4/resources' \
-X GET \
--data-urlencode 'query=["and",
["and",
["=", "type", "Check_mk::Host"],
["=", "exported", true]],
["=", ["parameter", "host_tags" ], "nat"]]'
----------8<------------------------------------------------------

But it does not return me anything, the previously matched nodes are not
returned, is such a filtering possible?

Was someone able to do something like this?

If someone have any idea on how to proceed with this... Thanks in advance!

Refs:
[1]- https://docs.puppetlabs.com/puppet/3.7/reference/lang_expressions.html#in
[2]- https://github.com/dalen/puppet-puppetdbquery
[3]- https://docs.puppetlabs.com/puppetdb/2.2/api/query/v4/operators.html#equality

Best,
Baptiste

--
\,,/_[-_-]_\,,/

"What has not been examined impartially has not been well
examined. Skepticism is therefore the first step toward truth."
[Denis Diderot, "Pensees philosophiques"]

[Felix Frank]

Hi,

I didn't read all of your message, but this first section had me
wondering: Are you aware that Puppet resources support actual tags?

@@check_mk::host { 'title': host_tags = [ ... ], tag => 'nat' }

You can then easily collect all tagged resources.

Check_mk::Host<<| tag == 'nat' |>>

The interesting part is that the tags actually *do* form an array, and
that the == operator in the collector *will* check to find wether the
RHS is in the array.

Come to think of it, you might even get away with

Check_mk::Host<<| host_tags == 'nat' |>>

Have you tried this?

Cheers,
Felix