basic question about hiera

[Albert Shih]

Hi,

Very basic question about hiera file.

I've

hieradata/my_host.yaml

I like at the end of my_host.yaml make something like

include my_host_add.yaml

is it possible ?

The reason is some data is automatically extract from a database, those
data is specific to that host. And because this file is pretty big I don't
known if it's a good idea to use hierachy to include in every host (knowing
I've no way to use those data outside to my_host)

Regards.

JAS

--
Albert SHIH
DIO bâtiment 15
Observatoire de Paris
5 Place Jules Janssen
92195 Meudon Cedex
France
Téléphone : +33 1 45 07 76 26/+33 6 86 69 95 71
xmpp: j...@obspm.fr
Heure local/Local time:
mer 8 jul 2015 15:31:36 CEST

[Toky]

You could use the data in my_host.yaml from multiple profiles/roles as long as you fullyqualify the data source. But to answer your hiera question, I do not think you can "include" another yaml file.

-Miguel

[jcbollinger]

On Wednesday, July 8, 2015 at 8:39:10 AM UTC-5, Albert Shih wrote:

Hi,

Very basic question about hiera file.

I've

  hieradata/my_host.yaml

I like at the end of my_host.yaml make something like

  include my_host_add.yaml

is it possible ?

No, at least not with HIera's built-in YAML back-end.  YAML has no facility such as you describe, so to get it you would need to engage a YAML parser that supported such a thing as an extension.

The reason is some data is automatically extract from a database, those
data is specific to that host. And because this file is pretty big I don't
known if it's a good idea to use hierachy to include in every host (knowing
I've no way to use those data outside to my_host)

The hierarchy is the main tool available for combining data from multiple sources.  If you don't want to use it for your purpose then you always have the alternative of writing a custom Hiera back-end, or, for some purposes, an altogether separate data lookup function.

With that said, I'm not really following the nature of your problem.  How would the availability of an 'include' statement mitigate the data-size issue you are concerned about?


John

[DJ]

are you saying those data is common for all hosts? then may be you can use common.yaml ?

[Albert Shih]

Le 09/07/2015 à 07:37:04-0700, DJ a écrit

> are you saying those data is common for all hosts? then may be you can use
> common.yaml ?

Well....no....

I've a puppet master who « manage » almost every my server. But I still get
lots of « thing » not manage by puppet, thing like networking hardware.

As you guest I can run puppet agent on a hp Procurve.

So because my monitoring server is under the control of puppet I would like
to use puppet to configure the nagios to monitor those hardware.

The list of those hardware is inside some csv file, I can extract a list of
all my Procurve. But I don't known where to put this file.

If I put the content inside common.yaml that « sucks » because those
information has no relation with all node.

If I put the content inside

my_miniroting_server.yaml

....I don't known how to do it because this file is edited by hand. Of
course I can make some very dirty but well...

> Hi,
>
> Very basic question about hiera file.
>
> I've
>
>   hieradata/my_host.yaml
>
> I like at the end of my_host.yaml make something like
>
>   include my_host_add.yaml
>
> is it possible ?
>
> The reason is some data is automatically extract from a database, those
> data is specific to that host. And because this file is pretty big I don't
> known if it's a good idea to use hierachy to include in every host (knowing
> I've no way to use those data outside to my_host)
>
Regards.

JAS

--
Albert SHIH
DIO bâtiment 15
Observatoire de Paris
5 Place Jules Janssen
92195 Meudon Cedex
France
Téléphone : +33 1 45 07 76 26/+33 6 86 69 95 71
xmpp: j...@obspm.fr
Heure local/Local time:

jeu 9 jul 2015 16:43:49 CEST

[DJ]

i did not understand your requirement clearly, if you have puppet agent running on all maachines then you can use facts to get hardware information and use that in hieara hierarchy to differentiate things.

Regards,
DJ

[Albert Shih]

Le 09/07/2015 à 10:55:50-0700, DJ a écrit

> i did not understand your requirement clearly, if you have puppet agent running
> on all maachines then you can use facts to get hardware information and use
> that in hieara hierarchy to differentiate things.

No puppet agent didn't work on the networking device (switch etc..) we got
ip-camera, ip-sensor etc...

That's exactly the point....I don't have the hability to run puppet agent
anywhere.

Regards.

[bert hajee]

Maybe this can help : https://forge.puppetlabs.com/hajee/connect

It is a replacement voor yaml in hiera. The connect language allows you to do an include statement. (See https://github.com/hajee/connect/blob/master/doc/nutshell.md#include-statement). You can also import data from external resources. (See https://github.com/hajee/connect/blob/master/doc/nutshell.md#importing-data)

Hope this helps. If there are any questions, please let me now.

Bert

Op donderdag 9 juli 2015 22:24:31 UTC+2 schreef Albert Shih:

[Albert Shih]

Le 09/07/2015 à 07:05:16-0700, jcbollinger a écrit

>
>
>
> No, at least not with HIera's built-in YAML back-end.  YAML has no facility
> such as you describe, so to get it you would need to engage a YAML parser that
> supported such a thing as an extension.

Ok thanks.

>
> The hierarchy is the main tool available for combining data from multiple
> sources.  If you don't want to use it for your purpose then you always have the
> alternative of writing a custom Hiera back-end, or, for some purposes, an
> altogether separate data lookup function.
>
> With that said, I'm not really following the nature of your problem.  How would
> the availability of an 'include' statement mitigate the data-size issue you are
> concerned about?
>

Well, they are two not very important point :

1/ If i put all data inside my

node.yaml

it's visualy hard to edit that file


2/ If I put (for example)

asset.yaml

in all node using the hiearchy I feel that's not very clean.

I agree...it's not very important thing.

Regards.

[jcbollinger]

On Friday, July 10, 2015 at 7:25:16 AM UTC-5, Albert Shih wrote:

Le 09/07/2015 à 07:05:16-0700, jcbollinger a écrit
>
>
>
> No, at least not with HIera's built-in YAML back-end.  YAML has no facility
> such as you describe, so to get it you would need to engage a YAML parser that
> supported such a thing as an extension.

Ok thanks.

>
> The hierarchy is the main tool available for combining data from multiple
> sources.  If you don't want to use it for your purpose then you always have the
> alternative of writing a custom Hiera back-end, or, for some purposes, an
> altogether separate data lookup function.
>
> With that said, I'm not really following the nature of your problem.  How would
> the availability of an 'include' statement mitigate the data-size issue you are
> concerned about?
>
Well, they are two not very important point :

  1/ If i put all data inside my

        node.yaml

    it's visualy hard to edit that file


  2/ If I put (for example)

      asset.yaml

    in all node using the hiearchy I feel that's not very clean.

If these data are pertinent only to a specific node, then no, you should not put them in a file that provides data for multiple nodes.  I think I'd characterize it a bit differently, though: the data appear to be pertinent to only one operational role (monitoring server), regardless of what machine is assigned that role.  If you happen to organize your manifests around the roles and profiles pattern, then it would be natural to put the monitoring data into a hierarchy level associated with the monitoring server role.

Alternatively, as long as you do not need to obtain the data via automatic class parameter binding, you can leave them completely out of your regular data hierarchy.  Instead, access them at need via one or more explicit hiera() calls, identifying the file containing them as an "override" data source via that function's third parameter.  Possibly the docs would be helpful, but the formatting seems to have been pretty badly borked.


John

[Toky]

This is exactly what I meant, just use explicit hiera calls that will pull from the yaml file that contains all the values.

In hiera data:
/somewhere/inyour/hierarchy/

devies.yaml
---
ipcam::location1::ipaddress: 10.0.0.1
ipcam::location2::ipaddress: 10.1.0.1
...
---EOF---

Then on multiple profiles/manifests you can retrieve the data with:

$ipcam1_address = hiera('ipcam::location1::ipaddress')


You can use the call above from multiple places to pull your data. At this point you would have 1 data file with all your devices data and a way to call that data from multple profiles if needed. There might be better ways to do this but that is how I do it on my ENV.

-Miguel

On Wednesday, July 8, 2015 at 9:39:10 AM UTC-4, Albert Shih wrote: