function that returns parameters from a webservice

[Pearl Raj]

I am trying to write a module containing a function that returns parameters from a webservice - http://localhost:5000/app/api/nodes/node_id. This function should return a hash of configuration settings specific to that host. How do I do this? I am using puppet 3.0.

pseudo code is as follows 

new module.pp 

function get_config_from_host(host_id):

    call http://localhost:5000/app/api/nodes/node_id and get the config hash of host_id

    return config hash

I am new to puppet, so confused whether it is possible to do this in puppet

[Purpose of this exercise is to replace hiera]

[Peter Kristolaitis]

You can write custom functions[1], but I don't think they can return a hash -- only a single value.  This functionality is designed for things like calculating password hashes, etc, not generalized data lookup.  This is probably also completely the wrong approach, in the Puppet model, if your goal is to parameterize entire classes.

A much better solution is to keep Hiera, but write a custom backend[2] to integrate the data returned by your API into Hiera's structure.  This keeps all the benefits of Hiera while also giving you back-end flexibility. 

[1] https://docs.puppet.com/guides/custom_functions.html
[2] https://docs.puppet.com/hiera/3.1/custom_backends.html

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/dc8258ac-59f6-4aa0-b3fa-1c989e39fee6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Henrik Lindberg]

In recent 4.x version of puppet, you can use "data in modules and
environment" using the lookup function. It provides an alternative way
of looking up data to hiera in that you can write a function (named
mymodule::data() for a module, and environment::data() in an
environment. Those functions can be written in the puppet language.

You will still need a function that makes the http call and transforms
the result back to a hash.

BTW: You can certainly return a hash from a custom function.

What you get with hiera (lookup in later versions of puppet) is also
automatic data binding of class parameters. That kind of data injection
is the recommended way to set parameters as it makes your code a lot
cleaner and more flexible to control.

If you just write an arbitrary function you miss out on the automatic
data binding. If you instead wrap that into the data() function
supported as a "backend" (data provider) for lookup you will get both
things.

- henrik

>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com

> <mailto:puppet-users...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/dc8258ac-59f6-4aa0-b3fa-1c989e39fee6%40googlegroups.com

> <https://groups.google.com/d/msgid/puppet-users/dc8258ac-59f6-4aa0-b3fa-1c989e39fee6%40googlegroups.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.

--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

[Pearl Raj]

Peter and Henrik, thanks for writing in.

Let me explain the scenario once. We use puppet 3 with Hiera for appX application. But due to some internal issues (management, company policy), the team which supports the application (Support-appX) do not have access to Hiera and those who manages Hiera (Support-Hiera) refuses to update Hiera parameters and instead has suggested a work around - to avoid Hiera and to replace it with a webservice which essentially mimics Hiera.

1) Sample Webservice which returns host specific json hash
-----------------------------

from flask import abort
{
  "hosts": [
    {
      "description": "host vm 1",
      "done": false,
      "id": 1,
      "title": "xyz",
      "role" : "agent"
     },
     {
      "description": "host vm 2",
      "done": false,
      "id": 2,
      "title": "abc",
      "role" : "loadbalancer" 
     }
  ]
}
@app.route('/appX/hosts/<int:host_id>', methods=['GET'])
def get_host(host_id):
    host = [host for host in hosts if host['id'] == host_id]
    if len(host) == 0:
        abort(404)
    return jsonify({'host': host[0]})

So calling http://localhost:5000/appX/hosts/2 will return the following json hash corresponding to host vm 2 (only representational)

    {
      "description": "host vm 2",
      "done": false,
      "id": 2,
      "title": "abc",
      "role" : "loadbalancer"
 }

2) Create a puppet module with a function "get_host_hash" which essentially returns the above json hash as in the following example :

(ruby example - this need to be replaced by a puppet function get_host_hash)

require 'net/http'

url = 'http://localhost:5000/appX/hosts/2'
resp = Net::HTTP.get_response(URI.parse(url)) # get_response takes an URI object

return resp.body 

3) Write a class to retrieve config for host

class new_appx_class {
    $config = get_host_hash ($::hostname)
    $role = $config['role']

    #calls roll specific wrapper class that passes parameters to underlying appx puppet classes

    class { "appx::$role":
        config => $config
    }
}

4) Wrapper classes for different roles

class appx::loadbalancer($config) {
    class {'lvs'::real_server':
      param1 => $config['lvs::real_server']['param1']
      paramx => $config['lvs::real_server']['paramx']
}

class appx::agent($config) {
    class {'lvs'::agent':
      param1 => $config['lvs::agent']['param1']
      paramx => $config['lvs::agent']['paramx']
}

My doubt is about step 2. Whether it is possible to write a puppet function get_host_hash to call the webservice. 

Thanks and Regards,
Pearl

> <mailto:puppet-users+unsub...@googlegroups.com>.

[Henrik Lindberg]

On 11/06/16 07:57, Pearl Raj wrote:
> Peter and Henrik, thanks for writing in.
>
> Let me explain the scenario once. We use puppet 3 with Hiera for appX
> application. But due to some internal issues (management, company
> policy), the team which supports the application (Support-appX) do not
> have access to Hiera and those who manages Hiera (Support-Hiera) refuses
> to update Hiera parameters and instead has suggested a work around - to
> avoid Hiera and to replace it with a webservice which essentially mimics
> Hiera.
>
> 1) Sample Webservice which returns host specific json hash
> -----------------------------
>
> |fromflask importabort|

> ||{"hosts":[{"description":"host vm 1","done":false,"id":1,"title":"xyz",

> ||||||"role" : "agent"
> || },{"description":"host vm 2","done":false,"id":2,"title":"abc",
> ||||||"role" : "loadbalancer"||
> }]}|
> @app.route('/appX/hosts/<int:host_id>',methods=['GET'])defget_host(host_id):host
> =[host forhost inhosts

> ifhost['id']==host_id]iflen(host)==0:abort(404)returnjsonify({'host':host[0]})|

>
>
> So calling http://localhost:5000/appX/hosts/2

> <http://localhost:5000/app/api/nodes/node_id>will return the following

> json hash corresponding to host vm 2 (only representational)
>
> ||{"description":"host vm 2","done":false,"id":2,"title":"abc",
> "role" : "loadbalancer"
> }||
>
>
> 2) Create a puppet module with a function "get_host_hash" which
> essentially returns the above json hash as in the following example :
>
> (ruby example - this need to be replaced by a puppet function get_host_hash)
>
> require 'net/http'
>
> url = 'http://localhost:5000/appX/hosts/2

> <http://localhost:5000/app/api/nodes/node_id>'

> resp = Net::HTTP.get_response(URI.parse(url)) # get_response takes an URI object
>
> return resp.body
>
>
> 3) Write a class to retrieve config for host
>
> class new_appx_class {
> $config = get_host_hash ($::hostname)
> $role = $config['role']
>
> #calls roll specific wrapper class that passes parameters to
> underlying appx puppet classes
>
> class { "appx::$role":
> config => $config
> }
> }
>
> 4) Wrapper classes for different roles
>
> class appx::loadbalancer($config) {
> class {'lvs'::real_server':
> param1 => $config['lvs::real_server']['param1']
> paramx => $config['lvs::real_server']['paramx']
> }
>
> class appx::agent($config) {
> class {'lvs'::agent':
> param1 => $config['lvs::agent']['param1']
> paramx => $config['lvs::agent']['paramx']
> }
>
> My doubt is about step 2. Whether it is possible to write a puppet
> function get_host_hash to call the webservice.
>

You can write a function like the one you want as a custom function
in Ruby - no problem.

You can also write it as a hiera backend. The hiera.yaml config then
needs to include that backend. This will then give you automatic data
binding. Your code above is a manual approach to achieve what data
binding gives you for free. Writing it that way would be idiomatic puppet.

In your logic above, you will run into errors if values are missing in
the json hash - for example $config['this']['that'] would error if there
is no value at $config['this'].

However...

This would be much easier if you were running 4.x as that would fix both
your technical issues as well as organizational constraints since it is
possible in 4.x to have both global hiera, and hiera data per
environment and per module each with its own autonomous hierarchy. The
two teams are then given control over the respective area. Done - no
need to jump through hoops.

- henrik

> Thanks and Regards,
> Pearl
>
> On Saturday, 11 June 2016 07:14:02 UTC+5:30, Henrik Lindberg wrote:
>
> On 10/06/16 17:53, Pearl Raj wrote:
> > I am trying to write a module containing a function that returns
> > parameters from a webservice
> > - http://localhost:5000/app/api/nodes/node_id

> <http://localhost:5000/app/api/nodes/node_id>. This function should

> > an email to puppet-users...@googlegroups.com <javascript:>
> > <mailto:puppet-users...@googlegroups.com <javascript:>>.

> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/puppet-users/dc8258ac-59f6-4aa0-b3fa-1c989e39fee6%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/dc8258ac-59f6-4aa0-b3fa-1c989e39fee6%40googlegroups.com>
>
> >

> <https://groups.google.com/d/msgid/puppet-users/dc8258ac-59f6-4aa0-b3fa-1c989e39fee6%40googlegroups.com?utm_medium=email&utm_source=footer

> <https://groups.google.com/d/msgid/puppet-users/dc8258ac-59f6-4aa0-b3fa-1c989e39fee6%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> > For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
>
> --
>
> Visit my Blog "Puppet on the Edge"
> http://puppet-on-the-edge.blogspot.se/

> <http://puppet-on-the-edge.blogspot.se/>

>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com

> <mailto:puppet-users...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/6be61bc1-acfa-41db-a530-57a6dec6d5e3%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/6be61bc1-acfa-41db-a530-57a6dec6d5e3%40googlegroups.com?utm_medium=email&utm_source=footer>.