Plugins and roles

[Stiv Ostenberg]

Having a devil of a time writing my first plugin.   The idea is simple, but the examples I find are not helping as much as I had hoped.

First off, to make life more interesting,  the plugin is one that is needed for a role.  I am trying to write an ansible cookbook to deploy Filebeat.  I was able to do this fairly easily on Chef (my first Chef project), but Ansible is proving more cantankerous.

So, the problem I am trying to solve is simple, and requires aggregation of values.  I found a plugin that will read the contents of a file, and load into into a variable value.  I plan to modify this so I can send it a list of files, and have it load the contents of ALL the files into a variable.

The problems I am currently struggling with are these:

Where does the plugin go within the role folder hierarchy,  or does it need to be global?  I have found various contradictory sources that state it goes in "plugins/lookup"  or "lookup_plugins",or  in Ansible root.  Below is my heirarchy, and the challenge is that I need to load a list of files to load based on the default, membership in groups or instanceID.  That list will be built by checking for and loading any group.yaml, EC2ID.yaml that exist.  Once I have that list, I need to aggregate the values from the corresponding files in the prospectors folders into a variable I can then plug into my template.

Ansible doesnt handle aggregation in loops well.  I am looking at one or two plugins to do this.  One would be a modification of the example that loads a file into a variable to it takes a file list instead.  

   .gitignore

│   filebeat.yml

│   OPS

│   OPS.pub

│   README.md

│

└───roles

    └───filebeat

        │   .travis.yml

        │   LICENSE

        │   README.md

        │

        ├───defaults

        │       main.yml

        │

        ├───handlers

        │       main.yml

        │

        ├───lookup_plugins

        │       hostip.py

        │       prospectors.py

        │

        ├───meta

        │       main.yml

        │

        ├───tasks

        │       debian.yml

        │       main.yml

        │       redhat.yml

        │

        ├───templates

        │       filebeat-config.yml.j2

        │

        ├───tests

        │       inventory

        │       test.yml

        │

        └───vars

            │   default.yaml

            │   main.yml

            │

            ├───groups

            │       filebeaters.yaml

            │       tomcatters.yaml

            │

            ├───instances

            │       i-e4b354f2.yaml

            │

            └───prospectors

                    Base.yaml

                    Tomcat7-Catalina.out.yaml

                    Tomcat7-custom_access.log.yaml

                    Tomcat7-Localhost.log.yaml

                    var_log_openvpn_log.yaml

[Kai Stian Olstad]

On 21. aug. 2017 20:00, Stiv Ostenberg wrote:
> Having a devil of a time writing my first plugin.

Ansible have a mailing list dedicated to development, you might have
better luck there.

> First off, to make life more interesting, the plugin is one that is needed
> for a role. I am trying to write an ansible cookbook to deploy Filebeat.

That's Chef, in Ansible we do play as in playbook.

> The problems I am currently struggling with are these:
>
> Where does the plugin go within the role folder hierarchy, or does it need
> to be global? I have found various contradictory sources that state it
> goes in "plugins/lookup" or "lookup_plugins",or in Ansible root.

Instead of looking around why not just go to _the source_, the Ansible
docmentation?
https://docs.ansible.com/ansible/latest/dev_guide/developing_plugins.html
https://docs.ansible.com/ansible/latest/dev_guide/developing_plugins.html#distributing-plugins


--
Kai Stian Olstad

[Stiv Ostenberg]

The documentation would indeed be nice, however it really doesn't go into great detail on how plugins are loaded, when they are part of  role. In my specific case, I used the example in _the source_ for a lookup.  I copied their plugin code exactly (to load the data from a file to a variable) since it is close to what I need.  I was able to run things just ducky.  Then I changed the name of my alleged lookup plugin to "prospectors.py" and changed the name of the call to {{ lookup('prospectors', 'vars/prospectors/Base.yaml') }}.  It fails.  That means either my plugin is in the wrong place (and since it is in a role that I need to be portable it should be in the role hierarchy) or there is some reference to the name in the code that I missed.  The documentation doesnt say anything about the relationship between the name of the lookup plugin, and how it is called in the playbook.  Since "file" is a built-in,  the example is not really valid.

Thanks for responding.  Appreciate your taking the time to look at my problem.

[Kai Stian Olstad]

On 21. aug. 2017 21:52, Stiv Ostenberg wrote:
> The documentation would indeed be nice, however it really doesn't go into
> great detail on how plugins are loaded, when they are part of role. In my
> specific case, I used the example in _the source_ for a lookup. I copied
> their plugin code exactly (to load the data from a file to a variable)
> since it is close to what I need. I was able to run things just ducky.

Ansible has a file lookup, so you probably just used that one.

> Then I changed the name of my alleged lookup plugin to "prospectors.py"
> and changed the name of the call to {{ lookup('prospectors',
> 'vars/prospectors/Base.yaml') }}. It fails. That means either my plugin
> is in the wrong place (and since it is in a role that I need to be portable
> it should be in the role hierarchy) or there is some reference to the name
> in the code that I missed. The documentation doesnt say anything about the
> relationship between the name of the lookup plugin, and how it is called in
> the playbook. Since "file" is a built-in, the example is not really valid.

The name of the file is the name of the plugin.

In your case it should be roles/filebeat/lookup_plugins/prospectors.py
and since the plugin is in a role you need to apply the role for the
plugin to be available.


--
Kai Stian Olstad

[Stiv Ostenberg]

Thanks for your help, Kai.  Couldn't have done it alone.  I am now able run the plugin using:

 - set_fact: contents2="{{ lookup('prospectors', 'Base.yaml' ) }}"

Question: What would be the correct syntax to send an array of files to load?  Plugin code appears to loop through (for term in terms) the arguments, but not sure how to send both Base.yaml and Base2.yaml.  Curly brackets, quotes, square brackets, commas, ... Too many options.

The value passed into the plugin for "terms" with the single value syntax above is [u'Base.yaml']

[Pshem Kowalczyk]

Hi,

I don't think there's an easy way of passing an array, the params is just a string that gets parsed in the plugin. I ended up writing my own modules (instead of plugins) for functions that required multiple parameters (or dict/list parameters).

kind regards

Pshem

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/d085c0d4-287f-41f9-9435-b39bc3a4d80f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.