Playing with Ansible mutiple callbacks

[grobs]

Hi,

As I began to introduce my projet in another post, I'm currently experimenting a blocking issue.
The thing that I want to do is pretty simple and I think that I am verry close to the solution but I clearly need your help :)

Here is the flow I want to create :
A playbook executes a module (that I quickly wrote) which runs the sell command apt-show-versions, parse the result and then return it into JSON format.
At the end, a callback is responsible for processing the result and then puting it into a database.

Info: this (great !) web page helped me to do it : http://jpmens.net/2012/09/11/watching-ansible-at-work-callbacks/
It almost works but I'm still have an issue (see under).

So here are the files:

ANSIBLE PLAYBOOK:

---
- hosts: webservers
  user: root
  vars:
     name: dpkg Tom
  tasks:
  - name: Verify that apt-show-versions is installed and install it if it is not present
    action: apt pkg=apt-show-versions state=present
  - name: Launch a custom dpkg-show-versions module
    action: dpkg_tom

 

 ANSIBLE MODULE (in bash language because I don't know how to do it in Python):

# DEFINITION DE LA FONCTION PRINCIPALE
main()
{
#    parseArguments $1
    echo -n "{\"changed\": \"False\", \"name\": \"dpkg_tom\", \"packages\": ["

    nombre_lignes=$(apt-show-versions --upgradeable | wc -l)
    compteur=1

    apt-show-versions --upgradeable | while read line
    do
        echo "$line" | sed 's@^\(.*\?\)/\(.*\?\) \(.*\?\) from \(.*\?\) to \(.*\)@{\"\1\":{\"distribution\": \"\2\",\"status\": \"\3\",\"installed_version\": \"\4\",\"available_version\": \"\5\"@';

        echo -ne "}}"

        if [ $compteur -lt $nombre_lignes ]
        then
            echo -ne ","
        fi

        compteur=$[$compteur + 1]
    done

#    echo -ne "\"\":{";
    echo -n "]}"
}

######################################################

main

Info : this module works just fine (tested without the callback).

ANSIBLE CALLBACK:

 import os
import time
import sqlite3

dbname = '/etc/ansible/cmdb.db'
TIME_FORMAT='%Y-%m-%d %H:%M:%S'

try:
    con = sqlite3.connect(dbname)
    cur = con.cursor()
except:
    pass

def log(host, data):

    if type(data) == dict:
        invocation = data.pop('invocation', None)
        if invocation.get('module_name', None) != 'setup':
            return

    facts = data.get('ansible_facts', None)

    now = time.strftime(TIME_FORMAT, time.localtime())

    try:
        # `host` is a unique index
        cur.execute("REPLACE INTO inventory (now, host, arch, dist, distvers, sys,kernel) VALUES(?,?,?,?,?,?,?);",
        (
            now,
            facts.get('ansible_hostname', None),
            facts.get('ansible_architecture', None),
            facts.get('ansible_distribution', None),
            facts.get('ansible_distribution_version', None),
            facts.get('ansible_system', None),
            facts.get('ansible_kernel', None),
        ))
        con.commit()
    except:
        pass

class CallbackModule(object):
    def runner_on_ok(self, host, res):
        log(host, res)

 The problem that I have is the following Python error (error raised during callback execution):

fatal: [127.0.0.1] => Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/ansible/runner/__init__.py", line 236, in _executor
    exec_rc = self._executor_internal(host)
  File "/usr/lib/pymodules/python2.6/ansible/runner/__init__.py", line 292, in _executor_internal
    return self._executor_internal_inner(host, self.module_name, self.module_args, inject, port)
  File "/usr/lib/pymodules/python2.6/ansible/runner/__init__.py", line 435, in _executor_internal_inner
    self.callbacks.on_ok(host, data)
  File "/usr/lib/pymodules/python2.6/ansible/callbacks.py", line 363, in on_ok
    super(PlaybookRunnerCallbacks, self).on_ok(host, host_result)
  File "/usr/lib/pymodules/python2.6/ansible/callbacks.py", line 191, in on_ok
    call_callback_module('runner_on_ok', host, res)
  File "/usr/lib/pymodules/python2.6/ansible/callbacks.py", line 50, in call_callback_module
    method(*args, **kwargs)
  File "/usr/lib/pymodules/python2.6/ansible/callback_plugins/inventory.py", line 43, in runner_on_ok
    log(host, res)
  File "/usr/lib/pymodules/python2.6/ansible/callback_plugins/inventory.py", line 18, in log
    if invocation.get('module_name', None) != 'setup':
AttributeError: 'NoneType' object has no attribute 'get'


FATAL: all hosts have already failed -- aborting

Then, I think that this issue is raised because of the lack of module_name in the result that my module give.
Have I to declare my custom module somewhere to make ansible know the module_name of my module?
Have I to put a module_name attribute in my JSON result?
I know that I have a verry poor level in Python and that I'm a beginner in playing with Ansible features so, here I am :)

Thank you for you awsome project anyway!
Tom

[Michael DeHaan]

Nope, it's saying there is no 'invocation' hash at all in what you
have run. This is probably a minor quirk that should not be the
case, but see example here:

https://github.com/ansible/ansible/blob/devel/plugins/callbacks/log_plays.py

I suspect adding the if clause about 'verbose_override' will fix you up.

> --
>
>

[grobs]

Thanks for your verry impressive reactivity which is really appreciated!
I see the condition that you're talking about, but I'm not sure to understand.
Could you please tell me what you mean by "invocation hash"?
Have I to declare something in my module?
The "setup" module (wrote in Python) don't causes any problem with the callback I gave below so how can I say "this callback has to work ONLY IF the name is 'setup' " if my module don't have a proper name?

Thank you for your help :)
Tom

[Dusan Sovic]

Hi Tom,

try something like this in your callback code (just put if invocation is None: and return......):

    if type(data) == dict:

        invocation = data.pop('invocation', None)

        if invocation is None:

            return

        if invocation.get('module_name', None) != 'setup':

            return

hope it helps you,

Dusan

Dňa piatok, 11. januára 2013 18:18:28 UTC+1 grobs napísal(-a):

[grobs]

Thank you verry much! It works properly :)

Now, how can I do if I want to write another callback which is executed when my module (written in bash) return it's result?
Because if my module dos'nt have any name, I don't see how I can link it to a callback in particular.

Tom

[grobs]

For information, I just rewritten the module in perl to make it just a bit better.
Here is the source code:
8<---------------------------------------------------------------------------------------------------------------------------

#!/usr/bin/perl

use strict;
use warnings;

sub main
{
    print "{\"changed\": \"False\", \"packages\": [";

    my $nbrLignes=`apt-show-versions | wc -l`;
    my $compteur=0;

    for(`apt-show-versions`) {
        $compteur++;

        # Formats attendus :
        #    zlib1g/squeeze uptodate 1:1.2.3.4.dfsg-3
        #    libnss3-1d/squeeze upgradeable from 3.12.8-1+squeeze5 to 3.12.8-1+squeeze6
        #    ansible-provisioning 0.9 installed: No available version in archive

        next if not $_ =~ /
        ^
        ([a-z0-9_-]*?)  # package
        \/              # slash
        (.*?)           # distribution
        \s              #
        (.*?)           # status
        \s              #
        (?:             # groupe non capturant
            from        # from
            \s          #
            (.*?)       # installed_version
            \s          #
            to          # to
            \s          #
        )?              # fin de groupe non capturant
        (.+$)           # available_version
        /ix;            # case Insensitive and eXtended spacing

        my $package             = $1;
        my $distribution        = $2;
        my $status              = $3;
        my $installed_version   = '';
        my $available_version   = '';

        if(defined $4) {
            $installed_version = $4;
            $available_version = $5;
        }
        else {
            $installed_version = $5;
        }

        print "{\"$package\":{\"distribution\": \"$distribution\",\"status\": \"$status\",\"installed_version\": \"$installed_version\",\"available_version\": \"$available_version\"}}";

        if($compteur < $nbrLignes) {
            print ","
        }
    }

    print "]}";
}

######################################################

main;

8<---------------------------------------------------------------------------------------------------------------------------

This module works fine in my playbook but I don't understand how I could give it a name to identify the module in my callback and then use the result of my module.

Any idea?

Tom

[Michael DeHaan]

Callback modules must be written in Python, so I'm fairly confused here.

Regular ansible modules which are executed on remote hosts are
completely different from callbacks.

> --
>
>

[grobs]

Yes, I'm ok with that I think, sorry if I'm not clear in what I say.

The source code that I gave before is the module, not the callback.
The module read the result of apt-show-versions and then return JSON formatted result.
I also have a callback (which is written in Python) which take the result and then put it in a database.
As I have several modules and several callback associated, I want to make my Python callback work ony if this or this module was launched.
To do this, I have those lines in my Python Callback to identify the module which was ran:
8<---------------------------------------------------------------------------------------------------------------------------

 if type(data) == dict:
        invocation = data.pop('invocation', None)
        if invocation is None:
            return

        if invocation.get('module_name', None) != 'the_name_of_my_module':
            return

8<---------------------------------------------------------------------------------------------------------------------------

But as my module is written in Perl (or bash or whatever is not Python), I don't know how to give a name to my module to verify in my callback that the modulethat was ran really is the one I want ("the_name_of_my_module" here).

Hope that my question is more understandable.

Tom

[Michael DeHaan]

On Tue, Jan 15, 2013 at 12:40 PM, grobs <tom.l...@gmail.com> wrote:
> Yes, I'm ok with that I think, sorry if I'm not clear in what I say.
>
> The source code that I gave before is the module, not the callback.
> The module read the result of apt-show-versions and then return JSON
> formatted result.
> I also have a callback (which is written in Python) which take the result
> and then put it in a database.
> As I have several modules and several callback associated, I want to make my
> Python callback work ony if this or this module was launched.
> To do this, I have those lines in my Python Callback to identify the module
> which was ran:
> 8<---------------------------------------------------------------------------------------------------------------------------
>>
>> if type(data) == dict:
>> invocation = data.pop('invocation', None)
>> if invocation is None:
>> return
>> if invocation.get('module_name', None) != 'the_name_of_my_module':
>> return
>
> 8<---------------------------------------------------------------------------------------------------------------------------
>
> But as my module is written in Perl (or bash or whatever is not Python), I
> don't know how to give a name to my module to verify in my callback that the
> modulethat was ran really is the one I want ("the_name_of_my_module" here).
>
> Hope that my question is more understandable.

I see.

Looks like if your module is named 'foo', it doesn't matter what
language it is, it will still appear as foo.

line 423 here

https://github.com/ansible/ansible/blob/devel/lib/ansible/runner/__init__.py

> --
>
>