ansible-playbook output grep/diff friendly

[Jeremy Schneider]

Something I've been doing a lot of is parsing the output of ansible-playbook with diff or grep.  Specific use cases:

1) auto-deploy script which triggers a subversion pull on a group of hosts after ever commit to the repo, grep for "changed" in the output of ansible-playbook and include this concise output in the short summary email

2) audit script which runs ansible-playbook --check twice daily and does a diff with expected output and emails differences only (if any are present)

Both cases are tricky since the line containing the server name and play outcome doesn't have the name of the play.  The second case is especially tricky since the host lines can appear in any order from ansible-playbook order, and diff doesn't like that.  In order to make my scripts work, I do this:

1) ansible-playbook ... | awk '/^TASK:/{task=$0;next}/^PLAY/{task=""}/^NOTIFIED/{task=""}{if(task&&$0){print$0" "task}}' | sort -k2,10 --stable | grep ^changed

2) ansible-playbook ... --check | awk '/^TASK:/{task=$0;next}/^PLAY/{task=""}/^NOTIFIED/{task=""}{if(task&&$0){print$0" "task}}'  | sort -k2,10 --stable

Question: is there any existing option in ansible-playbook that I just missed which can make the output more deterministic and line-processing-friendly/concise?  If not, would others find an option like this useful if it were proposed?

-Jeremy

[James Cammarata]

It might be a bit easier to write scripts in a higher level language like python, so that you could directly interpret the JSON output.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

James Cammarata <jcamm...@ansibleworks.com>
Sr. Software Engineer, AnsibleWorks, Inc.
http://www.ansibleworks.com/

[Jeremy Schneider]

How do you make ansible-playbook give JSON output?  I don't see the flag in the help output.

[James Cammarata]

My apologies, when using -v it does output the results in JSON however all of the output is not in JSON, so it would still require some parsing. However, I would still resort to a higher level language like python to do that. For direct access to the JSON, you could use the Ansible API which would allow you to bypass any additional parsing.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Jeremy Schneider]

My knee-jerk reaction is that switching to python will turn my 3-line bash script into something not quite so simple, but admittedly I'm a lot better with bash than python.  is this just as simple in python?

ansible-playbook config.yml --check --extra-vars audit=y|awk '/^TASK:/{task=$0;next}/^PLAY/{task=""}/^NOTIFIED/{task=""}{if(task&&$0){print$0" "task}}' |LC_ALL=C sort -k2,10 --stable >/tmp/audit-current.out

diff audit-expected.out /tmp/audit-current.out | grep "^>"

cp /tmp/audit-current.out audit-expected.out

it runs twice daily from crontab and only generates output if someone messed with a server's config outside of ansible.  where there is output, it's concise and cron emails it to the whole group.

[Michael DeHaan]

It's not going to be a core use case for us to output a JSON structure that is parseable by machines, because then the output would not be real time.

The AWX API provides a really good way to get job events if you want to do that from REST and don't want to use the Python API and writing your own callback.

[Jeremy Schneider]

Right - anything that changes the output (server-results) to come in a deterministic order would not be real-time.

However, going back to my original question, how about simply a flag to give line-based output?  That is, supress the headings and instead show the play and task name on the same line as the server name.  This is a simple change to the output formatting but would make a world of difference to all of us who use any sort of unix-y (i.e. line-based) tool.  It can still be real-time output but would be *much* more friendly to pipelines.

Example output, currently what I'm using awk to generate:

skipping: [server1] TASK: [get latest oracle scripts]   PLAY [environment for grid user]  

skipping: [server1] TASK: [get latest oracle scripts]   PLAY [environment for oracle user]  

ok: [server1] TASK: [iptables]   PLAY [networking, time, passwords, etc]  

ok: [server2] => (item=init.sql) TASK: [copy additional tpt files]   PLAY [environment for grid user]  

ok: [server2] => (item=init.sql) TASK: [copy additional tpt files]   PLAY [environment for oracle user]  

ok: [server2] => (item=snapper4.sql) TASK: [copy additional tpt files]   PLAY [environment for grid user]  

ok: [server2] => (item=snapper4.sql) TASK: [copy additional tpt files]   PLAY [environment for oracle user]  

changed: [server2] TASK: [ORAENV_ASK set correctly (uppercase) in bash_profile]   PLAY [environment for grid user]  

ok: [server2] TASK: [ORAENV_ASK set correctly (uppercase) in bash_profile]   PLAY [environment for oracle user]  

ok: [server2] TASK: [bg color - bash_logout]   PLAY [ansible setup, terminal colors]  

ok: [server2] TASK: [bg color - colorprompt]   PLAY [ansible setup, terminal colors]  

ok: [server2] TASK: [bg color - dircolor256 file]   PLAY [ansible setup, terminal colors]  

Having a flag to generate output like this wouldn't compromise the real-time nature of the output but it would be *much* easier to inject into a pipeline.  It should also be pretty trivial to implement... I might be able to put together a patch if it might be considered (i.e. not contrary to any design goals).

-J

[Brian Coca]

anisble has a -o --online option, if that helps

--
Brian Coca
Stultorum infinitus est numerus
0110000101110010011001010110111000100111011101000010000001111001011011110111010100100000011100110110110101100001011100100111010000100001

Pedo mellon a minno

[Jeremy Schneider]

didn't see that before - that's exactly it.  so basically all i'm asking for is a -o/--one-line option for ansible-playbook.

--
http://about.me/jeremy_schneider

--
You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/86ogFCYbWSM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.

[Brian Coca]

have you tried creating a callback plugin that does this?

[Jeremy Schneider]

guess it's time to learn about plugins.  :)  just tried a basic callback module (context_demo from repo) and i see that i'm able to customize the output.  however i don't see a way to suppress the normal output, so my one-line output is in addition rather than replacing the normal output.

--
http://about.me/jeremy_schneider

On Thu, Sep 19, 2013 at 9:07 AM, Brian Coca <bria...@gmail.com> wrote:

have you tried creating a callback plugin that does this?

--

[Michael DeHaan]

You would have to make your own API script, which /usr/bin/ansible-playbook basically is.

Or just use AWX to mine the event data via the API, which also is a easy way to do it.

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/