Ansible and configuration verification?

[fdavis]

Hello,

(short version)

We want to discuss adding a test parameter to playbooks
and tasks.

Verification of configuration is an important step to making sure
your systems are in the state you believe they are in.

An idea for testing would be a playbook directive to run some
test playbook, basically including the playbook after the current one.

An idea for testing tasks is a little more tricky, because sequence
matters, would be to specify a handler to run.

-vvv (very verbose version)

Something our group has discussed a few times is that Ansible
can be used to verify that your configuration is truly live on
the system.

For instance, in dealing with a new kernel update, kernel modules
such as Nvidia graphics, VMware, or VirtualBox need to be updated.

It would also be useful in most places where a conf file is templated or
copied over. Some services, like named or nagios, come with a checkconf
or pre-flight check in their command line util.

We tend to run tests at the end of the playbook, register the result,
and then send fail mail to our ticket queue so we can look into that
particular machine,
(why it failed, how to correct it ,
how to make the playbook more robust, etc...)

Part of the problem is specifying when tasks run. In some cases
it should directly after the task, sometimes it would need to be after
the handlers restarted services.

I'll put down an example of what setting up named looks like now,
and then a couple ideas for a test option.

# Current playbook single file
# conf playbook
---
  - hosts: dns-servers
    tasks:
      - name: template or copy a conf file
        copy: src=files/named.conf dest=/etc/named.conf backup=yes
        notify: restart dns

     -  name: check conf
        command: named-checkconf /etc/named.conf

    handlers:
      - name: restart dns
        service: name=named state=restarted
# test tasks that should come after handlers
---
  - hosts: dns-servers
    tasks:
       -  name: verify a few choice hosts
          local_action: command nslookup $item ${ansible_default_ipv4.address}
          with_items:
                 - important_hostname_1
                 - important_hostname_2


# test idea, add a test parameter for playbooks and for tasks
# playbook split into 2 files
# conf playbook
---
  - hosts: dns-servers
    test: test/playbook.yml
    tasks:
      - name: template or copy a conf file
        copy: src=files/named.conf dest=/etc/named.conf backup=yes
        notify: restart dns
        test: check conf

    handlers:
      - name: restart dns
        service: name=named state=restarted

     -  name: check conf
        command: named-checkconf /etc/named.conf

# ./test/playbook.yml
---
  - hosts: dns-servers
    tasks:
       -  name: verify a few choice hosts
          local_action: command nslookup $item ${ansible_default_ipv4.address}
          with_items:
                 - important_hostname_1
                 - important_hostname_2

[Michael DeHaan]

Replies inline...

On Sun, Feb 10, 2013 at 2:53 PM, fdavis <fdav...@ucr.edu> wrote:
> Hello,
>
> (short version)
>
> We want to discuss adding a test parameter to playbooks
> and tasks.
>
> Verification of configuration is an important step to making sure
> your systems are in the state you believe they are in.
>
> An idea for testing would be a playbook directive to run some
> test playbook, basically including the playbook after the current one.

Seems like this would just be:

ansible-playbook main.yml test.yml

test.yml would always run completely after main.yml

>
> An idea for testing tasks is a little more tricky, because sequence
> matters, would be to specify a handler to run.
>
> -vvv (very verbose version)
>
> Something our group has discussed a few times is that Ansible
> can be used to verify that your configuration is truly live on
> the system.
>
> For instance, in dealing with a new kernel update, kernel modules
> such as Nvidia graphics, VMware, or VirtualBox need to be updated.
>
> It would also be useful in most places where a conf file is templated or
> copied over. Some services, like named or nagios, come with a checkconf
> or pre-flight check in their command line util.

in some cases, the command module may be all we need here, because
they would fail and kill the playbook if they actually failed

but like you say, you would want it to run after any handlers...
(hence the above)

>
> We tend to run tests at the end of the playbook, register the result,
> and then send fail mail to our ticket queue so we can look into that
> particular machine,
> (why it failed, how to correct it ,
> how to make the playbook more robust, etc...)
>
> Part of the problem is specifying when tasks run. In some cases
> it should directly after the task, sometimes it would need to be after
> the handlers restarted services.

in case A, I'd interlace those tests with the main playbook, other per above...

I'm wondering what that may be missing?

This way your playbook still returns non-zero...