Integrating application level smoke tests with ansible runs

350 views
Skip to first unread message

Bryan Berry

unread,
Dec 15, 2014, 4:33:34 AM12/15/14
to ansible...@googlegroups.com
Dear Ansiblists,

I need to integrate application-level tests with ansible runs. These test are quite specific to that applications we are deploying and go quite beyond the existing test support in ansible. The purpose of these tests is not to test the ansible code but the entire deployment stack. Running these tests as part as part of the ansible playbook makes sense as the ansible run knows the location of all created resources. I am happy to go off and create this on my own, but I would really like to collaborate if there are others working on similar problems. I want to write the tests using plain python unittest so it will be familiar to my developers.

here is an example where i want to test a Consul cluster of 5 nodes. I want to verify that the Consul API functions as expected.

here is a code example of what I am looking to test

# tests/test_consul.py
import unittest
import os.environ
import requests
import consul
import time


class TestConsul(unittest.TestCase):

  def setUp(self):
    self.members = os.environ['CONSUL_CLUSTER'].split(',')

  def test_nodes_return_same_membership_list(self):
    results = []
    for member in self.members:
      member_list = requests.get('http://member:4444/v1/cluster/members' % member)
      member_list.sort()
      results.append(member_list)
    self.assertTrue(all([True for result in results if result == results[0]))

  def test_kv_put(self):
    c = consul.Consul(self.members[0])                                                              
    c.kv.put('foo', 'bar')
    # wait for replication                                                                                                                                                                               
    time.sleep(5)
    results = []
    for member in self.members:
      c = consul.Consul(member)                                                                     
      results.append(c.kv.get('foo'))
    self.assertTrue(all([True for result in results if result == 'bar']))
                                                                           

as you can see, I plan on passing the list of nodes as an environment variable, but I would love to hear alternate approaches.

I currently plan to trigger the running of these tests by an environment variable.

$ RUN_TESTS=TRUE ansible-playbook my-plays.yml

- name: run tests
  when: "{{ lookoup('ENV', 'RUN_TESTS')|bool}}"
  roles:
     - execute-tests    # executes tests in tests/test_*.py using unittest module, yet to be written

It's really key that my developers be able to write tests using plain python code rather than Ansible's YAML. This decouples the tests from ansible. AThis could we helpful later if we every choose to run the exact same tests later separately from ansible, for example executing them as part of regular monitoring.

Thoughts?

Regards,

Bryan W. Berry



Michael DeHaan

unread,
Dec 15, 2014, 7:10:42 AM12/15/14
to ansible...@googlegroups.com
This is a good read if you haven't seen it already -  http://docs.ansible.com/test_strategies.html 

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/02e8c1c1-cb08-416c-9bf5-907607b529f5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Bryan Berry

unread,
Dec 15, 2014, 7:15:28 AM12/15/14
to ansible...@googlegroups.com
Hi Michael,

I have read it and I believe we will use some of those testing strategies. Those tests are good for testing an ansible playbook while I am looking to test the actual API of the services I am spinning up, and in a far more flexible way. From that page:

"Something like an integration test battery should be written by your QA team if you are a production webservice. This would include things like Selenium tests or automated API tests and would usually not be something embedded into your Ansible playbooks."

This is my exact use case, though not using Selenium specifically. We have a QA team that will be writing integration tests. I would like run them at the end of the ansible run.

Michael DeHaan

unread,
Dec 15, 2014, 8:15:43 AM12/15/14
to ansible...@googlegroups.com
Yeah in that case, it probably doesn't make sense to have ansible launch that job, but it could.

"Those tests are good for testing an ansible playbook while I am looking to test the actual API of the services I am spinning up, and in a far more flexible way. From that page:"

In our case, we'll fire off an install of Ansible in Jenkins against a QA/Stage environment, and then a dependent Jenkins job will launch automated API and Selenium tests.



Reply all
Reply to author
Forward
0 new messages