SaltStack Test Suite

[sadde...@sparkcognition.com]

I installed Salt on centos 7 by adding rpm repository and installed salt master using yum. I have several state files and bash scripts which spans salt minions and installs components. Now I would like to configure unit testing in order to save time on manual testing. Based on the documentation from saltstack, I could not find the tests directory in my salt codebase. Is salt/tests directory available iff by cloning repo?? Is there any other way I can do saltstack unit and integration testing??? Can anyone please help me out. 

[Jeremy McMillan]

Your story isn't clear on whether you're intending to run existing test scripts or writing new ones, whether and how you are currently running tests, or how you would like salt to save time. Do you think you could provide a little more help understanding what you would like to do? It would be terrible to follow good advice for solving someone else's problems.

[sadde...@sparkcognition.com]

Hey, I'm sorry. I've installed salt master using rpm on my centos machine. I configured all my salt scripts to do required tasks on the minions. My salt scripts consist of state files and some shell scripts that are being called by the state file so that these scripts execute on the minion. Everything is fine till this point. In general, I do manual testing where I run the commands for:

1) Spinning up the minion

2) Run state files on the master to install on the minion.

I have several installations being done on the minion and the whole process takes around 30 min. Roughly I have around 15 state files to be executed on the minion via the master.

Now to make sure my state files work, I do testing. Testing is done by spinning up the instances and calling these 15 state files and see if it's working or not. Is there any better testing framework I can use for testing my state files?? Can I use unit testing or integration testing over here?? If I can use-> then where can I find the tests directory in salt codebase?? 

[dwal...@saltstack.com]

Check out test-kitchen with the kitchen-salt provisioner.

https://kitchen.saltstack.com/

This will allow you to run your states, and then write a test suite to run against it, to make sure that salt is doing what you expect.

[Christian McHugh]

Recently we've been using the saltcheck module to handle exactly this use case. I've recently submitted a bunch of fixes and documentation improvements to make it work even better for our needs, so future releases should just work, but for now I'd suggest backporting the saltcheck module to your environment.

Saltcheck allows you to write tests for your states in a very similar yaml (or supported renderer of choice) syntax, including having access to the same pillars and grains. 

For example, if you had a salt state which managed linux users where the data comes from a pillar, you could use:

# /srv/salt/users/init.sls

{% for user, uid in pillar.get('users', {}).items() %}

{{user}}:

  user.present:

    - uid: {{uid}}

{% endfor %}

You could then create a saltcheck-tests directory in the same location as your states, and create a users.tst file inside with the content:

# /srv/salt/users/saltcheck-tests/init.tst

{% for user, uid in pillar.get('users', {}).items() %}

check-user-{{ user }}:

  module_and_function: user.info

  args:

    - {{ user }}

  assertion: assertEqual

  expected-return: {{ uid }}

  assertion_section: uid

{% endfor %}

(This example relies on functionality submitted in https://github.com/saltstack/salt/pull/49395, so it might take a few more days for it to be merged in to develop)

If your state was run with:

salt <minion-id> state.sls users

The saltcheck could be run as:

salt <minion-id> saltcheck.run_state_tests users

The cool thing is that it uses the standard salt modules to perform all the checks, so you have all the power of salt behind your tests (it's just checking the output of user.info in this case). This example validates all users defined in the pillar have been added, so it works for 1 or 100 just as easily. This also allows you to output the results as you would any other salt command, so you can use the standard human text output to view green or red success messages, or output as json for further processing.

We started using saltcheck recently to validate application deployments to hadoop clusters using the hadoop.dfs and s3 modules. Where our QA team would spend time manually checking files in various locations, this task is now accomplished in seconds with greater reliability.

Saltcheck validations can then be combined with test kitchen or Jenkins pipelines for CI, but it's also a simple to use but super powerful tool for validating state results.

Good luck!