Salt CI/CD setup documentation

[Dave Jones]

I don't have much experience with CI/CD and i would like to setup a good salt development workflow based on git.  Does anyone know of any documentation or have any information on how to get started?  I have watched as many videos as I can find on Youtube and searched for general documentation but nothing is detailed enough for integration with salt.  I am not asking for command-level information but slightly higher to detail the workflow.

From my research, it looks like Jenkins with the salt plugin is the way to go.  Containers seem to be the fastest way to build and test individual formulas with some type of test suite for validation.  I am not sure how to glue all of this together properly with git branches.  For example, should I have a single top.sls manually compiled from the individual formula tests?  I am looking at simple dev and prod git branches to get started.

Thank you in advance,

Dave

[Dave Neeley]

I am just getting started with this as well. The design that the team at VirtuStream have presented a few times seems pretty good. They use cookiecutter templates to set up their formulas, plus Vagrant, and it seems kitchen.ci manages the setup, testing, and teardown. There is a saltstack-cookiecutter-template project on GitHub from someone at MIT. I was thinking our formulas would build on that and end up with salt packages in the Salt Package Manager format. The VirtuStream way has rpm packages as the deliverables.

[Dave Jones]

Unfortunately I need a little more detail to understand how to layout my code in git and how to test it.  For example I am looking at using testinfra for formula testing.  I can script the testing and understand git hooks to trigger Jenkins builds, but I don't want to start down a bad git layout and cause workflow problems down the road that will be hard to undo.

I was hoping that some had documented their setup at a high level and would be willing to share it with anonymous data to help not just me but the Salt community.

Thanks,

Dave

On Dec 22, 2016 3:05 AM, "Dave Neeley" <dave....@gmail.com> wrote:

I am just getting started with this as well. The design that the team at VirtuStream have presented a few times seems pretty good. They use cookiecutter templates to set up their formulas, plus Vagrant, and it seems kitchen.ci manages the setup, testing, and teardown. There is a saltstack-cookiecutter-template project on GitHub from someone at MIT. I was thinking our formulas would build on that and end up with salt packages in the Salt Package Manager format. The VirtuStream way has rpm packages as the deliverables.

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Arif Ali]

> --
>
Hi Dave,

I have recently start building salt formulas, for the work I am doing in
my company, and therefore have also started work on CI

I use a combination of GitLab, GitLab CI, test-kitchen, kitchen-salt,
gitshelf and serverspec. There are many examples going around in various
projects on github. Some URLs below that may help to get started. Some
use github and travis CI instead of gitlab. URL [3] might be useful,
where it is demonstrating all of these in the one repo

[1] https://github.com/simonmcc/kitchen-salt
[2] https://github.com/simonmcc/beaver-formula
[3] https://github.com/corux/seafile-formula

Hopefully, these would be useful

regards,
Arif

[Dave Neeley]

Here's the cookiecutter formula template I mentioned...Should help with layout concerns at least to get started.
https://github.com/mitodl/saltstack-formula-cookiecutter

[Dave Jones]

Hi Dave,

I have recently start building salt formulas, for the work I am doing in my company, and therefore have also started work on CI

I use a combination of GitLab, GitLab CI, test-kitchen, kitchen-salt, gitshelf and serverspec. There are many examples going around in various projects on github. Some URLs below that may help to get started. Some use github and travis CI instead of gitlab. URL [3] might be useful, where it is demonstrating all of these in the one repo

[1] https://github.com/simonmcc/kitchen-salt
[2] https://github.com/simonmcc/beaver-formula
[3] https://github.com/corux/seafile-formula

Hopefully, these would be useful

regards,
Arif

Thank you for this detail.  I am thinking about using testinfra which is a Python version of serverspec.

https://github.com/philpep/testinfra

Dave

[Dave Neeley]

Yup, that template uses testinfra as well. Take a look.

[Xandrellas]

For our CI, we use:

- Gitlab CI w/runner

- Test kitchen

- Saltscaffold (i love this)

- Kitchen openstack

- serverspec (could use testinfra since python but eh.. levels of maturity I suppose)

Lots and lots of webhooks:

- each repo (formula) has a gitlab-ci.yml that runs kitchen test; kitchen destroy on all commits.

- webhook receiver (inhouse written in python but there's ones out there) that listens for a merge and then git pulls and rsyncs out to other masters.

We're still figuring out logical CD, but for now our continuous integration game is fairly strong.  Need to learn cross-formula/server integrations.

This method of CI can be done solely through salt as well but we've put the leg work into using gitlab's CI runner.