Why salt?

[Jean-Philippe Ouellet]

Hello,

I am not criticizing the choice of salt, I am just curious about why
exactly it was chosen over the alternatives. From the list archives it
appears that it was just suddenly being worked on, and I'd be
interested to know what lead to choosing it instead of
{ansible,chef,puppet,etc.}. I could not find any public discussion on
our list archives evaluating their relative merits.

Regards,
Jean-Philippe

[Unman]

I suspect that, as with Fedora in dom0, it was what the developers were
comfortable with.
There have been discussions about other management stacks, and some
proposed solutions, but I haven't seen many of those since salt was
adapted to Qubes.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

See here: https://www.qubes-os.org/news/2015/12/14/mgmt-stack/

> From all the available projects, we have chosen Salt Stack, because of
> its wide adoption, ability to run without any network service (“local”
> mode), clean and easy to read configuration, and ease of extensibility
> (python API for modules). In addition, it has the ability to manage
> (possibly remote) system which doesn’t have Salt stack agent
> (“minion”), a mechanism known as salt-ssh (which in fact isn’t that
> unique to Salt Stack). With Salt Stack, we have implemented a module
> to handle Qubes configuration, specifically managing VMs.

An alternative we've considered was Ansible, which is also wide adopted,
but finally decided for Salt mostly for two reasons:
- Salt syntax (in our opinion) better encourage to write declarative
configuration (describe desired system state, not a sequence of
actions to perform)
- we (mostly nrgaway at that time) were more familiar with Salt
architecture

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYzycmAAoJENuP0xzK19csj1UIAII4lWFa2B+468sMV/KrqDZ1
8Sfe0qlBumcWb4La6ofNqa04F6ZUaqzsK4FaJfK5k2vKvlcw41z13TpQFQFUK/0A
bfbFqafBivr/C3h7x3JEf6KDcFyZQHJGUA4JdM5/08QuHWzr5Qi72tVKY5S+rR0N
zG6zAV5glSHpU5gPFkigbMpavi49q/yHE6uAdoT+92UMiLHhpE5l0sJ5xsFdgu3C
sumjQNrJEGlIZBeQe7qIY1Z3jKzuEArRNJhu4BqnQSqZ9k8QZC2BL0KQ2bC21LDz
OmhRb8jBaoLBv/b85DTrAjibP2U8TJdZK45kS5klOyEznuW0w7HYmhisdUBIbEg=
=7Q+S
-----END PGP SIGNATURE-----

[max....@gmail.com]

Sorry to piggyback on the thread, but I've been meaning to ask this. Have there been any notable usecases of end users using salt to coordinate Qubes VM lifecycles with salt, and orchestra/publish them into services? I imagine something like Joanna/Marek's qubes-app-linux-pdf-converter
which coordinates multiple VMs to produce a stronger security guarantee than a single VM could produce.

I am unfamiliar with the structure of salt formulas and pillars, or whether the end goal of a salt configuration can be to publish some service endpoint (though I'd imagine the answer is yes), nor am I certain the exact use salt has seen in qubes so far (it has been my impression that salt helps compose qubes VMs in a  way not fully encouraged for end user, but looking at the documentation that might have simply been my impression given my level of understanding at the time - it certainly seemed intimidating.) What I'm more familiar with is Docker, where most configuration management is done via Docker file, and coordination is done through Docker-Compose declarations.

Maybe I can show you guys such a file and see if Salt can do similar tasks?

This Docker-Compose specification defines a pair of containers that run in tandem - one is a container which sets up an openvpn link to a VPN provider and runs a torrent client through that tunnel, and the other container is a reverse proxy server that allows the local user to access the torrent client's web interface. The definitions for the component containers (in qubes they would be VMs) are referenced from a repository of container definitions via the "image" command, while the other commands determine the configuration parameters of the containers - their internal environment variables, ports that are open and the port forwardings for such, the external file resources that will be mounted in the containers, and also things like the behaviors of the containers - for example the the "transmission" container will "restart: always", meaning that if there is an internal error in the container, the execution engine should attempt to restart it, while the "proxy" container's "links: transmission" directive specifies that the transmission container will be visible to the proxy container via the hostname "transmission".

All in all it's a very light weight but surprisingly flexible ethod of coordinating apps that are built of multiple units, and I'm wondering if we can do similar declarative definitions of multiple VM systems via salt.

A couple motivating usecases that I can think of go as follows:

1. Specifying a proxy VM for a given VM, and optionally injecting a "man in the middle" utility VM which can do things like:
    - Provide packet/protocol capture and analysis (for example for testing whether a proxy VM leaks)
    - Provide a TLS termination proxy/mitmproxy

2. Easily define application specific processing pipelines and use them to compose/abstract services that span multiple VMs in a way that
    - Lets you specify very light, application specific VMs for individual steps
    - Lets you set up complex life cycles and data flows between these vms
    - Where all VM configurations are pulled automatically from a repository

Anyways, this is all kind of all over the place, and might need to be posed elsewhere, but I'm just wondering whether salt as deployed on qubes can covers these kinds of usecases in a fairly light weight way.

Regards,
Max

[Nicklaus McClendon]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I'm been poking at an implementation for Ansible currently:
https://github.com/kulinacs/ansible-qubes

In contrast to the current solution by Rudd-O, I'm attempting to avoid
any dependencies outside qubes-core-admin and Ansible itself. I
wouldn't recommend use outside of testing yet, but I have yet to have
any problems with it. It's more or less just a combination of
qvm-create, qvm-prefs, and qvm-remove at the current moment, driven by
Ansible. The next things on my list are an Ansible transport module,
which will hopefully mimic qubesctl, and qubes-prefs, with the end
goal of having 1:1 feature compatibility with qubes-core-admin.

- --
kulinacs <nick...@kulinacs.com>
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEPL+ie5e8l/3OecVUuXLc0JPgMlYFAljRzVcACgkQuXLc0JPg
MlbTmw//Wf5EVVNEVV9auK607l2YlG+7R4EsrvWdvuOltnZ6Zc5cywmk67DyIa6x
PnVA0LZXv14owMoPsxcDl9VtJKCoHp+aBao8FwXtZqimLA4lu1/68LF1/SCVb2zp
eBLgRYN1YbTAevZOPHGDQ4blfDgJdnRN3Wc3834S08IZLpWSuML9AH08UyOfzKEE
QGkLU+GPj0Bq5p7uZourJTvS0+dmquDeazVWPN+klMpuWLkrQ9QkhVJbOA0oX3JX
PW6A0Qdmbvha+kD6ugyckfwHvRpkfjYLoXlRQUupK3HKdXANfyjK1FSn/JdMc5EU
xt4scKi0dkvQdrZwPNC9VUUB0yTPeK47X/DVamx9l9gyy6hidh9qqwwsUlIywNaS
l2GpzNxW5pWU89/J9Sf0p3egZYKGHG0uzPcRy2+5TnuLWlcXVdL6VZTwt/5CwAfv
cfxMPeWED95PZ3d4OQdE3rdkUzcyBz8n1BbjULQiHhE6CsNTYnS8lde9rtFHAQmL
JPqnYtvOq+24AGJ7FOQ7TZfFaV5wpsmVAGE6NtEyawaz3zfa79FdwCK7TpgU3FLt
FPD6RqunnSM80T3sRmOP0lQM3IdUkYFPBXLhTileeTJv2/BvIJdwwAjnw1/yR478
gLCKl8NriwtPMGUASDRdsnSXSKn5K/s9l9eubGkmY9Ejl5jiqdM=
=P4oj
-----END PGP SIGNATURE-----

[Chris Laprise]

On 03/21/2017 03:40 AM, max....@gmail.com wrote:
> Sorry to piggyback on the thread, but I've been meaning to ask this.
> Have there been any notable usecases of end users using salt to
> coordinate Qubes VM lifecycles with salt, and orchestra/publish them
> into services? I imagine something like Joanna/Marek's

> <https://github.com/QubesOS/qubes-app-linux-pdf-converter>qubes-app-linux-pdf-converter
> <https://github.com/QubesOS/qubes-app-linux-pdf-converter>

> which coordinates multiple VMs to produce a stronger security guarantee
> than a single VM could produce.

I haven't seen any expressed on the lists.

> I am unfamiliar with the structure of salt formulas and pillars, or
> whether the end goal of a salt configuration can be to publish some
> service endpoint (though I'd imagine the answer is yes), nor am I
> certain the exact use salt has seen in qubes so far (it has been my
> impression that salt helps compose qubes VMs in a way not fully
> encouraged for end user, but looking at the documentation that might
> have simply been my impression given my level of understanding at the
> time - it certainly seemed intimidating.) What I'm more familiar with is
> Docker, where most configuration management is done via Docker file, and
> coordination is done through Docker-Compose declarations.

Let me be the first to admit that salt appears difficult to understand
at first. Their documentation drives me up the wall. It strikes me as
the kind of project that gained followers early when it was still easy
to grasp, then built up an obscurantist mindset.

Ansible is much easier to understand, IMHO.

> Maybe I can show you guys such a file and see if Salt can do similar tasks?
>
> This

> <https://github.com/haugene/docker-transmission-openvpn/blob/master/docker-compose.yml>
...

> Anyways, this is all kind of all over the place, and might need to be
> posed elsewhere, but I'm just wondering whether salt as deployed on
> qubes can covers these kinds of usecases in a fairly light weight way.

Salt probably does cover them. But I suspect Qubists have looked at the
salt concepts and syntax and decided its not yet worth the effort.


--

Chris Laprise, tas...@openmailbox.org
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Yes, we need better documentation to make Salt accessible:

https://github.com/QubesOS/qubes-issues/issues/1983

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJY1HmqAAoJENtN07w5UDAw3yUQAKJ/gSSrwhdOWifuSmeiu8vh
PxIXwtoc3tGXJIqUnNkcE9avTmCpzYhHp86WnXaK3vk1dfkr8F/RdSW0CGYkqHl2
2l83sY8Ol3qC1OqDhlegH9UDQCYh5UL7FzEWnr8OIXYrmlj67d1jpWJK2W8V7Cgd
U1COWlkWr+LnWJEyK1FBjlj9cMqPeaODcpI6DpjB6ZLizffRVywsYlhcQCDHSTS3
JK5wke9jX/6Em1RMdkR0WA6vL50qTLDPxprjW8KD4IGSWWijryvAVIQGmFn2J9WY
34ej9RJst1jmX0a1Z2fqFtRS8yrcrJJgxaJB7l1i1ucoCcpvdrL1qUwyA/6jjs6M
uDaCDthm/jfFl1VK4R+rYqwGB+/kr9l68dzO/Iauk4ZJKEkIKojmKAIRP975EpGT
3xJ0pW8f91pBsBc0aDOEmQAQSEYbGKpWShgtMGmhbvDRfGISPmAOUpI5p3WZd8/E
qW4F0GDkOKldkG6DZYl295skGZNaH7UFUy4+rELmgtleRKksFNL62ShDOb96wy+8
3Hci2paTXJR/pc5ha/pl3+nI7CeSwMNCHZqQ5zxrPCbnFS+PYrQsImoUALYaajEA
VSJViLQ3KZwyDzj1oOg9xe1aNXzIb9kjRTzesiPSaqblA5nAi4X1o69vAFVk2Bd8
C1Wyh5Ky1Gn3ytcOYT4q
=mdc5
-----END PGP SIGNATURE-----