salt-ssh data flow

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

What is designed data flow in salt-ssh based setup? Currently I see it
works like this:

1. Sends a thin wrapper to execute commands, it first checks:
- if thin minion is deployed, if not - emit "deploy" message and exit
- if required external modules are deployed, if not - emit "ext_mods"
message and exit

2. Thin wrapper execute requested function, and return output after
special delimiter.

In case of state.highstate, state.sls etc, this is somehow more complex:

1. salt-ssh first execute test.opts_pkg (through thin minion) to get
grains from target system.

2. It package state with all required files, suitable for state.pkg
function. This (tgz) package consists of:
- pillar.json
- lowstate.json
- additional files referenced with salt://
3. Send that tgz using scp.
4. Call thin minion again to actually execute state.pkg salt_state.tgz.
5. Receive the output.

Now the actual questions:

1. Is it design decision (something one can rely on) that only states
targeted to this particular system are included in salt_state.tgz? Or is
it only a coincidence that can change in the future?

2. Similar for additional files - is it guaranteed that only files
referenced from states targeted at this particular system will be
included in the package? In other words, is salt-ssh a "solution" to
file access control[2]?

3. Is it possible to render the states at the minion side (so salt-ssh
would not need to receive grains), while still holding properties 1 and
2? This would be at the cost of using grains to target states in top
files, but lets assume for now it isn't a problem (which is reasonable
assumption given [1]).

4. Is it possible to use salt_state.tgz building code outside of
salt-ssh? For example in some custom runner.
Especially valuable is the part for extracting referenced files.

5. Does "ext_mods" callback give all the additional modules, or only
those referenced from states targeted at given system?

6. And finally - is it possible to use salt-ssh (or equivalent) over
some other shell access protocol? I've managed to do that by writing a
wrapper replacing original /usr/bin/ssh (and scp), but it's rather ugly
solution...

[1]
https://docs.saltstack.com/en/latest/faq.html#is-targeting-using-grain-data-secure
[2]
https://docs.saltstack.com/en/latest/faq.html#is-it-possible-to-deploy-a-file-to-a-specific-minion-without-other-minions-having-access-to-it

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWyjOEAAoJENuP0xzK19csJswH/jF13RRSeUopG9hWt+zHdvYS
0LwaVeus6pcnne/mYmIR8h7ejd0StAjNZXbLVFiZEiR1dSwelQ8G74Q++5roP+6e
xTF1izOI3+vG4iyHKnp+UzvvNOgymXXF7Q0cKT9YoHdI8ts8iFkiF+5iVbOE+BPB
wWstETjN2Lo4IvRQsNbZbceBm/2UchgOH+vtsmLX0ANTCGf4tJ7TFh9/poZ5GRcM
vTtBGHEx+kPnW3YnDMUYhP40AJeFO7ayF9bIE18DNtP4UJn7f6VfxvxIQ9m/xAMS
cI9Jp/EsHVNne0GmnC+QJNN/nhUBrYSO2nH0WS2LZkMPMfUXY9RjbQBw25usccU=
=A3Mu
-----END PGP SIGNATURE-----

[Colton Myers]

Wow, you’ve done your homework! Your summary of data flow in salt-ssh is spot-on.

I’ll answer your questions inline:

1. Is it design decision (something one can rely on) that only states
targeted to this particular system are included in salt_state.tgz? Or is
it only a coincidence that can change in the future?

This is a design decision driven by keeping the archives as small as possible, not driven by security. Salt-SSH can be configured to include additional files not found by the state compiler if they are necessary for custom code or jinja imports.

 

2. Similar for additional files - is it guaranteed that only files
referenced from states targeted at this particular system will be
included in the package? In other words, is salt-ssh a "solution" to
file access control[2]?

As stated in (1), this decision was driven by transfer constraints, not security. That said, it could probably be relied on, because in this case the transfer constraints align nicely with your security desires.

3. Is it possible to render the states at the minion side (so salt-ssh
would not need to receive grains), while still holding properties 1 and
2? This would be at the cost of using grains to target states in top
files, but lets assume for now it isn't a problem (which is reasonable
assumption given [1]).

This is not currently possible. The state compiler is designed with on-demand access to the fileserver in mind. This is why the salt-ssh state compiler works the way it does — rather than re-implementing the state compiler, we tweaked it such that it could work master-side, so we could use it with minimal changes to the existing state compiler.

Even if we could move the compilation to the minion, you run into issues with states. We have to compile the states in order to resolve all salt:// references. Since the minion cannot request files from the master, we have to send all these files down with the state tar.

4. Is it possible to use salt_state.tgz building code outside of
salt-ssh? For example in some custom runner.
Especially valuable is the part for extracting referenced files.

This is something I’ve wanted to do. We could use this system to create portable state runs, with all files included, which could be very useful. There’s currently no interface for this mechanism, however.

 

5. Does "ext_mods" callback give all the additional modules, or only
those referenced from states targeted at given system?

I actually haven’t been in this code for awhile so I can’t remember exactly what triggers the ext_mods return. But I anticipate it syncs all custom modules in the same way as master-minion salt’s `saltutil.sync_all`.

 

6. And finally - is it possible to use salt-ssh (or equivalent) over
some other shell access protocol? I've managed to do that by writing a
wrapper replacing original /usr/bin/ssh (and scp), but it's rather ugly
solution...

It’s definitely designed with ssh in mind. But theoretically it is possible if you replaced salt/client/ssh/shell.py with routines for another system.

Hope that helps!

--

Colton Myers

@basepi

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Thanks for the answers! That's exactly what I wanted to know.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWy6jKAAoJENuP0xzK19csclwH/j6qBXIONbT8ZEZ6hLe7cAjv
MbGSqOJvWHu5I8Xnw+iJ/74ZM7jh3oA/bcaPFb2WF/T0HqyCDmL6eMxhRWXJtG1a
nnD20k8MgbbLtTPjIXrCdmFKvE3zxK8q0dQsZJ2KSUWisvdRcblzZGl8loBCYrSd
O/GafhRdNS3s6RPdQvpzOW5PVq2Wdt8NZIs0jhVxr2QHHmknPSkMMH+AVe7+K2lJ
cOOUTz/sQc35A2w4fBkiFB2My3yIDxj3xFp88nqVWN8wfHEVmFPNbqGY4LKqbL5W
Og3lc70awahzQxSwn+aMlLrBaoLUJOxrfUd5RvGkmiFuq4Uddy5pgO8vIwD/ZA8=
=azO1
-----END PGP SIGNATURE-----