salt-ssh to AlmaLinux 8

[Steffen Wurm]

Hello, 

I am having some issues with salt these days, after using it for 10+ years. 

For security reasons, I always avoid the classical master/minion like hell. Salt had repeatedly security vulnerabilities. Leaving the master exposed to (m)any minions is pretty risky, if your doing it based on zero trust. 

Lets say I want to control some AlmaLinux 8 minions via salt-ssh. It ships with python 3.6. 

However: the oldest installable salt-ssh installation I can find, is a "onedir" installation (3005), that comes with python 3.10 and can't handle my python 3.6. 

Installing python 3.11 from the appstream, does not help either, as it is not being used. 

Is there any way left to control the minions, without having an almighty system responding to incoming connections? Especially if your environment is very fragmented (Alpine Linux, AlmaLinux, Debian, Ubuntu)?

Best regards

Steffen

[Christopher Hawkins]

Hi Steffen, just a suggestion. We use salt master exposed to the
Internet with remote minions but we run it on a random high numbered
port to avoid all the traffic and scans that we would get on the usual
port. This has been fine for us for many years and I don't consider it
risky. SSH key based authentication is well established and it's
difficult to discover on a random port.

> --
> You received this message because you are subscribed to the Google Groups "Salt-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/salt-users/4e358a8a-8dd2-46e7-a224-e5837dcb3592n%40googlegroups.com.

[Steffen Wurm]

Hello Chawk, 

Thank you for your reply. 

I am not that afraid of an foreign IP, contacting the salt ports. In the end, iptables will only allow connections from our own systems. Although this additional configuration hastle would not be necessary, if the connection would be established in the opposite direction. 

Our environment is just huge and very heterogeneous. Ransomware groups (and other attackers with some brain) will infect your network and just wait. You wont notice them at first. Once on a minion, the attacker can see the masters port in the minions config and can access the master. 

Salt is single system, capable of destroying your/our entire infrastructure within mere seconds. An attacker waiting on a minion, for the next security vulnerability on the master, is my biggest nightmare. And this is their typical behavior with the MS ActiveDirectory, already. 

If the connection would be established in the other direction, the salt master could be hardened a lot more. You could even block *all* incoming connections, if the admin can access the master locally. 

https://www.cvedetails.com/vulnerability-list/vendor_id-12943/product_id-26420/Saltstack-Salt.html

I was able to get salt-ssh running a tiny bit better. "ssh_ext_alternatives" is completely useless, once you are using "onedir" installations on the master. They all ship with the same python version at the moment. 

AlmaLinux/RHEL 8 provides a python3.11 package, you can install additionally. Add "set_path: '$PATH:/opt/salt-ssh/bin/' to the roster for the affected minions and place a symlink called "python3" in that directory on the minion. 

Debian 12 is using a compatible python. However: Debian 11 is too old, already. 

ArchLinux will not be working. I guess, it does not have a "backports" module, as it is using the latest python version. 

Alpine Linux 3.16 up to 3.20 use a compatible python-version. But Alpine Edge is affected by the same issue as ArchLinux. 

Thats with Salt 3006. Still not a great situation. :(

Any ideas on improving this, are very welcome. 

Best regards

Steffen

[Steffen Wurm]

Hi out there 

Just in case, someone runs into the same issue, getting Alpine Edge or Arch Linux with Python 3.12 supported (while maintaining compability with older distros). 

Extract the 3007.1 rpm on the master and add it via the "ssh_ext_alternatives" I described as useless, earlier. 🙄

ssh_ext_alternatives:

    salt-onedir-3007:            # Must be a string - NOT a number/float!
        py-version: [3, 12]     # Constraint to specific interpreter version
        py-bin: /opt/salt-onedir/opt/saltstack/salt/bin/python3.10
        path:   /opt/salt-onedir/opt/saltstack/salt/lib/python3.10/site-packages/salt  # Main Salt installation directory.
        dependencies:           # List of dependencies and their installation paths
          jinja2:  /opt/salt-onedir/opt/saltstack/salt/lib/python3.10/site-packages/jinja2
          yaml:    /opt/salt-onedir/opt/saltstack/salt/lib/python3.10/site-packages/yaml
          tornado: /opt/salt-onedir/opt/saltstack/salt/lib/python3.10/site-packages/tornado
          msgpack: /opt/salt-onedir/opt/saltstack/salt/lib/python3.10/site-packages/msgpack
          certifi: /opt/salt-onedir/opt/saltstack/salt/lib/python3.10/site-packages/certifi
          distro:  /opt/salt-onedir/opt/saltstack/salt/lib/python3.10/site-packages/pip/_vendor/distro
          packaging:     /opt/salt-onedir/opt/saltstack/salt/lib/python3.10/site-packages/packaging
          markupsafe:    /opt/salt-onedir/opt/saltstack/salt/lib/python3.10/site-packages/markupsafe
          backports_abc: /opt/salt-onedir/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/backports_abc.py
          backports: /opt/salt-onedir/opt/saltstack/salt/lib/python3.10/site-packages/salt/ext/
          looseversion:  /opt/salt-onedir/opt/saltstack/salt/lib/python3.10/site-packages/looseversion

Best Regards

Steffen