Expose salt master on Internet

[Peng Zhao]

Hi,

I'm not sure this is possible, but I want to expose Salt master's zeromq on the Internet.

I read some on HN:

 """ Salt takes advantage of a number of technologies and techniques. The networking layer is built with the excellent ZeroMQ networking library, so the Salt daemon includes a viable and transparent AMQ broker. Salt uses public keys for authentication with the master daemon, then uses faster AES encryption for payload communication; authentication and encryption are integral to Salt. Salt takes advantage of communication via msgpack, enabling fast and light network traffic. """ 

And

" 0mq does not natively support encryption, so Salt includes its own AES implementation that it uses to protect its payloads. Recently, a flaw was discovered in this code along withseveral other remote vulnerabilities. Ansible is largely immune to such issues because its default configuration uses standard SSH and does not require any daemons to be running on the remote servers aside from OpenSSH (and I don’t think there is a single package in the entire world that I trust more). Regarding Fireball Mode, Ansible’s 0mq AES encryption is done using Keyczar instead of a homegrown solution, is not enabled by default, and its connections are ephemeral. "

Sounds like there are some security issues to do so and Salt's code has some bugs that makes it vulnerable.

Thanks,

Gnep

[Mrten]

On 03-07-2013 11:27:57, Peng Zhao wrote:
> Hi,
>
> I'm not sure this is possible, but I want to expose Salt master's zeromq on
> the Internet.

I wouldn't (and don't). If someone 0wns your salt-master, they own your
network and all the minions.


As stated in the comments you posted, Salt does use homebrew crypto with a
library that isn't developed anymore (M2Crypto), and though there are efforts
underway to replace that with more recent ones (PyCrypto), this is still
homebrew crypto. Audited crypto, yes, but still audited homebrew crypto.

There is also an effort to use ssh for transport, but I hear it would be slow.

(all information from the grapevine, I'm no Salt developer).


However, you don't need to expose the salt master, you can create your own
private network to run salt on; connect various sites via IPSec, or, way
easier, with a SSH layer 2 tunnel.

(for example; http://www.kernel-panic.it/openbsd/vpn/vpn5.html)

HTH,
Mrten.

[Peng Zhao]

Thanks, but that does fit my situation.

I need to expose it for my clients, not internal usage.

ZeroMQ makes things difficult to run Salt (or Ansible) for a hosted service like Hosted Chef. That said, I think Salt need a HTTPS module.

--
You received this message because you are subscribed to a topic in the Google Groups "Salt-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/salt-users/3Le4xYzK0rY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

==========================================

     MadeiraCloud - A river runs through the  cloud

[Peng Zhao]

Sorry, "that does not fit my situation"

[Valentin Bud]

Hello Peng,

You could protect access to your Master with iptables rules. For example if your Master is host
M and your minion is host m with M.M.M.M and m.m.m.m as IP addresses the following entries
on the Master will allow connections only from the specified minion:

iptables -A INPUT -s m.m.m.m/32 -d M.M.M.M/32 -p tcp -m state --state NEW -m tcp --dport 4505 -j ACCEPT
iptables -A INPUT -s m.m.m.m/32 -d M.M.M.M/32 -p tcp -m state --state NEW -m tcp --dport 4506 -j ACCEPT

The traffic between M and m is encrypted using M2Crypto.

This is not a suggestion but merely a way to connect your minion(s) with the master via the Internet in a more
secure fashion.

Cheers and Goodwill,

You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Valentin Bud

http://databus.pro | vale...@databus.pro

[Peng Zhao]

Thanks, but faking an IP is not difficult. 

[Avi Marcus]

displaying my ignorance here: You can fake the IP for what's shown in the packet, but won't the reply go to that actual IP?

So faking is good for DDOS that's not blockable or for MITM, but if you don't have a MITM situation, then is this a concern here?

-Avi

[Mrten]

On 03-07-2013 13:02:21, Peng Zhao wrote:
> Thanks, but that does fit my situation.
>
> I need to expose it for my clients, not internal usage.

Ah, then take a look at the syndic.

http://docs.saltstack.com/ref/syndic.html

As far as I understand it, you create a local master at your clients', and
have the master-master talk to that. This way you don't have to open up the
master fully, just to the sub-masters.

M.

[Peng Zhao]

No, you have been helpful, Avi.

However, to run a public service, my concern is that any potential security hole will create huge impact on the user's confidence on the service. So I am trying to scan and eliminate all possibilities.

Make any sense?

[Peng Zhao]

Does it use 0mq for communication? If yes, same situation.

The real problem is: running an Internet-facing server and having a port open for the outside to push message to the server, sounds risky.

M.

--
You received this message because you are subscribed to a topic in the Google Groups "Salt-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/salt-users/3Le4xYzK0rY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Mrten]

On 03-07-2013 13:48:23, Peng Zhao wrote:
> Does it use 0mq for communication? If yes, same situation.

yes, 0mq. no, not the same: you can firewall off the ports more easy then before.

> The real problem is: running an Internet-facing server and having a port open
> for the outside to push message to the server, sounds risky.

So is breathing ;)

(and openssh)

M.

[martin f krafft]

also sprach Peng Zhao <peng...@madeiracloud.com> [2013.07.03.1127 +0200]:

> Sounds like there are some security issues to do so and Salt's code
> has some bugs that makes it vulnerable.

At the moment, there are no known bugs, but it is true that Salt ues
homebrew, which isn't really optimal.

There's a discussion at
https://github.com/saltstack/salt/commit/5dd304276ba5745ec21fc1e6686a0b28da29e6fc

And I created an issue to track progress on switching Salt to TLS:

http://github.com/saltstack/salt/issues/5909

Cheers,

--
martin | http://madduck.net/ | http://two.sentenc.es/

# vim:tw=70

spamtraps: madduc...@madduck.net

[martin f krafft]

also sprach Peng Zhao <peng...@madeiracloud.com> [2013.07.03.1317 +0200]:

> Thanks, but faking an IP is not difficult.

It is also not trivial to take over a TCP connection, unless you are
in the direct line between the two peers, in which case you will
probably try a DNS attack instead, or a MITM attack.

I think 0MQ+TLS on the master should make it secure enough to be
ready to be exposed on the net. Now we just need to do that work.

--
martin | http://madduck.net/ | http://two.sentenc.es/

beauty, brains, availability, personality; pick any two.

spamtraps: madduc...@madduck.net

[Luminous Salt]

On 2013-07-03 08:06, martin f krafft wrote:
> also sprach Peng Zhao <peng...@madeiracloud.com> [2013.07.03.1127
> +0200]:
>> Sounds like there are some security issues to do so and Salt's code
>> has some bugs that makes it vulnerable.
>
> At the moment, there are no known bugs, but it is true that Salt ues
> homebrew, which isn't really optimal.
>
> There's a discussion at
> https://github.com/saltstack/salt/commit/5dd304276ba5745ec21fc1e6686a0b28da29e6fc
>
> And I created an issue to track progress on switching Salt to TLS:

While I'm no crypto geek, I'm not so sure TLS is something to be so
warm and fuzzy about - would we really be much better with the flaws TLS
has had over the last few years? Am I mistaken and ignorant of updates
for which these issues are no longer of concern? From my research, TLS
can be locked down and holes plugged (to some degree) through specific
configuration, but this does not seem to be sufficient or to be relied
upon.

There many more enlightening posts/articles available, but here is one
with info on the topic:
https://community.qualys.com/blogs/securitylabs/2013/03/19/rc4-in-tls-is-broken-now-what

Maybe it is considered the safer option, given that we know more about
its vulnerabilities?


Overall, it seems that the responsible thing to do would be to run salt
as is, over a crypt'd network linked via OpenVPN or IPSec? (provided you
could handle client/server configuration as needed). Salt should
definitely use less home brew in this area, and more common practice
with standard libraries, but maybe this is also the space where we
invest in helping libzmq secure itself.

Firewall filtering could be added on top of this, to only allow traffic
from the secured VPN subnet, and if you wanted to be really responsible,
having that also setup/run through a dedicated firewall (and not just
software running on the master). I would feel more comfortable with this
type of setup than putting salt through TLS. In fact, I would probably
still apply the above with salt on TLS, so that, as the mod noted
concern for, salt were not a service running 'out in the open' on de
tubez.

Thoughts / criticisms?

[martin f krafft]

also sprach Luminous Salt <sa...@luminous.io> [2013.07.03.1613 +0200]:

> Overall, it seems that the responsible thing to do would be to run
> salt as is, over a crypt'd network linked via OpenVPN or IPSec?

Why would OpenVPN or IPsec be any more secure?

Why would you be worried about having Salt on a relatively unknown
port if it uses 0mq+TLS, but are okay with OpenSSH and an SMTP
server on well-known ports?

> Salt should definitely use less home brew in this area, and more
> common practice with standard libraries, but maybe this is also the
> space where we invest in helping libzmq secure itself.

I agree, libzmq would be the best place for TLS integration.

--
martin | http://madduck.net/ | http://two.sentenc.es/

"the husbands of very beautiful women
belong to the criminal classes."
-- oscar wilde

spamtraps: madduc...@madduck.net

[Mrten]

On 03-07-2013 16:13:53, Luminous Salt wrote:
> On 2013-07-03 08:06, martin f krafft wrote:
>> also sprach Peng Zhao <peng...@madeiracloud.com> [2013.07.03.1127 +0200]:
>>> Sounds like there are some security issues to do so and Salt's code
>>> has some bugs that makes it vulnerable.
>>
>> At the moment, there are no known bugs, but it is true that Salt ues
>> homebrew, which isn't really optimal.
>>
>> There's a discussion at
>> https://github.com/saltstack/salt/commit/5dd304276ba5745ec21fc1e6686a0b28da29e6fc
>>
>>
>> And I created an issue to track progress on switching Salt to TLS:
>
>
> While I'm no crypto geek, I'm not so sure TLS is something to be so warm and
> fuzzy about - would we really be much better with the flaws TLS has had over
> the last few years? Am I mistaken and ignorant of updates for which these
> issues are no longer of concern? From my research, TLS can be locked down and
> holes plugged (to some degree) through specific configuration, but this does
> not seem to be sufficient or to be relied upon.

TLS is no cure-all, agreed, and using OpenSSL properly is a can of worms as
well (disclaimer: I know a little about using it in C, not python).

There's also a lot of details in getting TLS right, so I'm not sure if TLS is
the right approach.


In general, for crypto to be any good, the less knobs you can tune on a
library as programmer the better. For that reason I have championed the NaCl
library (or its crossplatform version Sodium) for salt (also because the names
go so awesomely together :)

http://labs.umbrella.com/2013/03/06/announcing-sodium-a-new-cryptographic-library/

M.

[Luminous Salt]

On 2013-07-03 10:55, martin f krafft wrote:
> also sprach Luminous Salt <sa...@luminous.io> [2013.07.03.1613 +0200]:
>> Overall, it seems that the responsible thing to do would be to run
>> salt as is, over a crypt'd network linked via OpenVPN or IPSec?
>
> Why would OpenVPN or IPsec be any more secure?

homebrew solution versus that + established/more widely used and
reviewed/etc projects/code.

> Why would you be worried about having Salt on a relatively unknown
> port if it uses 0mq+TLS, but are okay with OpenSSH and an SMTP
> server on well-known ports?

I'm certainly not ok with smtp, I don't believe any of us should be
using it. Particularly after the disclosures of the last two months.

Conversely, I have a great amount of respect for the OpenSSH team, and
feel far more comfortable with their interfaces out there than most all
other services/software. It's still a risk, but that is why you layer
security. Regardless, this does not detract from my concerns over
putting TLS in salt, that just seems silly, when libzmq deserves
something like NaCl.

>> Salt should definitely use less home brew in this area, and more
>> common practice with standard libraries, but maybe this is also the
>> space where we invest in helping libzmq secure itself.
>
> I agree, libzmq would be the best place for TLS integration.

So why advocate for TLS in salt?

[Seth House]

On Wed, Jul 3, 2013 at 5:02 AM, Peng Zhao <peng...@madeiracloud.com> wrote:
> I think Salt need a HTTPS module.

The current recommended way to expose a Salt master to a less secure
network (e.g., the internet) is via a REST interface through salt-api
[1] (which uses HTTPS). Of course this may not fit all situations but
it sounds like it may be appropriate for exposing to your clients.
There is a in-development CLI interface for accessing the REST API
called pepper [2]. It is functional but currently (very) light on
features -- it is in need of eyeballs and attention.

[1] https://github.com/saltstack/salt-api
[2] https://github.com/saltstack/pepper

[Peng Zhao]

I guess the REST API is for salt master admin purpose.

--
You received this message because you are subscribed to a topic in the Google Groups "Salt-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/salt-users/3Le4xYzK0rY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Seth House]

On Wed, Jul 3, 2013 at 7:56 PM, Peng Zhao <peng...@madeiracloud.com> wrote:
> I guess the REST API is for salt master admin purpose.

The REST API can execute all regular salt commands and salt-run
commands and you can configure a variety of user accounts with a
variety of permissions (using Salt's external_auth backend).

[Peng Zhao]

But the communication between master and minion is still using 0mq,right?

--
You received this message because you are subscribed to a topic in the Google Groups "Salt-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/salt-users/3Le4xYzK0rY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Seth House]

On Wed, Jul 3, 2013 at 8:59 PM, Peng Zhao <peng...@madeiracloud.com> wrote:
> But the communication between master and minion is still using 0mq,right?

Yes. The REST API just exposes running commands on the master on a
stock Salt installation.