Using the file server functionality with salt-syndic

[Gavrie Philipson]

Hi all,

I am using Salt with the Syndic functionality, since I have several servers on an internal network that cannot communicate with the outside world. There is a 'root' salt-master running on a regular server (we'll name it host1), and there is a gateway host which can access both networks, which runs a 'second-level' salt-master, as well as a salt-syndic and a salt-minion. The other servers run 'leaf' minions that connect to the second-level master:

host1: [master] => gateway: [syndic] [master] [minion] => internal: [minion]

Now, sending commands from the root master to all leaf minions works quite well, except for one thing, which is the file server functionality: When I send a command such as `cp.get_file salt://...` to a leaf minion, it downloads the file from the master on the gateway, instead of from the root master (on host1) as I would expect. 

I'd like to distribute files from the root master to all leaf minions, and having to transfer them first to the gateway is quite a pain point for me. Is there any simple way to make all file downloads access the root master seamlessly via the syndic?

Thanks,

-- Gavrie

[Thomas S Hatch]

Right now the answer is no, there is not. Apart from having the lower masters sync the file_root on a regular basis. I want to add this in but I have not gotten to it quite yet, the problem is if the lower master needs to check for file updates every time a file is requested. Basically there are some structural aspects that need to be considered to make this happen.

[Seth Madison]

Are there any plans to revisit this? Is it on the docket?

[Colton Myers]

Yes, we'd love to have some sort of syncing to the syndics, but it may be awhile before we have the bandwidth to tackle this.

In the meantime, I think the best way to handle this is to have a minion on each syndic that connects to the master of masters.  Then you can target those minions and sync the fileserver down with a `state.top` call or similar.

--

Colton Myers

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[nma...@g.clemson.edu]

I really think there should be some mechanism to transfer the files from the master of master to the syndics without making them minions of the master of master. What if the minions on the syndics point to the master on the localhost. Is there any progress on this? Any other way files could be shared?  

[Seth House]

On Wed, Apr 30, 2014 at 10:45 AM, <nma...@g.clemson.edu> wrote:
> What if the minions on the syndics point to the master on the
> localhost.

In this case you could run two minion daemons, one pointed to
localhost one pointed to the master-master.

Another good way to keep files on syndics in sync is to serve them
from Git -- or use GitFS on each Syndic so they update themselves
automatically. (Less applicable for large binary downloads, of
course.)

[Colton Myers]

Yep, gitfs is one way, or you can just use a `file.recurse` which only runs on syndic minions to mirror the `file_roots` from your master of masters.

--

Colton Myers

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Nitin Madhok]

That's what i have used to copy the file_roots down to the syndic minions. (file.recurse)

Thanks,
Nitin Madhok
Clemson University

You received this message because you are subscribed to a topic in the Google Groups "Salt-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/salt-users/-IAamZP7t38/unsubscribe.
To unsubscribe from this group and all its topics, send an email to salt-users+...@googlegroups.com.

[钟畅达]

"""you can just use a `file.recurse` which only runs on syndic minions to mirror the `file_roots` from your master of masters."""

sorry, can you describe the way more detailed. thank you.

在 2014年5月7日星期三 UTC+8上午2:51:58，basepi写道：

[Colton Myers]

The idea is to install the salt minion on your lower level master machines (syndic masters), but instead of connecting that minion to localhost (the local master), you connect it directly to the upper level master (master of masters). Now, you can target those minions with a file.recurse state [1], something like this:

sync_file_roots:

  file.recurse:

    - name: /srv/salt

    - source: salt://file_roots

Note that we want to support something like `salt://.` so you can just sync the file_roots as they exist on the master. There's an open issue here: https://github.com/saltstack/salt/issues/15713

For now, you just have to nest the roots under a single directory to make it work properly.

Once you run the above state, the syndic masters will have the correct contents in /srv/salt and will be ready to serve those files to their minions.

[1]: http://docs.saltstack.com/en/latest/ref/states/all/salt.states.file.html#salt.states.file.recurse

--

Colton Myers

Platform Engineer, SaltStack

@basepi on Twitter/Github/IRC

[herman....@cpap.com]

Hello, just reviving this thread for some more clarification. Let's say I have a file roots config like this:

file_roots:
  base:
    - /srv/salt/base
  dev:
    - /srv/salt/dev
  prod:
    - /srv/salt/prod

If I am to sync these dirs from the master of masters down to the syndic with file.recurse, how do I go about it? In particular, what should be my "source:"?

salt://. <-- only gets files and dirs that begin with .
salt://* or salt://\* <-- invalid

Even if using salt:// were to work, won't it just sync the dir for the particular environment? How would one sync all environments at once?

Thank you!