Example of using git as a remote location

[Daniel B]

Is there an example of using git as the remote location for the file server?  I see the notes in the file server setting file, but it's a bit unclear how to pull the files from the minion.

If this is in the setup:

fileserver_backend:
  - git

gitfs_remotes:
  - git://github.com/saltstack/salt.git

How would I checkout tag v0.12.0 onto a minion using a python module?

thanks

-daniel

[Thomas S Hatch]

You don't checkout a tag, technically the file server backend for git never checks out any tags and branches, the salt master automatically updates the git repo for you and then the files are available via salt:// in sls files and via the cp module.

Tags and branches become salt environments. What is the end goal that you have in mind for it? Perhaps we can shed some light on the underlying problem you are trying to solve.

--
 
 

--

Thomas S. Hatch  |  Founder, CTO

5272 South College Drive, Suite 301 | Murray, UT 84123

tha...@saltstack.com | www.saltstack.com

[Daniel B]

This is probably a very common use case; I'm trying to push code for a python project to the webserver minions.  The code is all in git, tagged by the CI server.  Right now I'm checking out the tag on the salt master and rsyncing the files to the minions.

I'd like to have the minions pull directly from git on the master and save the rsync step.  I don't want to have the minions have a clone git on the salt master so I don't have to maintain ssh keys for every server (that's already done by salt).

Thanks for any help

[Thomas S Hatch]

thats easy then! I am assuming you are using file.recurse:

/opt/code:

  file.recurse:

    - env: v0.12.0

    - source: salt:///

The env (the salt fileserver environment) becomes the tag in git, a branch works too, so you could specify the development branch

Thomas S. Hatch  |  Founder, CTO

5272 South College Drive, Suite 301 | Murray, UT 84123

tha...@saltstack.com | www.saltstack.com

On Wed, Jan 16, 2013 at 11:38 PM, Daniel B <dbond...@gmail.com> wrote:

--
 
 

[Daniel B]

If there were multiple gitfs_remotes, and I was checking out develop, then how would the file server tell them aparts?

Master config:

fileserver_backend:
  - git

gitfs_remotes:
  - git://github.com/saltstack/salt.git

  - git://github.com/saltstack/salt-states.git
  - git://github.com/saltstack/salt-ci.git

state file:

/opt/code:

  file.recurse:

    - env: develop

    - source: salt:///

which git repo would it pull from?  Or how would I specify pulling from salt-ci?

[Thomas S Hatch]

It will actually merge them together, so it searched for the file in the environment in the order in which the remotes are listed and the first time it finds the file, then that file become authoritative  Really though, I would discourage deploying salt projects directly from develop, since I reserve the right to break develop

--
 
 

--

[Daniel B]

I was just using the salt repos as an example.

It becomes problematic when I'm trying to deploy tag 1.0.0, and I have three listed gitfs remotes.   Is there anyway to control which repo it pulls from?

[Thomas S Hatch]

not right now, no. But this is something that I imagine we could get in

On Thu, Jan 17, 2013 at 11:48 AM, Daniel B <dbond...@gmail.com> wrote:

I was just using the salt repos as an example.

It becomes problematic when I'm trying to deploy tag 1.0.0, and I have three listed gitfs remotes.   Is there anyway to control which repo it pulls from?

--
 
 

[Erik Johnson]

So, say you have multiple git repos listed in your file_roots. How would it differentiate between branches/tags if the same branch/tag exists in both? If this is not currently something that it can do, what is the way in which salt decides which is authoritative?

--
 
 

--

-Erik

"For me, it is far better to grasp the universe as it really is than to persist in delusion, however satisfying and reassuring."  -- Carl Sagan

[Thomas S Hatch]

Everything is ordered, so if you have conflicting environments the first environment wins, git repos are ordered but file backends are also ordered:

so if you say:

fileserver_backend:

  - git

then the file_roots are never read,

if you say:

fileserver_backend:

  - roots

  - git

then the environment where the file is first found wins if a git repo and a file_roots have the same environment. This is useful because a file can be edited on the fly in file_roots to test it out, and then you know exactly what file(s) have been changed to diff against git

Thomas S. Hatch  |  Founder, CTO

5272 South College Drive, Suite 301 | Murray, UT 84123

tha...@saltstack.com | www.saltstack.com

--
 
 

[Erik Johnson]

On Fri, Jan 18, 2013 at 11:23 AM, Thomas S Hatch <that...@gmail.com> wrote:

Everything is ordered, so if you have conflicting environments the first environment wins, git repos are ordered but file backends are also ordered:

so if you say:

fileserver_backend:

  - git

then the file_roots are never read,

if you say:

fileserver_backend:

  - roots

  - git

then the environment where the file is first found wins if a git repo and a file_roots have the same environment. This is useful because a file can be edited on the fly in file_roots to test it out, and then you know exactly what file(s) have been changed to diff against git

Thomas S. Hatch  |  Founder, CTO

5272 South College Drive, Suite 301 | Murray, UT 84123

tha...@saltstack.com | www.saltstack.com

This is awesome stuff!

[Thomas S Hatch]

Thanks!

The goal was to make it so that a single environment can be spread across many sources, so you can store large files for an environment directly on the master and keep sls files in git of things like that

Thomas S. Hatch  |  Founder, CTO

5272 South College Drive, Suite 301 | Murray, UT 84123

tha...@saltstack.com | www.saltstack.com

--
 
 

[Sam Lai]

OP, I feel like I'm missing something obvious, but why not just use
the git states to pull your python project directly from git? You
don't need Salt to process the contents of your git repository and
there's already an established source serving the files.

http://docs.saltstack.org/en/latest/ref/states/all/salt.states.git.html#module-salt.states.git

> --
>
>

[Thomas S Hatch]

I think that he mentioned that he does not want to directly grant all target systems access to the git repo, but I might be mixing up conversations :)

Thomas S. Hatch  |  Founder, CTO

5272 South College Drive, Suite 301 | Murray, UT 84123

tha...@saltstack.com | www.saltstack.com

--

[Daniel Fackrell]

I may not have a handle on the request yet, and haven't been as involved in salt as I'd like for several months, but re-scanning the OP and related follow-ons, I wonder if a minion running on the same system as the master could be targeted for a git state that would check out the needed tree version into the master's file store.

With this approach, it may even be possible to have multiple versions deployed to different target systems, by checking out the code into different directories within the master's file store.

Daniel Fackrell / Intchanter

--
 
 

[Sam Lai]

Ah-ha, I knew I was missing something!

Maybe the addition of a path prefix property to each backend source, allowing you to map a backend source to a particular location on the file server, would be useful. I personally don't have a use case for this though :)

--