gitfs fails to pull sls files into salt cache

[rohit]

Running into an issue in using gitfs to invoke states.

Salt server version : salt 2014.1.5

On running highstate : salt '*' state.highstate -l debug

    Data failed to compile:
----------
    No matching sls found for 'vim' in env 'base'
----------
    No matching sls found for 'firefox' in env 'base'

Contents of top.sls :

base:
 'roles:******':
  - match: grain
  - vim
 '*':
  - firefox

gitfs configuration on /etc/salt/master :

     gitfs_remotes:

- git+ssh://git@gitlab.****.com/rsood/saltdev.git
- git://git@gitlab****.com/rsood/saltdev.git
- http://gitlab.*****.com/rsood/saltdev.git
- file:///gitLabRepos/saltdev

I have tried individually using each of the above syntax .

Noticed that the init.sls does not get pulled under individual folders in the Salt cache :

[root@ip-******]# ls -R /srv/salt/var/cache/salt/

master/gitfs/refs/master/

/srv/salt/var/cache/salt/master/gitfs/refs/master/:
firefox  top.sls  vim

/srv/salt/var/cache/salt/master/gitfs/refs/master/firefox:

/srv/salt/var/cache/salt/master/gitfs/refs/master/vim:

The local repo layout under /gitLabRepos/saltdev is :


[root@ip-*****]# ls -R
.:
aws  base  pillar  top.sls

./aws:
userData

./aws/userData:
cloud-init.txt

./base:
firefox  gitClient  mList  rmFile  vim

./base/firefox:
init.sls

./base/gitClient:
init.sls

./base/mList:
init.sls  serverPing

./base/rmFile:
init.sls

./base/vim:
init.sls

./pillar:
mdata.sls  top.sls

- show quoted text -

[Colton Myers]

Gitfs actually works with the raw git objects, rather than checking out a working copy of each branch or anything like that.  So just because you don't see the init.sls file where you expect it to be, doesn't necessarily mean that it's not being pulled down.

The way to check what files are available via gitfs is via the `cp` module.  Run `salt myminion cp.list_master saltenv=myenv` to see all the files available in an environment.  This works for all fileserver backends.  Run this, and see if the relevant files are there.  If not, you should probably file an issue on Github so we can investigate this further.

--

Colton Myers

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rohit Sood]

The cp.list_master state does come back with the correct list of files from git.

thanks,
Rohit

[root@ip-*** ~]# salt my-minion state.highstate

my-minion:

    Data failed to compile:
----------
    No matching sls found for 'vim' in env 'base'
----------
    No matching sls found for 'firefox' in env 'base'

[root@ip-**** ~]# salt my-minion cp.list_master

my-minion:
    - aws/userData/cloud-init.txt
    - base/firefox/init.sls
    - base/gitClient/init.sls
    - base/mList/init.sls
    - base/mList/serverPing
    - base/rmFile/init.sls
    - base/vim/init.sls
    - pillar/mdata.sls
    - pillar/top.sls
    - top.sls

[Rohit Sood]

I am able to do execute an explicit copy from the master :

[root@ip-** ~]# salt my-minion cp.get_file salt://base/firefox/init.sls /var/cache/salt/minion/files/base/firefox/init.sls

my-minion:
    /var/cache/salt/minion/files/base/firefox/init.sls

[Rohit Sood]

To isolate the issue, I disabled the gitfs backend and enabled local file root.

The error remained unchanged.

To resolve it,  I copied the /srv/salt/firefox/init.sls as  /srv/salt/firefox.sls and repeated that for .../vim/init.sls.

...Rohit

[root@ip-**** base]#  grep '^[^#]' /etc/salt/master
publish_port: 4505
user: root
ret_port: 4506
root_dir: /srv/salt
cachedir: /var/cache/salt/master
file_roots:
  base:
    - /srv/salt/
fileserver_backend:
  - roots
win_repo: '/srv/salt/win/repo'
win_repo_mastercachefile: '/srv/salt/win/repo/winrepo.p'
win_gitrepos:
  - 'https://github.com/saltstack/salt-winrepo.git'


[root@ip-**** salt]# ls -l /srv/salt
total 24
drwxr-xr-x. 7 root root 4096 Jul 24 22:47 base
drwxrwx---. 3 root root 4096 Jul 24 20:12 etc
-rw-r--r--. 1 root root   31 Jul 24 22:47 firefox.sls
-rw-r--r--. 1 root root   94 Jul 24 23:45 top.sls
drwxrwx---. 5 root root 4096 Jul 24 20:12 var
-rw-r--r--. 1 root root   25 Jul 24 22:47 vim.sls

[Rohit Sood]

Finally fixed it. The key was to understand the subtle dependency between the location of the top.sls and the invoked state sls files in the hierarchy.
The file_roots/gitfs_roots settings need to be tweaked.

Broken state :
   

Location of top.sls : /srv/salt/top.sls
Dependencies :

/srv/salt/base/firefox/init.sls
/srv/salt/base/vim/init.sls

Note the extra "base" folder created.

Fix : Move the top.sls file under the .../base/ folder and change file_roots for local files or gitfs_roots for remote file sourcing

Location of top.sls : /srv/salt/base/top.sls
Dependencies : Unchanged as above

Master configuration(/etc/salt/master) :

file_roots:
  base:
    - /srv/salt/base
fileserver_backend:
  - git
gitfs_remotes:
  - file:///gitLabRepos/saltdev
gitfs_root: base/

[Colton Myers]

Glad you got it figured out!  Sorry for my extremely slow reply.

--

Colton Myers