salt.states.file.recurse - any way to exclude an exact file path from the file tree?

Alex Bolgarov

unread,

Jan 4, 2013, 1:33:43 PM1/4/13

to salt-...@googlegroups.com

Hi,

'salt.states.file.recurse', that specifies that Salt should recursively copy a file tree to the minion node, has a 'exclude_pat' parameter that allows to give it a glob or regex to check the filenames to be excluded from the copying operation. I was trying to understand if I can specify an exact file path, starting from the root of the file tree (the file tree that I want to copy to the minion), if I want to exclude this exact file. So far my understanding is that glob or regex from the 'exclude_pat' parameter will be attempted to match only on the file basenames (without using full file path).

Here is an example of a situation I have:

I want Salt to copy bunch of a php files in a file tree from the salt fileserver area to the minion nodes (say, into a /var/www/htdocs/myapp/ directory). So initially I have to prepare a file tree in the salt fileserver area. I have a script that copies file tree from my development directory (where I develop those php files), into the salt fileserver area, so that as I'm making changes to the php code, I can easily copy these changes to the salt fileserver and then run highstate command to deploy them to minions.

The file tree has several files named 'config.php', in various places of the tree. Upon deployment to the minion nodes, one of these 'config.php' files has to be modified to include correct config information specific for this depolyment (such as database access info, etc.). Separately from the whole tree of the php files, I have a template for this config.php file, which is supposed to be processed by jinja template engine and get correct config information from the pillars data repository. I'm using salt.states.file.managed to copy this single config file and pass it through jinja engine.

So, in the sls file I have:

a): file.recurse statement to copy the whole tree of php files:

/var/www/htdocs/myapp:
        file: [ recurse, { source: "salt://files/myapp" }, ... ]

b) file.managed statement to copy and process by jinja the template of the config.php file, overwriting the config.php already present in the file tree previously copied:

/var/www/htdocs/myapp/config.php:
        file: [ managed, { source: "salt://files/templates/config.php" }, { template: jinja }, { require : [ { file: "/var/www/htdocs/myapp"} ] }, ... ]

The problem with this is, even if I did not make any changes, running highstate will always report changes back, because first 'file.recurse' will see that the config.php file was modified and overwrite it from salt://files/myapp/config.php; then file.manage will see that the file was modified and will overwrite it by the result of the template expansion of the salt://files/templates/config.php.

This is not fatal, because the result will be correct - that is, on the minions myapp/config.php file will always have correct contents after the template expansion, but I don't like the fact that Salt will be making unnecessary changes every time highstate is running (when the changes were not necessary, because there were no changes in the sources).

So I was thinking about using 'exclude_pat' in the file.recorse, to make Salt not to copy this config.php file to the minions from myapp/config.php, and then allowing salt to copy template-expanded config.php from templates/config.php using file.managed. So is this possible:

/var/www/htdocs/myapp:
        file: [ recurse,
{ source: "salt://files/myapp" },
            { exclude_pat : "/files/myapp/config.php" },
   ...
]

Note that { exclude_pat : "config.php" } will probably work - but I have more then one config.php file in other subdirectories of the tree, and this way all of them will be excluded, right?

Alex Bolgarov

unread,

Jan 7, 2013, 2:33:38 PM1/7/13

to salt-...@googlegroups.com

OK, I think I figured it out. I looked at the source code, it appears that you actually can use 'exclude_pat' to exclude a specific file on a specific path within your file tree. When salt processes excludes in 'recurse' , it tries to match to the pattern in 'exclude_pat' the path of the file starting from the path given in the 'source' parameter; and this path does not include '/' at the beginning. Which means if you have a file tree under "salt://files/myapp" and want to exclude file "salt://files/myapp/config/config.php", you use:

/var/www/htdocs/myapp:
file: [ recurse,
{ source: "salt://files/myapp" },

{ exclude_pat : "config/config.php" },
...
]

Notice that 'exclude_pat' starts where 'source' finishes, and there is no '/' at the beginning of the 'exclude_pat'.

This way file "salt://files/myapp/config/config.php" will be excluded, but file "salt://files/myapp/other-dir/config.php" will be not excluded.

BTW, the same is true for processing of 'include_pat' too.