Variables in pillar files or cross-referencing pillar vars

[xek xek]

Hi everybody.

Imagine I have a simple pillar fileset:

$ tree pillar
pillar
├── jabberd2.sls
├── mail.sls
├── mysql.sls
├── network.sls
└── top.sls

Some of these files have different vars with the same values. E. g.:

pillar/jabberd2.sls:
jabberd2_inner_login: 'jabberd_user'
jabberd2_inner_pass: 'jabberd_pass'
jabberd2_listen_ip: 'a.b.c.d'

cat pillar/network.sls
net_ip1: 'a.b.c.d'
net_netmask1: '255.255.0.0'
net_gateway1: 'a.b.c.0'

both having 'a.b.c.d' IP address.

I have a simple question:

Is there any solution with wich I can stop copying the same data across pillar files? I'm really a little bit annoyed to copy 'a.b.c.d' to be 'nginx_ip', 'ssh_listen_address', 'jabberd2_ip' and so on.

I really want some kind of variable/linkage/aliasing solution with wich I will still have different pillar files, one for nginx, one for jabberd with different pillar variables e t. c.

[xek xek]

Anybody?

[xek xek]

Answering my own question,
{{ pillar.get('var') }}
perfectly works.

On Thursday, March 21, 2013 5:35:42 PM UTC+4, xek xek wrote:

[Thomas S Hatch]

Sorry about no response, we are working to keep up! Thanks for posing your answer!

Thomas S. Hatch  |  Founder, CTO

5272 South College Drive, Suite 301 | Murray, UT 84123

tha...@saltstack.com | www.saltstack.com

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[xek xek]

No problem, anyway thanks for the great software.

Since you support {{ pillar.get('var') }} in 0.14.x, then please make both variants work:
{{ pillar['var'] }}
{{ pillar.get('var') }}

this will make the soft more predictable.

On Monday, March 25, 2013 6:53:56 AM UTC+4, Thomas Hatch wrote:

Sorry about no response, we are working to keep up! Thanks for posing your answer!

Thomas S. Hatch  |  Founder, CTO

5272 South College Drive, Suite 301 | Murray, UT 84123

[Thomas S Hatch]

we have always supported {{ pillar.get }}, that is native python, what we support now is {{ salt['pillar.get']('foo:bar', 'baz') }} and pillar['foo'] has always worked

Thomas S. Hatch  |  Founder, CTO

5272 South College Drive, Suite 301 | Murray, UT 84123

tha...@saltstack.com | www.saltstack.com

[xek xek]

Pardon, refined the text.

Sorry, {{ pillar['var'] }} is not working when it is used in a pillar .sls file referencing other pillar .sls file.

Here is a demo I just made:

ubuntu@ubuntu:~$ tree
.
└── pillar
    ├── file1.sls
    ├── file2.sls
    └── top.sls

1 directory, 3 files

ubuntu@ubuntu:~$ cat pillar/top.sls
base:
  '*':
    - file1
    - file2

ubuntu@ubuntu:~$ cat pillar/file1.sls
foo: bar

ubuntu@ubuntu:~$ cat pillar/file2.sls
foo2: {{ pillar['foo'] }}

ubuntu@ubuntu:~$ sudo salt-call pillar.data
[INFO    ] Loaded configuration file: /etc/salt/minion
[CRITICAL] Pillar render error: Rendering SLS file2 failed, render error:
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/salt/utils/templates.py", line 55, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/usr/lib/pymodules/python2.7/salt/utils/templates.py", line 98, in render_jinja_tmpl
    output = jinja_env.from_string(tmplstr).render(**context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 894, in render
    return self.environment.handle_exception(exc_info, True)
  File "<template>", line 2, in top-level template code
UndefinedError: 'dict object' has no attribute 'foo'

[CRITICAL] Pillar render error: Rendering SLS file2 failed, render error:
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/salt/utils/templates.py", line 55, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/usr/lib/pymodules/python2.7/salt/utils/templates.py", line 98, in render_jinja_tmpl
    output = jinja_env.from_string(tmplstr).render(**context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 894, in render
    return self.environment.handle_exception(exc_info, True)
  File "<template>", line 2, in top-level template code
UndefinedError: 'dict object' has no attribute 'foo'


ubuntu@ubuntu:~$ vi pillar/file2.sls
ubuntu@ubuntu:~$ cat pillar/file2.sls
foo2: {{ pillar.get('foo') }}

ubuntu@ubuntu:~$ sudo salt-call pillar.data
[INFO    ] Loaded configuration file: /etc/salt/minion
foo:
    bar
foo2:
    bar
master:
    ----------
    acceptance_wait_time:
        10
    arg:
    autoload_dynamic_modules:
        True
    backup_mode:
       
    cache_jobs:
        False
    cachedir:
        /var/cache/salt/minion
    caller:
        True
    clean_dynamic_modules:
        True
    conf_file:
        /etc/salt/minion
    config_dir:
        /etc/salt
    cython_enable:
        False
    default_include:
        minion.d/*.conf
    disable_modules:
    disable_returners:
    dns_check:
        True
    doc:
        False
    environment:
        None
    extension_modules:
        /var/cache/salt/minion/extmods
    external_nodes:
       
    failhard:
        False
    file_client:
        local
    file_roots:
        ----------
        base:
            - /home/ubuntu/pillar
    fun:
        pillar.data
    grains_run:
        False
    hash_type:
        md5
    id:
        ubuntu
    ipc_mode:
        ipc
    json_out:
        False
    local:
        False
    log_datefmt:
        %H:%M:%S
    log_datefmt_logfile:
        %Y-%m-%d %H:%M:%S
    log_file:
        /var/log/salt/minion
    log_fmt_console:
        [%(levelname)-8s] %(message)s
    log_fmt_logfile:
        %(asctime)s,%(msecs)03.0f [%(name)-17s][%(levelname)-8s] %(message)s
    log_granular_levels:
        ----------
    log_level:
        info
    loop_interval:
        60
    master:
        salt
    master_finger:
       
    master_port:
        4506
    module_dirs:
    multiprocessing:
        True
    no_color:
        False
    open_mode:
        False
    outputter_dirs:
    permissive_pki_access:
        False
    pidfile:
        /var/run/salt-minion.pid
    pillar:
        ----------
        foo:
            bar
        foo2:
            None
        master:
            ----------
            acceptance_wait_time:
                10
            arg:
            autoload_dynamic_modules:
                True
            backup_mode:
               
            cache_jobs:
                False
            cachedir:
                /var/cache/salt/minion
            caller:
                True
            clean_dynamic_modules:
                True
            conf_file:
                /etc/salt/minion
            config_dir:
                /etc/salt
            cython_enable:
                False
            default_include:
                minion.d/*.conf
            disable_modules:
            disable_returners:
            dns_check:
                True
            doc:
                False
            environment:
                None
            extension_modules:
                /var/cache/salt/minion/extmods
            external_nodes:
               
            failhard:
                False
            file_client:
                local
            file_roots:
                ----------
                base:
                    - /home/ubuntu/pillar
            fun:
                pillar.data
            grains_run:
                False
            hash_type:
                md5
            id:
                ubuntu
            ipc_mode:
                ipc
            json_out:
                False
            local:
                False
            log_datefmt:
                %H:%M:%S
            log_datefmt_logfile:
                %Y-%m-%d %H:%M:%S
            log_file:
                /var/log/salt/minion
            log_fmt_console:
                [%(levelname)-8s] %(message)s
            log_fmt_logfile:
                %(asctime)s,%(msecs)03.0f [%(name)-17s][%(levelname)-8s] %(message)s
            log_granular_levels:
                ----------
            log_level:
                info
            loop_interval:
                60
            master:
                salt
            master_finger:
               
            master_port:
                4506
            module_dirs:
            multiprocessing:
                True
            no_color:
                False
            open_mode:
                False
            outputter_dirs:
            permissive_pki_access:
                False
            pidfile:
                /var/run/salt-minion.pid
            pillar_roots:
                ----------
                base:
                    - /home/ubuntu/pillar
            pki_dir:
                /etc/salt/pki/minion
            providers:
                ----------
            raw_out:
                False
            recon_max:
                5000
            render_dirs:
            renderer:
                yaml_jinja
            retry_dns:
                30
            return:
               
            returner_dirs:
            root_dir:
                /
            saltversion:
                0.13.3
            selected_output_option:
                output_indent
            sls_list:
            sock_dir:
                /var/run/salt/minion
            startup_states:
               
            state_output:
                full
            state_top:
                salt://top.sls
            state_verbose:
                True
            states_dirs:
            tcp_keepalive:
                True
            tcp_keepalive_cnt:
                -1
            tcp_keepalive_idle:
                300
            tcp_keepalive_intvl:
                -1
            tcp_pub_port:
                4510
            tcp_pull_port:
                4511
            test:
                False
            text_out:
                False
            top_file:
               
            update_restart_services:
            update_url:
                False
            user:
                root
            verify_env:
                True
            whitelist_modules:
            win_repo_cachefile:
                salt://win/repo/winrepo.p
            yaml_out:
                False
    pillar_roots:
        ----------
        base:
            - /home/ubuntu/pillar
    pki_dir:
        /etc/salt/pki/minion
    providers:
        ----------
    raw_out:
        False
    recon_max:
        5000
    render_dirs:
    renderer:
        yaml_jinja
    retry_dns:
        30
    return:
       
    returner_dirs:
    root_dir:
        /
    saltversion:
        0.13.3
    selected_output_option:
        output_indent
    sls_list:
    sock_dir:
        /var/run/salt/minion
    startup_states:
       
    state_output:
        full
    state_top:
        salt://top.sls
    state_verbose:
        True
    states_dirs:
    tcp_keepalive:
        True
    tcp_keepalive_cnt:
        -1
    tcp_keepalive_idle:
        300
    tcp_keepalive_intvl:
        -1
    tcp_pub_port:
        4510
    tcp_pull_port:
        4511
    test:
        False
    text_out:
        False
    top_file:
       
    update_restart_services:
    update_url:
        False
    user:
        root
    verify_env:
        True
    whitelist_modules:
    win_repo_cachefile:
        salt://win/repo/winrepo.p
    yaml_out:
        False

[Thomas S Hatch]

Ahh, pillar does not have access to itself when rendering

Thomas S. Hatch  |  Founder, CTO

5272 South College Drive, Suite 301 | Murray, UT 84123

tha...@saltstack.com | www.saltstack.com

[xek xek]

Are you going to do smth about this?
I can file a bug on GitHUb if you want.

This is really the coolest feature for me — to reference one pillar vars from another file.

[Les Mikesell]

On Tue, Mar 26, 2013 at 12:06 PM, Thomas S Hatch <that...@gmail.com> wrote:
>
> Ahh, pillar does not have access to itself when rendering
>

Intentional? Shouldn't there be a way to express a hierarchy where
lower levels can inherit or override values?

--
Les Mikesell
lesmi...@gmail.com

[Thomas S Hatch]

This is something that could be added, go ahead and open an issue.

Thomas S. Hatch  |  Founder, CTO

5272 South College Drive, Suite 301 | Murray, UT 84123

tha...@saltstack.com | www.saltstack.com

[Kris Saxton]

Inheritance and overriding and both really useful features when pulling in external data, having it in the native pillar would be great. As a stopgap though, it's fairly easy to implement this (or any arbitrary behaviour you want) using external_pillar. Both features (inheritance and overriding) are present in the external LDAP pillar module for example.

--
Kris Saxton
e: kr...@automationlogic.com
m: +447932834856
t: @KrisSaxton

[Les Mikesell]

On Tue, Mar 26, 2013 at 12:51 PM, Kris Saxton <kr...@automationlogic.com> wrote:
> Inheritance and overriding and both really useful features when pulling in external data, having it in the native pillar would be great. As a stopgap though, it's fairly easy to implement this (or any arbitrary behaviour you want) using external_pillar. Both features (inheritance and overriding) are present in the external LDAP pillar module for example.

It's the behavior I would have guessed for the multi-level get function.

--
Les Mikesell
lesmi...@gmail.com

[John Albietz]

Kris, 

That sounds really interesting. Can you give an example of how to use external_pillar with inheritance and overriding?

Is this the LDAP pillar module you're referring to?

https://github.com/KrisSaxton/salt-ldap

- John

John Albietz

e:inthec...@gmail.com

t: @inthecloud247

l: www.linkedin.com/in/ydavid

[Kris Saxton]

Hi John,

Sure, I blogged about it at the time:

LDAP Backend for Salt Pillar
http://www.automationlogic.com/archives/345

and

Salt Pillar, LDAP and lists
http://www.automationlogic.com/archives/359

That's the correct repo yes, although these LDAP add-ons have been in the salt core since 0.10.4

I think it would be great to see these kind of patterns implemented in other external pillar modules (SQL, noSQL etc.) as well as in the native pillar.

Best,

Kris

--
Kris Saxton
e: kr...@automationlogic.com
m: +447932834856
t: @KrisSaxton

[xek xek]

Pardon me, dear salt developers, but now I've tried to run a full master-minion scheme and what I see is
    {{ pillar.get('var') }}
is only working in stand-alone mode. This is very-very pity ;(

Please make some functionality to include/reference variables in pillar from other pillar files. Please.

On Thursday, March 21, 2013 5:35:42 PM UTC+4, xek xek wrote:

[Mrten]

On 18/4/2013 21:38 , xek xek wrote:
> Pardon me, dear salt developers, but now I've tried to run a full
> master-minion scheme and what I see is
> {{ pillar.get('var') }}
> is only working in stand-alone mode. This is very-very pity ;(
>
> Please make some functionality to include/reference variables in pillar
> from other pillar files. Please.

You're not the first ;)

https://github.com/saltstack/salt/issues/1064
https://github.com/saltstack/salt/issues/4244
https://github.com/saltstack/salt/issues/3991
https://github.com/saltstack/salt/issues/1543

M.

[Mrten]

On 18/4/2013 22:08 , Mrten wrote:
> On 18/4/2013 21:38 , xek xek wrote:

> https://github.com/saltstack/salt/issues/1543

I needed this, too, and I should've refrained from just wisecracking
buglists. You can see how I've worked around it in a comment that I just
posted in the link above.

Basically: load the pillar you want to reference with yaml.safe_load().
Hacky, yes, and limits you to mako. But, it works. Also, take care not
to step into the trap of

https://github.com/saltstack/salt/issues/3991

and give your new pillar a unique name (but I see the bug is slated for
0.15)

M.

[Thomas S Hatch]

These are not simple problems to solve, which is why they are still in here, but we are working on them

Thomas S. Hatch  |  Founder, CTO

5272 South College Drive, Suite 301 | Murray, UT 84123

tha...@saltstack.com | www.saltstack.com

M.