jinja loops within loops?

[schlag]

Hi,  I am trying to write states that deal with creating databases, mysql user & grants.  My pillar stack looks like this:

multisite_mysql_dbs:

  database1:

    master_server:       server1

    users:

      user1:           

        password_hash:  XXXXXXXXXXXXXXXXX

        grants:               all

      user2:            

        password_hash:  XXXXXXXXXXXXXXXXX

        grants:                select,insert,update,delete

      user3:             

        password_hash:  XXXXXXXXXXXXXXXXX

        grants:                all

  database2:

    master_server:        server2

    users:

      user1:           

        password_hash:  XXXXXXXXXXXXXXXXX

        grants:               all

      user2:            

        password_hash:  XXXXXXXXXXXXXXXXX

        grants:                select,insert,update,delete

      user3:             

        password_hash:  XXXXXXXXXXXXXXXXX

        grants:                all

I envision the states looking somewhat like below, where a loop is started through each database, and a subloop started which cycles through users & their respective grants.   This is one way I'd do it with a bash script, but think I might be missing some jinja magic.  

{% for db, args in pillar['multisite_mysql_dbs'].iteritems %}

mysql db {{ db }}:

  mysql_database.present:

    - name:     {{ db }}

  {% for user in args['users'] %}

mysql user {{ users }}:

  mysql_user.present:

    - name:                   {{ user }}

    - host:                     localhost

    - password_hash:     {{ args['password'] }}

    - require:

      - file:                     /etc/salt/minion.d/mysql.conf 

mysql grants for {{ user }}:

  mysql_grants.present:

    - user:             {{ user }}

    - grant:            {{ args['grants'] }}

    - database:      {{ db }}

    - require:

      - mysql_user: {{ user }}

      - file:              /etc/salt/minion.d/mysql.conf

  {% endfor %}

{% endfor %}

The errors I get are:

local:

    Data failed to compile:

----------

    Rendering SLS environments.all.db.mysql failed, render error: Traceback (most recent call last):

  File "/usr/lib/python2.6/dist-packages/salt/utils/templates.py", line 63, in render_tmpl

    output = render_str(tmplstr, context, tmplpath)

  File "/usr/lib/python2.6/dist-packages/salt/utils/templates.py", line 116, in render_jinja_tmpl

    output = jinja_env.from_string(tmplstr).render(**context)

  File "/usr/lib/pymodules/python2.6/jinja2/environment.py", line 891, in render

    return self.environment.handle_exception(exc_info, True)

  File "<template>", line 82, in top-level template code

TypeError: 'builtin_function_or_method' object is not iterable

Traceback (most recent call last):

  File "/usr/lib/python2.6/dist-packages/salt/state.py", line 1877, in render_state

    rendered_sls=mods

  File "/usr/lib/python2.6/dist-packages/salt/template.py", line 68, in compile_template

    ret = render(input_data, env, sls, **render_kwargs)

  File "/usr/lib/python2.6/dist-packages/salt/renderers/jinja.py", line 41, in render

    tmp_data.get('data', 'Unknown render error in jinja renderer')

SaltRenderError: Traceback (most recent call last):

  File "/usr/lib/python2.6/dist-packages/salt/utils/templates.py", line 63, in render_tmpl

    output = render_str(tmplstr, context, tmplpath)

  File "/usr/lib/python2.6/dist-packages/salt/utils/templates.py", line 116, in render_jinja_tmpl

    output = jinja_env.from_string(tmplstr).render(**context)

  File "/usr/lib/pymodules/python2.6/jinja2/environment.py", line 891, in render

    return self.environment.handle_exception(exc_info, True)

  File "<template>", line 82, in top-level template code

TypeError: 'builtin_function_or_method' object is not iterable

Am I close or way off?

Thanks in advance

[Ethan Erchinger]

Hi,

It should be:

pillar['multisite_mysql_dbs'].iteritems()

Ethan

[schlag]

ah yeah, thanks, i fixed that.  that snuck in sometime during my many edits attempting to figure this out.  it seems to still have a problem with the second loop.  i've tweaked things a little to try to get at the user password_hashes & grants, but still doesn't seem to read {{ user }}

{% for db, args in pillar['multisite_mysql_dbs'].iteritems() %}

mysql db {{ db }}:

  mysql_database.present:

    - name:     {{ db }}

  {% for user in args['users'].iteritems() %}    <===added .iteritems()

mysql user {{ user }}:

  mysql_user.present:

    - name:         {{ user }}

    - host:         localhost

    - password_hash:     {{ args['user']['password'] }}   <==added ['user']

    - require:

      - file:       /etc/salt/minion.d/mysql.conf

mysql grants for {{ user }}:

  mysql_grants.present:

    - user:         {{ user }}

    - grant:        {{ args['user']['grants'] }}     <===added ['user']

    - database:     {{ db }}

    - require:

      - mysql_user: {{ user }}

      - file:       /etc/salt/minion.d/mysql.conf

  {% endfor %}

{% endfor %}

local:

    Data failed to compile:

----------

    Rendering SLS environments.all.db.mysql.noworky failed, render error: Traceback (most recent call last):

  File "/usr/lib/python2.6/dist-packages/salt/utils/templates.py", line 63, in render_tmpl

    output = render_str(tmplstr, context, tmplpath)

  File "/usr/lib/python2.6/dist-packages/salt/utils/templates.py", line 116, in render_jinja_tmpl

    output = jinja_env.from_string(tmplstr).render(**context)

  File "/usr/lib/pymodules/python2.6/jinja2/environment.py", line 891, in render

    return self.environment.handle_exception(exc_info, True)

  File "<template>", line 92, in top-level template code

  File "/usr/lib/pymodules/python2.6/jinja2/environment.py", line 352, in getitem

    return obj[argument]

UndefinedError: 'dict object' has no attribute 'user'

Traceback (most recent call last):

  File "/usr/lib/python2.6/dist-packages/salt/state.py", line 1877, in render_state

    rendered_sls=mods

  File "/usr/lib/python2.6/dist-packages/salt/template.py", line 68, in compile_template

    ret = render(input_data, env, sls, **render_kwargs)

  File "/usr/lib/python2.6/dist-packages/salt/renderers/jinja.py", line 41, in render

    tmp_data.get('data', 'Unknown render error in jinja renderer')

SaltRenderError: Traceback (most recent call last):

  File "/usr/lib/python2.6/dist-packages/salt/utils/templates.py", line 63, in render_tmpl

    output = render_str(tmplstr, context, tmplpath)

  File "/usr/lib/python2.6/dist-packages/salt/utils/templates.py", line 116, in render_jinja_tmpl

    output = jinja_env.from_string(tmplstr).render(**context)

  File "/usr/lib/pymodules/python2.6/jinja2/environment.py", line 891, in render

    return self.environment.handle_exception(exc_info, True)

  File "<template>", line 92, in top-level template code

  File "/usr/lib/pymodules/python2.6/jinja2/environment.py", line 352, in getitem

    return obj[argument]

UndefinedError: 'dict object' has no attribute 'user'

[Ethan Erchinger]

iteritems() always returns two items, so your second loop needs to account for that.  Also, there is no 'user' key under the 'users' key.  This should work.

{% for db, args in pillar['multisite_mysql_dbs'].iteritems() %}

mysql db {{ db }}:

  mysql_database.present:

    - name:     {{ db }}

  {% for user, args2 in args['users'].iteritems() %}

mysql user {{ user }}:

  mysql_user.present:

    - name:         {{ user }}

    - host:         localhost

    - password_hash:     {{ args2['password_hash'] }}

    - require:

      - file:       /etc/salt/minion.d/mysql.conf

mysql grants for {{ user }}:

  mysql_grants.present:

    - user:         {{ user }}

    - grant:        {{ args2['grants'] }}

    - database:     {{ db }}

    - require:

      - mysql_user: {{ user }}

      - file:       /etc/salt/minion.d/mysql.conf

  {% endfor %}

{% endfor %}

Generates this:

mysql db database1:

  mysql_database.present:

    - name:     database1

mysql user user2:

  mysql_user.present:

    - name:         user2

    - host:         localhost

    - password_hash:     XXXXXXXXXXXXXXXXX

    - require:

      - file:       /etc/salt/minion.d/mysql.conf

mysql grants for user2:

  mysql_grants.present:

    - user:         user2

    - grant:        select,insert,update,delete

    - database:     database1

    - require:

      - mysql_user: user2

      - file:       /etc/salt/minion.d/mysql.conf

mysql user user3:

  mysql_user.present:

    - name:         user3

    - host:         localhost

    - password_hash:     XXXXXXXXXXXXXXXXX

    - require:

      - file:       /etc/salt/minion.d/mysql.conf

mysql grants for user3:

  mysql_grants.present:

    - user:         user3

    - grant:        all

    - database:     database1

    - require:

      - mysql_user: user3

      - file:       /etc/salt/minion.d/mysql.conf

mysql user user1:

  mysql_user.present:

    - name:         user1

    - host:         localhost

    - password_hash:     XXXXXXXXXXXXXXXXX

    - require:

      - file:       /etc/salt/minion.d/mysql.conf

mysql grants for user1:

  mysql_grants.present:

    - user:         user1

    - grant:        all

    - database:     database1

    - require:

      - mysql_user: user1

      - file:       /etc/salt/minion.d/mysql.conf

mysql db database2:

  mysql_database.present:

    - name:     database2

mysql user user2:

  mysql_user.present:

    - name:         user2

    - host:         localhost

    - password_hash:     XXXXXXXXXXXXXXXXX

    - require:

      - file:       /etc/salt/minion.d/mysql.conf

mysql grants for user2:

  mysql_grants.present:

    - user:         user2

    - grant:        select,insert,update,delete

    - database:     database2

    - require:

      - mysql_user: user2

      - file:       /etc/salt/minion.d/mysql.conf

mysql user user3:

  mysql_user.present:

    - name:         user3

    - host:         localhost

    - password_hash:     XXXXXXXXXXXXXXXXX

    - require:

      - file:       /etc/salt/minion.d/mysql.conf

mysql grants for user3:

  mysql_grants.present:

    - user:         user3

    - grant:        all

    - database:     database2

    - require:

      - mysql_user: user3

      - file:       /etc/salt/minion.d/mysql.conf

mysql user user1:

  mysql_user.present:

    - name:         user1

    - host:         localhost

    - password_hash:     XXXXXXXXXXXXXXXXX

    - require:

      - file:       /etc/salt/minion.d/mysql.conf

mysql grants for user1:

  mysql_grants.present:

    - user:         user1

    - grant:        all

    - database:     database2

    - require:

      - mysql_user: user1

      - file:       /etc/salt/minion.d/mysql.conf

[schlag]

thank you!  that worked alot better.  much appreciated