Vars Module

[Kyle James Walker]

Original Post (Asked to post here): https://groups.google.com/d/msg/ansible-project/7NABAfO745E/O4xfFcDQotUJ

I have a fews questions:

1. Is there a way to turn on stdout for a task, to allow user prompting?

2. Is there support for creating a 'vars module', run before any tasks like vars and vars_prompt?

3. Any recommendations on another direction to get the desired results mentioned below?

I'm currently looking into creating a custom module that will read a file, take a user's input, and based on the input create a file on the system, with additional options like not prompting the user if the file already exists on the remote system.

The problem is I can't find a way to get the stdout to display when prompting the user, and it would be better to have a 'vars module' instead of 'task module' (run before the playbook, aka the same times vars_prompt is run).

I could do this with the vars_prompt, and a template, but then I loose flexibility and of the functionality currently in my app, and have more locations to update when changes are made.

Here's my example:

my_app/defaults.yaml (in revision control):  This has all the values that can be overridden, along with some defaults.  Some are perfectly fine to keep in revision control, so defaults are usually ok, others are not (passwords, etc).

# Program Defaults, do not edit this file!!!

# All values should be overridden in the 'settings.yaml' file.

sample_config:

  databases:

    database_name_1:

      user: user

      passwd: password

      host: db.example.com:5432

      db: postgres

      compress: true

      engine: postgresql

    database_name_2:

      user: user

      passwd: password

      host: anotherdb.example2.com:5432

      db: example

      compress: true

      engine: postgresql

  secret_key: example hard to guess flask secret key

  more_values: more_examples

remote_location/settings.yaml: Generated by user with, outside revision control. This doesn't need every values, as I want the defaults left out of this file, so if updated in rev control they won't be overridden.

sample_config:

  databases:

    database_name_1:

      user: kyle_walker

      passwd: my password is secure

    database_name_2:

      user: sample_app_login

      passwd: the apps password is secure also

  secret_key: secure flask secret key

Core logic of my module (work in progress): This would work well enough if the stdout can be enabled.  But I think it would be better to be have has run at prompt time, and then use a copy with no source, and this variable data as the default contents, or another custom module.

#!/usr/bin/python

# -*- coding: utf-8 -*-

import os

import yaml

from collections import OrderedDict

DOCUMENTATION = '''

---

module: yaml_prompt

... Not needed for example ...

author: Kyle Walker

'''

EXAMPLES = '''

... Not needed for example ...

'''

def ordered_load(stream, Loader=yaml.Loader, object_pairs_hook=OrderedDict):

    class OrderedLoader(Loader):

        pass

    OrderedLoader.add_constructor(

        yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,

        lambda loader, node: object_pairs_hook(loader.construct_pairs(node)))

    return yaml.load(stream, OrderedLoader)

def ordered_dump(data, stream=None, Dumper=yaml.Dumper, **kwds):

    class OrderedDumper(Dumper):

        pass

    def _dict_representer(dumper, data):

        return dumper.represent_mapping(

            yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,

            data.items())

    OrderedDumper.add_representer(OrderedDict, _dict_representer)

    return yaml.dump(data, stream, OrderedDumper, **kwds)

def prompt_user(module, settings, depth=1, defaults=True):

    if depth is 1:

        print 'Enter values for:'

    for key, val in settings.iteritems():

        indent = "  " * depth

        if isinstance(val, dict):

            print "{}{}:".format(indent, key)

            org_len = len(val)

            prompt_user(module, val, depth+1, defaults)

            # Remove if all values were default.

            if defaults is False and len(val) is 0 and org_len is not 0:

                settings.pop(key)

        else:

            new_val = raw_input("{}{}({}):".format(indent, key, val))

            if new_val:

                settings[key] = new_val

            elif defaults is False:

                settings.pop(key)

def main():

    module = AnsibleModule(

        # not checking because of daisy chain to file module

        argument_spec=dict(

            src=dict(required=True),

            dest=dict(required=True),

            copy_defaults=dict(default=True, type='bool'),

            override=dict(default=True, type='bool'),

        ),

        add_file_common_args=True,

    )

    src = os.path.expanduser(module.params['src'])

    dest = os.path.expanduser(module.params['dest'])

    copy_defaults = module.params.get('copy_defaults', None)

    override = module.params.get('validate', None)

    settings = ordered_load(open(src), yaml.SafeLoader, OrderedDict)

    prompt_user(module, settings, defaults=copy_defaults)

    content = ordered_dump(settings,

                           Dumper=yaml.SafeDumper,

                           default_flow_style=False)

    module.fail_json(msg="Still testing, stop here.\n{}".format(content))

    res_args = dict(

        dest=dest,

        src=src,

        md5sum=md5sum_src,

        changed=changed

    )

    # Copy the created contents to the server.

    module.params['src'] = None

    module.params['dest'] = dest

    module.params['content'] = content

    file_args = module.load_file_common_arguments(module.params)

    res_args['changed'] = module.set_file_attributes_if_different(

        file_args,

        res_args['changed']

    )

    module.exit_json(**res_args)

# import module snippets

from ansible.module_utils.basic import *

main()

[Michael DeHaan]

Replies inline.

On Fri, Feb 21, 2014 at 4:05 PM, Kyle James Walker <kylejam...@gmail.com> wrote:

Original Post (Asked to post here): https://groups.google.com/d/msg/ansible-project/7NABAfO745E/O4xfFcDQotUJ

I have a fews questions:

1. Is there a way to turn on stdout for a task, to allow user prompting?

There is not.    (And technically this would also require stdin)

 

2. Is there support for creating a 'vars module', run before any tasks like vars and vars_prompt?

There is a vars plugin infrastructure that powers group_vars and host_vars, but is a per-host mechanism.

You probably want to use vars_prompt or "-e", or maybe make a wrapper script.

 

3. Any recommendations on another direction to get the desired results mentioned below?

I'm currently looking into creating a custom module that will read a file, take a user's input, and based on the input create a file on the system, with additional options like not prompting the user if the file already exists on the remote system.

 

The problem is I can't find a way to get the stdout to display when prompting the user, and it would be better to have a 'vars module' instead of 'task module' (run before the playbook, aka the same times vars_prompt is run).

I could do this with the vars_prompt, and a template, but then I loose flexibility and of the functionality currently in my app, and have more locations to update when changes are made.

I'm a little unclear on the use case above, without dropping into the code can you explain more how this would be used by an end user?

Thanks!

[Kyle James Walker]

Here's my preferred use case:

When running the playbook the user will be prompted possible server variables (like what vars_prompt would do) but these will read from a file with defaults and nested variables, not just a flat file of variables.

Once the user has been prompted for all the values in the file, a file will be created on the remote system with only the values that are different from the input file.

Is there any documentation on hooking into the var plugin infrastructure? I found the VarsModule but not seeing any examples to hook into this, like there is for the AnsibleModule.

Hopefully this helps clear up what I'm trying to accomplish.

Thanks

[Michael DeHaan]

"When running the playbook the user will be prompted possible server variables (like what vars_prompt would do) but these will read from a file with defaults and nested variables, not just a flat file of variables."

There's no way for vars_prompt to take a list of files for things to prompt for, but I'd be willing to consider it via a pull request.

This should probably work as:

vars_prompt:

    - include: <filename>

   

If this doesn't work for you, you could still code generate the playbook.

"Once the user has been prompted for all the values in the file, a file will be created on the remote system with only the values that are different from the input file."

This would be up to you.

"Is there any documentation on hooking into the var plugin infrastructure?"

Unlike most of the plugins, this part is not intended to be a user-serviceable interface.   I'd recommend the above patch instead.

--
You received this message because you are subscribed to the Google Groups "Ansible Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-deve...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Kyle James Walker]

Thanks, for now I think I'll setup the playbook to fail if the files doesn't exist (then one I was hoping to generate).  Then I can simply manually create the file, or what you recommended.

Do you think it would be beneficial to allow VarsModules to be a user-serviceable interface in the future?  I feel support for vars_prompt to take a file as you mentioned with defaults would be a great feature, but it would still be great to have the ability to write a custom vars module for adding that extra level of configuration.

Thanks again for your help.

[Michael DeHaan]

=

Do you think it would be beneficial to allow VarsModules to be a user-serviceable interface in the future?  I feel support for vars_prompt to take a file as you mentioned with defaults would be a great feature, but it would still be great to have the ability to write a custom vars module for adding that extra level of configuration.

Usually most people handle these kinds of needs by inventory scripts.   The "vars" plugin allows an extra layer to hook this in, but it's only decided for reading host_vars and group_vars, and could be inefficient for various applications.  Since it's really there to only support that need I don't want to undertake the idea of supporting it as a documented external interface.

[Christopher O'Connell]

I'm not sure this justified creating an external vars plugin, but here's two specific use cases where we are currently using an external script to generate yml files:

1) Generating DNS zone files for domain names that point to DNS servers (e.g. we have a database with a list of domain names which need to be available for other people to put into the nameserver field at a registrar). We need to update the host names and glue records on these machines and at the registrar.
2) Generating a list of nodes for a ceph deployment from a web tool which allows us to assign servers to "roles".

I realize that both of these are edge cases, and but our current solution for this is to run a bash script which contacts the respective servers which render out vars files as yml and then run ansible. I think we definitely *could* use a "vars_plugin" type infrastructure for this.

Without having dug far into that, I suspect that a large part of the complexity of allowing vars plugins will be in resolving which version to use on conflicting keys.

All the best,

~ Christopher