Support for passing complex (nested dictionary) arguments to modules added

[Michael DeHaan]

Warning -- this a complex topic I'm adding to enable James Martin's
work on the Cloud Formations module. If the idea of this seems
confusing,
you don't have to use it. It will be best illustrated by live
examples that use it, rather than the technical example of how it
works.

What this is -- I've added the ability to specify complex arguments to modules.

What does it look like? Here's a really basic example:

https://github.com/ansible/ansible/blob/devel/examples/playbooks/complex_args.yml

As you notice, this gets more verbose than Ansible normally likes to
me. It's there to support those *complex* use cases, like the Cloud
Formations idea, rather than
everyday use cases. It's for when key=value arguments aren't good
enough -- which is basically only 1% of the time or something like
that.

We are still going to prefer the dead-simple "key=value" form, but
there are times when passing a complex datastructure is needed.

There are a few caveats to usage:

(A)

If the data structure is complex -- i.e. more than a dictionary of
strings, numbers, and booleans -- i.e. a nested data structure -- the
module must be written in Python
and use the standard ansible module common infrastructure or be an
action plugin that knows what to do with these things. This shouldn't
be a problem for many people,
but just be aware that a Bash module still gets the key=value stuff.
This is good actually, because you don't want to be parsing JSON in
bash most of the time :)

Simple data structures can still be passed to any old standard module,
though some clever hacks under the hood, just any members of those
hashes that are lists
or hashes themselves will not be forwarded along.

(B)

Right now this only is really working for "normal" remote modules.
This means it won't work as a way to feed data to, for example, fetch
or pause. This is easy to add
and just takes adding a few calls to self.runnner._complex_args_hack
like I have in lib/ansible/runner/action_plugins/normal.py, and we'll
do this.

Again, the goal here was to pass those arguments to things like the
CloudFormations module, so it didn't seem to be an immediate priority.

I'll file a ticket about this just the same, and I'm not documenting
this feature just yet for this reason.

(C)

It is possible to add both the key=value arguments and the extra
"args" arguments at the same time. I have coded this so the key=value
arguments can win, which provides
an extra feature where you can use the "args:" to supply defaults.

For instance, you could do, "args: ${file_defaults}" if you so chose,
and it would basically work like you would expect. I think this is
not something I would probably do, but
it's there, in case you want to do it.

(D)

In order to support this, the signature of action_plugins run() method
has changed. If you want to save your own custom action plugins from
API additions in the future, this is super
easy to do, just make sure your plugin ends with run(....,**kwargs) to
absorb keyword arguments. If you haven't done this, you will need to
make that change.

---

So, there you have it, complex argument support -- if anyone has
questions about implementation or usage, or find any interesting bugs,
let me know!

[Tomek Paczkowski]

I'm trying this out. How do I pass complex argument to hacking/test-module?

[Michael DeHaan]

You can't for now, you need to use a playbook. We could tweak that
to accept complex args from stdin or something though, maybe with an
extra flag?

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ansible-proje...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Tomasz Paczkowski]

From stdin or a file would be nice:

test-module -m ./module -a spam=good -e - # stdin
test-module -m ./module -a spam=good -e ./bacon.{yml|json}

[elektrokokke]

I find the example quite difficult to read and understand. I wondered why, and I think it's because the complex variables are not all in strict JSON format. I know, YAML is a superset of JSON, but somehow I think it would be clearer and better to read, if all complex variables would be in JSON format. At last, module output is in JSON, too...

Or did I completely misunderstand everything? "defaults" and "complex" are complex variables, right? As is the part after "args:", but unnamed in that case? I somehow get confused with the keywords looking very similar to variables.

Cheers

[Michael DeHaan]

On Fri, Feb 22, 2013 at 3:54 PM, elektrokokke <elektr...@gmail.com> wrote:
> I find the example quite difficult to read and understand. I wondered why,
> and I think it's because the complex variables are not all in strict JSON
> format. I know, YAML is a superset of JSON, but somehow I think it would be
> clearer and better to read, if all complex variables would be in JSON
> format. At last, module output is in JSON, too...

Nope, not going to happen.

We use YAML for playbooks and things are going to be in YAML.

>
> Or did I completely misunderstand everything? "defaults" and "complex" are
> complex variables, right? As is the part after "args:", but unnamed in that
> case? I somehow get confused with the keywords looking very similar to
> variables.

args is a dictionary, always, it may be a dictionary inline or one
referenced by a variable

[Arne Jacobs]

> Nope, not going to happen.
>
> We use YAML for playbooks and things are going to be in YAML.

Ok, but I guess I could always force myself to keep all complex
variables in JSON, right? :-P

> args is a dictionary, always, it may be a dictionary inline or one
> referenced by a variable

Yes of course, but it is a dictionary which is interpreted by Ansible in
a certain way, that's why I think of it as some kind of "keyword".

It just confuses me sometimes... is something such a "keyword", i.e. is
it expected that way by Ansible (like "args"), or is it expected by a
module as an argument (i.e., the module specifies the name), or is it
just a variable whose name I am free to choose, as long as I reference
it accordingly later (like "complex" in the example) etc...

Well, I'm relatively new to YAML, so sorry about the fuss.

[Michael DeHaan]

>
> Yes of course, but it is a dictionary which is interpreted by Ansible in a
> certain way, that's why I think of it as some kind of "keyword".

It's just a dictionary.

>
> It just confuses me sometimes... is something such a "keyword", i.e. is it
> expected that way by Ansible (like "args"), or is it expected by a module as
> an argument (i.e., the module specifies the name), or is it just a variable
> whose name I am free to choose, as long as I reference it accordingly later
> (like "complex" in the example) etc...

You have to say "args: "