shell/command json output into variables

[Stuart Reynolds]

(1) Many command line tools (such as the aws CLI tools) output results directory as JSON. Is there a way to slurp up stdout and json-to-pydict it?

In many cases this would be preferable to using modules which are i) highly coupled to fast-paced external code sources, ii) only implement a subset of the available functionality of tools they use. 

(2) Better yet if I had a command that returned a result as a JSON-like object, can I (in my task/playbooks) run in-place python to filter/modify that?

[Stuart Reynolds]

Re:(1) I've found that I can do this:

- hosts: localhost
- name: Describe instances
  hosts: localhost
  gather_facts: false # Prevents immediately logging in to hosts
  tasks:
  - command: aws ec2 describe-instances
    register: result
  - debug: var=result.stdout|from_json  

However, really what I want to do is:

- hosts: localhost
- name: Describe instance
  hosts: localhost
  gather_facts: false # Prevents immediately logging in to host
  tasks:
  - command: aws ec2 describe-instances
    register: x=from_json(result.stdout)
 - debug: var=x

Possible?

 

[Michael DeHaan]

callback plugins exist in ansible for this purpose and there is the '--tree' flag to ad-hoc /usr/bin/ansible mode.

However, we're not really trying to be a system that emits arbitrary JSON as many playbooks will include results for multiple systems and this would get very very large and unwieldy.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/448b7e2e-ebec-4b1e-9100-61bc8fd060bd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Peters]

Would it be possible to have an extra option to command to so that it
can work similar to the uri module's "return_content" option? That way
results that are registered would be the same except for an extra json
key with the structured data. Not sure what to call it as
"return_content" isn't particularly descriptive.

> https://groups.google.com/d/msgid/ansible-project/CA%2BnsWgzz8-7smB8CZWv7HtCFzk16CFq9vavcF-3vZ1EHMurepA%40mail.gmail.com.

[Michael DeHaan]

well it does return content already, all the stdout and such.

I think what you are saying is that have a flag that if the output of the command is already JSON and you requested this behavior, return the JSON datastructure in an element called "json"? 

I'm somewhat open to it, but also think that because it's possible to do the "from_json" stuff with set fact, we have a bit of a solution, and it *might* be used in such a minority set of use cases skimming over the option for most might cause greater confusion.

I don't know.

To be clear, I'm suggesting this:

- set_fact:

    foo: "{{ x | from_json }}"

Which I believe will keep data typed-ness if you use that form and not "foo=" on one line.

Would need to check.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAJQqANeKhy5aNqjobbeQJ%3DsUmGBMv2DNw_zhJ3mYKbxkBVrXHQ%40mail.gmail.com.

[Stuart Reynolds]

On Monday, July 14, 2014 12:39:23 PM UTC-7, Michael DeHaan wrote:

well it does return content already, all the stdout and such.

I see -- but I don't see how to usefully use that in subsequent tasks (beyond debug).

I can do:

   register: result=std_out|to_json

Or some such?

 

I think what you are saying is that have a flag that if the output of the command is already JSON and you requested this behavior, return the JSON datastructure in an element called "json"? 

Yes. That's my specific problem.

More generally, it raises the question: how can I operate on the results of an action (in code I write in playbooks and tasks vs modules)? The scope of what's possible isn't clear to me from the docs. Can I say, filter a list of results (e.g. from ec2_describe) and store the results in another list with an named value?

I'm somewhat open to it, but also think that because it's possible to do the "from_json" stuff with set fact, we have a bit of a solution, and it *might* be used in such a minority set of use cases skimming over the option for most might cause greater confusion.

I don't know.

To be clear, I'm suggesting this:

- set_fact:

    foo: "{{ x | from_json }}"

Doesn't this only assign a fact to the target host?

In many cases, that's not the desired behavior. Suppose I want to collect a list of RDS instances ids by tag and perform simple operations on them over and over?

Being able to:

 - add variables

 - trigger conditions

 - filter output, join lists... etc...

would be tremendously useful. I guess something like:

  

- hosts: localhost
- name: Describe instances
  hosts: localhost
  gather_facts: false # Prevents immediately logging in to hosts
  tasks:
  - command: aws ec2 describe-

instances
    after: >
        some python code to filter the results

        some python code to register 5 groups

        some python code to register 2 variables (register)

isn't possible? 

This is just an example, I'm already using ec2.py for much of this. But more fined-grained controlled is required (such as having tasks conditionally execute or wait on ec2_instance states which may change in response to prior tasks).

- Stu

[Michael DeHaan]

You cannot register the result of an expression.

See what was posted about "set_fact" above, however.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/6e552272-5da4-48b4-acf7-8389265441a2%40googlegroups.com.

[Mark]

I was searching for exactly this feature. set_fact doesn't work for me because in my case i need to create AWS API keys for a newly created IAM user. The docs in set_fact says that the new fact will be available between plays, which means those keys will be cached on disk which represents a security risk.

What I really would like is the same as the original poster - a way to get the stdout from the command module in a way that can be used later. E.g. after creating a new IAM user and registering the output as 'iam', and running:

- name: Create access keys for the new user
  local_action: command aws iam create-access-key --user-name {{ iam.stack_outputs.UserName }}
  register: keys

- name: debug keys
  debug: var=keys

I see:

ok: [localhost] => {
    "keys": {
        "changed": true,
        ....
        "stdout": "{\n    \"AccessKey\": {\n        \"UserName\": \"MyUser-UMF4KM2DRQWQ\", \n        \"Status\": \"Active\", \n        \"CreateDate\": \"2014-11-03T12:52:44.458Z\", \n        \"SecretAccessKey\": \"***\", \n        \"AccessKeyId\": \"AK***\"\n    }\n}",

I'd like to then pass the SecretAccessKey and AccessKeyId into another cloudformation template as parameters. The simplest way would be if this json could be parsed by the 'command' module and made available as a dict.

Alternatively, is there any way of using 'set_fact' in a transient mode so the fact won't persist between invocations?

[Matt Martz]

I think you are misunderstanding terminology slightly.

Vars set with set_fact will be across plays but not invocations of ansible.

A playbook can have multiple plays and multiple playbooks can be included and executed in 1 invocation of ansible.

set_fact is in memory only, and will not persist between invocations.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/2a3f8912-185c-444f-91d7-cb696022feae%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Matt Martz
@sivel
sivel.net

[Mark]

Ah OK. Yes, I understood "These variables will survive between plays" meant survive between invocations of ansible. If that's not the case and the fact is not cached to disk, then I can go with that approach.

Thanks

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/2a3f8912-185c-444f-91d7-cb696022feae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael DeHaan]

"The docs in set_fact says that the new fact will be available between plays, which means those keys will be cached on disk which represents a security risk."

Yeah don't assume that.  Not the case.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/bd6cba6a-2336-456c-938e-7975fb5cb95f%40googlegroups.com.