Requires Help ,using Python for accessing dictionary values from yaml file

61 views
Skip to first unread message

Pawan Kumar

unread,
Sep 2, 2019, 6:18:42 PM9/2/19
to Ansible Project
Using Python scriptĀ  to access values from yaml dictionary & print some of the values as list .Ā 

For Example -Ā  Parse the each key and extract the ā€œproduct_nameā€, ā€œrelease_candidateā€ and ā€œversionā€œ & Print results as list

The yaml file looks likeĀ 



- releases:
Ā  Ā  - gotcha:
Ā  Ā  Ā  Ā  file_name: gtp.tar
Ā  Ā  Ā  Ā  md5: 11f6987275613ca3ecf832ea59c1
Ā  Ā  Ā  Ā  product_name: gotcha
Ā  Ā  Ā  Ā  product_number: GTP 8881
Ā  Ā  Ā  Ā  release_candidate: rc19
Ā  Ā  Ā  Ā  version: 19.0.0
Ā  Ā  - tp:
Ā  Ā  Ā  Ā  file_name: tpr.tar
Ā  Ā  Ā  Ā  md5: ebd7742ed8fff0472733c0cd8e
Ā  Ā  Ā  Ā  product_name: tp
Ā  Ā  Ā  Ā  product_number: TTP 8057
Ā  Ā  Ā  Ā  release_candidate: rc25
Ā  Ā  Ā  Ā  version: 25.0.0

Dick Visser

unread,
Sep 2, 2019, 6:36:16 PM9/2/19
to ansible...@googlegroups.com
Hi

Kindly explain where ansible comes into play in this story?

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/cdcb12c4-b6f6-4dbb-be42-b781e7d576d3%40googlegroups.com.
--
Sent from a mobile device - please excuse the brevity, spelling and punctuation.

Pawan Kumar

unread,
Sep 2, 2019, 7:01:37 PM9/2/19
to Ansible Project
Hi Dick,

Thanks for replying .Ā 

Well said , i was working with Ansible to parse values from dictionary file .Ā 

working with the following ansible playbookĀ 



---
- hosts: 127.0.0.1
Ā  connection: local
Ā  vars_files:
Ā  Ā  - latest123.yaml
Ā  tasks:

Ā  Ā  - name: Printing "releases" value from yaml file...
Ā  Ā  Ā  debug:
Ā  Ā  Ā  Ā  msg: "Name of the Product is {{ releases[item]['product_name'] }}Ā  it's release candidate: {{ releases[item]['release_candidate'] }} & version: {{ releases[item]['version'] }}"
Ā  Ā  Ā  with_items: "{{ releases.keys() }}"




from the following latest123.yaml fileĀ 


---
- releases:
Ā  Ā  gotcha:
Ā  Ā  Ā  file_name: gtp.tar
Ā  Ā  Ā  md5: 11f6987275613ca3ecf832ea59c1
Ā  Ā  Ā  product_name: gotcha
Ā  Ā  Ā  product_number: GTP 8881
Ā  Ā  Ā  release_candidate: rc19
Ā  Ā  Ā  version: 19.0.0
Ā  Ā  tp:
Ā  Ā  Ā  file_name: tpr.tar
Ā  Ā  Ā  md5: ebd7742ed8fff0472733c0cd8e
Ā  Ā  Ā  product_name: tp
Ā  Ā  Ā  product_number: TTP 8057
Ā  Ā  Ā  release_candidate: rc25
Ā  Ā  Ā  version: 25.0.0
Ā  Ā  device_name:
Ā  Ā  Ā  file_name: dna.tar
Ā  Ā  Ā  md5: e464e69a07123de075778aa9b7
Ā  Ā  Ā  product_name: device_name
Ā  Ā  Ā  product_number: DNP 6746
Ā  Ā  Ā  release_candidate: rc25
Ā  Ā  Ā  version: 25.0.0




On Tuesday, September 3, 2019 at 4:06:16 AM UTC+5:30, Dick Visser wrote:
Hi

Kindly explain where ansible comes into play in this story?
On Tue, 3 Sep 2019 at 00:18, Pawan Kumar <pawa...@gmail.com> wrote:
Using Python scriptĀ  to access values from yaml dictionary & print some of the values as list .Ā 

For Example -Ā  Parse the each key and extract the ā€œproduct_nameā€, ā€œrelease_candidateā€ and ā€œversionā€œ & Print results as list

The yaml file looks likeĀ 



- releases:
Ā  Ā  - gotcha:
Ā  Ā  Ā  Ā  file_name: gtp.tar
Ā  Ā  Ā  Ā  md5: 11f6987275613ca3ecf832ea59c1
Ā  Ā  Ā  Ā  product_name: gotcha
Ā  Ā  Ā  Ā  product_number: GTP 8881
Ā  Ā  Ā  Ā  release_candidate: rc19
Ā  Ā  Ā  Ā  version: 19.0.0
Ā  Ā  - tp:
Ā  Ā  Ā  Ā  file_name: tpr.tar
Ā  Ā  Ā  Ā  md5: ebd7742ed8fff0472733c0cd8e
Ā  Ā  Ā  Ā  product_name: tp
Ā  Ā  Ā  Ā  product_number: TTP 8057
Ā  Ā  Ā  Ā  release_candidate: rc25
Ā  Ā  Ā  Ā  version: 25.0.0

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible...@googlegroups.com.

Pawan Kumar

unread,
Sep 2, 2019, 7:09:16 PM9/2/19
to Ansible Project
My Ansible code works fine , if i call dictionary values from file .Ā 

But requires guidance , how using the yaml file to pass as argument as INV file command line to execute playbook:

something like :

"ansible-playbook test.yml -e INV_FILE=latest123.yaml"

Zolvaring

unread,
Sep 3, 2019, 1:38:53 AM9/3/19
to ansible...@googlegroups.com
I think what you want to do to use cli is specify the path as your var name just like your example, and then in your code when you invoke the variable, use the file lookup:

- set_fact:
Ā  Ā  your_fact: "{{ lookup('file', INV_VAR) }}"

I'm not clear if you will need to do something beyond that for it to naturally interpret the yaml object though there's a chance you'll still end up with a string.



To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/486f2e57-bed0-4a07-a841-81d090665633%40googlegroups.com.

Pawan Kumar

unread,
Sep 3, 2019, 11:06:57 AM9/3/19
to Ansible Project
Thanks Zolvaring ,


Took input from the file & now output is coming in string format with following command ---
Now , Requires formatting of the output to a list .Ā Ā 

"ansible-playbook test.yml -e INV_FILE=latest123.yaml"

test.ymlĀ 
---
- hosts: 127.0.0.1
Ā  connection: local
Ā  tasks:

Ā  Ā  - name: Print
Ā  Ā  Ā  set_fact:
Ā  Ā  Ā  Ā  your_fact: "{{ lookup('file', INV_FILE) }}"

Ā  Ā  - name: Demonstrate
Ā  Ā  Ā  debug:
Ā  Ā  Ā  Ā  msg: "{{ your_fact }}"

===============================================

& Output gives values in string formatĀ 

TASK [Demonstrate] **************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
Ā  Ā  "msg": "---\n- releases:\nĀ  Ā  gotcha:\nĀ  Ā  Ā  file_name: gtp.tar\nĀ  Ā  Ā  md5: 11f6987275613ca3ecf832ea59c1\nĀ  Ā  Ā  product_name: gotcha\nĀ  Ā  Ā  product_number: GTP 8881\nĀ  Ā  Ā  release_candidate: rc19\nĀ  Ā  Ā  version: 19.0.0\nĀ  Ā  tp:\nĀ  Ā  Ā  file_name: tpr.tar\nĀ  Ā  Ā  md5: ebd7742ed8fff0472733c0cd8e\nĀ  Ā  Ā  product_name: tp\nĀ  Ā  Ā  product_number: TTP 8057\nĀ  Ā  Ā  release_candidate: rc25\nĀ  Ā  Ā  version: 25.0.0\nĀ  Ā  device_name:\nĀ  Ā  Ā  file_name: dna.tar\nĀ  Ā  Ā  md5: e464e69a07123de075778aa9b7\nĀ  Ā  Ā  product_name: device_name\nĀ  Ā  Ā  product_number: DNP 6746\nĀ  Ā  Ā  release_candidate: rc25\nĀ  Ā  Ā  version: 25.0.0\nĀ  Ā  abc:\nĀ  Ā  Ā  file_name: abc.tar.gz\nĀ  Ā  Ā  md5: 94d992844322a05e9f4bec250f\nĀ  Ā  Ā  product_name: abc\nĀ  Ā  Ā  product_number: None\nĀ  Ā  Ā  release_candidate: rc24\nĀ  Ā  Ā  version: Raka_24_A\nĀ  Ā  adaptor:\nĀ  Ā  Ā  file_name: aap.tar\nĀ  Ā  Ā  md5: 84f75908bac48ddaae564e63\nĀ  Ā  Ā  product_name: adaptor\nĀ  Ā  Ā  product_number: apt 7815\nĀ  Ā  Ā  release_candidate: rc25\nĀ  Ā  Ā  version: 25.1.1\nĀ  Ā  trai:\nĀ  Ā  Ā  file_name: trai.tar\nĀ  Ā  Ā  md5: d992f1a34320f0590b52f69\nĀ  Ā  Ā  product_name: trai\nĀ  Ā  Ā  product_number: trp 6290\nĀ  Ā  Ā  release_candidate: rc23\nĀ  Ā  Ā  version: 23.2.0\nĀ  Ā  prxy:\nĀ  Ā  Ā  file_name: prp.tar\nĀ  Ā  Ā  md5: dc345ef912779f6f2be252a79\nĀ  Ā  Ā  product_name: prxy\nĀ  Ā  Ā  product_number: ppp 903\nĀ  Ā  Ā  release_candidate: rc26\nĀ  Ā  Ā  version: 26.0.1\nĀ  Ā  luc:\nĀ  Ā  Ā  file_name: lxp.tar\nĀ  Ā  Ā  md5: 96f0e6f9ba753978e19d393cb\nĀ  Ā  Ā  product_name: luc\nĀ  Ā  Ā  product_number: lxp 7686\nĀ  Ā  Ā  release_candidate: RC26\nĀ  Ā  Ā  version: 26.2.0\nĀ  Ā  west:\nĀ  Ā  Ā  file_name: wxp.tar\nĀ  Ā  Ā  md5: 048e15e3717be5302471ccb83\nĀ  Ā  Ā  product_name: west\nĀ  Ā  Ā  product_number: wxp 8347\nĀ  Ā  Ā  release_candidate: rc26\nĀ  Ā  Ā  version: 26.1.1\nĀ  Ā  world:\nĀ  Ā  Ā  file_name: wxp1.tar\nĀ  Ā  Ā  md5: dbb86bf189bdb39b3f7a66150\nĀ  Ā  Ā  product_name: world\nĀ  Ā  Ā  product_number: wld 6384\nĀ  Ā  Ā  release_candidate: rc24\nĀ  Ā  Ā  version: 24.0.0\nĀ  Ā  breaking:\nĀ  Ā  Ā  file_name: brp.tar\nĀ  Ā  Ā  md5: 051656275ed763b34c8de98de\nĀ  Ā  Ā  product_name: breaking\nĀ  Ā  Ā  product_number: brp 8059\nĀ  Ā  Ā  release_candidate: RC25\nĀ  Ā  Ā  version: 25.0.0\nĀ  Ā  bad:\nĀ  Ā  Ā  file_name: bdp.tar\nĀ  Ā  Ā  md5: 2f163c117e5bfaa7d5b7700d8\nĀ  Ā  Ā  product_name: bad\nĀ  Ā  Ā  product_number: bdp 8056\nĀ  Ā  Ā  release_candidate: rc25\nĀ  Ā  Ā  version: 25.0.0\nĀ  Ā  flash:\nĀ  Ā  Ā  file_name: dc.tar\nĀ  Ā  Ā  md5: ee811c2e6f434e190aec46f\nĀ  Ā  Ā  product_name: flash\nĀ  Ā  Ā  product_number: fls 902 \nĀ  Ā  Ā  release_candidate: RC24\nĀ  Ā  Ā  version: 24.0.0\nĀ  Ā  arrow:\nĀ  Ā  Ā  file_name: arr.tar\nĀ  Ā  Ā  md5: 4e5ba127475c6242491a2cb1\nĀ  Ā  Ā  product_name: arrow\nĀ  Ā  Ā  product_number: aro 7655\nĀ  Ā  Ā  release_candidate: rc25\nĀ  Ā  Ā  version: 25.0.0\nĀ  Ā  legends:\nĀ  Ā  Ā  file_name: lxr.tar\nĀ  Ā  Ā  md5: 9f2dbed641703a75dc88ffc0\nĀ  Ā  Ā  product_name: legends\nĀ  Ā  Ā  product_number: lxr 6291\nĀ  Ā  Ā  release_candidate: rc18\nĀ  Ā  Ā  version: 18.0.0\nĀ  Ā  zoom:\nĀ  Ā  Ā  file_name: zxp.tar\nĀ  Ā  Ā  md5: ba3cd0da6e81df20afce3e2\nĀ  Ā  Ā  product_name: zoom\nĀ  Ā  Ā  product_number: zxp 7801\nĀ  Ā  Ā  release_candidate: rc25\nĀ  Ā  Ā  version: 25.0.0\nĀ  Ā  savitar:\nĀ  Ā  Ā  file_name: sav.tar\nĀ  Ā  Ā  md5: 666c7b470e47a3\nĀ  Ā  Ā  product_name: savitar\nĀ  Ā  Ā  product_number: CXP 5110\nĀ  Ā  Ā  release_candidate: RC23\nĀ  Ā  Ā  version: 23.0.0\nĀ  Ā  scarlett:\nĀ  Ā  Ā  file_name: sxp228.tar\nĀ  Ā  Ā  md5: 151127dfa1f5a335dd3052d1\nĀ  Ā  Ā  product_name: scarlet\nĀ  Ā  Ā  product_number: sxp 6292\nĀ  Ā  Ā  release_candidate: rc18\nĀ  Ā  Ā  version: 18.0.0"
}
Ā 

========================================================================================




Zolvaring

unread,
Sep 4, 2019, 1:15:45 AM9/4/19
to Ansible Project
Hm okay I'm not sure that would work, but gojng back to your original example, have you tried passing the yaml file like this when you run the playbook?

"--extra-vars=@your_yaml_file.yml"

Reading I think that should work the same as running it "with_vars"

Pawan Kumar

unread,
Sep 4, 2019, 2:17:50 AM9/4/19
to Ansible Project
Thanks @Zolvaring

I tried your previous suggestionĀ  ,Ā ansible-playbook ansible_dictionary.yml -e INV_FILE=dictionaryFile.yaml
But unable to print key values in dictonary format , For example -Ā Each key is product. Parse the each key and extract the ā€œproduct_nameā€, ā€œrelease_candidateā€ and ā€œversionā€œ



ansible_dictionary.yml


---
- hosts: 127.0.0.1
Ā  connection: local

Ā  tasks:


Ā  Ā  - name: Print
Ā  Ā  Ā  set_fact:
Ā  Ā  Ā  Ā  your_fact: "{{ lookup('file', INV_FILE) }}"
Ā  Ā  Ā  Ā  stdout_callback: yaml
Ā  Ā  - name: Output Formatting
Ā  Ā  Ā  set_fact:
Ā  Ā  Ā  Ā  filtered_data: "{{ your_fact.split('\n') }}"
Ā  Ā  - name: Testing
Ā  Ā  Ā  debug:
Ā  Ā  Ā  Ā  msg: "{{ filtered_data }}"


With Following Output......


TASK [Output Formatting] ********************************************************************************************************************************************************************************************************************
ok: [127.0.0.1]

TASK [Testing] ******************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
Ā  Ā  "msg": [
Ā  Ā  Ā  Ā  "- releases:",
Ā  Ā  Ā  Ā  "Ā  Ā  - gotcha:",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  file_name: gtp.tar",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  md5: 11f6987275613ca3ecf832ea59c1",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  product_name: gotcha",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  product_number: GTP 8881",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  release_candidate: rc19",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  version: 19.0.0",
Ā  Ā  Ā  Ā  "Ā  Ā  - tp:",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  file_name: tpr.tar",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  md5: ebd7742ed8fff0472733c0cd8e",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  product_name: tp",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  product_number: TTP 8057",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  release_candidate: rc25",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  version: 25.0.0",
Ā  Ā  Ā  Ā  "Ā  Ā  - device_name:",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  file_name: dna.tar",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  md5: e464e69a07123de075778aa9b7",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  product_name: device_name",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  product_number: DNP 6746",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  release_candidate: rc25",
Ā  Ā  Ā  Ā  "Ā  Ā  Ā  Ā  version: 25.0.0",

Zolvaring

unread,
Sep 4, 2019, 10:30:53 AM9/4/19
to Ansible Project
Hey Pawan,

I got the laptop out and tried:
ansible-playbook testplay.yml -i localhost, -e "@testvars.yml"

and got:

ERROR! Invalid extra vars data supplied. '@testvars.yml' could not be made into a dictionary

After digging, I found:Ā 
https://github.com/ansible/ansible/issues/38415
https://github.com/ansible/ansible/pull/31433

The above issues (Recent as of Dec 2018) point out that YAML was never accepted as CLI args, though at one point the documentation erroneously claimed it was.

What I did was this:
---
- hosts: 127.0.0.1
Ā  connection: local
Ā  vars_files:
Ā  Ā  - "{{ TEST }}"
Ā  tasks:


Ā  Ā  - name: Testing
Ā  Ā  Ā  debug:
Ā  Ā  Ā  Ā  msg: "{{ releases }}"

---
- releases:
Ā  Ā  gotcha:
Ā  Ā  Ā  file_name: gtp.tar
Ā  Ā  Ā  md5: 11f6987275613ca3ecf832ea59c1
Ā  Ā  Ā  product_name: gotcha
Ā  Ā  Ā  product_number: GTP 8881
Ā  Ā  Ā  release_candidate: rc19
Ā  Ā  Ā  version: 19.0.0
Ā  Ā  tp:
Ā  Ā  Ā  file_name: tpr.tar
Ā  Ā  Ā  md5: ebd7742ed8fff0472733c0cd8e
Ā  Ā  Ā  product_name: tp
Ā  Ā  Ā  product_number: TTP 8057
Ā  Ā  Ā  release_candidate: rc25
Ā  Ā  Ā  version: 25.0.0
Ā  Ā  device_name:
Ā  Ā  Ā  file_name: dna.tar
Ā  Ā  Ā  md5: e464e69a07123de075778aa9b7
Ā  Ā  Ā  product_name: device_name
Ā  Ā  Ā  product_number: DNP 6746
Ā  Ā  Ā  release_candidate: rc25
Ā  Ā  Ā  version: 25.0.0


And then ran:
ansible-playbook testplay.yml -i localhost, -e "TEST=testvars.yml"

And got:
TASK [Gathering Facts] **************************************************************************************************************************************************************
ok: [localhost]

TASK [Testing] **********************************************************************************************************************************************************************
ok: [localhost] => {
Ā  Ā  "msg": {
Ā  Ā  Ā  Ā  "device_name": {
Ā  Ā  Ā  Ā  Ā  Ā  "file_name": "dna.tar",
Ā  Ā  Ā  Ā  Ā  Ā  "md5": "e464e69a07123de075778aa9b7",
Ā  Ā  Ā  Ā  Ā  Ā  "product_name": "device_name",
Ā  Ā  Ā  Ā  Ā  Ā  "product_number": "DNP 6746",
Ā  Ā  Ā  Ā  Ā  Ā  "release_candidate": "rc25",
Ā  Ā  Ā  Ā  Ā  Ā  "version": "25.0.0"
Ā  Ā  Ā  Ā  },
Ā  Ā  Ā  Ā  "gotcha": {
Ā  Ā  Ā  Ā  Ā  Ā  "file_name": "gtp.tar",
Ā  Ā  Ā  Ā  Ā  Ā  "md5": "11f6987275613ca3ecf832ea59c1",
Ā  Ā  Ā  Ā  Ā  Ā  "product_name": "gotcha",
Ā  Ā  Ā  Ā  Ā  Ā  "product_number": "GTP 8881",
Ā  Ā  Ā  Ā  Ā  Ā  "release_candidate": "rc19",
Ā  Ā  Ā  Ā  Ā  Ā  "version": "19.0.0"
Ā  Ā  Ā  Ā  },
Ā  Ā  Ā  Ā  "tp": {
Ā  Ā  Ā  Ā  Ā  Ā  "file_name": "tpr.tar",
Ā  Ā  Ā  Ā  Ā  Ā  "md5": "ebd7742ed8fff0472733c0cd8e",
Ā  Ā  Ā  Ā  Ā  Ā  "product_name": "tp",
Ā  Ā  Ā  Ā  Ā  Ā  "product_number": "TTP 8057",
Ā  Ā  Ā  Ā  Ā  Ā  "release_candidate": "rc25",
Ā  Ā  Ā  Ā  Ā  Ā  "version": "25.0.0"
Ā  Ā  Ā  Ā  }
Ā  Ā  }
}

It looks to me like if you want to dynamically supply the vars file as YAML on cli, that is your best bet. Use 'vars_files' with a dynamic var and declare that var on the cli
Reply all
Reply to author
Forward
0 new messages