yum repolist -v Output to Vars/Dict and use the data

[Spoonless]

Hi all,

I am having some difficulty in getting my head around the use of variables, facts and dicts.

The Ansible yum module does not give me output from an equivalent yum repolist -v command, so I am using shell for that.

This is what I am trying to do:

yum repolist -v will give me a list of the enabled repos on a system, for example:

Repo-id      : jenkins
Repo-name    : Jenkins-stable
Repo-revision: 1585146777
Repo-updated : Wed Mar 25 15:32:57 2020
Repo-pkgs    : 102
Repo-size    : 6.1 G
Repo-baseurl : http://pkg.jenkins.io/redhat-stable/
Repo-expire  : 21,600 second(s) (last: Tue Apr 14 08:28:35 2020)
  Filter     : read-only:present
Repo-filename: /etc/yum.repos.d/jenkins.repo

Repo-id      : mongodb-org-4.2/7
Repo-name    : MongoDB Repository
Repo-revision: 1585187010
Repo-updated : Thu Mar 26 02:43:32 2020
Repo-pkgs    : 25
Repo-size    : 543 M
Repo-baseurl : https://repo.mongodb.org/yum/redhat/7/mongodb-org/4.2/x86_64/
Repo-expire  : 21,600 second(s) (last: Tue Apr 14 08:28:36 2020)
  Filter     : read-only:present
Repo-filename: /etc/yum.repos.d/mongodb-org-4.2.repo

I want to:

1: register, lets say for now, the Repo-id, Repo-name, Repo-baseurl, Repo-filename to variable/s, or a dictionary, not sure here ?.

2: Check if the Repo-baseurl of a Repo-id contains a certain url, if it does, register another variable, illegalurl = 1,  for that Repo

3: Grab the Repo-filename where the Repo-baseurl illegalurl is 1

4: Do something with that file:

  a: Generate a mail with that information

  b: Check if a repo file exists in a different location with a correct Repo-baseurl

  c: Change the Repo-baseurl in the originating .repo file.

  == I guess, if I have the variables, dictionary, facts of the individual blocks of data, then I can use them to do other tasks that I need.

I have tried:

Individual tasks to grab the data and register a variable:

example:

  tasks:
    - name: Collect Repo id  from yum repolist -v
      shell: "yum repolist -v | grep Repo-id | awk -F\": \" '{print $2}'"
      register: echo_repoids
#   - debug:
#        var: echo_repoids.stdout_lines

    - name: Collect Repo Name  from yum repolist -v
      shell: "yum repolist -v | grep Repo-name | awk -F\": \" '{print $2}'"
      register: echo_reponame
#    - debug:
#        var: echo_reponame.stdout_lines

    - name: Collect Repo base url  from yum repolist -v
      shell: "yum repolist -v | grep Repo-baseurl | awk -F\": \" '{print $2}'"
      register: echo_repobaseurl
#    - debug:
#        var: echo_repobaseurl.stdout_lines

    - name: Collect Repo filename  from yum repolist -v
      shell: "yum repolist -v | grep Repo-filename | awk -F\": \" '{print $2}'"
      register: echo_repofilename
#   - debug:
#       var: echo_repofilename.stdout_lines

I also tried:

  tasks:
    - name: Collect the enabled Repo information
      shell: "yum repolist -v | grep {{item}} | awk -F\": \" '{print $2}'"
      with_items:
        - "Repo-id"
        - "Repo-name"
        - "Repo-baseurl"
        - "Repo-filename"
      register: output     
#      debug: var=output

But I do not know how to match the blocks contents of Repo-id, Repo-name, Repo-baseurl, Repo-filename as keys and values so I can use them later in the tasks/ playbooks.

I hope that helps you to understand what I am trying to accomplish.

Thanks in advance for any advice anyone can offer. :)

Best Regards.

[Kai Stian Olstad]

If you use <variable>.stdout you get the value, .stdout_lines is a list where
the first line is the first element, so you would need to use
<variable>.stdout_lines.0

> I also tried:
>
> tasks:
> - name: Collect the enabled Repo information
> shell: "yum repolist -v | grep {{item}} | awk -F\": \" '{print $2}'"
> with_items:
> - "Repo-id"
> - "Repo-name"
> - "Repo-baseurl"
> - "Repo-filename"
> register: output
> # debug: var=output

Since you run this in a loop, the result is in results.

Repo-id is output.results.0.stdout
Repo-name is output.results.1.stdout
and so on

> But I do not know how to match the blocks contents of Repo-id, Repo-name,
> Repo-baseurl, Repo-filename as keys and values so I can use them later in
> the tasks/ playbooks.

You could just use the variables as shown above. You could use set_fact to
create new variable. Something like this for the last example.

- set_fact:
repoid: "{{ output.results.0.stdout }}"
reponame: "{{ output.results.1.stdout }}"


--
Kai Stian Olstad

[Spoonless]

Hi,

Thanks for the tip in the right direction:

Here is what I have so far..

But, my Debug output is showing me all of the repo's and not filtering. ?

tasks:
    - name: Set the required enabled repo information as facts
      shell: "yum repolist -v | grep {{item}} | awk -F\": \" '{print $2}'"
      with_items:
        - "Repo-id"
        - "Repo-name"
        - "Repo-baseurl"
        - "Repo-filename"
      register: output      
    - set_fact:
        repoid: "{{ output.results.0.stdout }}"
        reponame: "{{ output.results.1.stdout }}"
        repobaseurl: "{{ output.results.2.stdout }}"
        repofilename: "{{ output.results.3.stdout }}"

    - debug:
        msg: "{{reponame}} is an illegal url"
      when: repobaseurl is search("ftp://somehostname.*")

===

[Kai Stian Olstad]

On Fri, Apr 17, 2020 at 12:59:09AM -0700, 'Spoonless' via Ansible Project wrote:
> Hi again,

>
> Thanks for the tip in the right direction:
>

> I now have this:

>
> tasks:
> - name: Set the required enabled repo information as facts

> shell: "yum repolist -v | grep {{item}} | awk -F\": \" '{print $2}'"
> with_items:
> - "Repo-id"

> - "Repo-updated"
> - "Repo-pkgs"
> - "Repo-size"

> - "Repo-name"
> - "Repo-baseurl"
> - "Repo-filename"
> register: output

> - set_fact:
> repoid: "{{ output.results.0.stdout }}"
> # repoupdated: "{{ output.results.1.stdout }}"
> # repopkgs: "{{ output.results.2.stdout }}"
> # reposize: "{{ output.results.3.stdout }}"
> reponame: "{{ output.results.4.stdout }}"
> repobaseurl: "{{ output.results.5.stdout }}"
> repofilename: "{{ output.results.6.stdout }}"

>
> - debug:
> msg: "{{reponame}} is an illegal url"
> when: repobaseurl is search("ftp://somehostname.*")
>
>
>
> ==
>

> But..
> My debug output is now showing all of the repo's and not filtering.
> ?

Yes, the stdout will contain everything.

output.results.0.stdout_lines.0 will contain the repo-id for the first one.
output.results.0.stdout_lines.1 will contain the repo-id for the second one.

output.results.5.stdout_lines.0 will contain the repo-baseurl for the first one.
output.results.5.stdout_lines.1 will contain the repo-baseurl for the second one.


It isn't easy to parse text in Ansible, i would create a script that did the
parsing and returned a json that you can use a lot easier in Ansible.


--
Kai Stian Olstad

[Spoonless]

Hi,

So, if I get the output of my yum repolist -v to a json format then I can use the json query in Ansible, like here:

https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#json-query-filter

I guess, Python or jq or something to prep the output in the required format.

(I'll have to get to grips with that part.)

Thanks again.

[Spoonless]

Trying to get the yum repolist -v data to a json format that I can parse with Ansible  is tricky enough for a beginner.

[Spoonless]

If I am correct, I would need the output from yum repolist -v :

Loading "fastestmirror" plugin
Loading "langpacks" plugin
Adding en_US.UTF-8 to language list
Config time: 0.008
Yum version: 3.4.3
Loading mirror speeds from cached hostfile
Setting up Package Sacks
pkgsack time: 0.008

Repo-id      : docker-ce-stable/x86_64
Repo-name    : Docker CE Stable - x86_64
Repo-revision: 1583949071
Repo-updated : Wed Mar 11 18:51:11 2020
Repo-pkgs    : 70
Repo-size    : 1.6 G
Repo-baseurl : https://download.docker.com/linux/centos/7/x86_64/stable/
Repo-expire  : 21,600 second(s) (last: Tue Apr 21 11:06:47 2020)
  Filter     : read-only:present
Repo-filename: /etc/yum.repos.d/docker-ce.repo

Repo-id      : jenkins
Repo-name    : Jenkins-stable
Repo-revision: 1585146777
Repo-updated : Wed Mar 25 15:32:57 2020
Repo-pkgs    : 102
Repo-size    : 6.1 G
Repo-baseurl : http://pkg.jenkins.io/redhat-stable/

Repo-expire  : 21,600 second(s) (last: Tue Apr 21 11:06:48 2020)

  Filter     : read-only:present
Repo-filename: /etc/yum.repos.d/jenkins.repo

==

To look like this as a json file:

{
    "repos": [
        {
            "repoid_ref": "docker-ce-stable/x86_64",
            "reponame_ref": "Docker CE Stable - x86_64"
        },
        {
            "repoid_ref": "jenkins",
            "reponame_ref": "Jenkins-stable"       
        }
    ],
    "repoinfo": [
        {
            "repoid": "docker-ce-stable/x86_64",
            "reponame": "Docker CE Stable - x86_64",
            "repoupdated": "Wed Mar 11 18:51:11 2020",
            "repopkgs": "70",
            "reposize": "1.6 G",
            "repobaseurl": "https://download.docker.com/linux/centos/7/x86_64/stable/",
            "repofilename": "/etc/yum.repos.d/docker-ce.repo",
            "repoid_ref": "docker-ce-stable/x86_64",
            "reponame_ref": "Docker CE Stable - x86_64"
        },
        {
            "repoid": "jenkins",
            "reponame": "Jenkins-stable",
            "repoupdated": "Wed Mar 25 15:32:57 2020",
            "repopkgs": "102",
            "reposize": "6.1 G",
            "repobaseurl": "http://pkg.jenkins.io/redhat-stable/",
            "repofilename": "/etc/yum.repos.d/jenkins.repo",
            "repoid_ref": "jenkins",
            "reponame_ref": "Jenkins-stable"
        }
    ]
}

Any reasonable tips on doing that. ?

I created that manually, for now, to then try the Ansible part:

---

- hosts: "{{target_host}}"
  gather_facts: false
  become: true
  become_user: root
 
  tasks:
    - name: Display the JSON file content
      shell: cat /root/repoinfo_new.json
      register: result

    - name: save the Json data to a Variable as a Fact
      set_fact:
        jsondata: "{{ result.stdout | from_json }}"

    - name: Set facts - Repo id's
      set_fact: 
        repoids:  "{{ jsondata | json_query(jmesquery) }}"
      vars:
        jmesquery: '*.repoinfo[*].repoid'
     
    - name: Repo id's and Names
      set_fact: 
        repoinfo:  "{{ jsondata | json_query(jmesquery) }}"
      vars:
        jmesquery: '*.repoinfo[*].[repoid, reponame]'
     
    - name: Print all repoid names
      debug:
        msg: "{{ item }}"
      with_items:
        - "{{ repoids }}"

    - name: Print all repoids and Names
      debug:
        msg: "{{ item }}"
      with_items:
      - "{{ repoinfo }}"

But I am not getting any output as yet.

[Kai Stian Olstad]

No, repos should be a list with only 2 element.

>
> Any reasonable tips on doing that. ?

That is way out of the scope for this list, you should find some other
community to help with text parsing.

That being said, I had 20 minutes before bedtime I needed to kill I created this crude
thing that probably breaks between different yum versions.

I'm not going to explain how it work since this group is not the place for that.
Use it if it works or try to fix it yourself if it don't

Since the output of yum repolist is more or less yaml, it's a lot easier to
convert it to yaml than json.

- shell: yum repolist -v | sed -e '1,/Repo-filename/d' | head -n -2 | sed -e 's/^ \+//' -e 's/ \+:/:/' -e 's/^/ /' -e 's/^ Repo-id/- Repo-id/' -e 's/:\s/:/2g'
register: r

- set_fact:
repos: '{{ r.stdout | from_yaml }}'

- debug: var=repos
- debug: var=repos[0]['Repo-id']


--
Kai Stian Olstad