Issue with selectattr

[lift...@gmail.com]

I am trying to determine when user's password's are going to expire in the next 10 days.  After I traverse my FreeIPA users and store those users into a variable, I try to set a fact like so:

  - name: Find users who's password will expire in the next 10 days
    set_fact:
      pwd_expire_soon: "{{ user_show.results | json_query('[*].json.result.result.{uid: uid[0], pwdchg: krblastpwdchange[0].__datetime__}') | selectattr('pwdchg', 'lessthan', 'expire_date') | list }}"

When I run my playbook, I get the following error:

fatal: [localhost]: FAILED! => {"msg": "Unexpected templating type error occurred on ({{ user_show.results | json_query('[*].json.result.result.{uid: uid[0], pwdchg: krblastpwdchange[0].__datetime__}') | selectattr('pwdchg', 'lessthan', 'expire_date') | list }}): '<' not supported between instances of 'NoneType' and 'str'. '<' not supported between instances of 'NoneType' and 'str'"}

I can't seem to find what the issue is.  I originally  had '<' instead of 'lessthan' but got the same error.  Any ideas?

Thanks,

Harry

[Rowe, Walter P. (Fed)]

pwd_expire_soon: "{{ user_show.results | json_query('[*].json.result.result.{uid: uid[0], pwdchg: krblastpwdchange[0].__datetime__}') | selectattr('pwdchg', 'lessthan', 'expire_date') | list }}"

What are you expecting this red portion to do? I don't think it is valid in json_query.

Walter
--
Walter Rowe, Division Chief
Infrastructure Services Division
Mobile: 202.355.4123

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/a1131cb0-bc23-46bb-afbf-ca9ad6f4ce34n%40googlegroups.com.

[lift...@gmail.com]

Just pull out those fields from the returned user information.  I use that in 2 or 3 other playbooks so I know that it works.

Thanks,

Harry

[Rowe, Walter P. (Fed)]

{{ user_show.results | json_query('[*].json.result.result.{uid: uid[0], pwdchg: krblastpwdchange[0].__datetime__}') }}

I would display this info in a debug to see what the resulting data stream looks like. Maybe the selectattr('pwdchg') is in inaccurate reference to pwdchg?

Walter
--
Walter Rowe, Division Chief
Infrastructure Services Division
Mobile: 202.355.4123

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/c0b4de3d-50e2-4fff-85b1-0437076137dcn%40googlegroups.com.

[lift...@gmail.com]

Looks OK to me:

TASK [Show user info] **************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": [
        {
            "pwdchg": "20210416141027Z",
            "uid": "user1"
        }
    ]
}

Thanks,

Harry

[Todd Lewis]

Without showing us the expression you used in your debug's "msg:", this doesn't tell us anything.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/f76c158f-1107-4d10-8977-12638128d056n%40googlegroups.com.

-- 
Todd

[lift...@gmail.com]

I'm not including the entire playbook, but the URI module call where user_show gets registered, then the debug statements:

  - name: Run user_show from IDM API using previously stored session cookie
    ansible.builtin.uri:
      url: "https://{{idmfqdn}}/ipa/session/json"
      method: POST
      headers:
        Cookie: "{{ login.set_cookie }}"
        Referer: "https://{{idmfqdn}}/ipa"
        Content-Type: "application/json"
        Accept: "application/json"
      body_format: json
      body: "{\"method\": \"user_show\",\"params\": [[ \"{{ item }}\"],{\"all\": true,\"version\": \"{{ api_vers }}\"}]}"
    register: user_show
    loop: "{{ uid | json_query('[:10]') }}"


  - name: Set expire date
    ansible.builtin.set_fact:
      expire_date: '{{ lookup(''pipe'', ''date -u --date="today + 10 days" +%Y%m%d000000Z'') }}'

  - name: Show expire date
    ansible.builtin.debug:
      msg: "{{ expire_date }}"

  - name: Show user info
    debug:
      msg: "{{ user_show.results | json_query('[*].json.result.result.{uid: uid[0], pwdchg: krblastpwdchange[0].__datetime__}') }}"

Thanks,

Harry

[Todd Lewis]

The original problem is you're comparing 'NoneType' and 'str'. So, for at least one of your principals there's no krblastpwdchange. You need to work on the subset of data relevant to the comparison.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/6c29210e-e854-414c-9007-13c37edb3265n%40googlegroups.com.

-- 
Todd

[lift...@gmail.com]

So it looks like the VERY 1st user in our system has never logged in, so the krblaspwdchange property has never gotten set.  Is there a way to ignore when that field doesn't exist or is null?

Thanks,

Harry

[lift...@gmail.com]

I'm including the playbook below.  I'm still unable to find a way to ignore any user that does not have the krblastpwdchange property set.  When I run the playbook, I still get the following error:

TASK [Find users who's password will expire in the next 10 days] *******************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "Unexpected templating type error occurred on ({{ user_show.results | json_query('[*].json.result.result.{uid: uid[0], pwdchg: krblastpwdchange[0].__datetime__}') | selectattr('pwdchg', 'defined') | selectattr('pwdchg', '<', expire_date) | list }}): '<' not supported between instances of 'NoneType' and 'AnsibleUnsafeText'. '<' not supported between instances of 'NoneType' and 'AnsibleUnsafeText'"}

Playbook:

---
- name: Gather User Password Expiration information from IDM server
  hosts: localhost
  gather_facts: no

  pre_tasks:
    - setup:
        filter: 'ansible_date_time'

  vars_files:
    - /etc/ansible/vault.yml

  vars:
    idmfqdn: idmserver
    binduser: 'admin'
    bindpasswd: '{{ secure_ipa_pass }}'
    warning_days: 10

  tasks:

  - name: Set list of users to ignore
    ansible.builtin.set_fact:
      ignore_users:
        - "root"
        - "some.user"
        - "some.c-ctr.user"
        - "test.redhat"
        - "admin"

  - name: Login to IDM use returned cookie to access the API in later tasks
    ansible.builtin.uri:
      url: "https://{{idmfqdn}}/ipa/session/login_password"
      method: POST
      headers:

        Referer: "https://{{idmfqdn}}/ipa"

        Content-Type: "application/x-www-form-urlencoded"
        Accept: "text/plain"
      body_format: form-urlencoded
      body:
        user: "{{binduser}}"
        password: "{{bindpasswd}}"
      status_code: 200
      follow_redirects: all
    register: login

  - name: Get IDM API version using previously stored session cookie

    ansible.builtin.uri:
      url: "https://{{idmfqdn}}/ipa/session/json"
      method: POST
      headers:
        Cookie: "{{ login.set_cookie }}"
        Referer: "https://{{idmfqdn}}/ipa"
        Content-Type: "application/json"
        Accept: "application/json"
      body_format: json

      body: '{"method": "ping","params": [[],{}]}'
    register: api_vers_out

  - name: Set fact for api version
    ansible.builtin.set_fact:
      api_vers: "{{ api_vers_out.json.result.messages|json_query('[*].data.server_version')|join() }}"

  - name: Run user_find from IDM API using previously stored session cookie

    ansible.builtin.uri:
      url: "https://{{idmfqdn}}/ipa/session/json"
      method: POST
      headers:
        Cookie: "{{ login.set_cookie }}"
        Referer: "https://{{idmfqdn}}/ipa"
        Content-Type: "application/json"
        Accept: "application/json"
      body_format: json

      body: "{\"method\": \"user_find/1\",\"params\": [[],{\"version\": \"{{ api_vers }}\"}]}"
    no_log: true
    register: user_find

  - name: Set users fact
    ansible.builtin.set_fact:
      uid: "{{ user_find.json.result.result|map(attribute='uid')|flatten|difference(ignore_users) }}"

  - name: Run user_show from IDM API using previously stored session cookie
    ansible.builtin.uri:
      url: "https://{{idmfqdn}}/ipa/session/json"
      method: POST
      headers:
        Cookie: "{{ login.set_cookie }}"
        Referer: "https://{{idmfqdn}}/ipa"
        Content-Type: "application/json"
        Accept: "application/json"
      body_format: json
      body: "{\"method\": \"user_show\",\"params\": [[ \"{{ item }}\"],{\"all\": true,\"version\": \"{{ api_vers }}\"}]}"
    register: user_show

    loop: "{{ uid | json_query('[:1]') }}"

  - name: Set expire date
    ansible.builtin.set_fact:

      expire_date: '{{ lookup(''pipe'', ''date -u --date="today + {{ warning_days }} days" +%Y%m%d000000Z'') }}'

  - name: Show expire date
    ansible.builtin.debug:
      msg: "{{ expire_date }}"

  - name: Show user info
    debug:
      msg: "{{ user_show.results | json_query('[*].json.result.result.{uid: uid[0], pwdchg: krblastpwdchange[0].__datetime__}') }}"

  - name: Find users who's password will expire in the next {{ warning_days }} days
    ansible.builtin.set_fact:
      pwd_expire_soon: "{{ user_show.results | json_query('[*].json.result.result.{uid: uid[0], pwdchg: krblastpwdchange[0].__datetime__}') | selectattr('pwdchg', 'defined') | selectattr('pwdchg', '<', expire_date) | list }}"

  - name: Show accounts that are due to expire in the next {{ warning_days }} days
    ansible.builtin.debug:
      msg: "{{ pwd_expire_soon }}"

Thanks,

Harry

[Todd Lewis]

Normally I would change

krblastpwdchange[0].__datetime__

to

krblastpwdchange[0].__datetime__ | default('0')

but your expression is part of json_query(). Perhaps there's an equivalent of Jinja's default() in json_query(); I just don't know.

Without standing up an IPA instance and experimenting, I don't see a way to derive what your user_show data looks like. (I did that a year or two ago to answer a similar question, but I'm a bit swamped at the moment.) Perhaps if you could give us the first couple of records from your registered user_show data — the first one that's throwing the error and another one or two that has what you expect, anonymized as appropriate of course — that would give us enough information about your data's structure to suggest some expressions to try.

Including the playbook helps a whole lot; alas, without sample data or an IPA instance to play with it's not sufficient to give you a definitive answer. Well, at least I can't.

I know this sort of problem can be extremely frustrating, and you've been patient in asking. We really do want to help, honestly. But the IPA thing is one level of niche, and very few people are versed in json_query() so that adds yet another level of niche. But if you can provide the sample data I mentioned before I think we can make some progress.
--
Todd

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/11a5adb1-a9ff-4e30-93ec-ab7b44488c04n%40googlegroups.com.

-- 
Todd

[lift...@gmail.com]

I totally understand where you're coming from.  You guys have always been great and extremely helpful, and I appreciate that.  I have adjusted the playbook to look at the first 5 users, and the anonymized output is below.

Thanks,

Harry

TASK [Show user info] **************************************************************************************************************************************************************************************
ok: [localhost] => {
    "msg": [
        {

            "pwdchg": null,
            "uid": "user1"
        },
        {
            "pwdchg": "20240124144409Z",
            "uid": "user2"
        },
        {
            "pwdchg": "20231212115535Z",
            "uid": "user3"
        },
        {
            "pwdchg": null,
            "uid": "user4"
        },
        {
            "pwdchg": "20240115133407Z",
            "uid": "user5"
        }
    ]
}

[Todd Lewis]

Assuming what you posted is the result of:

  - name: Show user info
      debug:
        msg: "{{ user_show.results | json_query('[*].json.result.result.{uid: uid[0], pwdchg: krblastpwdchange[0].__datetime__}') }}"

Then throw this onto the end:

  | selectattr("pwdchg", "!=", None)

You could put that new expression in a set_fact for use later on.
--
Todd

[lift...@gmail.com]

I think I got it.  I set the fact like so:

  - name: Find users who's password will expire in the next {{ warning_days }} days
    ansible.builtin.set_fact:

      pwd_expire_soon: "{{ user_show.results | json_query('[*].json.result.result.{uid: uid[0], pwdchg: krblastpwdchange[0].__datetime__}') | selectattr('pwdchg', '!=', None) | selectattr('pwdchg', '<', expire_date) | list }}"

And that returns those 3 that did not have null for krblastpwdchange.  Now, those 3 had values that were within the last 30/60 days, but I think I may be looking at the wrong attribute at this point.  But that's something that I'll work on next.

Thank you very much for your help and time!  I always appreciate you guys!

Harry