with_dict expects a dict...can a dict be too large

[Joanna Delaporte]

I messed up and deleted my first post. Sorry! 

I think I need another pair of eyes here. I am attempting to run a dict of users through an ipa user-add command, but it fails with the error "with_dict expects a dict." I believe I have formatted my dict correctly (it's about 40,000 lines long, generated from a python3 script): 

---
students:
  username1:
      lastname: jibberish
      firstname: jibberish
      firstinitial: jibberish
      studentid: jibberish
      dob: jibberish
      dateexpires: 2018-06-19
      epochexpiry: 1529415778
      uid: jibberish
      passwordhash: "jibberish"
  username2:
      lastname: jibberish
      firstname: jibberish
      firstinitial: jibberish
      studentid: jibberish
      dob: jibberish
      dateexpires: 2018-06-19
      epochexpiry: 1529416593
      uid: jibberish
      passwordhash: "jibberish"
  username3:
      lastname: jibberish
      firstname: jibberish
      firstinitial: jibberish
      studentid: jibberish
      dob: jibberish
      dateexpires: 2018-06-19
      epochexpiry: 1529416755
      uid: jibberish
      passwordhash: "jibberish"

And here is the play where I am trying to use it: 

  - name: Add existing student users to ipaserver

    command: "/bin/ipa user-add 

            {{ item.key }} 

            --first='{{ item.value.firstname }}'

            --last='{{ item.value.lastname }}'

            --cn='{{item.value.firstname}} {{item.value.lastname}}'

            --email={{ item.value.name }}@domain

            --shell=/bin/bash

            --homedir='/home/students/{{ item.value.firstinitial }}/{{ item.key }}'

            --uid='{{ item.value.uid }}'

            --gidnumber='{{ item.value.uid }}'

            --skeleton=/etc/skel

            --setattr userpassword='{crypt}{{ item.value.passwordhash}}'

            --title=student"

    with_dict: "{{students}}"

    register: added_students

And the error:

TASK [Add existing student users to ipaserver] *********************************

fatal: [216.125.253.133]: FAILED! => {"failed": true, "msg": "with_dict expects a dict"}

Running it verbose only gives me the task path in addition to the above error.

Is there a maximum size a dict can be? Is there something else wrong with my dict? I have tried it as a list (with a -<space> before each username and with_items in the play), but got a unicode error. I'm stumped.

Thanks for taking a look!

[Dick Davies]

I always find with_dict fiddly, it'll work eventually. I can't see
anything obvious at first glance
(
I do think

--email={{ item.value.name }}@domain

should be

--email="{{ item.key }}@domain"

but I don't think Ansible is getting as far as even opening the dict yet
)

When I get this kind of issue, I cut down the size of the dict to a
little sample set, just to verify the YAML
is formatted correctly (no whitespace gotchas?).

Also, are you sure the dict is in scope at this point?

> --
> You received this message because you are subscribed to the Google Groups
> "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ansible-proje...@googlegroups.com.
> To post to this group, send email to ansible...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ansible-project/2fe49d88-e756-43ab-b6e6-e99ac62b6ffc%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Joanna Delaporte]

Hi Dick, 

Thanks for replying! Good catch on the email address. I'll tweak that. 

I tried even just a debug of printing each item and that had the same with_dict error.

It's a good idea to try a truncated dict; then at least I'll have a better idea whether it's the length breaking it. That's my next step.

Thanks!

Joanna 

[Joanna Delaporte]

Good question about whether the dict is in scope. 

I import the vars this way at the top of the playbook:

  vars_files:

    - vars/studentdict.yml

    - vars/staffdict.yml

I run the playbook with become: yes. I verified that root and the local ansible user can read the dict.

I wonder if dashes in my date fields are breaking it. I am testing quoting those entries right now.

On Wednesday, June 22, 2016 at 7:04:09 PM UTC-5, Dick Davies wrote:

[Matt Martz]

I don't have an exact answer as to why currently, but the problem is with your `dateexpires` values.  That value gets converted to a python datetime object.  And because of it being automatically parsed as a datetime object, the dict is transformed into a string representation of the dict.  The conversion to a string representation is the part that I don't fully understand right now.  It seems to be something inside of jinja2 itself that is causing this behavior.

So effectively you end up with a list that has 1 element, and that 1 element is a string representation of your dict.

Simply quoting the value of `dateexpires` resolves the issue, such as:

  username1:

      lastname: jibberish

      firstname: jibberish

      firstinitial: jibberish

      studentid: jibberish

      dob: jibberish

      dateexpires: "2018-06-19"

      epochexpiry: 1529415778

      uid: jibberish

      passwordhash: "jibberish"

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/4eeb5fb9-f761-49ec-b8b6-ea0b74567246%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Matt Martz
@sivel
sivel.net

[Joanna Delaporte]

Yup, a test with a short dict of only three records failed. Until I quoted all the date fields containing dashes. 

Each user record contains two dates that formatted like YYYY-MM-DD, which apparently breaks yaml. It makes sense now, but hadn't occurred to me until I was driving home a while ago.

I was using this:

    dob: 1962-01-21
    dateexpires: 2018-06-19
    epochexpiry: 1529415778

Dashes break hashes, so it fixed it to quote the values:  

    dob: '1962-01-21'
    dateexpires: '2018-06-19'
    epochexpiry: 1529415778

I have yet to tweak my 40,000 lines (about 4000 records) and try with those.

On Wednesday, June 22, 2016 at 7:04:09 PM UTC-5, Dick Davies wrote:

[Joanna Delaporte]

It turns out I had various dashes and apostrophes (isolated single quotes spell trouble, too) smattered throughout the user's first and last names. So, after quoting dates, password hashes, and name fields, I tried with the full large dictionary. Now my play works! Even more exciting, I have new users appearing on my IPA server with appropriate data. Thanks Dick and Matt! 

If you're ever in Champaign, IL USA, look me up. I'll buy you a coffee.

Joanna