Reading in extra files, as or into dicts?

[Richard Hector]

Hi all,

I have created a directory 'users' alongside my inventory. It has a
directory 'user_vars', intended to be used like host_vars, but for
users, obviously.

In there, I have files like this:

=====================
---
name: richard
gecos: 'Richard Hector,,,'
shell: '/bin/bash'
ssh_keys:
- richard@foo
- richard@bar
=====================

Then in host_vars/all, I have this kind of thing:

=====================
---
users:
- richard
admins:
- richard
ansible_users:
- richard
=====================

I also have users/public_keys, which has a file for each of
'richard@foo' etc, containing one key.

Where I'm stuck is reading in the user_vars file(s).

I want to get rid of what I used to have:

=====================
- name: users
user:
name: '{{ item.name }}'
comment: '{{ item.gecos }}'
shell: '{{ item.shell }}'
createhome: yes
state: present
groups: '{{ item.groups }}'
append: yes
with_items:
- { name: 'richard', gecos: 'Richard Hector,,,', shell:
'/bin/bash', groups: [ 'sudo', 'adm' ] }
tags:
- users
======================

since I want to separate data from the rest of my config.

So I'd like to either read all the user_vars files into a single
dictionary before I run that loop, or read each file in its own
iteration of the loop - or something better if that's the answer.

I thought about using set_fact in a loop, but that would give me
separate facts/variables for each user, making it harder(?) to index
them (but maybe by text templating the variable name?)

I also thought about doing a lookup in every line of the user loop
above, but that seems wasteful, and I'm not sure how I'd do it anyway.

Any suggestions?

Thanks,
Richard

[Richard Hector]

I've got this, but it looks horrible:

==================
- name: set up user dicts
set_fact:
"user_{{ item }}": "{{ lookup('file', inventory_dir +
'/users/user_vars/' + item) |from_yaml }}"
with_items: "{{ users }}"
tags:
- users

- name: users
user:
name: "{{ lookup('vars', 'user_' + item).name }}"
comment: "{{ lookup('vars', 'user_' + item).gecos }}"
shell: "{{ lookup('vars', 'user_' + item).shell }}"
createhome: yes
state: present
append: yes
with_items: "{{ users }}"
tags:
- users

- name: admin groups
user:
name: "{{ item }}"
append: yes
groups:
- sudo
- adm
when: item in admins
with_items: "{{ users }}"
tags:
- users

- name: ansible group
user:
name: "{{ item }}"
append: yes
groups:
- sudo
- adm
when: item in ansible_users
with_items: "{{ users }}"
tags:
- users
=========================

I'm still to do the ssh key stuff - that's going to be pretty ugly too,
I think.

Are there ways to make this cleaner?

Cheers,
Richard

[Michael Ströder]

On 4/16/22 13:52, Richard Hector wrote:
> On 16/04/22 22:13, Richard Hector wrote:
>> I have created a directory 'users' alongside my inventory. It has a
>> directory 'user_vars', intended to be used like host_vars, but for
>> users, obviously.
>

> I've got this, but it looks horrible:
>
> ==================
> - name: set up user dicts
>   set_fact:

You could avoid this by just using regular group_vars/.

> Are there ways to make this cleaner?

Probably not the answer you're looking for: While I use ansible for many
things I'd always prefer a decent user management system over hacking
this into ansible playbooks. Maintaining users, groups and maybe sudoers
in a database gives you required features like enforcing unique
constraints (username, POSIX-UID/-GID), data references, audit reports
etc. There are various ready-to-use systems (most times based on some
LDAP server) to choose from.

Ciao, Michael.

[Todd Lewis]

> Any suggestions?

Questions more than suggestions, but you can take them as such anyway.

1) How many users are we talking about? If the user data is smaller than the code it takes to manipulate it, maybe it'd be better to make the data more complex and the code simpler.

2) Are these files the canonical source of truth, or are they downstream? If downstream, perhaps it would be better to pull the data from upstream as needed. If these files are canonical truth, then either restructure them in a way that supports cleaner access, or consider creating an upstream data source. The latter would only make sense if the answer to #1 was large enough to justify it.

If it were me, and assuming a fairly small number of users and/or relatively static data (i.e. I'm going to maintain this by hand for the foreseeable future), I'd put all these user data into defaults/main.yml of a custom "my_users" role, probably in a single dict keyed by ID, or maybe a list. Make everything that might have multiple values for any user a list - ssh keys for example - so even if a user has only one (or none) you access that data the same way for everybody.

The fact that the expressions you're having to use to access the data currently are unwieldy indicates the initial data structures are a problem. Having user data split into multiple files may seem like a simplification, but I think it's just the opposite.

Good luck!

[Vladimir Botka]

On Sat, 16 Apr 2022 23:52:14 +1200
Richard Hector <ric...@walnut.gen.nz> wrote:

For example, given the tree

shell> tree inventory/
inventory/
├── group_vars
│   └── all
├── hosts
└── users
├── public_keys
│   ├── alice@bar
│   ├── alice@foo
│   ├── richard@bar
│   └── richard@foo
└── user_vars
├── alice
└── richard


and the files for testing

shell> cat inventory/users/user_vars/alice
name: alice
gecos: 'Alice Beatrice,,,'
shell: '/bin/bash'
ssh_keys:
- alice@foo
- alice@bar
shell> cat inventory/users/user_vars/richard

name: richard
gecos: 'Richard Hector,,,'
shell: '/bin/bash'
ssh_keys:
- richard@foo
- richard@bar

shell> cat inventory/users/public_keys/
alice@bar alice@foo richard@bar richard@foo
shell> cat inventory/users/public_keys/alice@bar
ssh-rsa djflskdjflsdkjflsdjflkdjl alice@bar
shell> cat inventory/users/public_keys/alice@foo
ssh-rsa djflskdjflsdkjflsdjflkdjl alice@foo
shell> cat inventory/users/public_keys/richard@bar
ssh-rsa djflskdjflsdkjflsdjflkdjl richard@bar
shell> cat inventory/users/public_keys/richard@foo
ssh-rsa djflskdjflsdkjflsdjflkdjl richard@foo


* Create variables

inventory_dir: inventory
dyn_vars_list: [user_vars, public_keys]


* Get the names of the files

- set_fact:
keys: "{{ keys|d({})|
combine({item: files}) }}"
loop: "{{ dyn_vars_list }}"
vars:
fileglob: "{{ inventory_dir }}/users/{{ item }}/*"
files: "{{ query('fileglob', fileglob)|
map('basename')|list }}"

gives

keys:
public_keys:
- richard@bar
- alice@bar
- alice@foo
- richard@foo
user_vars:
- alice
- richard


* Get the content of the files

- set_fact:
values: "{{ values|d([]) +
[{'var': item.0.key,
'key': item.1,
'value': value}] }}"
with_subelements:
- "{{ keys|dict2items }}"
- value
vars:
path: "{{ inventory_dir }}/users/{{ item.0.key }}/{{ item.1 }}"
value: "{{ lookup('file', path)|from_yaml }}"

gives

values:
- key: alice
value:
gecos: Alice Beatrice,,,
name: alice
shell: /bin/bash
ssh_keys:
- alice@foo
- alice@bar
var: user_vars
- key: richard
value:
gecos: Richard Hector,,,
name: richard

shell: /bin/bash
ssh_keys:
- richard@foo
- richard@bar

var: user_vars
- key: richard@bar
value: ssh-rsa dkjfslkdjflskdjflsdkjflsdjflkdjl richard@bar
var: public_keys
- key: alice@bar
value: ssh-rsa dkjfslkdjflskdjflsdkjflsdjflkdjl alice@bar
var: public_keys
- key: alice@foo
value: ssh-rsa dkjfslkdjflskdjflsdkjflsdjflkdjl alice@foo
var: public_keys
- key: richard@foo
value: ssh-rsa dkjfslkdjflskdjflsdkjflsdjflkdjl richard@foo
var: public_keys


* Group the values by 'var' and create the dictionary

- set_fact:
dyn_vars: "{{ dyn_vars|d({})|
combine({item.0: item.1|
items2dict(key_name='key',
value_name='value')}) }}"
loop: "{{ values|groupby('var') }}"

gives

dyn_vars:
public_keys:
alice@bar: ssh-rsa djflskdjflsdkjflsdjflkdjl alice@bar
alice@foo: ssh-rsa djflskdjflsdkjflsdjflkdjl alice@foo
richard@bar: ssh-rsa djflskdjflsdkjflsdjflkdjl richard@bar
richard@foo: ssh-rsa djflskdjflsdkjflsdjflkdjl richard@foo
user_vars:
alice:
gecos: Alice Beatrice,,,
name: alice
shell: /bin/bash
ssh_keys:
- alice@foo
- alice@bar
richard:
gecos: Richard Hector,,,
name: richard

shell: /bin/bash
ssh_keys:
- richard@foo
- richard@bar

The usage of the dictionary should be trivial.

--
Vladimir Botka