new Debconf module - please review / comment

[Sébastien Bocahu]

Hi,

Debconf and its pair tools can be used to reconfigure a Debian package, or to
preconfigure packages before they get installed.

It can be really handy, especially for web application packages in Debian such
as roundcube or phpmyadmin.

Until recently, I used to transfert a subset of the output of
debconf-get-selections in a file (copy) and pass its content to
debconf-set-selections (shell), then call dpkg-reconfigure package (command)...

I wrote a simple module that makes all these steps and checks the current
configuration before calling debconf-set-selections, so that ansible reports
whether there were changes or not.

It requires debconf-utils on the target machine.

For example:

- name: configure roundcube
debconf: pkg=roundcube keys='$TEMPLATE(roundcube)' reconfigure=no

Use a template to feed debconf-set-selections with debconf keys/values.

The format of 'keys' is the same as the output of debconf-get-selections, except
the first column (the package name) is removed.

For example:

roundcube/mysql/admin-pass password {{ mysql_root_password }}
roundcube/mysql/method select unix socket
roundcube/dbconfig-install boolean true
roundcube/db/dbname string roundcube
roundcube/dbconfig-upgrade boolean true
roundcube/mysql/admin-user string root
roundcube/pgsql/method select unix socket
roundcube/hosts string 127.0.0.1
roundcube/database-type select mysql
roundcube/db/app-user string roundcube
roundcube/language select fr_FR.UTF-8

Using TEMPLATE or FILE macros are not required, but makes it easier to pass
multiple debconf keys.

'reconfigure' defaults to yes and if so, calls 'dpkg-reconfigure $pkg'.


So:

* Do you guys believe it would be useful ?
* Any comments on the debconf/feature side ?
* I'm not used to coding in python. Any comments on the style would be appreciated

Source: https://github.com/zecrazytux/ansible/blob/library/debconf/library/debconf
Pull request: https://github.com/ansible/ansible/pull/2456

Michael: yup, I'll change the example part of the documentation to the new format :)


Thanks,

--
Sebastien

[seb....@gmail.com]

Am Montag, 25. März 2013 17:07:17 UTC+1 schrieb Sébastien 'zecrazytux' Bocahu:

Hi,

Debconf and its pair tools can be used to reconfigure a Debian package, or to
preconfigure packages before they get installed.
 

I would love to see a module that deals with debconf, because I think it's a pain to do by hand.

However, I see a problem with this: Once dpkg-reconfigure finds a value in a packages config file it will default to that value, no matter what you seed it with debconf-set-selections.
Therefore, I have to change both, the value in the debconf database as well as the "real" config file for the package. Here's an example for the package matlab-support:

The preseed file looks like this:

matlab-support  matlab-support/default-version  select  Matlab {{ matlab_version }} @ {{ matlab_instdir }}/{{ matlab_version }}
matlab-support  matlab-support/rename-libs      boolean false
matlab-support  matlab-support/matlab-install-glob      string  {{ matlab_instdir }}/{{ matlab_version }}
matlab-support  matlab-support/mexbuild-user    string

And the related tasks:

- name: Preseed matlab-support
action: shell debconf-set-selections </var/local/preseed/matlab-support
register: matlab_selections
when_boolean: ${matlab_seed.changed}
tags:
- matlab
- name: Dpkg-reconfigure matlab-support
action: command dpkg-reconfigure -fnoninteractive matlab-support
when_boolean: ${matlab_selections.changed}
tags:
- matlab

This _won't_ change the setting I'm interested in because it is also set in /etc/matlab/debconf:

- name: Configure matlab-support
  action: lineinfile dest=/etc/matlab/debconf regexp="^MATLAB_INSTALL_GLOB=" line="MATLAB_INSTALL_GLOB=${matlab_instdir}/${matlab_version}"
  when_boolean: ${matlab_selections.changed}
  tags:
    - matlab

This has been confusing some people including me (see also http://lists.debian.org/debian-user/2007/10/msg00480.html). Given you find a solution to that problem (or show me that I'm wrong) I would really appreciate such a module!


Sebastian

[Sébastien Bocahu]

Hey Sebastian,

Glad to see I'm not the only (Debian + Ansible) user :)

> I would love to see a module that deals with debconf, because I think it's
> a pain to do by hand.
>
> However, I see a problem with this: Once dpkg-reconfigure finds a value in
> a packages config file it will default to that value, no matter what you
> seed it with debconf-set-selections.

IMHO it's a bug. Devs who wrote the configure scripts haven't thought about
unatended (re)configuration...

There is no such problem with dovecot, for example (afaik).

> Therefore, I have to change both, the value in the debconf database as well
> as the "real" config file for the package.

> This has been confusing some people including me (see also
> http://lists.debian.org/debian-user/2007/10/msg00480.html).

I've run into similar issues as well. I've mentionned the tzdata issue on the
pull request.

I just had a look to how these debconf thingies work. For tzdata, the keys are
initialised with values computed from /etc/timezone.

I'll propose the following fix to Debian shortly:

/var/lib/dpkg/info/tzdata.config:

# Initializes debconf default values from the ones found in
# configuration files
-db_set tzdata/Areas "$AREA"
-db_set tzdata/Zones/$AREA "$ZONE"
+if [ -z "$DEBCONF_RECONFIGURE" ]; then
+ db_set tzdata/Areas "$AREA"
+ db_set tzdata/Zones/$AREA "$ZONE"
+fi

I haven't checked matlab, but it certainly has something like that which prevents
non-interactive reconfiguration.

Hints:

* check the configure (perhaps the postinst too) scripts (/var/lib/dpkg/info/)
* play with 'debconf-set-selections' and
'DEBCONF_DEBUG=developer dpkg-reconfigure -fnoninteractive'

> Given you find
> a solution to that problem (or show me that I'm wrong) I would really
> appreciate such a module!

You are right but here is a solution :)
report problems (and if possible, provide a fix) to upstream ;)


Thanks,

Sebastien

[seb....@gmail.com]

Hope you're not disappointed, but I use Ubuntu :) I fear I have to disagree with you that this behavior is a bug:

https://bugs.launchpad.net/ubuntu/+source/pam/+bug/682662

Here it's clearly stated that

"by design of debconf that settings on the system take precedence over any values set in the debconf database"

This won't-fix-bug-report also contains a proposal how to deal with this and answering debconf question via an EDITOR script, which seems like a workable approach for an ansible module to me.

Sebastian

[Sébastien Bocahu]

Hmm, good point.
It's more like a cache than a configuration registry/database.

I'll have a deeper look at this method.

Thanks :)


Sebastien

[Sébastien Bocahu]

Hi,

> > Here it's clearly stated that
> >
> > "by design of debconf that settings on the system take precedence over any values set in the debconf database"
> >
> > This won't-fix-bug-report also contains a proposal how to deal with this and answering debconf question via an EDITOR script, which seems like a workable approach for an ansible module to me.

I've reworked my debconf module, and actually renamed it to 'dpkg-reconfigure'.

https://github.com/zecrazytux/ansible/commit/8717044b6ecdbd65b842d778a22c48a81153c0f4

It can't be used for preseed package installation anymore (not sure if it would
have made sense anyway), and the format of keys/values has changed a bit.
The good news is it's shorter and cleaner (no need for the entry type (password,
select, multiselect...)

Example:

- name: Set default locale to fr_FR.UTF-8 and generate en_US.UTF-8 as well
dpkg-reconfigure: pkg=locales keys="locales/default_environment_locale fr_FR.UTF-8\n
locales/locales_to_be_generated en_US.UTF-8 UTF-8, fr_FR.UTF-8 UTF"

Some keys have to be answered for dpkg-reconfigure to be happy (for the above
example, locales_to_be_generated is mandatory. you can't change the default
locale only, even if the locale was previously available).

For finding keys/values that are interesting:

DEBIAN_FRONTED=editor dpkg-reconfigure $pkg

in your tested. Then use the simple format:
debconf_key debconf_value
as the 'keys' option. (multiple (key/value)s separated by \n or stored in a
file ($FILE macro))

As usual, tests, review & comments would be greatly appreciated.

Thanks,

--
Sebastien

[GW]

Hi,

I tested your module and it seems to work, although it is sad that there isn't a better way to implement this.

Some suggested improvements:

- rename it to `dpkg_reconfigure`, because other modules also use underscores in their names

- parameter `keys` could also accept a list (eg. keys={{ var_with_list }} or keys=["locales/default_environment_locale fr_FR.UTF-8", "locales/locales_to_be_generated en_US.UTF-8 UTF-8, fr_FR.UTF-8 UTF"], if it works?)

- parameter `keys` could also accept a dict (eg. keys={"locales/default_environment_locale": "fr_FR.UTF-8", "locales/locales_to_be_generated": "en_US.UTF-8 UTF-8, fr_FR.UTF-8 UTF"], if it works?)

- parameter `keys` could be renamed to `values` or `answers`, because debconf man pages refer to this as being values (not keys)

- maybe `debconf-shows` could be used instead of `debconf-get-selections`, because it is part of the standard `debconf` package and there is no need to install additional packages for `debconf-utils`

Greetings,
   gw 

[Sébastien Bocahu]

Hi,

> I tested your module and it seems to work, although it is sad
> that there isn't a better way to implement this.
>
> Some suggested
> improvements:

Thanks !

> - rename it to `dpkg_reconfigure`, because other modules
> also use underscores in their names

Agreed.

> - parameter `keys` could also
> accept a list (eg. keys={{ var_with_list }} or
> keys=["locales/default_environment_locale fr_FR.UTF-8",
> "locales/locales_to_be_generated en_US.UTF-8 UTF-8, fr_FR.UTF-8 UTF"],
> if it works?)
>
> - parameter `keys` could also accept a dict (eg.
> keys={"locales/default_environment_locale": "fr_FR.UTF-8",
> "locales/locales_to_be_generated": "en_US.UTF-8 UTF-8, fr_FR.UTF-8
> UTF"], if it works?)

Great suggestion. I'ld rather use either a dict or a list, not both.

> - parameter `keys` could be renamed to `values`
> or `answers`, because debconf man pages refer to this as being values
> (not keys)

Right, I used 'keys' as it sounded familiar to me but 'answers' is definitely
more debconf-compliant ;)

> - maybe `debconf-shows` could be used instead of
> `debconf-get-selections`, because it is part of the standard `debconf`
> package and there is no need to install additional packages for
> `debconf-utils`

Totally agreed !


Thank you very much, Gw.
I'll update soon.

Cheers,

Sebastien

[Sébastien Bocahu]

Hi,

> > - rename it to `dpkg_reconfigure`, because other modules
> > also use underscores in their names
>
> Agreed.

> > - parameter `keys` could be renamed to `values`
> > or `answers`, because debconf man pages refer to this as being values
> > (not keys)
>
> Right, I used 'keys' as it sounded familiar to me but 'answers' is definitely
> more debconf-compliant ;)

> > - maybe `debconf-shows` could be used instead of
> > `debconf-get-selections`, because it is part of the standard `debconf`
> > package and there is no need to install additional packages for
> > `debconf-utils`
>
> Totally agreed !

Done. https://github.com/ansible/ansible/pull/2805

> > - parameter `keys` could also
> > accept a list (eg. keys={{ var_with_list }} or
> > keys=["locales/default_environment_locale fr_FR.UTF-8",
> > "locales/locales_to_be_generated en_US.UTF-8 UTF-8, fr_FR.UTF-8 UTF"],
> > if it works?)
> >
> > - parameter `keys` could also accept a dict (eg.
> > keys={"locales/default_environment_locale": "fr_FR.UTF-8",
> > "locales/locales_to_be_generated": "en_US.UTF-8 UTF-8, fr_FR.UTF-8
> > UTF"], if it works?)
>
> Great suggestion. I'ld rather use either a dict or a list, not both.

It seems to me that it is not supported by the module boilerplate, and I'm not
going to dig too much into this. I want it to not be intrusive, to be
mainstream, while still useable. I hope that's good enough as is.

Thanks,

Sebastien

[Michael DeHaan]

"It seems to me that it is not supported by the module boilerplate, and I'm not

going to dig too much into this. I want it to not be intrusive, to be
mainstream, while still useable. I hope that's good enough as is."

Complex args do support passing lists and arrays, and these things "just work" in modules.

- action: blarg

  args:

     some_variable:

        key1:  [ 1,2,3,4]

     

etc

Sebastien

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

[Sébastien Bocahu]

> "It seems to me that it is not supported by the module boilerplate, and I'm
> not
> going to dig too much into this. I want it to not be intrusive, to be
> mainstream, while still useable. I hope that's good enough as is."
>
> Complex args do support passing lists and arrays, and these things "just
> work" in modules.
>
> - action: blarg
> args:
> some_variable:
> key1: [ 1,2,3,4]

Sweet ! I did not catch this change yet !

I commited support for the extended arguments so that we can do now:

- name: test
dpkg_reconfigure:
pkg: locales
answers:
locales/default_environment_locale: fr_FR.UTF-8
locales/locales_to_be_generated: en_US.UTF-8 UTF-8, fr_FR.UTF-8 UTF-8


Thanks !

--
Sebastien Bocahu

[Sébastien Bocahu]

> This is great! I had the idea for this module a couple of weeks ago and
> noticed you were already working on it. I was playing around with it last
> night to setup ldap and it worked well. Only a couple of annoyances I ran
> into:

Great "I'm not the only one" :)

> It hanging if you did not supply all of the right arguments

Ok. A few extra checks could probably prevent that... But have you got a simple
testcase (LDAP is probably not :p) ?

I couldn't get any hang while testing myself.


> The state always showing up as changed (I think this is a result of passwords
> not getting stored in debconf-shows for obvious security reasons. Not sure
> if there would be anyway around this).

Yup, that's the reason.

Passwords are stored in the password db but removed as soon as possible (and if
not, that's a bug in the packaging), so asking to read the password database
(debconf-show only look at the 'config', default database) won't help.

--
Sebastien Bocahu

[Sébastien Bocahu]

For whose (about to rock) who are interested by this module:

Michael does not like this module so I'm hosting it in an extra repository:
https://github.com/zecrazytux/ansible-library-extra

Cheers,

--
Sebastien Bocahu