Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Bug#1053502: mailman3-web: Package failed to install during upgrade from Debian 11 to 12

80 views
Skip to first unread message

Steven Verhulst

unread,
Oct 5, 2023, 4:50:06 AM10/5/23
to
Package: mailman3-web
Version: 0+20200530-2.1
Severity: important

Dear Maintainer,

When we upgraded our test mailingserver from Debian 11 to 12 we noticed the following error:

Setting up mailman3-web (0+20200530-2.1) ...
Determining localhost credentials from /etc/mysql/debian.cnf: failed (hosts not equal).
dbconfig-common: writing config to /etc/dbconfig-common/mailman3-web.conf
dbconfig-common: flushing administrative password
sed: -e expression #2, char 82: unterminated `s' command
dpkg: error processing package mailman3-web (--configure):
installed mailman3-web package post-installation script subprocess returned error exit status 1

From what I understand there seems to be an issue with the post-installation script.

Since the post-installation script did not run it left us with a broken listserver.
Further it is not possible to use APT to update packages as it keep warning about mailman3-web package being broken.

-- System Information:
Debian Release: 12.1
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 6.1.0-12-amd64 (SMP w/2 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages mailman3-web depends on:
ii dbconfig-sqlite3 2.0.24
ii debconf [debconf-2.0] 1.5.82
ii init-system-helpers 1.65.2
ii lsb-base 11.6
ii python3 3.11.2-1+b1
ii python3-django-hyperkitty 1.3.7-1
ii python3-django-postorius 1.3.8-3
ii python3-mysqldb 1.4.6-2+b1
ii python3-psycopg2 2.9.5-1+b1
ii python3-whoosh 2.7.4+git6-g9134ad92-7
ii sysvinit-utils [lsb-base] 3.06-4
ii ucf 3.0043+nmu1
ii uwsgi-core 2.0.21-5.1
ii uwsgi-plugin-python3 2.0.21-5.1

Versions of packages mailman3-web recommends:
ii libapache2-mod-proxy-uwsgi 2.4.57-2

Versions of packages mailman3-web suggests:
ii mariadb-server [virtual-mysql-server] 1:10.11.3-1

-- Configuration Files:
/etc/mailman3/apache.conf changed [not included]
/etc/mailman3/uwsgi.ini changed [not included]

-- debconf information excluded

Pierre-Elliott Bécue

unread,
Oct 5, 2023, 10:30:05 AM10/5/23
to
tags 1053502 +moreinfo
thanks

Hi,

Steven Verhulst <sver...@vub.be> wrote on 05/10/2023 at 10:37:05+0200:

> Package: mailman3-web
> Version: 0+20200530-2.1
> Severity: important
>
> Dear Maintainer,
>
> When we upgraded our test mailingserver from Debian 11 to 12 we
> noticed the following error:
>
> Setting up mailman3-web (0+20200530-2.1) ...
> Determining localhost credentials from /etc/mysql/debian.cnf: failed (hosts not equal).
> dbconfig-common: writing config to /etc/dbconfig-common/mailman3-web.conf
> dbconfig-common: flushing administrative password
> sed: -e expression #2, char 82: unterminated `s' command
> dpkg: error processing package mailman3-web (--configure):
> installed mailman3-web package post-installation script subprocess returned error exit status 1
>
> From what I understand there seems to be an issue with the
> post-installation script.
>
> Since the post-installation script did not run it left us with a
> broken listserver. Further it is not possible to use APT to update
> packages as it keep warning about mailman3-web package being broken.

When I read

> Determining localhost credentials from /etc/mysql/debian.cnf: failed (hosts not equal).
> dbconfig-common: writing config to /etc/dbconfig-common/mailman3-web.conf
> dbconfig-common: flushing administrative password

I tend to think some weird stuff happened on your host.

But to try understanding further the issue, could you please give me
some hints about what this file contains?

--
PEB
signature.asc

Pierre-Elliott Bécue

unread,
Oct 6, 2023, 9:10:04 AM10/6/23
to

Steven Verhulst <sver...@vub.be> wrote on 06/10/2023 at 13:16:09+0200:

> Hi,
>
>
>
> /etc/mysql/debian.cnf is a config file auto generated by some debian scripts.
>
> It contains host / user information for mysql connection.
>
> According to the contents this file is deprecated and should no longer be used:
>
>
>
> # THIS FILE IS OBSOLETE. STOP USING IT IF POSSIBLE.
>
> # This file exists only for backwards compatibility for
>
> # tools that run '--defaults-file=/etc/mysql/debian.cnf'
>
> # and have root level access to the local filesystem.
>
> # With those permissions one can run 'mariadb' directly
>
> # anyway thanks to unix socket authentication and hence
>
> # this file is useless. See package README for more info.
>
> # THIS FILE WILL BE REMOVED IN A FUTURE DEBIAN RELEASE.
>
>
>
>
>
> “sed: -e expression #2, char 82: unterminated `s' command”
>
> Makes me believe that there is an issue with with one the sed expression in the post-installation script.
>
>
>
> Kind regards,
>
>
>
> Steven Verhulst
>
>
>
>
>
> On 05/10/2023, 16:23, "Pierre-Elliott Bécue" <p...@debian.org> wrote:
>
> tags 1053502 +moreinfo
>
> thanks
>
>
>
> Hi,
>
>
>
> Steven Verhulst <sver...@vub.be> wrote on 05/10/2023 at 10:37:05+0200:
>
>
>
>> Package: mailman3-web
>
>> Version: 0+20200530-2.1
>
>> Severity: important
>
>>
>
>
>
>> Dear Maintainer,
>
>>
>
>
>
>> When we upgraded our test mailingserver from Debian 11 to 12 we
>
>> noticed the following error:
>
>>
>
>
>
>> Setting up mailman3-web (0+20200530-2.1) ...
>
>> Determining localhost credentials from /etc/mysql/debian.cnf: failed (hosts not equal).
>
>> dbconfig-common: writing config to /etc/dbconfig-common/mailman3-web.conf
>
>> dbconfig-common: flushing administrative password
>
>> sed: -e expression #2, char 82: unterminated `s' command
>
>> dpkg: error processing package mailman3-web (--configure):
>
>> installed mailman3-web package post-installation script subprocess returned error exit status 1
>
>>
>
>
>
>> From what I understand there seems to be an issue with the
>
>> post-installation script.
>
>>
>
>
>
>> Since the post-installation script did not run it left us with a
>
>> broken listserver. Further it is not possible to use APT to update
>
>> packages as it keep warning about mailman3-web package being broken.
>
>
>
> When I read
>
>
>
>> Determining localhost credentials from /etc/mysql/debian.cnf: failed (hosts not equal).
>
>> dbconfig-common: writing config to /etc/dbconfig-common/mailman3-web.conf
>
>> dbconfig-common: flushing administrative password
>
>
>

Pierre-Elliott Bécue

unread,
Oct 6, 2023, 9:20:04 AM10/6/23
to
Please keep the bug report CC-ed.

Steven Verhulst <sver...@vub.be> wrote on 06/10/2023 at 13:16:09+0200:

> Hi,
>
>
>
> /etc/mysql/debian.cnf is a config file auto generated by some debian scripts.
>
> It contains host / user information for mysql connection.
>
> According to the contents this file is deprecated and should no longer be used:
>
>
>
> # THIS FILE IS OBSOLETE. STOP USING IT IF POSSIBLE.
>
> # This file exists only for backwards compatibility for
>
> # tools that run '--defaults-file=/etc/mysql/debian.cnf'
>
> # and have root level access to the local filesystem.
>
> # With those permissions one can run 'mariadb' directly
>
> # anyway thanks to unix socket authentication and hence
>
> # this file is useless. See package README for more info.
>
> # THIS FILE WILL BE REMOVED IN A FUTURE DEBIAN RELEASE.
>
>
>
>
>
> “sed: -e expression #2, char 82: unterminated `s' command”
>
> Makes me believe that there is an issue with with one the sed
> expression in the post-installation script.

Yes, I am aware of your beliefs, and they're probably founded, but to be
able to dig in I need some context.

If you did not fix manually the issue, could you add "set -x" at the
beginning of /var/lib/dpkg/info/mailman3-web.postinst script and run a
dpkg --configure mailman3-web and give me the output?

If some passwords fall in the output, of course feel free to censor
them.

--
PEB
signature.asc

Boud Roukema

unread,
Oct 10, 2023, 4:10:05 PM10/10/23
to
Package: mailman3-web
Version: 0+20200530-2.1
Followup-For: Bug #1053502

Dear Maintainer,

SUMMARY:
----------------------------------------------------------------------
For me, this bug is confirmed, but solved. There appear to be three separate issues:

* the need to add DEFAULT_AUTO_FIELD in /etc/mailman3/mailman-web.py
as recommended in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1041496

* the need to have Q_CLUSTER retry > Q_CLUSTER timeout
e.g. by setting override values in /etc/mailman3/mailman-web.py, again
as recommended in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1041496

* There is a sed script (I don't know in which file) that should
ignore comment lines in /etc/mailman3/mailman-web.py but does not
ignore them. The result is that a parameter value (the value of
EMAILNAME) is interpreted as a command. This is correctly trapped
as an error, and could potentially lead to bad consequences if
an EMAILNAME (domain) is a dangerous command.

Details below.
----------------------------------------------------------------------


* What led up to the situation?

I did a system upgrade from bullseye (with an existing install of mailman3) to bookworm.

* What exactly did you do (or not do) that was effective (or
ineffective)?

(1) Following (and extending) from PEB's request on Fri, 06 Oct 2023
15:07:50 +0200, I did (1a) and (1b). Inspired by discussion of bug
#1041496, I did (1c).

(1a) insert 'set -x' near the beginning of /var/lib/dpkg/info/mailman3-web.postinst

(1b) insert 'set -x' near the beginning of /usr/share/debconf/confmodule

(1c) insert the following lines at the end of /etc/mailman3/mailman-web.py:

# per https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1041496
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
# Does this override the value in /usr/share/mailman3-web/settings.py?
Q_CLUSTER = {
'timeout': 300,
'retry': 600,
'save_limit': 100,
'orm': 'default',
'poll': 5,
}


(2) dpkg --configure mailman3-web |& tee log.dpkg.config.mailman3web.5

* What was the outcome of this action?

This gave a configure error, with the redacted log at https://paste.debian.net/hidden/51a5c38c

* What outcome did you expect instead?

I expected to succesfully finish the configuring of mailman3-web.

* What I did next (after reading the log):

The file /etc/mailman3/mailman-web.py originally contained the two successive lines:

# EMAILNAME = localhost
EMAILNAME = 'subdomain.example.com'

(This is because somehow my initial install/upgrade was left with 'localhost' instead
of 'subdomain.example.com'.)

(3) I removed the line '# EMAILNAME = localhost' from /etc/mailman3/mailman-web.py .

(4) dpkg --configure mailman3-web |& tee log.dpkg.config.mailman3web.6

This configured correctly, with the redacted log at https://paste.debian.net/hidden/6990cef5 .

After 'systemctl restart mailman3; systemctl restart mailman3-web' (and a few unrelated minor
fixes) my mailman3 install appears to be fully functional; and 'aptitude safe-upgrade'
says that no packages are broken and nothing remains to be installed.

So this is solved for me.

(5) I reverted (1a) and (1b).


ANALYSIS:
The bug appears to be the regexp

+ sed -n -e s/\s*EMAILNAME\s*=\s*'\(.*\)'\s*$/\1/p /etc/mailman3/mailman-web.py

in line 2934 of https://paste.debian.net/hidden/51a5c38c, with the result in line 2966

+ RET=20 Unsupported command "subdomain.example.com" (full line was "subdomain.example.com") received from confmodule.

The commented line is not excluded by the regexp rule, and subdomain.example.com
becomes a command instead of the value of EMAILNAME.


COMMENT: I'm not sure where the script with the 'sed' is. For
robustness, the script needs to also work if the user has
*uncommented* multiple lines setting a value to a variable.
Either the final value should override earlier ones (common convention),
or the first should override later ones, or an error should occur,
depending on whatever conventions are defined and documented and
what the user is likely to expect.


-- System Information:
Debian Release: 12.2
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 5.10.0-26-amd64 (SMP w/2 CPU threads)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE=fr_FR:fr
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages mailman3-web depends on:
pn dbconfig-sqlite3 | dbconfig-pgsql | dbconfig-mysql <none>
| dbconfig-no-thanks
ii debconf [debconf-2.0] 1.5.82
ii init-system-helpers 1.65.2
ii lsb-base 11.6
ii python3 3.11.2-1+b1
ii python3-django-hyperkitty 1.3.7-1
ii python3-django-postorius 1.3.8-3
ii python3-psycopg2 2.9.5-1+b1
ii python3-whoosh 2.7.4+git6-g9134ad92-7
ii sysvinit-utils [lsb-base] 3.06-4
ii ucf 3.0043+nmu1
ii uwsgi-core 2.0.21-5.1
ii uwsgi-plugin-python3 2.0.21-5.1

Versions of packages mailman3-web recommends:
ii libapache2-mod-proxy-uwsgi 2.4.57-2

Versions of packages mailman3-web suggests:
ii postgresql 15+248

-- Configuration Files:
/etc/mailman3/apache.conf changed:
Alias /mailman3/favicon.ico /var/lib/mailman3/web/static/postorius/img/favicon.ico
Alias /mailman3/static /var/lib/mailman3/web/static
<Directory "/var/lib/mailman3/web/static">
Require all granted
</Directory>
<IfModule mod_proxy_uwsgi.c>
ProxyPass /mailman3/favicon.ico !
ProxyPass /mailman3/static !
ProxyPass /mailman3 unix:/run/mailman3-web/uwsgi.sock|uwsgi://localhost/
</IfModule>


-- debconf information:
* mailman3-web/db/app-user: mailman3web@localhost
mailman3-web/db/basepath:
mailman3-web/upgrade-backup: true
* mailman3-web/db/dbname: mailman3web
mailman3-web/mysql/admin-user:
* mailman3-web/pgsql/admin-user: postgres
mailman3-web/nginx-choice:
mailman3-web/passwords-do-not-match:
* mailman3-web/emailname: #localhost
* mailman3-web/remote/port:
mailman3-web/pgsql/changeconf: false
mailman3-web/purge: false
* mailman3-web/dbconfig-reinstall: true
mailman3-web/internal/reconfiguring: false
mailman3-web/remove-error: abort
* mailman3-web/remote/host: localhost
* mailman3-web/superuser-name: admin
* mailman3-web/configure-webserver: apache2
mailman3-web/upgrade-error: abort
mailman3-web/pgsql/no-empty-passwords:
mailman3-web/remote/newhost: localhost
* mailman3-web/restart-webserver: true
* mailman3-web/pgsql/method: TCP/IP
* mailman3-web/superuser-mail: root@localhost
* mailman3-web/pgsql/authmethod-admin: ident
mailman3-web/mysql/authplugin: default
mailman3-web/dbconfig-upgrade: true
mailman3-web/mysql/method: Unix socket
* mailman3-web/dbconfig-install: true
mailman3-web/internal/skip-preseed: true
mailman3-web/install-error: abort
mailman3-web/missing-db-package-error: abort
mailman3-web/pgsql/manualconf:
* mailman3-web/database-type: pgsql
* mailman3-web/pgsql/authmethod-user: password
* mailman3-web/dbconfig-remove: true

Cheers
Boud

Kalle Kivimaa

unread,
Nov 1, 2023, 7:40:05 AM11/1/23
to
For me the postinst failed in the multi-sed at # Set custom settings in settings_local.py

The failing part is the second expression doing ADMINS, which expaned as:

-e /^ADMINS = ($/,/^)$/{s|'root@localhost
('Kalle Kivimaa', 'kalle....@iki.fi'),'|'root@localhost’|}

If I simply removed this -e expression the dpkg-configure finished fine.

--
Kalle Kivimaa
+358-40-5007717
0 new messages