Bug#1010273: mailman3-web: There seems to be a clas between mailman3-web and python3-django-hyperkitty about the /static/ dir

[Peter Gervai]

Package: mailman3-web
Version: 0+20200530-2
Severity: normal

Dear Maintainer,

This was a fresh install, and hyperkitty archive was forever spinning.
Apache was configured according to the official docs. So I have upgraded
from stable to testing (to fix one bug I thought to be related).

It tuned out that there are two separate sets of web-static directories:
- /var/lib/mailman3/web/static (the "wrong" one)
- /usr/share/python3-django-hyperkitty/static/

Same goes with posterius and others. The problem is that they _look_ like
they are okay but have different content, in fact one had 1.xx jquery
while the other had the new 3.6 one, and it caused all kinds of weird
failures on the web interface.

I believe there ought to be only one set of these, and the other,
for whatever purposes it's requird shall be symlinked over.

User error is possible! But I haven't really done anything out of the
ordinary.


-- System Information:
Debian Release: 11.3
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 5.13.19-3-pve (SMP w/4 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_FIRMWARE_WORKAROUND, TAINT_OOT_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)

Versions of packages mailman3-web depends on:
ii dbconfig-pgsql 2.0.19
ii dbconfig-sqlite3 2.0.19
ii debconf [debconf-2.0] 1.5.77
ii init-system-helpers 1.60
ii lsb-base 11.1.0
ii python3 3.9.2-3
ii python3-django-hyperkitty 1.3.5.0-2
ii python3-django-postorius 1.3.4-2+deb11u1
ii python3-psycopg2 2.8.6-2
ii python3-whoosh 2.7.4+git6-g9134ad92-5
ii ucf 3.0043
ii uwsgi-core 2.0.19.1-7.1
ii uwsgi-plugin-python3 2.0.19.1-7.1

Versions of packages mailman3-web recommends:
ii libapache2-mod-proxy-uwsgi 2.4.53-1~deb11u1

Versions of packages mailman3-web suggests:
ii postgresql 13+225

-- Configuration Files:
/etc/init.d/mailman3-web changed [not included]
/etc/mailman3/apache.conf changed [not included]
/etc/mailman3/uwsgi.ini changed [not included]

-- debconf information excluded

[Marco Steinacher]

Hi,

I can confirm this problem. I think the following is happening:

After updating mailman3-web, the static files in
/usr/share/python3-django-postorius/static/ (and
python3-django-hyperkitty, and maybe others) have an older date than the
collected files in /var/lib/mailman3/web/static from the previous version.

Because of that, "./manage collectstatic" skips the files with "not
modified", and is not updating the static files, even though the files
have been updated (but have an older modified date).

To fix this, you can touch the static files in
/usr/share/python3-django-postorius/static/ (to set a newer modification
date) and then run "./manage collectstatic". Then the files will be
updated, since collectstatic seems to compare the file modification
dates and not the actual file contents.

I'm not sure how the Debian package installer should avoid this problem.
Probably it should either make sure that the updated files have a newer
date before running collectstatic or delete the old files before doing that.

Below you find some additional notes that I wrote down while
investigating the situation. Maybe that is helpful.

Marco

--- Additional notes: ---

I noticed strange CSS problems after upgrading from buster to bullseye.
With the information from this bug report, I found the following issue:

- /var/lib/mailman3/web/static is used, but seems to contain old
versions (e.g. Bootstrap v3.3.5 at
postorius/libs/bootstrap/css/bootstrap.min.css)

- /usr/share/python3-django-postorius/static/ seems to contain the
correct version (e.g. Bootstrap v4.0.0) but is not used.

Further digging into the issue, I found that
/usr/lib/python3/dist-packages/postorius/static is a soft-link to the
correct location at /usr/share/python3-django-postorius/static/. So that
seems to be right.

Hence, I suspect that "./manage.py collectstatic" for some reason does
not update the static files. Running "./manage.py collectstatic -v 3"
skips all files with "not modified":

----------------------------------------------------------------------
www-data@mail:/usr/share/mailman3-web$ ./manage.py collectstatic -v 3

You have requested to collect static files at the destination
location as specified in your settings:

/var/lib/mailman3/web/static

This will overwrite existing files!
Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel: yes

[...]
Skipping 'postorius/libs/bootstrap/css/bootstrap.min.css' (not modified)
[...]
----------------------------------------------------------------------

Removing the static files at /var/lib/mailman3/web/static before running
collectstatic solves the problem:

----------------------------------------------------------------------
www-data@mail:/usr/share/mailman3-web$ ./manage.py collectstatic -v 3

You have requested to collect static files at the destination
location as specified in your settings:

/var/lib/mailman3/web/static

This will overwrite existing files!
Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel: yes
[...]
Copying
'/usr/lib/python3/dist-packages/postorius/static/postorius/libs/bootstrap/css/bootstrap.min.css'
[...]
----------------------------------------------------------------------

Now Bootstrap has the correct version (v4.0.0 in this case) and the CSS
issues are gone!

So the workaround is to delete the static files in
/var/lib/mailman3/web/static before collecting them with "./manage.py
collectstatic" or to touch the files in
/usr/share/python3-django-postorius/static/ so they have a newer
modification date.



--
OpenPGP Key ID: 0x62937F7F

[Marco Steinacher]

Hi again,

Here is an update to my previous message:

I think the easiest workaround is to add the --clear flag to the
collectstatic command, i.e.

www-data@mail:/usr/share/mailman3-web$ ./manage.py collectstatic --clear
www-data@mail:/usr/share/mailman3-web$ ./manage.py compress

With the --clear flag the existing files are deleted before trying to
copy the original file.
https://docs.djangoproject.com/en/4.2/ref/contrib/staticfiles/#collectstatic

This should solve the problem of old static files not being updated
because of source files that are older than the files installed by the
previous version.

The --clear flag should probably be added to
debian/mailman3-web.postinst in update_django(), line 198, to fix the
problem in the mailman3-web Debian package.
https://salsa.debian.org/mailman-team/mailman-suite/-/blob/master/debian/mailman3-web.postinst#L198

Marco

[Pierre-Elliott Bécue]

Hello,

I saw this bug report, but I admit it slipped out of my mind.

I have updated the django-mailman3 package recently and was planning to
finish with a sweep on postorius and hyperkitty and then the web package.

I'll include these fixes, and then try to have the relevant fixes
backported into bookworm via a stable upload.

Thanks for your mails.
--
PEB