Bug#1032654: aptitude: missing message about the Debian bookworm change concerning non-free-firmware

[Vincent Lefevre]

Package: aptitude
Version: 0.8.13-5
Severity: important

I'm using testing+unstable, and Debian changed the non-free component
to "non-free non-free-firmware" some time ago, and nothing has been
announced. I've recently learnt that with apt 2.6, a message is output
about that with "apt update", but I got nothing with updates from the
aptitude UI.

I have just tried "apt update" and got:

15 packages can be upgraded. Run 'apt list --upgradable' to see them.
N: Repository 'Debian bookworm' changed its 'non-free component' value from 'non-free' to 'non-free non-free-firmware'
N: More information about this can be found online in the Release notes at: https://www.debian.org/releases/bookworm/amd64/release-notes/ch-information.html#non-free-split

This message should have been displayed by aptitude. Without it, the
user who uses aptitude only may never be aware of this change and will
miss future updates of the concerned packages (which are important for
security reasons).

-- Package-specific info:
Terminal: xterm-256color
$DISPLAY is set.
which aptitude: /usr/bin/aptitude

aptitude version information:
aptitude 0.8.13
Compiler: g++ 12.1.0
Compiled against:
apt version 6.0.0
NCurses version 6.3
libsigc++ version: 2.10.8
Gtk+ support disabled.
Qt support disabled.

Current library versions:
NCurses version: ncurses 6.4.20221231
cwidget version: 0.5.18
Apt version: 6.0.0

aptitude linkage:
linux-vdso.so.1 (0x00007ffdc6f9e000)
libgtk3-nocsd.so.0 => /usr/lib/x86_64-linux-gnu/libgtk3-nocsd.so.0 (0x00007f381c337000)
libapt-pkg.so.6.0 => /usr/lib/x86_64-linux-gnu/libapt-pkg.so.6.0 (0x00007f381bc05000)
libncursesw.so.6 => /lib/x86_64-linux-gnu/libncursesw.so.6 (0x00007f381c2fd000)
libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007f381c2cb000)
libsigc-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libsigc-2.0.so.0 (0x00007f381c2c2000)
libcwidget.so.4 => /usr/lib/x86_64-linux-gnu/libcwidget.so.4 (0x00007f381bb13000)
libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f381b9b4000)
libboost_iostreams.so.1.74.0 => /usr/lib/x86_64-linux-gnu/libboost_iostreams.so.1.74.0 (0x00007f381c2a8000)
libxapian.so.30 => /usr/lib/x86_64-linux-gnu/libxapian.so.30 (0x00007f381b600000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f381c2a3000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f381b200000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f381b8d5000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f381c281000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f381b41f000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f381c27c000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f381c25d000)
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f381c24a000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f381c219000)
liblz4.so.1 => /usr/lib/x86_64-linux-gnu/liblz4.so.1 (0x00007f381b8af000)
libzstd.so.1 => /usr/lib/x86_64-linux-gnu/libzstd.so.1 (0x00007f381b144000)
libudev.so.1 => /usr/lib/x86_64-linux-gnu/libudev.so.1 (0x00007f381b882000)
libsystemd.so.0 => /usr/lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007f381b075000)
libgcrypt.so.20 => /usr/lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007f381af2e000)
libxxhash.so.0 => /usr/lib/x86_64-linux-gnu/libxxhash.so.0 (0x00007f381b86b000)
/lib64/ld-linux-x86-64.so.2 (0x00007f381c364000)
libuuid.so.1 => /usr/lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f381b861000)
libcap.so.2 => /lib/x86_64-linux-gnu/libcap.so.2 (0x00007f381b855000)
libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f381af06000)

-- System Information:
Debian Release: 12.0
APT prefers unstable-debug
APT policy: (500, 'unstable-debug'), (500, 'stable-updates'), (500, 'stable-security'), (500, 'unstable'), (500, 'testing'), (500, 'stable'), (1, 'experimental')
merged-usr: no
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 6.1.0-6-amd64 (SMP w/12 CPU threads; PREEMPT)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=POSIX, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages aptitude depends on:
ii aptitude-common 0.8.13-5
ii libapt-pkg6.0 2.6.0
ii libboost-iostreams1.74.0 1.74.0+ds1-20
ii libc6 2.36-8
ii libcwidget4 0.5.18-6
ii libgcc-s1 12.2.0-14
ii libncursesw6 6.4-2
ii libsigc++-2.0-0v5 2.12.0-1
ii libsqlite3-0 3.40.1-1
ii libstdc++6 12.2.0-14
ii libtinfo6 6.4-2
ii libxapian30 1.4.22-1

Versions of packages aptitude recommends:
ii libdpkg-perl 1.21.21
ii sensible-utils 0.0.17+nmu1

Versions of packages aptitude suggests:
pn apt-xapian-index <none>
ii aptitude-doc-en [aptitude-doc] 0.8.13-5
pn debtags <none>
ii tasksel 3.72

-- no debconf information

--
Vincent Lefèvre <vin...@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

[Axel Beckert]

Control: clone -1 -2
Control: retitle -2 aptitude: "aptitude -u" does not show download warnings emitted by "aptitude update" after the successful download of package lists.
Control: clone -1 -3
Control: retitle -3 aptitude: "aptitude update" does not show all NOTICE level messages if apt would change the default messages threshold to NOTICE
Control: severity -3 minor
Control: submitter -3 !
Control: clone -1 -4
Control: retitle -4 aptitude: apt's message threshold level should be configurable in aptitude (especially for the TUI), maybe via --log-level?
Control: severity -4 wishlist
Control: submitter -4 !
Control: severity -1 minor
Control: tag -1 + wontfix

Hi Vincent,

thanks for making us aware of the subtle difference between apt and
aptitude with regards to warnings and notices.

> This message should have been displayed by aptitude.

Did you test "aptitude update" or "aptitude -u" or both? It's not
clear from your bug report so far. (They behave differently in this
hindsight as I found out while digging into this topic, see below.)

> Without it, the user who uses aptitude only may never be aware of

> this change […]

This is wrong. Without doubt this type of information will be part of
the release notes — which is usually the _primary_ source for these
type of changes. I'm actually surprised that apt added such a message.

Besides that, the apt developers decided to output this only as LOW
severity message on the "notice" level (prefix by "N:" and in normal
font face; AFAIK the lowest level).

Aptitude shows "error" messages from apt to the user, but only those
at the "WARNING" level or above (or at least only from somewhere above
the "NOTICE" level). It is currently unclear to me if this is
historically grown as apt (not aptitude) defines the _default_ message
threshold by apt in /usr/include/apt-pkg/error.h as "WARNING" in at
least three places. I at least currently assume that this is what
aptitude uses when it calls DumpErrors. According the apt's changelog,
"WARNING" was for quite some time also the lowest possible message
level. ("NOTICE" and "DEBUG" levels got added in 0.7.26~exp8.)

I also do see some sense to suppress some less important messages in
aptitude: All those messages would cause popup window in the TUI mode
which the user has to acknowledge (aka "to click away"). That can get
very annoying if there are too many low severity messages. So IMHO
it's not so bad that aptitude only shows more important messages.
Besides, it's the default threshold from apt.

Then again, "aptitude -u" does not show any of the warnings (or
notices) I currently get with "aptitude update", so "aptitude -u" only
seems to cause popups of warnings which are caused directly during the
download and not after a successful download. Cloning this as a
separate bug report.

Anyway, back to the original topic. So I edited
/usr/include/apt-pkg/error.h temporarily and replaced all but the
first occurence (which is the definition) of WARNING with NOTICE and
then compiled aptitude against it. aptitude update indeed
showed notices now. But to my surprise not all of them, just this one:

N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'file:/var/cache/apt-build/repository apt-build InRelease' doesn't support architecture 'i386'

Not yet sure if this is a bug in the way how the missing notices are
generated in apt or if it is a bug in aptitude not coping with the
changed default message threshold. Likely needs deeper investigation
(or better overview of the code), probably by Manuel. (I already put
hours in investigating this IMHO rather minor issue, just to
understand a bit how the innards of messages inside apt and aptitude
are working.) Filing as yet another separate bug report against
aptitude (for now at least), but with severity minor, as it currently
does only show up after a hypothetical modification in apt.

I'm also surprised that, despite the default threshold in
/usr/include/apt-pkg/error.h is WARNING, both, "apt update" and
"apt-get update" show notices. So the apt developers changed the
thresholds for both CLI frontends individually? Weird.

And JFTR: "aptitude update" does show the WARNING level messages from
apt about "Key is stored in legacy trusted.gpg keyring
(/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for
details." which apt also shows in a bold font face compared to
seemingly not important messages like these about the
non-free-firmware repo. (Not "aptitude -u", though.)

Admittedly the message level threshold of aptitude should be user
configurable so that every user/admin can adjust their own TUI popup
annoyance level.

Actually I would have expected to see those notices on
--log-level=info, but I didn't and from the code it also seems that
this option has no influence on how aptitude uses apt's DumpErrors().

Anyway, despite I uncovered some more severe issues while trying to
understand how apt's and aptitude's message handling works internally,
I strongly disagree about the severity of the initial bug report (at
least those parts which were clear enough) and agree with the apt
developers that the message about the Debian bookworm change
concerning non-free-firmware is only a notice, not a warning.

And I don't expect that Manuel (and surely not me myself) will change
something in the way aptitude handles messages from apt for bookworm
as it likely will be way more invasive than is good at this stage of
the freeze. Hence tagging it as wontfix, too.

Regards, Axel
--
,''`. | Axel Beckert <a...@debian.org>, https://people.debian.org/~abe/
: :' : | Debian Developer, ftp.ch.debian.org Admin
`. `' | 4096R: 2517 B724 C5F6 CA99 5329 6E61 2FF9 CD59 6126 16B5
`- | 1024D: F067 EA27 26B9 C3FC 1486 202E C09E 1D89 9593 0EDE

[Vincent Lefevre]

On 2023-03-11 19:16:52 +0100, Axel Beckert wrote:
> > This message should have been displayed by aptitude.
>
> Did you test "aptitude update" or "aptitude -u" or both?

I've tried 'u' from the aptitude TUI, "aptitude update" and
"aptitude -u". No messages.

> > Without it, the user who uses aptitude only may never be aware of
> > this change […]
>
> This is wrong. Without doubt this type of information will be part of
> the release notes — which is usually the _primary_ source for these
> type of changes. I'm actually surprised that apt added such a message.

A sid user is not concerned by the stable releases, thus is not
supposed to read the release notes (and there would potentially
much duplicate information with past changes). The announces of
the main changes for sid (and testing) are normally done via the
NEWS.Debian files, and nothing has been announced there about
non-free-firmware.

> Besides that, the apt developers decided to output this only as LOW
> severity message on the "notice" level (prefix by "N:" and in normal
> font face; AFAIK the lowest level).
>
> Aptitude shows "error" messages from apt to the user, but only those
> at the "WARNING" level or above (or at least only from somewhere above
> the "NOTICE" level). It is currently unclear to me if this is
> historically grown as apt (not aptitude) defines the _default_ message
> threshold by apt in /usr/include/apt-pkg/error.h as "WARNING" in at
> least three places. I at least currently assume that this is what
> aptitude uses when it calls DumpErrors. According the apt's changelog,
> "WARNING" was for quite some time also the lowest possible message
> level. ("NOTICE" and "DEBUG" levels got added in 0.7.26~exp8.)
>
> I also do see some sense to suppress some less important messages in
> aptitude: All those messages would cause popup window in the TUI mode
> which the user has to acknowledge (aka "to click away"). That can get
> very annoying if there are too many low severity messages. So IMHO
> it's not so bad that aptitude only shows more important messages.

But do "NOTICE" messages occur often?

> Besides, it's the default threshold from apt.

What do you mean? There doesn't seem to be any default in
the "apt-config dump" output. So I get "NOTICE" messages
from apt by default.

> Then again, "aptitude -u" does not show any of the warnings (or
> notices) I currently get with "aptitude update", so "aptitude -u" only
> seems to cause popups of warnings which are caused directly during the
> download and not after a successful download. Cloning this as a
> separate bug report.
>
> Anyway, back to the original topic. So I edited
> /usr/include/apt-pkg/error.h temporarily and replaced all but the
> first occurence (which is the definition) of WARNING with NOTICE and
> then compiled aptitude against it. aptitude update indeed
> showed notices now. But to my surprise not all of them, just this one:
>
> N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'file:/var/cache/apt-build/repository apt-build InRelease' doesn't support architecture 'i386'
>
> Not yet sure if this is a bug in the way how the missing notices are
> generated in apt or if it is a bug in aptitude not coping with the
> changed default message threshold. Likely needs deeper investigation
> (or better overview of the code), probably by Manuel. (I already put
> hours in investigating this IMHO rather minor issue, just to
> understand a bit how the innards of messages inside apt and aptitude
> are working.) Filing as yet another separate bug report against
> aptitude (for now at least), but with severity minor, as it currently
> does only show up after a hypothetical modification in apt.

[...]

FYI, the implementation is apt seems to have been done by this commit:

https://salsa.debian.org/apt-team/apt/-/commit/9712edf6151308148518058bfbd5ccd937509143

[David Kalnischkies]

On Sat, Mar 11, 2023 at 07:16:52PM +0100, Axel Beckert wrote:
> > Without it, the user who uses aptitude only may never be aware of
> > this change […]
>
> This is wrong. Without doubt this type of information will be part of
> the release notes — which is usually the _primary_ source for these
> type of changes. I'm actually surprised that apt added such a message.

The message is primarily for people who have missed the release notes
or any other form of announcement – which in the end is their fault,
using sid isn't a hall pass for not actively interacting with Debian.

I added it specifically to avoid at least a bit the flood of bugreports
against apt we had after the security archive changed layout last
release because "apt is throwing errors". Okay, this change isn't
throwing any errors, so we would probably be mostly fine… so that is
less "announcing a change" and more a "needless support and bugreport
prevention attempt". That is also why the message sounds a bit strange –
I am reusing existing message templates as I really didn't want to
bother translators with a message which in an ideal world nobody sees.

If I get the memo before a release, I can add such messages even if it
is strictly a layer violation out of pure self-interest…
while writing this code I was actually retiring code which detected
sources.list entries referring to Debian mirrors via ftp:// and pointing
to a Debian News entry announcing the decommissioning of ftp access.

Another example of a notice from apt is if you have a sources.list entry
like "http://user:pass...@example.org/". In that case we are
advertising apt_auth.conf(5) usage.


You can find me rambling about n-f-f transition some more e.g. in:
https://lists.debian.org/debian-devel/2022/10/msg00217.html
https://salsa.debian.org/apt-team/apt/-/merge_requests/282
https://bugs.debian.org/1030189#42
But the gist is what I said here already as well: It isn't really apts
job to announce such changes, but as nothing else really does it and
apt is the thing affected and used by users, I implemented it anyhow
in the least annoying and cost-efficient way (for me) last minute as
nobody else showed any interest in another possible solution.

> Besides that, the apt developers decided to output this only as LOW
> severity message on the "notice" level (prefix by "N:" and in normal
> font face; AFAIK the lowest level).

We have sorta a lot of messages at the Notice level nowadays as not
everything we want to bring to the attention of a user is a Warning –
and Notices are shown only on interactive usage, not in logs or anywhere
where they could confuse scripts and such. So, some Notices even
accompany Warnings/Errors to explain them further as changing the
initial message is hard(er) as it would break other stuff relying on it…

> Anyway, back to the original topic. So I edited
> /usr/include/apt-pkg/error.h temporarily and replaced all but the
> first occurence (which is the definition) of WARNING with NOTICE and
> then compiled aptitude against it. aptitude update indeed
> showed notices now. But to my surprise not all of them, just this one:
>
> N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'file:/var/cache/apt-build/repository apt-build InRelease' doesn't support architecture 'i386'
>
> Not yet sure if this is a bug in the way how the missing notices are
> generated in apt or if it is a bug in aptitude not coping with the
> changed default message threshold. Likely needs deeper investigation

It is actually (in a way) a small bug in your personal configuration. :)

libapt is trying to tell you that you have a Multi-Arch system (I guess
amd64 and i386 configured), but this repository does not support the
i386 architecture – which might be completely harmless or it could be
a problem later on if you try to install stuff from it, depending on
what you install from it (imagine trying to install a newer version of
a library you have installed from both arches, but now only amd64 is
available…).

You can silence this notice either by telling apt which architectures
you wanna get from that repository (depending on how you like things,
something like [arch=amd64] or [arch-=i386] would work I guess) OR
[preferably] the repository declares support for i386: That would
automatically happen if you would build & ship packages for that
architecture, but you can also just provide an empty Packages file or no
Packages file at all and just add i386 to the Architectures field in the
Release file. That tells apt that the repository owner cares for users
with that architecture, so presumably a user wont run into issues – or
at least will have a better experience than "closed, unsupported, go
away" to bug reports if something goes wrong.

While this is technically predicting a possible problem in the future,
it is so unspecific that annoying users with a warning wouldn't be right.

> I'm also surprised that, despite the default threshold in
> /usr/include/apt-pkg/error.h is WARNING, both, "apt update" and
> "apt-get update" show notices. So the apt developers changed the
> thresholds for both CLI frontends individually? Weird.

Well, imagine we would have changed them for ALL libapt users… given
that you speak about popups and annoyance levels in your own mail…
What should unattended-upgrades do with those notices? What should
a software-center-like thing do with them?

So until we have better control over messages, so that a frontend can
actually more programmatically decide which to show, which to hide and
how to display them, I decided a while ago to bundle them into a new
level and hide it away by default. Similar to how some front ends decide
to not even bother with warnings (and some not even with errors, but
that is a different story).

> least those parts which were clear enough) and agree with the apt
> developers that the message about the Debian bookworm change
> concerning non-free-firmware is only a notice, not a warning.

On a not unimportant side note: This notice (as well as my explicit
examples ftp & auth.conf earlier) is apt{,-get} specific, so even
if aptitude would show notices, it wouldn't show this one as the code
generating them is not in libapt, mostly because it can not really be
there. The update-code shared in libapt is only concerned with updating
the files, but which sources.list files are (not) involved and if and
how the caches are (re)build is an individual application decision. The
notice generation needs to know this through, so we would need a new API
for that… and if that new API is just generation messages, shouldn't it
be better at generating them. I mean, giving you a couple lengthy
strings to display as-is to the user? Shouldn't the URI be clickable?
And why isn't the package name? Oh, and the filename of the sources.list
file? Heh, how about opening it… (see the last paragraph) WAY to much
design work for something I initially didn't even want to implement in
apt to begin with.

I hope that clear this sorta misunderstanding up at least a bit.


Best regards

David Kalnischkies