Single broken package blocks whole package management

[Urs Thuermann]

After an dist-upgrade from Raspian 8 (jessie) to 9.13 (stretch)
hundreds of packages still need to be upgraded and aptitude reports
numerous conflicts.

I first wanted to upgrade everything which doesn't causes any
conflicts, which fails because of problems in wolfram-engine:

root@uranus:~# aptitude safe-upgrade --no-new-installs -RZ
Resolving dependencies...
open: 2242; closed: 1906; defer: 121; conflict: 311
The following packages will be upgraded:
acl <+36.9 kB> adduser <-222 kB> alacarte <-20.5 kB>
alsa-utils <+198 kB> aspell-en <+95.2 kB> avahi-daemon <+53.2 kB>
[...]
xserver-common <+53.2 kB> xserver-xorg <+1024 B>
xserver-xorg-input-all <+1024 B> xterm <+163 kB>
The following partially installed packages will be configured:
debconf-utils
The following packages are RECOMMENDED but will NOT be installed:
at-spi2-core bsd-mailx cinnamon dunst exfat-utils fonts-piboto
gdisk gnome-accessibility-themes gnome-flashback gnome-screenshot
[...]
xfce4-notifyd xfonts-base xserver-xorg-input-wacom
xserver-xorg-legacy
663 packages upgraded, 0 newly installed, 0 to remove and 344 not upgraded.
Need to get 0 B/283 MB of archives. After unpacking 89.8 MB will be used.
Do you want to continue? [Y/n/?]
Reading changelogs... Done
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/^(.*?)(\\)?\${ <-- HERE ([^{}]+)}(.*)$/ at /usr/share/perl5/Debconf/Question.pm line 72.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/\${ <-- HERE ([^}]+)}/ at /usr/share/perl5/Debconf/Config.pm line 30.
Extracting templates from packages: 100%
Preconfiguring packages ...
dpkg: unrecoverable fatal error, aborting:
files list file for package 'wolfram-engine' contains empty filename
E: Sub-process /usr/bin/dpkg returned an error code (2)
Failed to perform requested operation on package. Trying to recover:

root@uranus:~#

I tried to remove packages wolfram-engine and wolframscript, also
tried to remove debconf-utils, but everything fails with the same
error message:

root@uranus:~# dpkg --force-all -P wolfram-engine
dpkg: unrecoverable fatal error, aborting:
files list file for package 'wolfram-engine' contains empty filename

Also, upgrading a single package that's completely unrelated, is not
possible:

root@uranus:~# aptitude install acl
The following packages will be upgraded:
acl libacl1
The following partially installed packages will be configured:
debconf-utils{b}
2 packages upgraded, 0 newly installed, 0 to remove and 1005 not upgraded.
Need to get 0 B/80.7 kB of archives. After unpacking 49.2 kB will be used.
The following packages have unmet dependencies:
debconf-utils : Depends: debconf (= 1.5.61) but 1.5.56+deb8u1 is installed and it is kept back.
The following actions will resolve these dependencies:

Remove the following packages:
1) debconf-utils



Accept this solution? [Y/n/q/?]
The following packages will be REMOVED:
debconf-utils{a}
The following packages will be upgraded:
acl libacl1
2 packages upgraded, 0 newly installed, 1 to remove and 1005 not upgraded.
Need to get 0 B/80.7 kB of archives. After unpacking 58.4 kB will be freed.
Do you want to continue? [Y/n/?]
Reading changelogs... Done
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/^(.*?)(\\)?\${ <-- HERE ([^{}]+)}(.*)$/ at /usr/share/perl5/Debconf/Question.pm line 72.
Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/\${ <-- HERE ([^}]+)}/ at /usr/share/perl5/Debconf/Config.pm line 30.
dpkg: unrecoverable fatal error, aborting:
files list file for package 'wolfram-engine' contains empty filename
E: Sub-process /usr/bin/dpkg returned an error code (2)
Failed to perform requested operation on package. Trying to recover:

root@uranus:~#

What else can I do to get the package management working again?

urs

[Richard Hector]

On 6/01/22 02:32, Urs Thuermann wrote:
> After an dist-upgrade from Raspian 8 (jessie) to 9.13 (stretch)
> hundreds of packages still need to be upgraded and aptitude reports
> numerous conflicts.

Firstly, the standard response is that Raspbian is not Debian :-) There
are differences which might be related to your problem.

> I first wanted to upgrade everything which doesn't causes any
> conflicts, which fails because of problems in wolfram-engine:

wolfram-engine appears not to be a debian package, for starters.

[snip]

> Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/^(.*?)(\\)?\${ <-- HERE ([^{}]+)}(.*)$/ at /usr/share/perl5/Debconf/Question.pm line 72.
> Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/\${ <-- HERE ([^}]+)}/ at /usr/share/perl5/Debconf/Config.pm line 30.

Looks bad, but deprecation isn't an error.

[snip]

> dpkg: unrecoverable fatal error, aborting:
> files list file for package 'wolfram-engine' contains empty filename
> E: Sub-process /usr/bin/dpkg returned an error code (2)
> Failed to perform requested operation on package. Trying to recover:

Ah. So perhaps a bug in that package. Or it's corrupted.

If you look at /var/lib/dpkg/info/wolfram-engine.list does it have an
empty line in it? What happens if you edit that out?

>
> root@uranus:~#
>
> I tried to remove packages wolfram-engine and wolframscript, also
> tried to remove debconf-utils, but everything fails with the same
> error message:
>
> root@uranus:~# dpkg --force-all -P wolfram-engine
> dpkg: unrecoverable fatal error, aborting:
> files list file for package 'wolfram-engine' contains empty filename

Same thing

Still comes back to the same thing.

>
> root@uranus:~#
>
> What else can I do to get the package management working again?

I'd try deleting any blank lines from that file, and trying again.

Or maybe apt-get install --reinstall wolfram-engine

Or ask on a Raspbian list :-)

A related question here:

https://unix.stackexchange.com/questions/425355/x11-common-contains-empty-filename

Cheers,
Richard

[Andrew M.A. Cater]

On Thu, Jan 06, 2022 at 03:20:38AM +1300, Richard Hector wrote:
> On 6/01/22 02:32, Urs Thuermann wrote:
> > After an dist-upgrade from Raspian 8 (jessie) to 9.13 (stretch)
> > hundreds of packages still need to be upgraded and aptitude reports
> > numerous conflicts.
>
> Firstly, the standard response is that Raspbian is not Debian :-) There are
> differences which might be related to your problem.
>

Raspbian _isn't_ Debian. Wolfram-engine is a third party (commercial) app -
Wolfram Mathematica which the Raspberry Pi foundation licences with a
special educational arrangement.

You might be able to force a reinstall of wolfram-engine to produce
something consistent to then remove but this is probably wasted
effort.


Rather than trying to update from 8 to 9, unless you have significant
data invested, I would suggest simply downloading a copy of the latest
release of Raspberry Pi OS based on Debian 11.

Although I can guarantee that Debian should be upgradable between major
releases, I can't be sure for other Debian derivatives.

As ever, for Debian: https://wiki.debian.org/DontBreakDebian

Debian installers which will install something closer to vanilla Debian
are available - but don't include Wolfram's software. They're unofficial
in that they're dependnet on non-free software to install. See
https://cdimage.debian.org/cdimage/unofficial/raspi/

Quickest solution: install Raspberry Pi OS latest version - available
from https://www.raspberrypi.com/software/

With every good wish, as ever,

Andy Cater

[David Wright]

Debian buster here.

$ ls -l /usr/share/perl5/Debconf/Question.pm
-rw-r--r-- 1 root root 5867 Oct 1 04:39 /usr/share/perl5/Debconf/Question.pm
$

$ grep -A1 -B1 '(\\\\)' /usr/share/perl5/Debconf/Question.pm
my $escape;
while ($rest =~ m/^(.*?)(\\)?\$\{([^{}]+)\}(.*)$/sg) {
$result.=$1; # copy anything before the variable
$

So edit /usr/share/perl5/Debconf/Question.pm and insert a \ before
the {, just as the message indicates.

Cheers,
David.

[Urs Thuermann]

Richard Hector <ric...@walnut.gen.nz> writes:

> Firstly, the standard response is that Raspbian is not Debian :-)
> There are differences which might be related to your problem.

Yes, of course. I know. But my question was not about the Raspbian
specific packages but on apt, dpkg, aptitude & co. I assume, these
work mostly (or exactly) like in Debian.

> > I first wanted to upgrade everything which doesn't causes any
> > conflicts, which fails because of problems in wolfram-engine:
>
> wolfram-engine appears not to be a debian package, for starters.
>

> [...]

>
> Ah. So perhaps a bug in that package. Or it's corrupted.
>
> If you look at /var/lib/dpkg/info/wolfram-engine.list does it have an
> empty line in it? What happens if you edit that out?

OK, thanks for that hint. This file indeed not only contained a blank
line, but it seems to be completely corrupted. Roughly 5-10% of all
bytes in random positions are replaced with random (some ASCII, some
non-ASCII) values, making *lots* corrupted path names, which I can
still recognize/guess, but corrupted. And one line had its last
character replaced by a new-line, thus making a blank line. A quick
check

# cd /var/lib/dpkg/info; file *.list | grep -v ASCII
ca-certificates.list: UTF-8 Unicode text
libmodule-build-perl.list: empty
python-apt-common.list: data
wolfram-engine.list: data

showed another corrupted file. python-apt-common.list has the same
random errors. Just curious, I wanted to know, whether all other
files contain valid and existing path names:

# for f in *list; do echo $f; cat $f | tr '\n' '\0' |
xargs -0 ls -d >/dev/null ; done 2>&1 | less

Many package list files contain path names which seem valid but are
non-existent in the file system. Is that normal? On another
Debian stretch/amd64 system with 1868 packages installed I got only 3
packages each containing only 1 non-existent path name.

> I'd try deleting any blank lines from that file, and trying again.

Yes, I will try that. Just to learn if it will fix things. I expect
many errors because of the many broken path names which cannot be
removed.

However, since this system looks somewhat suspicious already and I
wanted to go to buster (or maybe bullseye) anyway, I will save /home
and relevant stuff from /var and then reinstall from scratch. After
testing the SD card. Seems to be easier.

> Or maybe apt-get install --reinstall wolfram-engine

It all leads to the same error message as above.
I don't need Wolfram anyway, it's been installed by default.

> Or ask on a Raspbian list :-)

Hm, probably I wasn't clear enough. My question was not about
Raspbian or the wolfram package, but about dpkg and friends. Isn't
there a better way to remove packages if the .list file corrupted?
And isn't there a way to temporarily ignore such errors and install or
upgrade other packages despite such an error? Even dpkg --force-all
didn't help so I couldn't do aynthing. I found it surpring that a
single broken package leaves you in a state where you're stuck and
cannot go out, easily.

> https://unix.stackexchange.com/questions/425355/x11-common-contains-empty-filename

Looks very similar to my problem.

urs

[Urs Thuermann]

"Andrew M.A. Cater" <amac...@einval.com> writes:

> Raspbian _isn't_ Debian. Wolfram-engine is a third party (commercial) app -
> Wolfram Mathematica which the Raspberry Pi foundation licences with a
> special educational arrangement.
>
> You might be able to force a reinstall of wolfram-engine to produce
> something consistent to then remove but this is probably wasted
> effort.

Yes, and my question should be about dpkg & friends how to get out of
that state where a single corrupted package blocks everything.

> Rather than trying to update from 8 to 9, unless you have significant
> data invested, I would suggest simply downloading a copy of the latest
> release of Raspberry Pi OS based on Debian 11.
>
> Although I can guarantee that Debian should be upgradable between major
> releases, I can't be sure for other Debian derivatives.

I hoped for the chain of upgrades to work since that would be easier
than installing from scratch. Since that Raspberry Pi is head-less in
the room next-door. I need to pull the SD card, write the image onto
it, put the Raspi to some screen at my desk, find a keyboard etc. Not
much of a problem, but dist-upgrading through remote-login would be
less effort. Now, I'll do a installation from scratch.

> As ever, for Debian: https://wiki.debian.org/DontBreakDebian

Yes, sure. AFAIK, nothing of that sort has been done to that Raspi,
at least, as long as I was the only root on it. I gave to my son a
year ago or so, but I think he didn't anything wrong to it.

> Debian installers which will install something closer to vanilla Debian
> are available - but don't include Wolfram's software. They're unofficial
> in that they're dependnet on non-free software to install. See
> https://cdimage.debian.org/cdimage/unofficial/raspi/

I think I'll give it a try. On one of my old Raspis idling in my home
network or lying around and gathering dust. Especially the 64 bit
Debian should be interesting on my Raspberry Pi 3. So thank for that
pointer and suggestion.

urs

[Dan Ritter]

Urs Thuermann wrote:
> "Andrew M.A. Cater" <amac...@einval.com> writes:
>
> > Raspbian _isn't_ Debian. Wolfram-engine is a third party (commercial) app -
> > Wolfram Mathematica which the Raspberry Pi foundation licences with a
> > special educational arrangement.
> >
> > You might be able to force a reinstall of wolfram-engine to produce
> > something consistent to then remove but this is probably wasted
> > effort.
>
> Yes, and my question should be about dpkg & friends how to get out of
> that state where a single corrupted package blocks everything.
>

Here's the dangerous way:

In /var/lib/dpkg/info, dpkg and therefore apt keep the install
and removal scripts for all packages.

You could download a copy of the wolfram-engine .deb manually,
use ar to pull out a copy of the package list, confirm that it
is unmunged and replace the broken one in /var/lib/dpkg/info.

Then remove it, do your upgrades, and try reinstalling it.

-dsr-

[Greg Wooledge]

On Wed, Jan 05, 2022 at 05:26:21PM +0100, Urs Thuermann wrote:
> Yes, and my question should be about dpkg & friends how to get out of
> that state where a single corrupted package blocks everything.

Use dpkg to purge it. With force if needed.

dpkg --force-help