Bug#1035949: mariadb: upgrade issue: mariadb-server-10.5 fails to stop after all other -10.5 packages were removed,Date: Thu, 11 May 2023 16:46:43 +0200

[Frédéric Loyer]

Hi,

I have just upgraded Debian from 11 to 12, and its mariadb-server. The
command was simply "apt full-upgrade".

I have get the #1035949 bug even if I havn't the zoph package, nor
default-mysql-server.

I didn't manage to make it works with:

# apt --fix-broken install mariadb-client


However, I could continue the installation by killing the mariadb-server
process, then the "apt install mariadb-server" didn't have to stop the
server and can install successfully the mariadb-server.

[Otto Kekäläinen]

Thanks Frederick for the report!

Do you still have the output of apt? Can you copy-paste here or attach log to show exactly what happened?

It is likely that you hit this bug, but exact details would help confirm, and also help build CI test/scenario to ensure we have realistic upgrade testing.

[Otto Kekäläinen]

Can you attach the full log as an attachment?

The current short log does not show what apt told you in the beginning
about what it plans to do, nor can I see if mariadb-client-10.5 was
uninstalled or what happened.

But it did at least hit a bug where the uninstall fails on
`invoke-rd.c stop mariadb` failing.

Is your system running systemd? What can you tell about the environment?

[Otto Kekäläinen]

Hi!

I did a bunch of reproducible experiments using Salsa-CI in
https://salsa.debian.org/mariadb-team/mariadb-server/-/pipelines/536587
testing:

## upgrade to Bookworm

* cacti and Bullseye upgrade
- apt install -qq --yes cacti
-> - apt full-upgrade -qq --yes
* default-mysql-server and Bullseye upgrade
- apt install -qq --yes default-mysql-server
-> - apt full-upgrade -qq --yes
* mariadb-server and Bullseye upgrade
- apt install -qq --yes mariadb-server
* zoph and Bullseye upgrade
- apt install -qq --yes zoph
-> - apt full-upgrade -qq --yes
* zoph and Bullseye upgrade with mariadb-server explicitly
- apt install -qq --yes zoph mariadb-server
-> - apt install -qq --yes mariadb-server + apt full-upgrade -qq --yes
* zoph with mariadb-server and Bullseye upgrade
- apt install -qq --yes zoph mariadb-server
-> - apt full-upgrade -qq --yes

Only the last scenario failed.

This suggests that perhaps users hit this issue only when having
indirect dependency on default-mysql-server (via e.g. zoph), and
upgrading only it or doing a full-upgrade.


## Recommendation how to avoid this issue

I would recommend this as the best way as of today to update MariaDB
10.5.19/20 in Buster to 10.11.3 in Debian 12 "Bookworm":

# Ensure Bullseye was running latest of everything, makes upgrade smoother
$ sudo apt upgrade --yes
# Ensure clean and safe shutdown before doing major version upgrade -
this may take several minutes on large and busy database
$ sudo systemctl stop mariadb || sudo /etc/init.d/mariadb stop
# Enable new release
$ sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list
$ sudo apt update
# Just upgrade MariaDB first so it can be brought back online as fast
as possible
$ sudo apt install mariadb-server
# Then upgrade everything else
$ sudo apt full-upgrade


## Recommendation how to recover if suffered this issue


If you did not prepare along the lines of above and just upgraded, and
if failed with error message:

dpkg: mariadb-server-10.5: dependency problems, but removing anyway as
you requested:
zoph depends on default-mysql-server | virtual-mysql-server; however:
Package default-mysql-server is not configured yet.
Package virtual-mysql-server is not installed.
Package mariadb-server-10.5 which provides virtual-mysql-server is
to be removed.

(Reading database ... 16559 files and directories currently installed.)
Removing mariadb-server-10.5 (1:10.5.19-0+deb11u2) ...
Stopping MariaDB database server: mariadbd failed!
invoke-rc.d: initscript mariadb, action "stop" failed.
dpkg: error processing package mariadb-server-10.5 (--remove):
installed mariadb-server-10.5 package pre-removal script subprocess
returned error exit status 1
dpkg: too many errors, stopping
Errors were encountered while processing:
mariadb-server-10.5
Processing was halted because there were too many errors.
E: Sub-process /usr/bin/dpkg returned an error code (1)

..in that case the easiest way to recover is simply to manually stop
the server and continue upgrade:

$ sudo pkill -ef mariadbd || sudo pkill -ef mysqld
$ sudo apt --fix-broken install
$ sudo apt full-upgrade


Workaround suggested to be included in Bookworm release notes at
https://salsa.debian.org/ddp-team/release-notes/-/merge_requests/197