upgrade bookworm->trixie

[Graham Eddy]

i just did an in situ upgrade from bookworm to trixie (edit sources.list, update, dist-upgrade) and hit a snag:

my old ~weewx/venv had been effectively wiped, so i had to reinstall weewx5 (i use pip method) plus my own private modules. it also needed module ’six’ reinstalled, which i thought would have been picked up in dependencies but…

smooth sailing apart from that.

cheers

⊣GE⊢

[vince]

You're going to have to describe with 'effectively wiped' means. 

A weewx venv installation is not consistent in where it pins things to a specific python version vs. where it relies on the os default python version (perhaps a bug). Long answer follows, but there might be a quick fix...

Background:

• bookworm uses python3.11 so your original venv expected python3.11 for everything under the hood, as well as its systemd service files
•  Unfortunately bookworm => trixie changes your system python version from 3.11 to 3.13
• so you've created a mismatch between the os python (new) and the weewx configuration that still expects the older 3.11 version of python
• and weewx being inconsistent in where it relies on the os python version is what you are likely running into

Looking in weewx-venv/bin:

lrwxrwxrwx 1 root root    7 Sep  1 17:14 python -> python3
lrwxrwxrwx 1 root root   16 Sep  1 17:14 python3 -> /usr/bin/python3
lrwxrwxrwx 1 root root    7 Sep  1 17:14 python3.11 -> python3

But the /etc/systemd/system weewx service files (there are three) contain:

ExecStart=/home/pi/weewx-venv/bin/python3 \

    /home/pi/weewx-venv/lib/python3.11/site-packages/weewxd.py \

    /home/pi/weewx-data/weewx.conf

So when you walk the symlinks the ExecStart is using the 'os' python version not the 'venv' python version.  That (to me) is a bug.

• weewx-venv/bin/python3 points to /usr/bin/python3 (the os version)
• when it should point to /usr/bin/python3.11 (the specific version the venv was created under)

A quick fix is to change one symlink to make things consistently point to the original 3.11 python still present (but not the default) of your os:

• deactivate the venv
• remove the weewx-bin/python3 symlink
• add a new symlink pointing it to /usr/bin/python3.11
• activate the venv
• run "python3 --version" and you'll see it points to the original 3.11
• so everything will be consistently pointing to the original 3.11 version

I'm thinking this might be a weewx bug - the symlink in the venv should point to the specific python version the venv was created in, once you read through all the symlinks.   If apps are going to set up a venv to be immune from the os default python version changing, they need to do so consistently to effectively pin themselves end-to-end to a specific version of python.

[vince]

Graham - in experimenting a little, I think this isn't necessarily a weewx bug but perhaps a minor documentation tweak might help.

• If you create the venv ala "python3 -m venv weewx_venv" then you are using the os python version, which can float upward if you update the os
  
• If you create the venv ala "python3.11 -m venv weewx_venv" then you are hard pinning it to a particular version of python that is stable.

In the latter example, the weewx-venv/bin symlinks look much better:

lrwxrwxrwx 1 vagrant vagrant   10 Sep  1 18:52 python -> python3.11
lrwxrwxrwx 1 vagrant vagrant   10 Sep  1 18:52 python3 -> python3.11
lrwxrwxrwx 1 vagrant vagrant   19 Sep  1 18:52 python3.11 -> /usr/bin/python3.11

And the service files also look better after running setup-daemon.sh:

ExecStart=/home/vagrant/weewx-venv/bin/python3.11 /home/vagrant/weewx-venv/lib/python3.11/site-packages/weewxd.py /home/vagrant/weewx-data/weewx.conf

So I have to retract my guess that this is a weewx 'code' bug but perhaps a minor docs tweak might help venv installations be a little more upgrade safe.

Sorry for any confusion previously.  FWIW in the original case I replied with earlier, I still think that one symlink change should have gotten the pre-existing condition to better pin to a specific python version.  The procedural change above here is perhaps what a new installation might better use to be os upgrade safe.

[Graham Eddy]

it is neither bug nor specific to weewx, but it applied to all my python venvs on multiple machines.

just something i think folk should be aware of, when considering/planning in situ upgrade to trixie, that

 (a) i found the process quite painless on weewx machine (and on multiple machines nothing to do with weewx)

 (b) rebuild your venv afterwards (and there might be other things if you have other apps/dependencies)

cheers

⊣GE⊢

[gary....@gmail.com]

Happily I'm waiting on mosquitto repo to be updated for trixie.

When that is in place, what is the verdict here? Am I going to lose my install?
I did a venv using the command from the docs, I believe it is the plain python3 version, not a pinned python3 version.

Good/not good?

It isn't clear from this thread, to me at least.

[vince]

My testing looked ok.  Tom's testing looked not so ok.  So it's uncertain.

I think Tom has stated his recommendation is to always rebuild your venv if the python version changes, as it does for bookworm => trixie.

There isn't an explicit 'how to rebuild your venv' in the docs that I can find, but one way (untested) might be:

• copy your weewx-venv and weewx-data someplace just-in-case
• move weewx-venv aside by renaming it
•  do the 'install in a virtual environment' and 'run as a daemon' steps from the pip quickstart instructions
  
• if you have any other python modules previously installed in your existing venv you'd have to of course install them too

If your weewx-data tree is unchanged (it should be fine), you'd be done.

FWIW - I never do os upgrades either if they can be avoided.  I always stash the data I care about (weewx-data) offline and do a full wipe+install of the new os.  On a raspi of course this is as easy as swapping in a new SD card and then ssh'ing the old weewx-data tree onto it once I did the new installation procedure etc. on the new SD card.

[Chuck Rhode]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 3 Sep 2025 08:57:22 -0700 (PDT)
vince <vince...@gmail.com> wrote:

> I think Tom has stated his recommendation is to always rebuild your
> venv if the python version changes, as it does for bookworm =>
> trixie.

Why isn't swapping python versions just this simple:

> python3 -m venv --upgrade weewx-venv

- --
.. Be Seeing You,
.. Chuck Rhode, Sheboygan, WI, USA
.. Weather: https://LacusVeris.com/Wx
.. 62° — Wind W at 10 mph. Sky overcast.
-----BEGIN PGP SIGNATURE-----

iF0EARECAB0WIQT+MY/5I/LMPSswTbVg2/xipKOWUgUCaLhtLwAKCRBg2/xipKOW
Uk6GAJ9U2IdzEYu/rucdtM0uopVZ7W/NygCfZTla43Xe9R5HwAWQjupxemMufHw=
=pWGo
-----END PGP SIGNATURE-----

[vince]

That results in a python3.13/site-packages tree that is empty other than the pip module

[Pablo Sanchez]

A bit OT: I use a rolling distro[1] (Tumbleweed) and I frequently roll changes. The last time I updated the OS was nearly eight days ago. :)

References
1 - https://get.opensuse.org/tumbleweed/

---
pablo

[vince]

Chuck:

If you do a "pip3 freeze > ~/requirements.txt" to quantify your venv to a file before upgrading, then a "pip3 install -r ~/requirements.txt" after upgrading, you 'do' get all the same modules+versions installed into the new 3.13 venv subdirectories (tested), which isn't too bad.

Rerunning setup-daemon.sh afterward does 'not' alter the weewx service files unfortunately.  I guess a minimal sudo hand editing is needed there.

[Chuck Rhode]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 3 Sep 2025 09:55:07 -0700 (PDT)
vince <vince...@gmail.com> wrote:

> That results in a python3.13/site-packages tree that is empty other
> than the pip module

IC. They don't seem to 'splain that in the *python* docs.

- --
.. Be Seeing You,
.. Chuck Rhode, Sheboygan, WI, USA
.. Weather: https://LacusVeris.com/Wx

.. 66° — Wind W at 16 mph. Sky partly cloudy.
-----BEGIN PGP SIGNATURE-----

iF0EARECAB0WIQT+MY/5I/LMPSswTbVg2/xipKOWUgUCaLiKwwAKCRBg2/xipKOW
Uqm5AJ9Jt+ZruDgUyRHqhcET7J9ZOMkgDgCfekgt1uFcHq5Nah1OlWSIhOQXo9g=
=6FTN
-----END PGP SIGNATURE-----

[cric...@pobox.com]

I just love how python completely ignores backwards compatibility.