Heads up: Debian 13.6 python3.13 update (3.13.5-2+deb13u3) causes random weewxd segfaults

[John Kline]

If you run WeeWX on Debian 13 (trixie) and your system installed the

python3.13 update from the 13.6 point release (version 3.13.5-2+deb13u3,

rolling out via unattended-upgrades since July 11), be aware that it can

crash weewxd with a segmentation fault.

What it looks like

------------------

weewxd dies with SIGSEGV -- no Python traceback in the log, the process

just exits.  systemd shows "Main process exited, code=dumped,

status=11/SEGV".  If you look closely at the timing, the crash comes

milliseconds after WeeWX's periodic garbage collection pass (the

"Garbage collected N objects" INFO line, every 3 hours by default).

weewxd is unusually good at tripping this bug precisely because it calls

gc.collect() on that schedule while its other threads keep working.

I run seven WeeWX machines (Raspberry Pi 5, all on trixie).  Four crashes

in the ~48 hours after the update reached them; the crash takes down maybe

1 in 25 GC passes, so it looks random and can hit anywhere from hours to

days after a restart.  The crashing thread differs every time (sqlite

reads, report generation, image generation), so don't chase whatever

happens to be in your traceback if you have faulthandler enabled -- it's

the victim, not the cause.

The cause

---------

It is not WeeWX and not any extension.  Debian's deb13u3 update backported

a garbage-collector fix into Python 3.13.5 but omitted a second upstream

commit that the fix depends on, so a GC pass can free memory that is still

in use.  It's tracked as Debian bug #1141977, the missing commit has been

identified by bisection, and the maintainer is engaged, so I'd expect a

fixed package (deb13u4) reasonably soon:

  https://bugs.debian.org/1141977

Am I affected?

--------------

  dpkg -l python3.13 | grep deb13u3

Two things must both be true: the deb13u3 package is installed AND weewxd

was (re)started after it was installed.  A weewxd that has been running

since before the upgrade still uses the old library and is safe until its

next restart.

What to do

----------

1. If your weewxd predates the upgrade, simply don't restart it until the

   fixed package arrives.

2. Otherwise, make crashes self-healing while you wait.  A systemd

   drop-in (systemctl edit weewx) with:

     [Service]

     Restart=on-failure

     RestartSec=10

   turns each crash into ~15 seconds of downtime.  I also recommend

   Environment=PYTHONFAULTHANDLER=1 there -- it logs a full thread dump

   on any future native crash, which is invaluable for problems like

   this one.

3. Raising gc_interval in weewx.conf reduces exposure but does not

   eliminate it (Python's automatic collections use the same broken

   code path), so I wouldn't rely on it.

4. When the fixed python3.13 package arrives, remember that a running

   weewxd keeps the old library until it is restarted -- do a

   "systemctl restart weewx" after the upgrade to actually get off the

   broken version.

Downgrading to deb13u2 also works but re-opens the security fixes that

update carried; with the auto-restart in place I don't think it's worth

it.

John

[Tom Keffer]

Thanks, John, for your detailed report.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/92B83A5B-A968-4531-8DF2-223C1E9D1ED7%40johnkline.com.

[Greg from Oz]

So when does the new fix arrive?

I upgraded the python 2 days ago and weewx has been running for 4 days....I better not restart weewx.

At least if it doesn't start I will know what to do...Thanks for the heads up.

[John Kline]

I believe the maintainers have what they need to push a fix.  I’m hoping that will happen in a week or two.

>> At least if it doesn't start I will know what to do

It will start just fine.  The issue is, each time weewx requests garbage collection (which, by default, weewx does every 3 hours), there is a small chance of a segment violation soon after.  As such, the symptom will be that weewx stops running for no obvious reason (and nothing in the log after a garbage collection message).

It is hard to say how often this might happen for a single instance of weewx running.  The frequency may depend on what extensions you are running (and how many of them).  It is conceivable that some won’t see the issue at all before the fix arrives.

I was seeing the issue too often (as I run quite a few weewx instances);  so I resorted to downgrading the python packages and put them on hold while waiting for the update.

John

On Jul 15, 2026, at 2:53 AM, Greg from Oz <ubea...@gmail.com> wrote:



To view this discussion visit https://groups.google.com/d/msgid/weewx-user/ae5a994b-b4f0-4c6b-aa50-91baedd28a34n%40googlegroups.com.

[Greg from Oz]

apt-cache policy python3.13
python3.13:
  Installed: 3.13.5-2+deb13u3
  Candidate: 3.13.5-2+deb13u3
  Version table:
 *** 3.13.5-2+deb13u3 500
        500 http://deb.debian.org/debian trixie/main amd64 Packages
        100 /var/lib/dpkg/status

Looks like I cannot downgrade...I will just have to now restart the weewx.

I do have weewx set up in monit and if it does crash in theory it should restart again.

[John Kline]

Well, you can from a Debian snapshot, but I think if I listed instructions on how to do that, I dig myself quite a hole in having to support the process; and, I suspect, some will never undo putting the packages on hold, thus, never get a python upgrade again.

On Jul 15, 2026, at 4:05 PM, Greg from Oz <ubea...@gmail.com> wrote:

apt-cache policy python3.13

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/a03d6aaa-a55f-4120-83bd-d808c3eb80cfn%40googlegroups.com.

[John Kline]

Hi Greg,

I pinged the maintainer (Stefano) on the bug.  Stefano already has a test build of 3.13.5-2+deb13u4.  See his reply here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1141977 for a link to it (if you want to install it).  I have it running on 7 machines and will do the rest after the first weewx gc run in about 2 and 1/2 hours.

John

On Jul 15, 2026, at 6:48 PM, John Kline <jo...@johnkline.com> wrote:



[Vince Skahan]

I know John strongly disagrees but this is another reason why I do not do ‘any’ package updates on my LAN-only weewx system. I get it working and leave it alone.  Granted my risk profile permits this for me. I permit ‘no’ incoming traffic to my LAN that the weewx system is on,

Running in docker is another way to avoid these kinds of ‘sand shifted under your feet’ problems, making it easier to revert to a previously stable setup.

(but wow quite the elaborate investigation figuring out all the gory internals things for this case)

[Greg from Oz]

Thanks John. I have read all of that on the link you gave me but it makes no sense to me at all as I am no programmer. I just use the software. I guess I will wait for the fix to appear. 

[Greg from Oz]

If the system does restart what will happen to weewx? Will it still run for a while and then just stop?

[John Kline]

It will definitely restart and will run for a while.  How long is not clear.  Perhaps a few days,  Perhaps a week or longer.  Perhaps you won’t see it at all.

BTW, they were chasing another bug (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1142056):

> Some pipeline jobs failed, because python3.13 interpreter segfaults. This is
> a different issue, and it looks more like a bug in python3.13, not in sphinx.

Hi. This is probably the python3.13 bug:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1141977

and at this point I really wonder if there was a bug in sphinx at all
(in which case we would not need an upload for trixie at all).

That is when they bumped our bug from important to serious.  It is now marked as fixed, but I’m still waiting to hear when they will push it.

On Jul 17, 2026, at 4:25 PM, Greg from Oz <ubea...@gmail.com> wrote:

If the system does restart what will happen to weewx? Will it still run for a while and then just stop?

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/8a1f1ef1-6eaa-45e1-9f5a-4a4ce6b77e02n%40googlegroups.com.

[John Smith]

Looks like it's being distributed, because apticron just said there is new python packages available

python3.13 3.13.5-2+deb13u4
etc

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/87A5BB68-408C-47AD-A6AE-22D3C895153C%40johnkline.com.

[Greg from Oz]

Yes I am seeing it and installing it now... Thanks

[John Kline]

Good news: this is fixed.  Debian has released python3.13 3.13.5-2+deb13u4 ("Fix use-after-free in dict.clear() with embedded values.  Resolves a regression in 3.13.5-2+deb13u3").  It reached trixie-updates today.

‘sudo apt update && sudo apt upgrade’ gets you the fix.

The breakage showed up all over the archive -- the original report was SQLAlchemy/greenlet test suites segfaulting, and five packages simply stopped building because their build-time test suites crashed ("FTBFS" = fails to build from source):

  #1141977  python3.13: segfaults with sqlalchemy/greenlet (original)

  #1142056  python3.13: heap corruption (SIGSEGV in cyclic GC)

  #1142104  flask-migrate: FTBFS: test failure

  #1142105  neutron: FTBFS: test failure

  #1142106  python-oslo.cache: FTBFS: segmentation fault

  #1142107  qtwebengine-opensource-src: FTBFS: segmentation fault

  #1142110  tahoe-lafs: FTBFS: segmentation fault

They needed to get this out quickly—and they did.

John

On Jul 17, 2026, at 5:41 PM, John Kline <jo...@johnkline.com> wrote:



[Nate Bargmann]

Thanks, John, for the initial heads-up. I'd not upgraded my WeeWX
server to Debian 13.6 as the secure boot update failed on my main
desktop and I'd not gotten around to the update a week ago when your
message arrived. I did its update this morning after confirming apt was
upgrading Python to the 'u4' version.

Most appreciated.

- Nate

--
"The optimist proclaims that we live in the best of all
possible worlds. The pessimist fears this is true."
Web: https://www.n0nb.us
Projects: https://github.com/N0NB
GPG fingerprint: 82D6 4F6B 0E67 CD41 F689 BBA6 FB2C 5130 D55A 8819

[Greg from Oz]

Would the broken version would have affected my weewx if it is running in a venv environment?

cat pyvenv.cfg
home = /usr/bin
include-system-site-packages = false
version = 3.13.5
executable = /usr/bin/python3.13
command = /usr/bin/python3 -m venv /opt/weewx/weewx-venv

[John Kline]

If your machine updated to the bad python release and you restarted weewx, you would have been running it.

On Jul 20, 2026, at 11:10 PM, Greg from Oz <ubea...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/e0f4b2fd-aac3-49ce-9260-a3a39eb85ad2n%40googlegroups.com.