Attention MySQL/MariaDB Debian 13 Trixie users!

[John Smith]

After using weeWX for nearly a decade now, I came across a problem with weeWX that I can only describe as a time bomb waiting to go off, and it was finally realised after upgrading to Debian 13 Trixie.

What ever change was made to SystemD caused weeWX to no longer start after MariaDB, and error handling in weeWX seems very brittle and not fault tolerant at all. The outcome of this particular situation is weeWX exits after failing to connect to MariaDB, no waiting or retrying, it just exits and then needs to be manually started once you notice no updates happening.

I proposed a simple change to the SystemD service file to re-introduce the previous behaviour during boot up, but 2 commentators objected because they have a SQL DB installed, but aren't using it for weeWX and for whatever reason assumed the delay would be an inconvenience to them, all without any actual proof to back their claims.

Then there was further comments pushing things into the bizarre with all sorts of incredibly unlikely edge cases.

In any case, no one seems to have any proof that a slight boot up delay would be an inconvenience let alone detrimental but since the issue has been closed with the only resolution being a small documentation change among the bazillion other things in documentation it will be hard to come across unless you know what you are looking for.

I think this is an unacceptable solution for the vast majority of people with a SQL DB installed specifically for use with weeWX to find one day, like I did, that weeWX will now require manual intervention after their system boots.

I still don't understand how the wants of 2 users with a weird setup trumps the needs of all the users that will suffer detrimentally without such a delay.

The proposed changes will have no effect on systems that don't have a SQL DB installed such as using weeWX with the default SQLite back end, it will however have a major positive impact on those running weeWX with a SQL DB back end.

You can find my PR here: https://github.com/weewx/weewx/pull/1030

I ended up implementing a much saner solution to the problem, and those now, or in the future running SystemD version 257.8-1~deb13u2 or later might also be interested in my solution...

[Greg Troxel]

In Home Assistant, using pgsql, I have the HA start script just wait
until pgsql comes up. But that's my private script for my environment,
so that's ok. It might be that HA deals with this properly now.

Overall, I would think the right answer is that weewx should try to
connect to the db, and if that doesn't work, just retry every minute.
While not connected, it wouldn't do anything. And, if the db connection
closes, it would go back to this waiting for db mode.

Generally, I think programs that connect to other programs should have
this sort of logic so there is no need for synchronization. This is
really the only workable approach when services are on different
machines, and especially if the network to the server might have
issuees.

[John Smith]

Yes fault tolerant code would be definitely be the best solution, but until that happens a couple of extra lines in the SystemD service file is the simplest fix at this time.

You could add similar lines to the HA service file instead of a watch dog script, SystemD is coded very well to handle such situations.

[Tom Keffer]

John, slow down there, dude.

Let me set the record straight. As I stated in the PR, if you want an out-of-the-box solution, use SQLite. MySQL was never intended to be that. To use it, it must be first installed, then a root password set up then, then using the mysql client, set up an account to be used with WeeWX, then set proper permissions for that account. Modifying the services file is just one more step in that process. It's not a "time bomb waiting to go off."

Also, weewxd does restart if MySQL is not ready. Or, if it doesn't, that's a bug. I just checked it on my Mac, and it works as it is supposed to.

Let's keep this constructive.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/CAGTinV4Kq9SRAWX%3DRxGgqbHYxyT4dLBNtbe3FoUyWGsSD11Vpg%40mail.gmail.com.

[John Smith]

On Mon, 17 Nov 2025 at 13:03, Tom Keffer <tke...@gmail.com> wrote:

John, slow down there, dude.

You didn't respond to any of my comments\ until I made them more widely known.

 

Let me set the record straight. As I stated in the PR, if you want an out-of-the-box solution, use SQLite. MySQL was never intended to be that. To use it, it must be first installed, then a root password set up then, then using the

Then why is it in core code and not in an extension?

 

mysql client, set up an account to be used with WeeWX, then set proper permissions for that account. Modifying the services file is just one more step in that process. It's not a "time bomb waiting to go off."

Yes it was a time bomb and it went off for me, hence the PR to restore previous system behaviour, I don't have some bent to go out of my way filing frivolous PRs, these take time and effort.

 

Also, weewxd does restart if MySQL is not ready. Or, if it doesn't, that's a bug. I just checked it on my Mac, and it works as it is supposed to.

Not on Debian 13 Trixie running SystemD 257.8-1~deb13u2 hence the PR.... 

[Pablo Sanchez]

On 2025-11-16 21:12, John Smith wrote:

Also, weewxd does restart if MySQL is not ready. Or, if it doesn't, that's a bug. I just checked it on my Mac, and it works as it is supposed to.

Not on Debian 13 Trixie running SystemD 257.8-1~deb13u2 hence the PR.... 

Hi everyone,

I see the same issue on openSUSE Tumbleweed. If I bring down MariaDB, wait for WeeWX to spew an error, then restart MariaDB, WeeWX does not re-connect to the DB. I have to manually restart WeeWX.

Cheers,
-pablo

[John Smith]

For what it's worth Python was also upgraded in Trixie to 3.13 from 3.11

However in the past I have noticed weeWX not waiting or retrying a database connection but it never happened during boot up before...

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/3d9cb1e4-0a35-4903-b6ce-d253a536589e%40hillsandlakes.com.

[Cameron D]

The quick response to whether it might be core or extension is that it is the systemD component that is new and is somehow seems to be causing problems. But I don't recall that capability (waiting for Mysql) even being possible in sysv-init code so I think the problem has always been there.

I moved weewx to systemd before it became part of the basic weewx system - I think precisely because I was seeing that problem whenever there was an upgrade to MariaDB.

I have kept using my own modified service files since then and update them manually if I see changes in the shipped version.

My impression was that weewx had become more resilient to the DB server disappearing for a bit, but I also have restart parameters set in my unit file. Maybe it is a difference between failing on initial connection or failing at a later stage.

My other comment (and the reason I did not reply in support of your PR) is that your changes would only be a partial solution, because it cannot account for a db server running on a different host.  This is how I was running my system at one stage and the weewx side must rely on some sort of restart/retry process when a remote mysql goes down, whether it is the server itself or the network connectivity.

[John Smith]

On Mon, 17 Nov 2025 at 16:58, 'Cameron D' via weewx-user <weewx...@googlegroups.com> wrote:

The quick response to whether it might be core or extension is that it is the systemD component that is new and is somehow seems to be causing problems. But I don't recall that capability (waiting for Mysql) even being possible in sysv-init code so I think the problem has always been there.

That was 1 reason given for SystemD's creation, it starts as much as it can in parallel, SYS-V ordered startup by sorting filenames and launching them one after another...

 

I have kept using my own modified service files since then and update them manually if I see changes in the shipped version.

The weeWX Debian package has pre/post scripts that install the service files during updates... I can't ever remember being prompted to approve/deny SystemD service files...

 

My impression was that weewx had become more resilient to the DB server disappearing for a bit, but I also have restart parameters set in my unit file. Maybe it is a difference between failing on initial connection or failing at a later stage.

I can't recall weeWX waiting and retrying at all, and I manually had to start it after resolving any issues, the difference now is weeWX is starting before MariaDB during boot up... That's what's new here...

 

My other comment (and the reason I did not reply in support of your PR) is that your changes would only be a partial solution, because it cannot account for a db server running on a different host.

If the most common use case is running weeWX using SQLite, and following that would be by people running weeWX with a SQL DB, I don't think such a setup as you described would be used by your average weather hobbyist... At most it'd be 2 docker containers but even that would still be on the same physical system...

 

  This is how I was running my system at one stage and the weewx side must rely on some sort of restart/retry process when a remote mysql goes down, whether it is the server itself or the network connectivity.

Please don't also stray into weird and wonderful use cases that aren't common, people running those setups usually know what they're doing and can adjust SystemD files to suit, what you described isn't a typical use case by a very long margin, but using a SQL DB on the same system for weeWX would be...

[Graham Eddy]

my take on mysql in weewx: weewx internal data store is sqlite3; it’s nice to offer mysql stubs but you’re on your own (or using 3rd party extensions) to utilise them

⊣GE⊢

[Cameron D]

" The weeWX Debian package has pre/post scripts that install the service files during updates... I can't ever remember being prompted to approve/deny SystemD service files..."

Because unlike config files, the user-modified unit files should be placed in  dirs like /etc/systemd/system where they simply override any unit file that is installed/updated in the default locations assigned to packages.

I don't think apt/dpkg does any checks for such situations.

[John Smith]

On Mon, 17 Nov 2025 at 18:42, 'Cameron D' via weewx-user <weewx...@googlegroups.com> wrote:

" The weeWX Debian package has pre/post scripts that install the service files during updates... I can't ever remember being prompted to approve/deny SystemD service files..."

Because unlike config files, the user-modified unit files should be placed in  dirs like /etc/systemd/system where they simply override any unit file that is installed/updated in the default locations assigned to packages.

I don't think apt/dpkg does any checks for such situations.

That is straying beyond the scope of PR I filed, by all means file a feature request, but let's stick to the topic at hand...

SystemD or Python or anything else upgraded has changed somehow and people expect weeWX will start on boot up, but that stopped happening with Debian Trixie for me at least...

[matthew wall]

on the contrary, cameron's comment is exactly on point. systemd unit files are *not* conf files, so they may be replaced when you update/upgrade the weewx package. the pattern for customizing systemd units is the .d pattern.

adding a dependency in systemd is a hack, not a real fix, for a number of reasons:

1) it does not fix the problem for every operating system and init system
2) it is a dependency that does not apply to every installation,
3) it is a dependency that is not part of weewx and that could change depending on a non-weewx component of the system

we can provide examples, but each file that is actually activated on a user's system is a potential support problem for us.

just as we have to minimize python module and os dependencies, we have to minimize init dependencies. otherwise long-term support and robustness suffer. a lot. we could take the route of not having any deb/rpm packages for weewx, and let someone else create those to include in debian or redhat or whatever. but some time ago we chose to do them, to make life easier for users and so that users could have weewx updates more frequently (if they choose) than the os releases. we have been fairly successful in keeping weewx packages robust wrt to the changes between os releases, and consistent across multiple operating systems (and their different uses of systemd, for that specific init system)

is not the fix to make weewx retry mysql/maria/postgres connections and do nothing else until that connection is successful?

m

[John Smith]

On Mon, 17 Nov 2025 at 23:09, matthew wall <mwall...@gmail.com> wrote:

on the contrary, cameron's comment is exactly on point.  systemd unit files are *not* conf files, so they may be replaced when you update/upgrade the weewx package.  the pattern for customizing systemd units is the .d pattern.

Why is everyone trying to dream up the worst configuration possible that someone maybe sort of kind of might do?

The core issue is very very simple weeWX no longer starts on boot up with out intervention, be it changes to the SystemD file or manually starting it.

 

adding a dependency in systemd is a hack, not a real fix, for a number of reasons:

It would immediately restore previous functionality.

 

1) it does not fix the problem for every operating system and init system

That's a straw man argument, the only problem is with the order that SystemD starts things.

 

2) it is a dependency that does not apply to every installation,

Since not every installation has SystemD involved this is again a straw man argument.

 

3) it is a dependency that is not part of weewx and that could change depending on a non-weewx component of the system

Have we moved on to buzz word bingo or something?

  

just as we have to minimize python module and os dependencies, we have to minimize init dependencies.  otherwise long-term support and robustness suffer.  a lot.  we could take the route of not having any deb/rpm

More straw man arguments, a slight change to service file will NOT increase your support work load, it will decrease it because people won't complain about weeWX not starting on system boot.

 

packages for weewx, and let someone else create those to include in debian or redhat or whatever.  but some time ago we chose to do them, to make life easier for users and so that users could have weewx updates more

What's any of that got to do with the PR I made? weeWX doesn't start when the OS starts... You are creating support headaches for yourself by trying to defend a point absent logic and there isn't any data showing how 2 extra lines in a SystemD service file will be even be noticed let alone detrimental...

 

frequently (if they choose) than the os releases.  we have been fairly successful in keeping weewx packages robust wrt to the changes between os releases, and consistent across multiple operating systems (and their different uses of systemd, for that specific init system)

There aren't any problems in an any other init system, so why keep making straw man arguments without actually addressing my comments based on evidence rather than going off on frequent tangents?

 

is not the fix to make weewx retry mysql/maria/postgres connections and do nothing else until that connection is successful?

Finally something on topic at last, but then you immediately brought up a red herring by including pgSQL...

[Tom Keffer]

John, time to give it a rest. You were heard. 

I would suggest focusing on the restart mechanism. The executable weewxd should restart if a database connection cannot be made. If that is not the case,  then that's a bug. Document it and let us know.

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/CAGTinV4xKmhq%2BtXgN%2B1awU3LYpZPi%3DS%3DvU8h39TV5PRny0vHzA%40mail.gmail.com.

[John Smith]

On Tue, 18 Nov 2025 at 00:11, Tom Keffer <tke...@gmail.com> wrote:

I would suggest focusing on the restart mechanism. The executable weewxd should restart if a database connection cannot be made. If that is not the case,  then that's a bug. Document it and let us know.

As per previous emails, I've never experienced weeWX ever waiting or retrying, it's just now not starting on boot up either... 

[Pablo Sanchez]

Hi Tom,

I will file a bug because that is not happening in my environment.

Thx!

---
pablo

[Tom Keffer]

On Mon, Nov 17, 2025 at 5:21 AM John Smith <deltafo...@gmail.com> wrote:

As per previous emails, I've never experienced weeWX ever waiting or retrying, it's just now not starting on boot up either... 

Forgive me for not remembering the details. Document it and post. I'm particularly interested in your comment that it is not starting on boot up. 

[John Smith]

On Tue, 18 Nov 2025 at 00:30, Tom Keffer <tke...@gmail.com> wrote:

Forgive me for not remembering the details. Document it and post. I'm particularly interested in your comment that it is not starting on boot up. 

I did and you didn't bother to fully read or comprehend my posts.

[matthew wall]

more precisely, weewxd should stop if it fails to connect. if loop-on-init is enabled, then it should restart.

for robustness it might be good to have retries at the transaction/query level, then if n retries fail, weewxd fails and stops (or restart if loop-on-init is enabled)

its been a long time since i was in the mysql code, but i thought that is what we implemented.

and systemd hacks (or sysv init hints) wont handle the case where mysqld/whatever is remote (which was the primary use case for using mysqld/whatever instead of sqlite)

m

><https://groups.google.com/d/msgid/weewx-user/CAGTinV4xKmhq%2BtXgN%2B1awU3LYpZPi%3DS%3DvU8h39TV5PRny0vHzA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .

>>
>
>--
>You received this message because you are subscribed to the Google
>Groups "weewx-user" group.
>To unsubscribe from this group and stop receiving emails from it, send
>an email to weewx-user+...@googlegroups.com.
>To view this discussion visit

>https://groups.google.com/d/msgid/weewx-user/CAPq0zEDGCS8kHdZwK1xr9P6rZFyjddL003GZ7LPZr97T4eGcMA%40mail.gmail.com.

[Tom Keffer]

John, cool it, man. I'm just trying to be helpful. We're all volunteers here.

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/CAGTinV5PNaz%3DQne-QAcGjyoXa6hw7T0te1ue7tHEKnDVjLOz4g%40mail.gmail.com.

[Pablo Sanchez]

Hi everyone,

Database puke here.

Nearly all applications are database applications. For any db application, the question is, what to do if the database is no longer available?  Do we fail or go into a retry-loop until the db comes back.

It is a design decision and I would also say an operational decision. That is, do we want to encumber the end-user with restarting database dependencies too? In this case, weewx? Generally, no.

Personally, I would continue with the design decision to do retry. This way the end-users, who by and large I would say are non-technical, have less to worry about. It would also solve any boot-up synchronization issues.

I captured forensics of what I witnessed (the weewxd not reconnecting on db becoming available) here - https://github.com/weewx/weewx/issues/1036

Thx!

---
pablo

[michael.k...@gmx.at]

If a user chooses a database other than sqlite, this is an active decision, and the database must be provided by the user—unlike the default sqlite. That said, I would expect there to be a defined behavior for weewx if the database connection cannot be established. This behavior should be described in the documentation, and if the reality differs from the desired/documented behavior, there is a bug that needs to be fixed. The user who has actively chosen an alternative to sqlite should be able to judge whether they need to take additional measures to ensure that the application works under the given circumstances.

[Pablo Sanchez]

Hi Michael,

Tom confirmed in the incident that it is a bug and has provided a fix. I will roll it in as soon as I can. W00t!

Cheers!

---
pablo

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/48532b96-eb34-46d1-b2e3-5b4190347f01n%40googlegroups.com.

[John Smith]

On Tue, 18 Nov 2025 at 04:05, 'michael.k...@gmx.at' via weewx-user <weewx...@googlegroups.com> wrote:

If a user chooses a database other than sqlite, this is an active decision, and the database must be provided by

The flaw in that logic is the SQL DB code is core code, it's not offered as an extension.

[michael.k...@gmx.at]

No, that's makes no difference. Think of a meeting you need a special person for: I he he's not there at the appointment, what you're gonna do? Wait forever? Leave immediately? Wait for a certain time and start the meeting as he arrives within that period?

Core code or not: the only thing that changes might the person you'd be filing your complaints if it doesn't work as you'd expect it. So now we're here, knowing it's a bug and it'd be still a bug if it was in an extension.

[John Smith]

On Tue, 18 Nov 2025 at 15:44, 'michael.k...@gmx.at' via weewx-user <weewx...@googlegroups.com> wrote:

No, that's makes no difference. Think of a meeting you need a special person for: I he he's not there at the appointment, what you're gonna do? 

That's my point, the person needed is already there, it just needed to be made more fault tolerant...  or 3 lines different more or less it turns out...

[michael.k...@gmx.at]

[ ] You understand analogies
[x] You don't

[John Smith]

So you can't actually point out what I got wrong, but you still want to win an internet argument?

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/6f2121bd-9b26-48e6-8bda-543a7154cfa5n%40googlegroups.com.

[michael.k...@gmx.at]

Sure, I can. But for someone complaining about his posting avalanche hasn't been read fully/comprehensively, I was expecting he was able to do so with a simple two-liner.

[John Smith]

On Tue, 18 Nov 2025 at 16:43, 'michael.k...@gmx.at' via weewx-user <weewx...@googlegroups.com> wrote:

Sure, I can. But for someone complaining about his posting avalanche hasn't been read fully/comprehensively, I was expecting he was able to do so with a simple two-liner.

Well posting a PR didn't result in a solution that would restore previous functionality... ie weeWX would no longer start on boot up...

Since you seem intent on arguing based on flawed logic let's revisit your analogy that I didn't comprehend fully or comprehensively, weeWX doesn't implement all SQLite  code for it to function, it imports modules from python libraries, so based on your point I guess they need a special person to be available for SQLite too then.

Just like SQLite support available in weeWX the mMySQL/MariaDB functions are distributed with other essential code needed to function, ie it is in the core code distributed

The MySQL/MariaDB code isn't an extension you have to install to be used. It is for those reason that special people needing to be available are already employed by the company, they are not very expensive outsourced consultants as you seemed to think was needed.

[michael.k...@gmx.at]

You are right that the code for connecting to a MySQL DB is there, But the Database isn't. And it is literally impossible for any program to work with a MySQL database that hasn't been deployed alongside the installation process by the program itself. With SQLite, all you need is write access to a file system and there you go. It is not about if the hitch was already sold with the car, or if it is an aftermarket one, it's all about setting up the trailer.

[John Smith]

On Tue, 18 Nov 2025 at 17:12, 'michael.k...@gmx.at' via weewx-user <weewx...@googlegroups.com> wrote:

You are right that the code for connecting to a MySQL DB is there, But the Database isn't. And it is literally

Neither is SQLite, the python libraries act in a similar fashion between weeWX code and the file system... The only difference is that SQLite uses a file and MariaDB does a whole lot more.

SQLite isn't somehow more supported just because there is a file involved compared to using an API.

In both cases weeWX doesn't try and do everything like you seem to think it does, it uses libraries to achieve that.

[p q]

I'm not invested in this thread and am not really following it. I use sqlite. If I was going to use a "real" server SQL DB, I would use postgres but no matter what, the server would be on a different machine than weewx or possibly in a container. Regardless I d need a different startup process than with sqlite. 

Peter Quinn
(415)794-2264

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/9c9323fe-26f4-4a6b-b841-ef38d98920c1n%40googlegroups.com.

[John Smith]

On Tue, 18 Nov 2025 at 17:26, p q <peterq...@gmail.com> wrote:

DB, I would use postgres but no matter what, the server would be on a different machine than weewx or possibly

This is a bit of a red herring unless and until some geek with an itch generates a PR for weeWX to support the pgSQL API. That's assuming Tom would even accept it, and not require it to be turned into an extension.

Tom has already stated he should have just made the core program and everything else should have been an extension, that includes SQLite functionality I assume...

I'm curious why you'd want, or even consider, splitting the storage and weeWX on separate physical systems?

Containers I understand as they usually get used in a fashion akin to a single program and not much else, but using containers on the same system it would still be easy to control the order things are started, multiple physical systems, not so much...

 

in a container. Regardless I d need a different startup process than with sqlite. 

You would still need some kind of middle-ware library to handle the file or API, just as already happens with both SQLite and MariaDB, I very much doubt weeWX will never have raw support for any storage method...

As far as I know weeWX has no code to do raw processing of anything, not even the config files... weeWX uses libraries to handle the content of config files, and parsing config files would be a much easier undertaking compared to something as complicated as a SQLite file...

Why would Tom reinvent the wheel when others have produced very robust and extensively tested libraries?

[John Smith]

On Tue, 18 Nov 2025 at 17:54, John Smith <deltafo...@gmail.com> wrote:

As far as I know weeWX has no code to do raw processing of anything, not even the config files... weeWX uses libraries to handle the content of config files, and parsing config files would be a much easier undertaking compared to something as complicated as a SQLite file...

My statement above failed to take into account the raw processing in core code of weather data of weather data from select weather station consoles because there isn't a suitable library that already exists...

However everything else predominately is handled in libraries...