Stuck at 3:00 AM on change to Daylight Savings time; Path issues under FreeBSD

[Marius Schamschula]

Something broke my old installation of weewx 4.10.1. It stopped at 3:00 AM with the change to CDT around here.

As someone who spends a lot of time avoiding pip and venvs as a port maintainer on MacPorts I have held back updating to 5.x. I don't like the Python maintenance nightmare of having multiple (incompatible) versions of packages installed under various paths.

Despite the above, I tried installing 5.3.1 using pip and venv as a regular user.

The result: bad idea!

Under FreeBSD the unprivileged user can't open the device (in this case /dev/ttyU0)

Root can open the device, but the paths for root are broken.

Even after adding the path to /home/marius/.local/bin I get 

# weewxd

Traceback (most recent call last):

  File "/home/marius/.local/bin/weewxd", line 5, in <module>

    from weewxd import main

ModuleNotFoundError: No module named 'weewxd'

I have no idea how to tell python 3.11 where that module lives. In the olden days of 4.10.1, that wouldn't have been an issue.

So I uninstalled the enduser venv, and installed it as root.

I installed weewx-venv outside of the /root space, but sure enough weewx-data appeared under /root. This is a security issue. So I moved it under the same directory as weewx-venv and used --config to start weewxd. So far so good.

I see the live LOOP data come in on the console, the NOAA and html pages are created. 

However, I'm back at square one: the time is also stuck at 3:00 AM CDT.

Amy ideas?

[Marius Schamschula]

It looks like the database is not being updated.

ls -l /tank/www/weewx/archive/

total 162211

-rw-rw-rw-  1 root wheel 197065728 Mar  8 03:00 weewx.sdb

I tried using the same .sdb file, as it has all the historical data, but it failed to update with 5.3.1.

I have now move that out of the way, and am testing to see if new data is use to update the web site:

That works!

However, I need to import data from the old database. I don't want to loose eighteen year of data.

[Vince Skahan]

You generally can't avoid venvs on a modern python on a current os.  Nobody here did that.  The python project forced that on everybody.

Installing as root is generally a bad idea.  One big improvement in weewx v5 is that it no longer needs to run as root nor even 'require' root other than to install system-level things.  But to answer:

• no module named 'weewxd' is due to you not activating your python venv.    Always run 'source /some/path/here/bin/activate' to activate your venv before installing any modules with pip3 or running weewxd or weectl. 

• for a pip installation, weewxd will run as the user you used when you ran the pip command to install weewx itself.

• if you installed weewx as a non-privileged user, you will usually need to add that user to whatever group has permissions to access the serial or usb devices. Typically adding them to group dialout helps.  Alternately you can set up udev rules to do the right thing. There are wiki and faq pages with a variety of ways to do this and which groups can typically do what (varies os by os).

• you shouldn't need to import data from an old db.  The old db should work fine.

• I have no idea what 'the time is also stuck at 3:00 AM' means.

• I have no idea what '(the .sdb) failed to update with 5.3.1' means.  Your system logs should show what happened if anything.  My wild guess is your weewx.conf file is pointing someplace else.  Set debug=1 in weewx.conf and restart it and check your logs.

[Marius Schamschula]

Sure enough the rc.d script is rather broken. It has the path to UTIL_ROOT hardcoded to the user directory. A quick edit got around this.

I also edited the paths in /usr/local/etc/rc.d/weewx, but service weewx status is not supported.

There is something else missing, as the daemon mode didn't update the database or web pages.

Further experimentation showed that when I restarted manually the updates also fail.

I moved the database file aside and restarted the daemon.  As expected, a new database was created. I'm just concerned what will happen when I need to restart weewx the next time.

[Marius Schamschula]

Indeed, the venv mess is not of weewx's making.

I'm somewhat surprised at the permissions issue for the USB socket, as my regular user is a member of wheel.

I'll try setting up as regular user again.

[Marius Schamschula]

After running (and restarting) weewx with the new database file I tried replacing it with the old file: Back to 3:00 AM!

Moved back the new database file: it works again!

[Vince Skahan]

Marius we cannot read minds. ‘What’ is back to 3am ? Your computer clock ? The timestamp on a webpage ?

[Marius Schamschula]

The time stamp on the web page is 3:00 AM, no change in the size of the weewx.sdb file since that time.

I tried to make sure there was no corruption to the database file, i.e. I ran sqlite3 weewx.sdb.old ".dump" | sqlite3 weewx.sdb.new before moving it back into place. No difference, i.e. same time stamp as when I ran the command.

The folder listing clearly shows that this new file was also not modified after being created.

I moved back the new database file, and data is being saved and the web page is being updated.

[Vince Skahan]

If you will not share your logs nobody can help. Best of luck.

[Greg Troxel]

Vince Skahan <vince...@gmail.com> writes:

> You generally can't avoid venvs on a modern python on a current os. Nobody
> here did that. The python project forced that on everybody.

That's not strictly true. pkgsrc has packages for a vast number of
py-foo all installed in the system site-packages, and it works fine. I
am actually running weewx that way, with the weewx code in
/usr/pkg/lib/python3.13/site-packages/weewx and so on.

In my case, the weewx progarm files are owned by root and live in the
system. I am running it in a data directory (with config file and
database) that is owned by a non-root user. Stepping back from weewx
and python, this is totally normal, to use installed programs with your
own data.

<rant>

I find that venvs are required because python culture says it is ok to
have requirements as foo==x.y.z, rather than foo>=x.y. Thus, there is
no way to have everything needed installed, and python packages with
unreasonably specific dependencies (Home Assistant) have to be in a venv
for isolation.

The root cause of pinned deps, besides people thinking it is ok, is API
instability within modules.

</rant>

[Vince Skahan]

Sure.  The other os have python libs they install at the system level too.  Run 'pip3 list --verbose' to see what's where.

I'm personally ok with things installing as root.  I just like not 'running' as root whenever it can be avoided.

FWIW - I have run into the problems venv solve quite a lot.  Things needing certain versions of a library (or minimum versions thereof) and the os freezing to something lesser.  Any RHEL-like system or LTS debian is a good example.  They go for many year stability and freeze to old versions.  The venv thing is a good way to not be limited by the os vendor's choices.

[Marius Schamschula]

I'm not sure what the log would tell you in this case. In the weewx.conf file for 4.10.1 had debug = 1 set. I have now also set that for the new install. I see nothing useful in the log file since I did that, i.e. no errors other than the ones I always see (e.g. ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 46 instead). It did show that an old cron job that was used to track the active tty caused multiple instances of weewx to run (FreeBSD changes the port from /dev/ttyU0 to /dev/ttyU1 or in reverse at random times). I commented that out in crontab to make sure only one instance is active. I tried one more time with the old .sdb file. Monitoring the /var/log/message file, I see no errors in the log. As before, the database doesn't update and a web page for 3:00 AM is rendered.

As the same thing happens with both weewx 4.10.1 and 5.3.1, but a clean database file works correctly with 5.3.1, so there is no problem with the basic functionality of both FreeBSD packages and weewx.

P.S. I'm leaving the root based install in place, as that is no different than what I was using for 4.10.1. No warnings of the sort:

Installing collected packages: pyserial, ephem, pyusb, PyMySQL, CT3, weewx

  WARNING: The scripts pyserial-miniterm and pyserial-ports are installed in '/home/marius/.local/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

  WARNING: The scripts weectl and weewxd are installed in '/home/marius/.local/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

when I installed as root. Of course, I've moved everything onto the ZFS array, rather than leaving it on the boot drive.

BTW: What happened to $HOME/bin? That's already in the $PATH.

In my experience some of the python version requirements are out of date or too strict, i.e. often you can ignore them and everything works correctly.

And yes, the venv install installed a newer version of py311-pillow than the current one provided by FreeBSD ports. So much for the install instructions for FreeBSD!

I now have to redo all the customization of the skin for both the private and public versions of the web site. It's been a while since I first did that so I have to rediscover what all needs to be edited.

I still need to get my all old data back!

I see that 5.3.1 uses an extended schema by default, while 4.10.1 was still using the old wview schema (yes, I did start out with wview). Could there be a problem with the database using the old schema?

[Marius Schamschula]

Changing the schema under [[wx_binding]] doesn't make any difference. Back to the new default setting.

However, after running weectl database reconfigure I get a database with the new schema that updates, i.e. creates weewx.sdb-journal. Unfortunately, when it is done, there are no live updates, and the web page again is stuck at 3:00 AM. Nothing in the log file.

I have no idea what else I could try to get at the old database and continue to update it.

Perhaps, I'll need to repurpose some of my own code (wx.aamu.edu) to access the old data and just use weewx for the new data.

I certainly won't update the Raspberry Pi that is running weewx 4.10.1 at my other house. It didn't have an issue at the start of CDT.

[Tom Keffer]

Your symptoms smell like corrupted data in your logger, but without the logs, we cannot be sure. Humor me, set debug=1, restart weewxd, let it run through the first reporting cycle, then post the logs.

The database is not your problem. If it's working for you, there is no reason to "upgrade" from the old schema to the new schema. It just offers more types. 

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/1e3e0173-f581-45cd-a954-d8ec514b558dn%40googlegroups.com.

[Marius Schamschula]

Thanks for the insights Tom!

The thought of corrupted data on the logger has also crossed my mind, but weewx builds a new database right past the problematic timestamp.

However, the old database file stops at 1772604000, which is March 4, 2026 at 12:00:00 AM CST, rather than March 8, 2026 03:00:00 AM CDT.

The new database goes back to February 27, 2026 at 3:20:00 PM CST.

As far as the logs are concerned, setting debug = 1, doesn't print any information on startup. I have restarted 5.3.1 several times w/o any output.

The only thing in the /var/log/messages file are comm errors:

Mar  8 23:48:18 mars weewxd[23459]: CRITICAL weewxd: Caught WeeWxIOError: LOOP max batch errors (3) exceeded.

Mar  8 23:48:18 mars weewxd[23459]: CRITICAL weewxd:     ****  Waiting 60.0 seconds then retrying...

Mar  9 00:00:04 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 0 instead

Mar  9 00:05:03 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 27 instead

Mar  9 00:05:23 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 0 instead

Mar  9 00:38:47 mars syslogd: last message repeated 1 times

Mar  9 00:39:05 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #2; error: Expected to read 99 chars; got 0 instead

Mar  9 00:56:51 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 0 instead

Mar  9 01:13:05 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 62 instead

Mar  9 01:29:04 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 0 instead

Mar  9 01:47:53 mars syslogd: last message repeated 1 times

Mar  9 02:03:13 mars syslogd: last message repeated 1 times

Mar  9 02:03:59 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #2; error: Expected to read 99 chars; got 0 instead

Mar  9 02:37:24 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 0 instead

Mar  9 02:38:00 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #2; error: Expected to read 99 chars; got 0 instead

Mar  9 03:28:44 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 0 instead

Mar  9 03:29:38 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #2; error: Expected to read 99 chars; got 0 instead

Mar  9 03:29:48 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #3; error: Expected to read 99 chars; got 0 instead

Mar  9 03:29:48 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP max batch errors (3) exceeded.

Mar  9 03:29:48 mars weewxd[23459]: CRITICAL weewxd: Caught WeeWxIOError: LOOP max batch errors (3) exceeded.

Mar  9 03:29:48 mars weewxd[23459]: CRITICAL weewxd:     ****  Waiting 60.0 seconds then retrying...

Mar  9 03:45:44 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 0 instead

Mar  9 04:02:08 mars syslogd: last message repeated 1 times

Mar  9 04:03:30 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #2; error: Expected to read 99 chars; got 0 instead

Mar  9 04:19:54 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 0 instead

Mar  9 04:37:04 mars syslogd: last message repeated 1 times

Mar  9 04:54:04 mars syslogd: last message repeated 1 times

Mar  9 04:54:14 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #2; error: Expected to read 99 chars; got 0 instead

Mar  9 04:54:50 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #3; error: Expected to read 99 chars; got 0 instead

Mar  9 04:54:50 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP max batch errors (3) exceeded.

Mar  9 04:54:50 mars weewxd[23459]: CRITICAL weewxd: Caught WeeWxIOError: LOOP max batch errors (3) exceeded.

Mar  9 04:54:50 mars weewxd[23459]: CRITICAL weewxd:     ****  Waiting 60.0 seconds then retrying...

Mar  9 05:27:20 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 0 instead

Mar  9 05:27:38 mars weewxd[23459]: ERROR weewx.drivers.vantage: LOOP batch try #2; error: Expected to read 99 chars; got 0 instead

Mar  9 05:45:06 mars weewxd[55633]: ERROR weewx.drivers.vantage: LOOP batch try #1; error: Expected to read 99 chars; got 0 instead

I restarted weewxd toward the end of this timeframe. Nothing here.

[Tom Keffer]

If all you are seeing is log level ERROR and CRITICAL, then the log level is too restrictive. I know nothing about FreeBSD, so you're going to have to figure that out on your own. One thing you could do is have weewx log to a file. See the wiki for how to do that.

However you do it, it would be very useful to see the whole log. It might show nothing, but that's a clue in its own way.

What kind of logger do you have? Serial, or USB? If the former, are you using a serial-to-USB converter?

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/0e70d3d6-062b-49ed-a8cf-580f9bdd5b64n%40googlegroups.com.

[Greg Troxel]

Tom Keffer <tke...@gmail.com> writes:

> If all you are seeing is log level ERROR and CRITICAL, then the log level
> is too restrictive. I know nothing about FreeBSD, so you're going to have
> to figure that out on your own. One thing you could do is have weewx log to
> a file. See the wiki

> <https://github.com/weewx/weewx/wiki/WeeWX-v4-and-logging#logging-to-rotating-files>

> for how to do that.

I've been running weewx on NetBSD since the very end of 2017.

FWIW:

I am using a USB/serial adaptor and a Davis VP2 serial logger.

I am now using a RPI3 (without RTC) and my boot scripts wait for NTP
sync before weewx is started.

The RPI3 is on a UPS.

With this setup, I have very little trouble.


It is totally normal on *BSD to use syslog, but syslog config out of the
box limits to INFO. So if the OP is not used to configuring syslog
(putting weewx into its own log file, allowing debug), files is a good
approach.

[Marius Schamschula]

First of all, I'm using the Davis logger, however with a USB cable,

Indeed, adding the custom log file gives more useful information for 5.3.1 with the new database:

2026-03-09 07:39:30 weewxd[58005]: INFO weewx.engine: Using binding 'wx_binding' to database 'weewx.sdb'

2026-03-09 07:39:30 weewxd[58005]: INFO weewx.manager: Starting backfill of daily summaries

2026-03-09 07:39:30 weewxd[58005]: INFO weewx.manager: Daily summaries up to date

2026-03-09 07:39:30 weewxd[58005]: DEBUG weewx.drivers.vantage: Getting archive packets since 2026-03-09 07:35:00 CDT (1773059700)

2026-03-09 07:39:31 weewxd[58005]: DEBUG weewx.drivers.vantage: Successfully woke up Vantage console

2026-03-09 07:39:31 weewxd[58005]: DEBUG weewx.drivers.vantage: Retrieving 0 page(s); starting index= 0

2026-03-09 07:39:31 weewxd[58005]: INFO weewx.engine: Starting main packet loop.

2026-03-09 07:39:31 weewxd[58005]: DEBUG weewx.drivers.vantage: Successfully woke up Vantage console

2026-03-09 07:39:31 weewxd[58005]: DEBUG weewx.drivers.vantage: Requesting 200 LOOP packets.

2026-03-09 07:39:32 weewxd[58005]: DEBUG weewx.drivers.vantage: Successfully woke up Vantage console

2026-03-09 07:40:14 weewxd[58005]: DEBUG weewx.drivers.vantage: Getting archive packets since 2026-03-09 07:35:00 CDT (1773059700)

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.drivers.vantage: Successfully woke up Vantage console

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.drivers.vantage: Retrieving 1 page(s); starting index= 0

2026-03-09 07:40:15 weewxd[58005]: INFO weewx.manager: Added record 2026-03-09 07:40:00 CDT (1773060000) to database 'weewx.sdb'

2026-03-09 07:40:15 weewxd[58005]: INFO weewx.manager: Added record 2026-03-09 07:40:00 CDT (1773060000) to daily summary in 'weewx.sdb'

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.drivers.vantage: DMPAFT complete: page timestamp 2026-02-28 09:25:00 CST (1772292300) less than final timestamp 2026-03-09 07:40:00 CDT (1773060000)

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.drivers.vantage: Catch up complete.

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.reportengine: Running reports for latest time in the database.

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.drivers.vantage: Requesting 200 LOOP packets.

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.reportengine: Running report 'SeasonsReport'

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.reportengine: Found configuration file /tank/www/weewx-data/skins/Seasons/skin.conf for report 'SeasonsReport'

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.reportengine: Unable to set locale 'en': unsupported locale setting. Using default locale instead ('C.UTF-8').

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.reportengine: Running generators for report 'SeasonsReport' in directory '/tank/www/weewx-data/skins/Seasons' with locale 'C.UTF-8'

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.cheetahgenerator: Using search list ['weewx.cheetahgenerator.Almanac', 'weewx.cheetahgenerator.Current', 'weewx.cheetahgenerator.DisplayOptions', 'weewx.cheetahgenerator.Extras', 'weewx.cheetahgenerator.Gettext', 'weewx.cheetahgenerator.JSONHelpers', 'weewx.cheetahgenerator.PlotInfo', 'weewx.cheetahgenerator.SkinInfo', 'weewx.cheetahgenerator.Station', 'weewx.cheetahgenerator.Stats', 'weewx.cheetahgenerator.UnitInfo']

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.manager: Daily summary version is 4.0

2026-03-09 07:40:15 weewxd[58005]: DEBUG weewx.drivers.vantage: Successfully woke up Vantage console

2026-03-09 07:40:16 weewxd[58005]: INFO weewx.cheetahgenerator: Generated 8 files for report SeasonsReport in 1.41 seconds

2026-03-09 07:40:16 weewxd[58005]: DEBUG weewx.manager: Daily summary version is 4.0

2026-03-09 07:40:17 weewxd[58005]: INFO weewx.imagegenerator: Generated 12 images for report SeasonsReport in 0.27 seconds

2026-03-09 07:40:17 weewxd[58005]: DEBUG weewx.reportengine: Copied 5 files to /tank/www/weewx-data/public_html

2026-03-09 07:40:17 weewxd[58005]: DEBUG weewx.reportengine: Report 'SmartphoneReport' not enabled. Skipping.

2026-03-09 07:40:17 weewxd[58005]: DEBUG weewx.reportengine: Report 'MobileReport' not enabled. Skipping.

2026-03-09 07:40:17 weewxd[58005]: DEBUG weewx.reportengine: Report 'StandardReport' not enabled. Skipping.

2026-03-09 07:40:17 weewxd[58005]: DEBUG weewx.reportengine: Report 'FTP' not enabled. Skipping.

2026-03-09 07:40:17 weewxd[58005]: DEBUG weewx.reportengine: Report 'RSYNC' not enabled. Skipping.