V5.0 release candidate available

697 views
Skip to first unread message

Tom Keffer

unread,
Dec 21, 2023, 6:07:40 PM12/21/23
to weewx-development
Celebrate the solstice (coming up in 4 hours) and the start of Winter with a WeeWX release candidate!

Changes since b17
We gave up on a standalone logger and have gone back to whatever your system uses. This is more robust to permission problems, but may make it harder to get the logs back out. See the wiki article View logs for tips on how to convince your system to let you have a look.

As always, you can add a [Logging] section to weewx.conf and set up your own rotating log system. See the article WeeWX V4 and logging for how to do this.

Pip

For pip installs, please delete your old virtual environment, then install from scratch by following the pip install instructions. While upgrading should work, we are particularly interested in the experience of a new install, including setting up a daemon and udev files. Make sure to follow the new instructions that use a daemon setup script.


Debian

For Debian package installs, modify /etc/apt/sources.list as follows:
echo "deb [arch=all] https://weewx.com/apt-test/python3 buster main" | sudo tee /etc/apt/sources.list.d/weewx.list
Note the "apt-test". This tells apt to look there for the beta release, instead of the normal repository. You may want to change it back when you're done.


Red Hat

For Red Hat package installs, put this in /etc/yum.repos.d/weewx.repo
[weewx]
name=weewx
baseurl=http://weewx.com/yum-test/weewx/el9
enabled=1
Note the "yum-test". This tells yum to look there for the beta release, instead of the normal repository. Again, you may want to change things back when you're done.


What we're looking for

1. We are interested in your upgrade experience. Did the installers get your configuration file right? Skins? Is the daemon configured correctly?

2. With this release, we have included udev rules to set the correct permissions for devices. Did it work? If not, did unplugging then replugging the device work?

3. How about logging? Is it going to your system logger? Do the labels look reasonable?

Thanks!

-Tom & Matt

Al Barnes

unread,
Dec 22, 2023, 12:46:58 AM12/22/23
to weewx-de...@googlegroups.com
First off, many thanks to all involved in developing weewx V5! After
reading the git logs for the past few months I can see that this has
taken a ton of time and effort.

I've finished a pip install and here are my findings.

I'm running Raspberry Pi OS (Debian Bookworm) on a pi 3b that has been
used for testing of the alpha versions of weewx. The station is a
Vantage Pro2 with the official Davis logger.

After deleting weewx-data and weewx-venv and a weewx.service file in
/etc I followed the instructions at
https://weewx.com/docs/5.0/quickstarts/pip/ and everything went as
expected. I created the station with 'weectl station create'. The
command 'sudo sh ~/weewx-data/scripts/setup-daemon.systemd' set up
systemd logging.

After this point I made no changes whatsoever to weewx.conf as I wanted
to see how it would run with the default settings. Before starting
weewx, I ran 'journalctl -f -u weewx in another terminal and first
received a message that I've never seen before:

/etc/systemd/system/weewx.service:15: Standard output type
syslog+console is obsolete, automatically updating to journal+console.
Please update your unit file, and consider removing the setting altogether.

After starting weewx the logging was normal.

As I had not copied over my old database, weewx faithfully starting
downloading records from the Vantage Pro2 starting with 2023-12-12
22:35:00 PST.

It can take quite a while to download almost 9 days of data from the
logger, and here is where I ran in to a serious problem. The downloading
of records stopped at 2023-12-21 19:50:00 when the actual time was
20:38:44. At this point the next line in the logs was 'Starting main
packet loop.'
Shortly thereafter it added records at 20:38:00 and 20:40:00, so we are
now missing all records from 19:50 through 20:35.
A minute later I received the dreaded 'Expected to read 99 chars; got 0
instead' message, followed by 'Main loop exiting. Shutting engine down'.

Weewx restarted on it's own and seems to be downloading a new archive
record every five minutes, but of course we still have the problem of
the missing records. I had the same problem in early version 5 alpha
releases when weewx was downloading many records and couldn't complete
everything within the five minute cycle. Not a problem if you are
copying over your old database as you are most likely only downloading
an hour or two of records from the logger, but this could bite someone
who's station has been down for a long time due to a power outage or
whatever. I've attached the log file so that you can see the details.

Anyhow, hopefully this is helpful and thanks again for all of the hard
work that has been done to come up with a new version of weewx!

Al
weewx.log

Hartmut Schweidler

unread,
Dec 22, 2023, 1:05:49 AM12/22/23
to weewx-development
Guten Morgen,

Zunächst einmal vielen Dank an alle, die an der Entwicklung von weewx V5 beteiligt waren!

Mein Installation erfolgte per "apt upgrade". auf einem Banana Pi + 1TB HDD

Die folgenden Pakete werden aktualisiert (Upgrade):
  weewx
1 aktualisiert, 0 neu installiert, 0 zu entfernen und 2 nicht aktualisiert.
Es müssen 2.278 kB an Archiven heruntergeladen werden.
Nach dieser Operation werden 2.048 B Plattenplatz zusätzlich benutzt.
Holen:1 https://weewx.com/apt-test/python3 buster/main all weewx all 5.0.0rc1-1 [2.278 kB]
Es wurden 2.278 kB in 2 s geholt (948 kB/s).
Preconfiguring packages ...
(Lese Datenbank ... 75361 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von .../weewx_5.0.0rc1-1_all.deb ...
Entpacken von weewx (5.0.0rc1-1) über (5.0.0b17-4) ...
weewx (5.0.0rc1-1) wird eingerichtet ...
Neue Version der Konfigurationsdatei /etc/weewx/skins/Ftp/skin.conf wird installiert ...
Neue Version der Konfigurationsdatei /etc/weewx/skins/Mobile/skin.conf wird installiert ...
Neue Version der Konfigurationsdatei /etc/weewx/skins/Rsync/skin.conf wird installiert ...

Konfigurationsdatei »/etc/weewx/skins/Seasons/skin.conf«
 ==> Geändert (von Ihnen oder von einem Skript) seit der Installation.
 ==> Paketverteiler hat eine aktualisierte Version herausgegeben.
   Wie möchten Sie vorgehen? Ihre Wahlmöglichkeiten sind:
    Y oder I : Die Version des Paket-Betreuers installieren
    N oder O : Die momentan installierte Version beibehalten
       D     : Die Unterschiede zwischen den Versionen anzeigen
       Z     : Eine Shell starten, um die Situation zu begutachten
 Der Standardweg ist das Beibehalten der momentanen Version.
*** skin.conf (Y/I/N/O/D/Z) [Vorgabe=N] ? n
...
Neue Version der Konfigurationsdatei /etc/weewx/weewx.conf.dist wird installiert ...
Copying previous config file to /etc/weewx/weewx.conf-5.0.0b18-5.0.0rc1
Saving distribution config file as /etc/weewx/weewx.conf-5.0.0rc1
Creating maintainer config as /etc/weewx/weewx.conf-5.0.0b18-5.0.0rc1
Using configuration file /etc/weewx/weewx.conf-5.0.0b18-5.0.0rc1
Finished upgrading configuration file /etc/weewx/weewx.conf-5.0.0b18-5.0.0rc1
Saving configuration file /etc/weewx/weewx.conf-5.0.0b18-5.0.0rc1
Created symlink /etc/systemd/system/multi-user.target.wants/weewx.service → /lib/systemd/system/weewx.service.

root@ba002:/etc/weewx# systemctl restart weewx@weewx
Warning: The unit file, source configuration file or drop-ins of we...@weewx.service changed on disk. Run 'systemctl daemon-reload' to reload units.
root@ba002:/etc/weewx# systemctl daemon-reload
root@ba002:/etc/weewx# systemctl restart weewx@weewx
root@ba002:/etc/weewx# systemctl restart weewx@weeusb

Meine Anpassungen:
Anpassungen nur in weewx@.service

"
# systemd service template file for running multiple instances of weewxd
#
# Each instance XXX must have its own config, database, and HTML_ROOT:
#
#  item            name                           where to specify
#  --------        -----------------------------  ----------------------------
#  config          /etc/weewx/XXX.conf            configuration directory
#  database_name   /var/lib/weewx/XXX.sdb         specified in XXX.conf
#  HTML_ROOT       /var/www/html/XXX              specified in XXX.conf

[Unit]
Description=WeeWX %i
Documentation=https://weewx.com/docs
Requires=time-sync.target
After=time-sync.target
PartOf=weewx.service

[Service]
ExecStart=weewxd --log-label weewx5-%i /etc/weewx/%i.conf
StandardOutput=null
StandardError=journal+console
#User=weewx
#Group=weewx

[Install]
WantedBy=multi-user.target

"

Es läuft hervorragend.

2023-12-22T07:00:24.155271+01:00 ba002 weewx5-weewx[5037]: INFO weewx.manager: Added record 2023-12-22 07:00:24 CET (1703224824) to database 'weewxDavis'
2023-12-22T07:00:24.393038+01:00 ba002 weewx5-weewx[5037]: INFO weewx.manager: Added record 2023-12-22 07:00:24 CET (1703224824) to daily summary in 'weewxDavis'
2023-12-22T07:00:25.032855+01:00 ba002 weewx5-weewx[5037]: INFO weewx.manager: Added record 2023-12-22 07:00:00 CET (1703224800) to database 'weewxGW2000'
2023-12-22T07:00:25.642882+01:00 ba002 weewx5-weewx[5037]: INFO weewx.manager: Added record 2023-12-22 07:00:00 CET (1703224800) to daily summary in 'weewxGW2000'
2023-12-22T07:00:34.314116+01:00 ba002 weewx5-weewx[5037]: INFO weewx.cheetahgenerator: Generated 9 files for report SeasonsReport in 8.51 seconds
2023-12-22T07:00:45.239521+01:00 ba002 weewx5-weewx[5037]: INFO weewx.imagegenerator: Generated 24 images for report SeasonsReport in 10.92 seconds
2023-12-22T07:00:45.241651+01:00 ba002 weewx5-weewx[5037]: INFO weewx.reportengine: Copied 0 files to /var/www/html/weewx
2023-12-22T07:00:48.646048+01:00 ba002 weewx5-weeusb[4799]: INFO weewx.drivers.fousb: synchronising to the weather station (quality=1)
2023-12-22T07:01:34.532773+01:00 ba002 weewx5-weeusb[4799]: INFO weewx.manager: Added record 2023-12-22 06:57:00 CET (1703224620) to database 'weewx.sdb'
2023-12-22T07:01:34.687560+01:00 ba002 weewx5-weeusb[4799]: INFO weewx.manager: Added record 2023-12-22 06:57:00 CET (1703224620) to daily summary in 'weewx.sdb'
2023-12-22T07:01:41.011145+01:00 ba002 weewx5-weeusb[4799]: INFO weewx.cheetahgenerator: Generated 8 files for report StandardReport in 5.96 seconds
2023-12-22T07:01:43.778354+01:00 ba002 weewx5-weeusb[4799]: INFO weewx.imagegenerator: Generated 13 images for report StandardReport in 2.76 seconds
2023-12-22T07:01:43.784111+01:00 ba002 weewx5-weeusb[4799]: INFO weewx.reportengine: Copied 0 files to /var/www/html/weewx/wx2013

Noch einmal Danke und ein gesundes Weihnachtsfest, verbunden mit einem Guten Rutsch in Jahr 2024

Hartmut

Greg

unread,
Dec 22, 2023, 3:45:30 AM12/22/23
to weewx-development
I installed weewx using the pip method and followed the instructions.
I did a pip list before installing to get the list of what other dependencies I am using.
 When I installed paho-mqtt and pyephem I got these errors:

Installing collected packages: paho-mqtt
  DEPRECATION: paho-mqtt is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for paho-mqtt ... done
Successfully installed paho-mqtt-1.6.1

/opt/weewx$ python3 -m pip install pyephem
Collecting pyephem
  Using cached pyephem-9.99.tar.gz (1.4 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: ephem in ./weewx-venv/lib/python3.11/site-packages (from pyephem) (4.1.5)
Installing collected packages: pyephem
  DEPRECATION: pyephem is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559

I am currently running it in simulation mode just to see what happens.
On my real system I have installed all my environment and data under /opt
eg /opt/weewx/weewx-venv and /opt/weewx/weewx-data

Other than that it seems to run.

My question is should I run the install of those packages above that had the error with the --use-pep517 option? I read the github information that was on the link contained in the error message but it made no sense to me.

My pip version is: 3.11

Tom Keffer

unread,
Dec 22, 2023, 8:09:15 AM12/22/23
to weewx-development
Thanks, all! Keep them coming. 

1. The syslog comment is annoying, but harmless. We've changed the weewx unit service file to specify StandardError of journal+console. Commit 940eff4.

2. Missing Vantage records. I've noticed variants of this problem when it takes a long time to download records from the console. My best is that while the console is busy emitting historic records, it neglects to create new ones. Hence, you miss a few records. I don't think it has anything to do with V5, as the driver code hasn't changed much.

3. Greg, I would guess that your problems are due to using a very old version of pip. You're at 3.11, but the current version is 23.3. See the wiki article Troubleshooting pip installs and see if that helps.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/3f813b17-0b9b-49b4-b9af-97dcd0a4f69bn%40googlegroups.com.

Paul R Anderson

unread,
Dec 22, 2023, 8:31:47 AM12/22/23
to Tom Keffer, weewx-development
Extremely nitpicky, this could be just me :)
On startup __init__.py now logs this:
Dec 21 06:47:19 hall-9000 weewxd[4484]: INFO weewx: Adding 'user' directory '/home/panders/weewx-data/bin'

First time I saw it I thought wait that directory should exist, it's where my extensions live, hope they're OK. Then I realized it's just adding the dir to PYTHONPATH
Perhaps change  __init__.py logging line to slightly more descriptive
log.info("Adding 'user' directory '%s' to PYTHONPATH" % lib_dir)
So it produces:
Dec 22 07:52:36 hall-9000 weewxd[508]: INFO weewx: Adding 'user' directory '/home/panders/weewx-data/bin' to PYTHONPATH
Paul

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.

matthew wall

unread,
Dec 22, 2023, 8:38:26 AM12/22/23
to weewx-development
On Friday, December 22, 2023 at 8:31:47 AM UTC-5 Paul R Anderson wrote:
On startup __init__.py now logs this:
Dec 21 06:47:19 hall-9000 weewxd[4484]: INFO weewx: Adding 'user' directory '/home/panders/weewx-data/bin'

First time I saw it I thought wait that directory should exist, it's where my extensions live, hope they're OK. Then I realized it's just adding the dir to PYTHONPATH
 

Steeple Ian

unread,
Dec 22, 2023, 10:48:05 AM12/22/23
to weewx-development
Installed the release candidate using pip. All working fine using simulator driver. Then tried to install Gary's weewx-gw1000.py driver and repeatedly get this error: -

(weewx-venv) Ian@bookworm:~$ weectl extension install https://github.com/gjr80/weewx-gw1000/releases/download/v0.6.0b2/gw1000-0.6.0b2.tar.gz
Using configuration file /home/Ian/weewx-data/weewx.conf
Request to install 'https://github.com/gjr80/weewx-gw1000/releases/download/v0.6.0b2/gw1000-0.6.0b2.tar.gz'.
Extracting from tar archive /tmp/tmpdoa3q052
Traceback (most recent call last):
  File "/home/Ian/weewx-venv/bin/weectl", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/Ian/weewx-venv/lib/python3.11/site-packages/weectl.py", line 66, in main
    namespace.func(namespace)
  File "/home/Ian/weewx-venv/lib/python3.11/site-packages/weectllib/__init__.py", line 96, in dispatch
    namespace.action_func(config_dict, namespace)
  File "/home/Ian/weewx-venv/lib/python3.11/site-packages/weectllib/extension_cmd.py", line 116, in install_extension
    ext.install_extension(namespace.source)
  File "/home/Ian/weewx-venv/lib/python3.11/site-packages/weecfg/extension.py", line 114, in install_extension
    extension_name = self._install_from_file(test_fd.name, info.get_content_subtype())
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Ian/weewx-venv/lib/python3.11/site-packages/weecfg/extension.py", line 154, in _install_from_file
    extension_name = self.install_from_dir(extension_dir)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Ian/weewx-venv/lib/python3.11/site-packages/weecfg/extension.py", line 165, in install_from_dir
    installer_path, installer = weecfg.get_extension_installer(extension_dir)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Ian/weewx-venv/lib/python3.11/site-packages/weecfg/__init__.py", line 738, in get_extension_installer
    installer = loader()
                ^^^^^^^^
  File "/tmp/tmpekv6izgc/gw1000/install.py", line 377, in loader
  File "/tmp/tmpekv6izgc/gw1000/install.py", line 382, in __init__
  File "/home/Ian/weewx-venv/lib/python3.11/site-packages/setuptools/_distutils/version.py", line 54, in __init__
    self.parse(vstring)
  File "/home/Ian/weewx-venv/lib/python3.11/site-packages/setuptools/_distutils/version.py", line 157, in parse
    raise ValueError("invalid version number '%s'" % vstring)
ValueError: invalid version number '5.0.0rc1'
(weewx-venv) Ian@bookworm:~$

I guess the second last line is the key one. Being trying to dig my way out of this one without success. I never had a problem with any of the other recent builds of version 5.

Any help would be appreciated,

Thanks,
Ian

matthew wall

unread,
Dec 22, 2023, 11:20:36 AM12/22/23
to weewx-development
I guess the second last line is the key one. Being trying to dig my way out of this one without success. I never had a problem with any of the other recent builds of version 5.

Any help would be appreciated,

hi ian,

we just discovered this as well.  the 'StrictVersion' parser does not handle 'rc' correctly.  here are some details:


as a workaround, you can extract the gw1000 tarball/zip, modify the install.py, then use 'weectl extension' to load from the directory instead of directly from the url.

in install.py, replace the StrictVersion conditional at line 382 with a simple "if False:"

m

Steeple Ian

unread,
Dec 22, 2023, 11:27:17 AM12/22/23
to weewx-development
Thanks Matthew, I have a fork of gw1000 so I can do it directly from there.

Ian
Message has been deleted

Steeple Ian

unread,
Dec 22, 2023, 11:54:21 AM12/22/23
to weewx-development
Can confirm that worked,
Thanks,
Ian

Vince Skahan

unread,
Dec 22, 2023, 12:34:42 PM12/22/23
to weewx-development
On Friday, December 22, 2023 at 5:09:15 AM UTC-8 Tom Keffer wrote:
3. Greg, I would guess that your problems are due to using a very old version of pip. You're at 3.11, but the current version is 23.3. See the wiki article Troubleshooting pip installs and see if that helps.

On Fri, Dec 22, 2023 at 12:45 AM Greg <ubea...@gmail.com> wrote:
I installed weewx using the pip method and followed the instructions.
I did a pip list before installing to get the list of what other dependencies I am using.
 When I installed paho-mqtt and pyephem I got these errors:

Installing collected packages: paho-mqtt
  DEPRECATION: paho-mqtt is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for paho-mqtt ... done
Successfully installed paho-mqtt-1.6.1 
 
My question is should I run the install of those packages above that had the error with the --use-pep517 option? I read the github information that was on the link contained in the error message but it made no sense to me.


I see these too on debian12 with pip 23.0.1 for those two packages...
The warnings do go away if you add the switch ala:

(weewx-venv) vagrant@deb12:~$ pip install --use-pep517 paho-mqtt
Collecting paho-mqtt
  Using cached paho-mqtt-1.6.1.tar.gz (99 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: paho-mqtt
  Building wheel for paho-mqtt (pyproject.toml) ... done
  Created wheel for paho-mqtt: filename=paho_mqtt-1.6.1-py3-none-any.whl size=62120 sha256=f329e400c6f3932151a07b46b5f098cd025392b454e94d874d48d7cc6f2c2d91
  Stored in directory: /home/vagrant/.cache/pip/wheels/29/ea/a5/ba9a63aaf4cd4e16e8a87ee31fb4d11b04ff5e1735d312619a
Successfully built paho-mqtt
Installing collected packages: paho-mqtt
Successfully installed paho-mqtt-1.6.1

 

Vince Skahan

unread,
Dec 22, 2023, 1:12:47 PM12/22/23
to weewx-development
I set up a couple instances via weewx-multi following https://github.com/weewx/weewx/wiki/weewx-multi and it looks like the generated weewx@.service file needs a little tweak.

My pip installation generated a weewx@.service file that had ExecStart pointing to weewx.conf, but it should point to %i.conf so the instance name is used.  After making that change multi works just amazingly well.  So easy.

Message has been deleted

Claudio

unread,
Dec 22, 2023, 1:57:42 PM12/22/23
to weewx-development
does it work on Raspberry 5 and Debian version: 12 (bookworm)?
Claudio

Vince Skahan

unread,
Dec 22, 2023, 2:13:59 PM12/22/23
to weewx-development
yes

Greg

unread,
Dec 22, 2023, 2:19:13 PM12/22/23
to weewx-development
Oops...I meant python version is 3.11 not pip. Pip is at the latest version.
So I guess I install this way  pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517'?
I am running debian 12 by the way. Anyone else seen these messages?

Tom Keffer

unread,
Dec 22, 2023, 2:42:48 PM12/22/23
to Greg, weewx-development
I was able to install both paho-mqtt and ephem (NB: these days, it's just "ephem", not "pyephem") without warnings on Debian 12 by installing "wheel" first:

python3 -m pip install wheel
python3 -m pip install ephem
python3 -m pip install paho-mqtt

Does this work for others? If so, we can put "wheel" in the requirements list for pip installs.

-tk


Vince Skahan

unread,
Dec 22, 2023, 2:53:11 PM12/22/23
to weewx-development
Works for me on deb12 vagrant and pi5 deb12 based raspios

Greg

unread,
Dec 22, 2023, 5:24:18 PM12/22/23
to weewx-development
Yes having install wheel first removes all my errors. I think it should be installed by default to save any questions from people like me who don't really know how this all works and what depends on what. :)

Because I was already running the beta V5 I just renamed my venv and recreated it so my weewx data would work and also if it didn't work I could rename it back and all would be good. (in theory) I have installed in /opt/weewx

All looks good so far.


Thanks for the support.

Tom Keffer

unread,
Dec 22, 2023, 5:34:23 PM12/22/23
to Greg, weewx-development
Hmmm, that didn't work. If you list "wheel" as a dependency, pip isn't smart enough to install it first before attempting the others. So, you get the same error. I'm reluctant to create yet another step in the install process.



Tom Keffer

unread,
Dec 22, 2023, 6:07:37 PM12/22/23
to Greg, weewx-development
After a bit more investigation, it appears that some systems install python3-wheel when they install python3-pip, some don't. Hence the confusing results.

Vince Skahan

unread,
Dec 22, 2023, 7:31:19 PM12/22/23
to weewx-development
Tom - your xaggs extension isn't installing.  I see the identical issue with one of my custom extensions too.

(weewx-venv) vagrant@deb12:~/adds$ weectl extension install --config=/home/vagrant/weewx-data/simulator.conf weewx-xaggs-master/
Using configuration file /home/vagrant/weewx-data/simulator.conf
Request to install 'weewx-xaggs-master/'.

Traceback (most recent call last):
  File "/home/vagrant/weewx-venv/bin/weectl", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/vagrant/weewx-venv/lib/python3.11/site-packages/weectl.py", line 66, in main
    namespace.func(namespace)
  File "/home/vagrant/weewx-venv/lib/python3.11/site-packages/weectllib/__init__.py", line 96, in dispatch
    namespace.action_func(config_dict, namespace)
  File "/home/vagrant/weewx-venv/lib/python3.11/site-packages/weectllib/extension_cmd.py", line 116, in install_extension
    ext.install_extension(namespace.source)
  File "/home/vagrant/weewx-venv/lib/python3.11/site-packages/weecfg/extension.py", line 125, in install_extension
    extension_name = self.install_from_dir(extension_path)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vagrant/weewx-venv/lib/python3.11/site-packages/weecfg/extension.py", line 201, in install_from_dir
    save_config |= self._inject_config(installer['config'], extension_name)
                                       ~~~~~~~~~^^^^^^^^^^
KeyError: 'config'

(weewx-venv) vagrant@deb12:~/adds$ pip3 list
Package            Version
------------------ ----------
certifi            2023.11.17
charset-normalizer 3.3.2
configobj          5.0.8
CT3                3.3.3
ephem              4.1.5
idna               3.6
paho-mqtt          1.6.1
Pillow             10.1.0
pip                23.0.1
pyephem            9.99
PyMySQL            1.1.0
pyserial           3.5
pyusb              1.2.1
requests           2.31.0
setuptools         66.1.1
six                1.16.0
urllib3            2.1.0
weewx              5.0.0rc1
wheel              0.42.0


Tom Keffer

unread,
Dec 22, 2023, 7:51:49 PM12/22/23
to Vince Skahan, weewx-development
Thanks for spotting that, Vince!

Fixed in commit 256cac5.

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.

Vince Skahan

unread,
Dec 22, 2023, 8:30:21 PM12/22/23
to weewx-development
I hand-edited the diff into weewx-venv/lib/python3.11/site-packages/weecfg/extension.py which worked for installing the extensions, but I get a similar message trying to run reports manually....

You might try this on a current fully patched up version to see if it works with all the updates you made after release to pypi just in case...

(weewx-venv) vagrant@deb12:~/weewx-data$ weectl report run --config=simulator.conf
Using configuration file simulator.conf
All enabled reports will be run.
Generating as of last timestamp in the database.

Traceback (most recent call last):
  File "/home/vagrant/weewx-venv/bin/weectl", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/vagrant/weewx-venv/lib/python3.11/site-packages/weectl.py", line 66, in main
    namespace.func(namespace)
  File "/home/vagrant/weewx-venv/lib/python3.11/site-packages/weectllib/__init__.py", line 96, in dispatch
    namespace.action_func(config_dict, namespace)
  File "/home/vagrant/weewx-venv/lib/python3.11/site-packages/weectllib/report_cmd.py", line 92, in run_reports
    weectllib.report_actions.run_reports(config_dict,
  File "/home/vagrant/weewx-venv/lib/python3.11/site-packages/weectllib/report_actions.py", line 84, in run_reports
    engine = weewx.engine.DummyEngine(config_dict)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vagrant/weewx-venv/lib/python3.11/site-packages/weewx/engine.py", line 89, in __init__
    self.loadServices(config_dict)
  File "/home/vagrant/weewx-venv/lib/python3.11/site-packages/weewx/engine.py", line 157, in loadServices
    obj = weeutil.weeutil.get_object(svc)(self, config_dict)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vagrant/weewx-venv/lib/python3.11/site-packages/weeutil/weeutil.py", line 1404, in get_object
    module = importlib.import_module(module_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1142, in _find_and_load_unlocked

Greg Troxel

unread,
Dec 22, 2023, 8:38:46 PM12/22/23
to Tom Keffer, Greg, weewx-development
Tom Keffer <tke...@gmail.com> writes:

> Hmmm, that didn't work. If you list "wheel" as a dependency, pip isn't
> smart enough to install it first before attempting the others. So, you get
> the same error.

Is it the case that new enough pip installs it? I just did

$ virtualenv-3.11 FOO
created virtual environment CPython3.11.7.final.0-64 in 451ms
creator CPython3Posix(dest=/home/gdt/FOO, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/gdt/.local/share/virtualenv)
added seed packages: pip==23.3.1, setuptools==69.0.2, wheel==0.41.3
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
$ virtualenv-3.11 --version
virtualenv 20.25.0 from /usr/pkg/lib/python3.11/site-packages/virtualenv/__init__.py

so it may be that people with older virtualenv are having trouble?

> I'm reluctant to create yet another step in the install process.

I'd say you aren't creating it, you are describing something that the
python world thinks is a feature!

Tom Keffer

unread,
Dec 22, 2023, 8:43:56 PM12/22/23
to Vince Skahan, weewx-development
Hmm, "weectl report run" works perfectly on my system, using Python 3.11.

It's not clear from your stack trace what raised the exception. Can you dig deeper?

-tk

Vince Skahan

unread,
Dec 22, 2023, 8:46:31 PM12/22/23
to weewx-development
It worked with the --config switch specified ?

If so, let me know (email is fine) and I can try to dig deeper tomorrow....

Tom Keffer

unread,
Dec 23, 2023, 7:56:46 AM12/23/23
to Vince Skahan, weewx-development
On Fri, Dec 22, 2023 at 5:46 PM Vince Skahan <vince...@gmail.com> wrote:
It worked with the --config switch specified ?

Yes. I tried macOS, Debian 12, Ubuntu 23.04, with a mix of Python 3.7 and 3.11 --- all worked with the --config switch.

Vince Skahan

unread,
Dec 23, 2023, 1:09:50 PM12/23/23
to weewx-development
Thanks.   I can't recreate yesterday's boom but I did find one bug.

In weewx@.service, the .conf file it uses for the exec line should say %i.conf (the config file for the instance being started)

matthew wall

unread,
Dec 23, 2023, 3:53:46 PM12/23/23
to weewx-development
On Saturday, December 23, 2023 at 1:09:50 PM UTC-5 Vince Skahan wrote:
In weewx@.service, the .conf file it uses for the exec line should say %i.conf (the config file for the instance being started)

this has been addressed in commit 33f18e8

jpb...@gmail.com

unread,
Dec 24, 2023, 1:17:41 PM12/24/23
to weewx-development
Install of 5.0.0rc1 worked just fine for me. I deleted my weewx-venv tree, and followed the instructions. No errors.

Cameron D

unread,
Dec 25, 2023, 2:34:29 AM12/25/23
to weewx-development
I tried an upgrade from a 4.10.2 simulator system just to see what happened. It looked OK, so I tried on my main system.
  • Debian 12, using the apt package upgrade.
  • I have two hardware units - Oregon WMR300 and EcoWitt 1000 (v0.6.0b2), both with custom databases and the gw1000 has remapped names.
  • I have been running as non-root, but not user  or group named "weewx"
  • I have been running the systems via systemd-multi (for just on 12 months).
I accept many of my problems relate to my non-standard systems and so are of my own making, but here goes...

User name:
I had WEEWX_USER defined in /etc/default/weewx  to be "weather" because that user is also attached to other software unrelated to weewx.  I was slightly surprised that the installer did not check, but that's not a big deal, bacause I doubt there are many in my situation. I modified my operation to use user 'weewx', who becomes a member of group 'weather'.  The installation process  changed ownership of  files under  /etc/weewx, but did not touch anything under HTML_ROOT, so reports failed from then on.

My only suggestion here is it would be good if you mention in the upgrade notes that the ownership change will happen unconditionally and that if people do have weewx running under a non-root user of any other name then convert first.

Udev rules:
The upgrade notes state that the example udev rules allow for group ownership. My installation did not get any rules with owner or group changes in them. The example rules just  give read-write permission to the world.

The sample rules under /etc/weewx/udev are mainly dated 2018 for individual devices, a vantage rule is dated Nov 2021 and an all-in-one "weewx.rules" dated Jan 2021.  None of them sets owner or group.

On my test system, the sample udev rules are all dated Feb 2023, which I guess is from v4.10.2

Reports not completing:
Both systems report via customised Seasons skins, and they appear to be giving reasonable updates to the reports. The WMR300 merges data from both databases and has a separate issue that I will report in detail later. The gw100 seems to give updated reports, including sensible plots and current conditions every 2 minutes.
They sample at 1 minute intervals and  every 2 minutes both systems independently report:
INFO weewx.engine: Launch of report thread aborted: existing report thread still running

Both instances are each consuming 60 to 100% cpu permanently.

When I stop (allow it to die) and restart, there is not much cpu activity for the first minute, so I would guess it is tied into thje report generation.

I'll have a go with the default skin...

Cameron D

unread,
Dec 25, 2023, 4:41:15 AM12/25/23
to weewx-development
A bit more info on the reports.
  1. I disabled any reports from the GW1000 and cpu usage dropped to zero.
  2. I set the WMR300 to just use the Seasons skin as-shipped, but there was no improvement.
  3. I changed wmr300 config to write to a new empty directory (with suitable ownership and permissions) and things got much worse:
  • NOAA dir was created and populated.
  • A set of PNG files and inxex.html were written as expected.
  • the reports now timeout every minute, rather than every 2 minutes.
  • various support files are never copied across - seasons.css and js; background, icons, font, lang folders do not exist (not sure whether some are hangovers from prev versions.)
  • Most files are not updated when expected.

 For example, I deleted all day*.png files and, at the next update, noted the following:
  • the barometer and tempdew were the only images to reappear. 
  •  rss.xml was updated
  • html files celestial, tabular and telemetry were 1 minute old
  • statistics.html was 2 minutes old
  • index.html was 4 min old
  • NOAA had not been updated for 7 min
  • other files were older - css and js files were dated from when I copied them over manually.
  • the files that do get updated within a report period are often dated within 0.1 seconds of each other, but sometimes are seconds apart.

Cameron D

unread,
Dec 25, 2023, 9:05:37 PM12/25/23
to weewx-development
I got basically this same result as Vince, trying to run a report manually.
Running as user weewx or root gave same result.
I have no venv, and no pip install.

A pointer to the cause would be that renaming /usr/share/weewx/user-2023xxx  back to user fixed the problem.

I also tried a symlink from /usr/share/weewx/user pointing to /etc/weewx/bin/user and that worked also.  Including being able to run as user weewx.

And got me back to my other problem, that the report took a long time to run - 1 minute and 30 seconds to generate the full list. I'll report on that separately.

Cameron D

unread,
Dec 25, 2023, 9:42:59 PM12/25/23
to weewx-development
Back to the reports themselves.
I am running the GW1000 system without any reports, and will probably end up running cron jobs to occasionally update the reports.

  • I created an empty folder and pointed the config file there.
  • This is a heavily customised Seasons report, because it is basically indoor air/living quality only.
  • I ran    weectl report run SeasonsReport  --config=/etc/weewx/ewitt.conf
  • then  ls -lrt --full-time to show the increment in file creation times.
  • my customised report has 5 daily, 4 weekly, 3 monthly and 5 yearly plots, so not a particular burdon.
The results show the report  files are generated in short bursts, then a gap, then a few more, etc.  The sequence is...
  • start at 2:14.6 seconds
  • 14.8 s NOAA folder populated by 14.8 seconds
  • 2.5 s later index.html, followed statistics
  • 0.9 s gap to  telemetry, tabular &  celestial, which were within 0.02 s