forcast wunderground for weewx 4.2.x using python 3

256 views
Skip to first unread message

Michael Waldor

unread,
Nov 20, 2020, 4:16:05 AM11/20/20
to weewx-user
Just some days ago I've upgraded my raspberry pi 3 installation of weewx from 3.x to 4.x (forced to because my SD card seemed to be corrupted).

Since I had to reinstall everything I decided to go with the most recent versions - i.e. upgrade raspian from stretch to buster, upgrade weewx from 3.x to 4.x, switch from python 2 to python 3.

Getting weewx 4.x was not straightforward to me (I read my own documentation when I did my first install more then 2 years back and the current upgrade instructions) - by my fault I installed the python 2 weewx by apt-get, but finally I discovered the current apt-get repository, and now weewx-4.2.0 using python 3 is up and running.

I had an additional problem. My weather station is the Renkforce WH2300 using the wh23xx extension. As far as I understand this extension is as of now not fully ported to python 3. But I discovered a fork an github https://github.com/EdwinGH/weewx-wh23xx
That's running smooth but (being beta) floods the logs - thus I've changed its function logmsg to to nothing (python command pass).

And now finally to my remaining problem: I also use the forcast extension with wunderground. Here, too, I'm using a temporary version from https://github.com/chaunceygardiner/weewx-forecast to get early python 3 support. It seems to work - I get forcast messages. But the data shown seem to be partly off. If I do a direct WU query using json like


I get different max/min temperatures compared with the data shown within the seasons skin. Especially the forcast for the current day looks always bad. Did I make a mistake within weewx.conf - it contains only the WU api_key (no separate location)?

John Kline

unread,
Nov 20, 2020, 8:20:43 AM11/20/20
to weewx...@googlegroups.com
Would you please get specific about what you think is wrong?  Include the WU output when you use the URL directly.

Let’s concentrate on temperature, since you’ve called that out.  You’ll see something like this in the output:
"temperatureMax":[7,7,9,8,9,11],"temperatureMin":[-3,-1,4,2,1,4],
Please include that line and the numbers you are seeing in the forecast.  The code is grabbing the high for the daytime 12-hour period and the low form the nighttime 12-hour period.  That probably needs to change as, for example, the low for the day could be lower than the low for the nighttime period.  Let’s see your specific numbers to determine if that is the issue.

On Nov 20, 2020, at 1:16 AM, 'Michael Waldor' via weewx-user <weewx...@googlegroups.com> wrote:


--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/5cacd1fa-51c2-4ad6-b995-5d12d7aa5a7dn%40googlegroups.com.

Michael Waldor

unread,
Nov 20, 2020, 9:59:58 AM11/20/20
to weewx-user
Disclaimer: This is my first usage of google groups - hope the answer reaches this thread.

Ok, from json I get (using my api_key):
"temperatureMax":[null,7,9,8,9,11],"temperatureMin":[-3,-1,4,2,1,4]
which sounds reasonable for my location (see my original post).

Within forecast seasons I see the appended kind of table (I use forecast_compact.css within seasons, but as is from weewx 3.x - no update of skin skin files) - part from /var/www/html/weewx/index.html with the forecast table. To compare the HTML more easily I've extracted the relevant part from my rendered weather page :

Fr FW -2 -2 6,4 10
Sa SC  7 -1 7,2 10
So OV  9 -1 6,4 10
Mo OV  8 -1 6,4 20
Di SC  9 -1 8,0 10

As you see the (rendered) temperatureMin is -2,-1,-1,-1,-1
5day.html

John Kline

unread,
Nov 20, 2020, 10:12:08 AM11/20/20
to weewx...@googlegroups.com
> but as is from weewx 3.x - no update of skin skin files)

You are going to need to install this extension, not just copy over the python file, so we can compare apples to apples here.  I know I had to make more changes than just the python file to get this working.  Please do a proper install and report back.

On Nov 20, 2020, at 7:00 AM, 'Michael Waldor' via weewx-user <weewx...@googlegroups.com> wrote:



Michael Waldor

unread,
Nov 20, 2020, 10:46:51 AM11/20/20
to weewx-user
As I wrote in the original post I did re-install the forecast module using wee_extension. But I use it as part of Seasons skin within the left area below "Current conditions". To get that I had to copy forecast_compact.css, forecast_compact.inc, and the icons into Seasons skin. I only skipped the update of these skin files for my new weewx 4 environment. Now I've compared forecast_compact.* from weewx 3.x with their new successors from weeewx 4 - css is unchanged while there are many changes within inc. Thus I now use that updated forecast_compact.inc. Now the compact forecast contains only 4 instead of 5 columns, but with unchanged data (column Tuesday is missing).

I've re-engaged the forecast skin to get "raw" forecast without Seasons skin. It, too, contains the wrong data with 4 columns (instead of 5) and with "wrong" data.

But to me it's still unclear if I'm using the "correct" recent forecast module weewx-forecast-3.4.0b10.zip
 (https://github.com/chaunceygardiner/weewx-forecast/releases/tag/v3.4.0b10). Is this the correct one?

John Kline

unread,
Nov 20, 2020, 9:53:50 PM11/20/20
to weewx...@googlegroups.com
This is my repository and that is the latest release.  I recall that I had to remove a column because it was not available in the new WU forecast (the old API no longer exists).  You don’t mention the column removed; but if you research it, I think you’ll find it’s not in the forecast returned.  If not, let me know (and please do mention the column you are referring to).

If you want to give me a reproducible case for the lows; using just the forecast pages (i.e., not copying files to seasons) and tell me what page is wrong; I will look at it.  I already have your lat long; so it will be able to reproduce here.

On Nov 20, 2020, at 7:46 AM, 'Michael Waldor' via weewx-user <weewx...@googlegroups.com> wrote:



D R

unread,
Nov 20, 2020, 9:57:55 PM11/20/20
to weewx...@googlegroups.com
The weewx is 4.2.0  Installed quickly, no errors, and is running in test
mode with the simulator showing what I'd expect. I have downloaded all
the install files within the last two days.

I did an update and upgrade for all things on the Rasp400 before
installing it.

The github location was: https://github.com/matthewwall/weewx-ip100
<https://github.com/matthewwall/weewx-ip100>

Yes, and I did try giving it the absolute location as you indicated. 
And I tried moving the file to the /weewx directory, but it said I
didn't have permission to do that.  Not knowing Desbian very well, I
just assumed I made a mistake that I couldn't surmount at the time being.

Dale

John Kline

unread,
Nov 20, 2020, 10:14:56 PM11/20/20
to weewx...@googlegroups.com
I’ve had this driver running for some time; but Matthew never took my pull request; so I just did a release on my clone.  You can try it here:


On Nov 20, 2020, at 6:57 PM, D R <daleea...@gmail.com> wrote:

The weewx is 4.2.0  Installed quickly, no errors, and is running in test mode with the simulator showing what I'd expect. I have downloaded all the install files within the last two days.
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

John Kline

unread,
Nov 20, 2020, 10:15:42 PM11/20/20
to weewx...@googlegroups.com
Oops.  Wrong thread for this reply.

On Nov 20, 2020, at 7:14 PM, John Kline <jo...@johnkline.com> wrote:



John Kline

unread,
Nov 20, 2020, 10:40:27 PM11/20/20
to D R, weewx-user
It’s a release.  It’s on the right side of the page I sent you.

You can also get to it directly here:


You don’t need to get.  Just download it in your browser.



On Nov 20, 2020, at 7:29 PM, D R <daleea...@gmail.com> wrote:



Gosh, now I'm confused.

The readme and the changes say that I should download the 'lastest release" but the change removes the link to the file, and there isn't one substituted to get the new file,  the 0.5 release.  But a search for that file brings me back to Mathew's page and file.

Thanks for the help so far.  Maybe it will gel overnight for me.

Dale
wget -O weewx-ip100.zip https://github.com/matthewwall/weewx-ip100/archive/master.zip
1) download the lastest release
1) download the driver

Michael Waldor

unread,
Nov 21, 2020, 12:18:12 PM11/21/20
to weewx-user
Sorry for my late response. My problem is still there. Just now I've run weewx-4.2.0 using "original" weewx.conf/original skins (as they are after fresh installation) - only changes are my 2 API keys for wunderground. Still I feel that the data look wrong if compared with direct JSON output. I've appended both.
The rendered forecast says
Max -2 8 7 9 °C
Min -2 3 2 1  "C
while the json says
"temperatureMax":[null,8,7,9,10,11],
"temperatureMin":[-2,3,1,1,1,2],

Currently I'm trying to merge "my" old weewx.conf with the one from my fresh weex-4.2.0 installation. Here, too, I encounter spurious behaviour. Both work, but as soon as I merge in all new things from original 4.s.0 into my weewx.conf aftrewards weewx silently dies during start. From my perspective the only changes are whitespace (I know that python is crucial with indentation) and some added units. Ok, within my weewx.conf I want to disable as much logging as possible. These are the kind of changes, e.g.
NEW:                 group_distance     = km                   # Options are 'mile' or 'km'
SPURIOUS:      [[[[[DegreeDays]]]]] and [[[[[Trend]]]]] looses one level of brackets (I've tried both - no improvement)
NEW:                 many prefer_hardware entries
NEW:                xtype_services = weewx.wxxtypes.StdWXXTypes, weewx.wxxtypes.StdPressureCooker, weewx.wxxtypes.StdRainRater (I've tried with and without this line - no improvement)
Currently I've no clue why my settings and the original both work, while their merge not.

To get more information I've also tried debug=1 - no additional message, silent death.
But guess that's another topic that I have to check on my own. I just report it for completeness.

Hope you can reproduce my forecast problem. And thanks for your support,

Michael
5day.json
forcast.tgz

John Kline

unread,
Nov 21, 2020, 1:14:00 PM11/21/20
to weewx...@googlegroups.com
You’ll need to ask about your new issues, unrelated to the forecast extension, on a new thread.

As for what you are seeing in the forecast, the way the extension was written, the two 12 hour periods are reported and the high and low come from that data (NOT the temperatureMin and temperatureMax fields.

I ran your forecast and got:
"daypartName":[null,"Tonight","Tomorrow","Tomorrow night","Monday","Monday night","Tuesday","Tuesday night","Wednesday","Wednesday night","Thursday","Thursday night"]
"temperature":[null,-2,8,2,7,2,9,1,10,1,11,2],
Reformatted for the discussion as:
Null     :  Null, Tonight        : -2
Tomorrow :  8   , Tomorrow Night :  2
Monday   :  7   , Monday Night   :  2
Tuesday  :  9   , Tuesday Night  :  1
Wednesday: 10   , Wednesday Night:  1
Thursday : 11   , Thursday Night :  2
The extension uses the day temp for high and the night temp for low.  Thus:
Day         H/L
Tonight  : -2/-2 (There is no longer a day as it has passed.)
Tomorrow :  8/2
Monday   :  7/2
Tuesday  :  9/1
Wednesday: 10/1
Thursday : 11/2
This matches your report, save for the low tomorrow (which differs by one degree); but that is likely because the forecast used in the report was older than when you got the forecast in your browser (they are only requested periodically, not on every reporting cycle).
On Nov 21, 2020, at 9:18 AM, 'Michael Waldor' via weewx-user <weewx...@googlegroups.com> wrote:


To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/3befaee2-44e0-4a8c-ba78-ba758f85bd02n%40googlegroups.com.
<5day.json>
<forcast.tgz>

Michael Waldor

unread,
Nov 22, 2020, 2:24:46 AM11/22/20
to weewx-user
Thanks for you deeper investigation of my forecast problem. Thus everything is fine within the forcast module. I myself did now increase the number of forecast days to 6 since there is free rendering space on generated HTML page anyway.

And I could resolve my weewx startup issue. I simply re-worked all setup files starting from original 4.2 files and merged in only those changes really needed. And then everything works as expected.

Michael Waldor

unread,
Nov 29, 2020, 4:21:59 AM11/29/20
to weewx-user
I still have trouble with the data displayed from WU forecast. The displayed values for temperature max/min are constant for the current week at 4/-3 °C. Thus I've analyzed the data within /var/lib/forecast.sdb. And here there are the 4/-3 values stored at fields tempMax/tempMin as 40.0/26.0 °F. As you wrote the forecust extension does NOT use tempMin/tempMax, and the values stored within the DB field temp look great - the temperatures are not constant, and their values are more plausible for our area. And they match the json data for field temperature. But those values are NOT shown within the rendered nginx Seasons window.

Maybe I'm using the wrong forecast_compact.inc (see my included file)? And see als my forecast.sdb and 5day.json with the presumably correct temp values, but not shown within the rendered window.

Michael Waldor

unread,
Nov 29, 2020, 4:23:34 AM11/29/20
to weewx-user
Sorry, but google doesn't allow me to append the data. Thus I try again ...

5day.json

Michael Waldor

unread,
Nov 29, 2020, 4:26:00 AM11/29/20
to weewx-user
Tried to rename forecast.sdb into forecast.sdb.txt ....
forecast.sdb.txt

Michael Waldor

unread,
Nov 29, 2020, 4:27:24 AM11/29/20
to weewx-user
Sorry for posting multiple times - but google did abort any post if containing *.json or *.inc files:-(
forecast_compact.inc.txt

John Kline

unread,
Nov 29, 2020, 8:36:07 AM11/29/20
to weewx...@googlegroups.com
> Maybe I'm using the wrong forecast_compact.inc (see my included file)? And see als my forecast.sdb and 5day.json with the presumably correct temp values, but not shown within the rendered window.

Did you diff forecast_compact.inc with the like named file in the latest forecast extension?  Are they identical?  If not, please provide the diff.

Does the compact forecast in the forecast skin have the correct values?

On Nov 29, 2020, at 1:27 AM, Michael Waldor <michael...@gmx.de> wrote:

Sorry for posting multiple times - but google did abort any post if containing *.json or *.inc files:-(
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/23bc8b05-639e-4a2b-9860-0e9b92ca6e8cn%40googlegroups.com.
<forecast_compact.inc.txt>

Michael Waldor

unread,
Nov 30, 2020, 3:09:58 AM11/30/20
to weewx-user
Yes, I did diff both - they are identical. But there is a minor issue: You did update install.py to include frzgrain.png. This is still missing within your  https://github.com/chaunceygardiner/weewx-forecast/releases/tag/v3.4.0b10 - the png is include but it will not be installed. Is there any other important change maybe missing within your last zip release?

Of course that does not explain the wrong temperature numbers. I've checked also the foreground skin. The data are in sync with my change within Seasons skin. But the data do not contain the values from temperature field within forecast.sdb.

Don't know the internal flow of weewx/forecast. I assume that first you fetch json contents from WU. Then that data is uploaded into forecast.sdb. Afterwards during creation of HTML output the data from forecast.sdb is read and added into the HTML templates by forecast.py. If my assumption is correct there might be a problem during the step from forecast.sdg into the HTML output.

John Kline

unread,
Nov 30, 2020, 10:15:24 AM11/30/20
to weewx...@googlegroups.com
frzngdrzl.png was added to the repository after the b10 release.  If you want it, you’ll have to download the repository at head.

I cannot decipher from your reply if the compact forecast is correct in the forecast skin.  Is it?

On Nov 30, 2020, at 12:10 AM, Michael Waldor <michael...@gmx.de> wrote:



Michael Waldor

unread,
Nov 30, 2020, 11:20:10 AM11/30/20
to weewx-user
Regarding missing icon... I already copied it from your zip, and it's working fine. I just thought that the missing icon might be a hint that some important code might be missing, too. But probably not. 

I did use the original forcast skin creating the subfolder forecast within HTML area. And it, too, contains the "wrong" temperature values. Identical to those values as been shown within my usecase of forecast_compact. 

Yesterday I've validated that forcast.sdb contains the "correct" forecast temperatures within the SQL database field temperature (there are 12entries per timestamp for 6days). Hopefully my forecast.sdb.txt posted yesterday is still a valid binary SQL database, then you can inspect it e. g. using sqlbrowser. 

Thus I assume that somewhere within weewx/forecast the wrong data are forwarded into HTML output. 

John Kline

unread,
Nov 30, 2020, 6:23:46 PM11/30/20
to weewx...@googlegroups.com
Thanks, I was just looking for you to confirm that you were seeing the issue in the forecast skin itself.

I looked at forecast_compact.inc.  It is looking for temperatureMin and temperatureMax (which are in the beginning section of the WU forecast).  Furthermore, when I looked at the tempMin and tempMax columns in the database, they were wrong.  There was a bug in parsing for these values.  I’ve fixed that now.

Please pull forecast.py at head and give it a try.  Copy it to <weewx-home>/bin/user.

You’ll need to wait for a new WU forecast to download and then for a report to generate.  This download may take some time.  If you’re comfortable mucking with the database, you could delete the latest WU forecast.  If you’re not comfortable doing that, and you don’t want to wait, you could simply delete the forecast.db file and then restart weewx (which you have to restart anyway after you copy over the new forecast.py file).

On Nov 30, 2020, at 8:20 AM, 'Michael Waldor' via weewx-user <weewx...@googlegroups.com> wrote:

Regarding missing icon... I already copied it from your zip, and it's working fine. I just thought that the missing icon might be a hint that some important code might be missing, too. But probably not. 

John Kline

unread,
Dec 1, 2020, 9:47:50 AM12/1/20
to weewx...@googlegroups.com
This is now released as 3.4.0b11:

On Nov 30, 2020, at 3:23 PM, John Kline <jo...@johnkline.com> wrote:



Michael Waldor

unread,
Dec 3, 2020, 8:42:20 AM12/3/20
to weewx-user
I've pulled your change and waited for some days: Now the temperatures shown look much better. No longer constant values. And the values match with our weather😊

Thanks for your support, 

Michael 

John Kline

unread,
Dec 3, 2020, 9:55:02 AM12/3/20
to weewx...@googlegroups.com
Thanks for getting back to me.

On Dec 3, 2020, at 5:42 AM, 'Michael Waldor' via weewx-user <weewx...@googlegroups.com> wrote:

I've pulled your change and waited for some days: Now the temperatures shown look much better. No longer constant values. And the values match with our weather😊
Reply all
Reply to author
Forward
0 new messages