windGustDir is always set to NULL in database after upgrating to weewx version 4.6.0/4.6.2

[Aslak Vaa]

After upgrating to weewx version 4.6.0/4.6.2 is windGustDir is always set to NULL in database (sqlite3).

Please see results of query from the database and extracts from the syslog file.

I stopped the version 4.5.1 at feb 5 15:02:05, and I started the version 4.6.0 same date half an hour later at 15:32:19. The windDir is saved as expected, only windGustDir is missing.

My weatherstation is model WH1080 , and I am using the FineOffsetUSB driver.

Contents of database:

sqlite3 /var/lib/weewx/weewx.sdb
SQLite version 3.34.1 2021-01-20 14:10:07
Enter ".help" for usage hints.
sqlite> select  dateTime(archive.dateTime, 'unixepoch', 'localtime') dateTime,
   ...>    archive.windSpeed, archive.windDir, archive.windGust, archive.windGustDir

   ...> from archive
   ...> where archive.datetime >

   ...>   (select max(archive.dateTime) from archive where windGustDir is not null) - 1000
   ...> limit 10;
2022-02-05 14:50:00|17.1634447548798|234.745604391204|29.7513266307928|247.5
2022-02-05 15:00:00|17.6108331252676|221.995376457057|27.2906905936595|247.5
2022-02-05 15:14:51|24.3826661861384|202.5|32.6593510383139|
2022-02-05 15:24:51|21.2509475934234|247.5|22.8168068897809|
2022-02-05 15:34:51|22.8168068897809|180.0|27.2906905936595|
2022-02-05 15:40:00|24.3826661861384|225.0|28.1854673344352|
2022-02-05 15:50:00|19.2987074317309|222.892960584793|34.8962928902532|
2022-02-05 16:00:00|19.6444166270306|220.208490631264|34.2252103346714|
2022-02-05 16:10:00|17.3871389400737|226.684593888771|23.4878894453627|
2022-02-05 16:20:00|13.7673603069355|222.317738592905|25.0537487417202|
sqlite>

Extracts from syslog:

aslak-pi0:~ $ grep '4\.5\.' /var/log/syslog.1 | tail
Jan 30 09:51:04 aslak-pi0 weewx[952] INFO __main__: Starting up weewx version 4.5.1
Jan 30 19:29:02 aslak-pi0 weewx[976] INFO __main__: Initializing weewx version 4.5.1
Jan 30 19:29:02 aslak-pi0 weewx[979] INFO __main__: Starting up weewx version 4.5.1
Jan 31 19:28:05 aslak-pi0 weewx[965] INFO __main__: Initializing weewx version 4.5.1
Jan 31 19:28:05 aslak-pi0 weewx[968] INFO __main__: Starting up weewx version 4.5.1
Feb  1 20:32:31 aslak-pi0 weewx[1032] INFO __main__: Initializing weewx version 4.5.1
Feb  1 20:32:31 aslak-pi0 weewx[1037] INFO __main__: Starting up weewx version 4.5.1
Feb  2 15:44:42 aslak-pi0 weewx[15495] INFO __main__: Initializing weewx version 4.5.1
Feb  2 15:44:43 aslak-pi0 weewx[15498] INFO __main__: Starting up weewx version 4.5.1
Feb  5 15:02:05 aslak-pi0 weewx[15498] INFO __main__: Terminating weewx version 4.5.1
aslak-pi0:~ $ grep '4\.6\.0' /var/log/syslog.1 | head
Feb  5 15:32:19 aslak-pi0 weewx[37838] INFO __main__: Initializing weewx version 4.6.0
Feb  5 15:32:19 aslak-pi0 weewx[37841] INFO __main__: Starting up weewx version 4.6.0
Feb  5 15:33:45 aslak-pi0 weewx[37841] INFO __main__: Terminating weewx version 4.6.0
Feb  5 15:35:28 aslak-pi0 weewx[37964] INFO __main__: Initializing weewx version 4.6.0
Feb  5 15:35:28 aslak-pi0 weewx[37967] INFO __main__: Starting up weewx version 4.6.0
Feb  5 16:46:50 aslak-pi0 weewx[37967] INFO __main__: Terminating weewx version 4.6.0
Feb  5 16:46:54 aslak-pi0 weewx[38640] INFO __main__: Initializing weewx version 4.6.0
Feb  5 16:46:54 aslak-pi0 weewx[38643] INFO __main__: Starting up weewx version 4.6.0
Feb  5 16:57:25 aslak-pi0 weewx[38643] INFO __main__: Terminating weewx version 4.6.0
Feb  5 16:57:29 aslak-pi0 weewx[38793] INFO __main__: Initializing weewx version 4.6.0
aslak-pi0:~ $

[Tom Keffer]

FO stations do not supply windGustDir, so the software should substitute windGust in its place.

Could you please run weewxd directly from the command line? Watch what it prints out for records (marked with "REC:"). What does it show for windGust and windGustDir?

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/1acfbc59-01a5-44a9-b1a8-bd000823204fn%40googlegroups.com.

[Aslak Vaa]

Here is the result from running  weewxd  from the command line:

REC:    2022-02-12 14:45:49 CET (1644673549) 'altimeter': '29.559632125170864', 'appTemp': '21.798150800000002', 'barometer': '29.68039690383254', 'cloudbase': '3612.989261474405', 'dateTime': '1644673549', 'delay': '10', 'dewpoint': '27.72361104088051', 'ET': 'None', 'heatindex': '32.0', 'humidex': '32.0', 'inDewpoint': '20.331537695862405', 'inHumidity': '40.0', 'inTemp': '42.980000000000004', 'interval': '10', 'maxSolarRad': '123.63840457180561', 'outHumidity': '84.0', 'outTemp': '32.0', 'outTempBatteryStatus': '0', 'pressure': '26.836852640000004', 'ptr': '37776', 'radiation': 'None', 'rain': 'None', 'rainRate': '0.0', 'rainTotal': '104.33999999999999', 'rxCheckPercent': '100', 'status': '0', 'usUnits': '1', 'UV': 'None', 'windchill': '23.361958924222293', 'windDir': '225.0', 'windGust': '12.079486000472244', 'windGustDir': 'None', 'windrun': '1.7895534815514438', 'windSpeed': '10.737320889308663'

Until version 4.6.0 the windGustDir was recorded in the database with a value different from NULL. It seems for me that something is changed.

Do you mean that  windGustDir should have the value of windDir (windGust is a speed, not a direction)?

[Tom Keffer]

Sorry, yes, that's what I meant. If windGustDir is missing, it should substitute windDir in its place.

Let me think about this for a bit.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7e95cedd-6db0-4bd2-9209-babeee3218e1n%40googlegroups.com.

[Tom Keffer]

Could you do the same again, except this time show the LOOP data as well? 

I'm not exactly sure what's happening, but my operating theory is that windGust and windDir never appear in the same LOOP packet. When the software sees windGust, but no windGustDir, it looks to substitute windDir, but it's not available. We can tell if this theory is correct by looking at the LOOP packets.

[Aslak Vaa]

The result with a couple of LOOP records.

Both windGust and windGustDir occur in both REC and Loop records.

REC:    2022-02-12 19:15:49 CET (1644689749) 'altimeter': '29.447857403881475', 'appTemp': '20.60149362123522', 'barometer': '29.56391226224894', 'cloudbase': '3549.153942263896', 'dateTime': '1644689749', 'delay': '10', 'dewpoint': '28.364486445406747', 'ET': 'None', 'heatindex': '32.36', 'humidex': '32.36', 'inDewpoint': '20.586469406020147', 'inHumidity': '41.0', 'inTemp': '42.620000000000005', 'interval': '10', 'maxSolarRad': '0.0', 'outHumidity': '85.0', 'outTemp': '32.36', 'outTempBatteryStatus': '0', 'pressure': '26.733497683750002', 'ptr': '38208', 'radiation': 'None', 'rain': 'None', 'rainRate': '0.0', 'rainTotal': '104.33999999999999', 'rxCheckPercent': '100', 'status': '0', 'usUnits': '1', 'UV': 'None', 'windchill': '22.558136878652178', 'windDir': '202.5', 'windGust': '15.882287148769064', 'windGustDir': 'None', 'windrun': '2.274224216138293', 'windSpeed': '13.645345296829756'
LOOP:   2022-02-12 19:19:05 CET (1644689945) 'altimeter': '29.444663747054953', 'appTemp': '18.256881727966974', 'barometer': '29.56173534294736', 'cloudbase': '3548.394255363211', 'dateTime': '1644689945', 'delay': '3', 'dewpoint': '28.187829067769762', 'ET': 'None', 'heatindex': '32.18', 'humidex': '32.18', 'inDewpoint': '20.586469406020147', 'inHumidity': '41.0', 'inTemp': '42.620000000000005', 'maxSolarRad': '0.0', 'outHumidity': '85.0', 'outTemp': '32.18', 'outTempBatteryStatus': '0', 'pressure': '26.730544685', 'ptr': '38224', 'radiation': 'None', 'rain': 'None', 'rainRate': '0.0', 'rainTotal': '104.33999999999999', 'rxCheckPercent': '100', 'status': '0', 'usUnits': '1', 'UV': 'None', 'windchill': '20.989026611354667', 'windDir': '202.5', 'windGust': '19.68508829706588', 'windGustDir': 'None', 'windrun': 'None', 'windSpeed': '17.448146445126575'
LOOP:   2022-02-12 19:20:06 CET (1644690006) 'altimeter': '29.441470085033213', 'appTemp': '22.918881727966976', 'barometer': '29.558469574570985', 'cloudbase': '3548.394255363211', 'dateTime': '1644690006', 'delay': '3', 'dewpoint': '28.187829067769762', 'ET': 'None', 'heatindex': '32.18', 'humidex': '32.18', 'inDewpoint': '20.586469406020147', 'inHumidity': '41.0', 'inTemp': '42.620000000000005', 'maxSolarRad': '0.0', 'outHumidity': '85.0', 'outTemp': '32.18', 'outTempBatteryStatus': '0', 'pressure': '26.72759168625', 'ptr': '38224', 'radiation': 'None', 'rain': '0.0', 'rainRate': '0.0', 'rainTotal': '104.33999999999999', 'rxCheckPercent': '100', 'status': '0', 'usUnits': '1', 'UV': 'None', 'windchill': '24.387120697115183', 'windDir': '180.0', 'windGust': '11.408403444890455', 'windGustDir': 'None', 'windrun': 'None', 'windSpeed': '9.171461592951148'

[Remy Lavabre]

Same problem for me !
Since weewx 4.6.x windGustDir is always N/A but the value is correct.
I never had this problem before with 4.5.x or before... Only since 4.6.x

WindGustDir is always None in loop package values :

LOOP:   2022-02-13 10:32:17 CET (1644744737) 'altimeter': '30.122934097139712', 'appTemp': '45.455613403122314', 'barometer': '30.14900779712183', 'beaufort': '1', 'cloudbase': '3288.4136917377255', 'dateTime': '1644744737', 'dayRain': '0.0', 'dewpoint': '41.36', 'ET': 'None', 'extraHumid1': '65', 'extraHumid2': '91', 'extraHumid3': 'None', 'extraHumid4': 'None', 'extraHumid5': 'None', 'extraHumid6': 'None', 'extraHumid7': 'None', 'extraTemp1': '51.980000000000004', 'extraTemp2': '51.980000000000004', 'extraTemp3': 'None', 'extraTemp4': 'None', 'extraTemp5': 'None', 'extraTemp6': 'None', 'extraTemp7': 'None', 'heatindex': '46.88600000000001', 'humidex': '48.74', 'inDewpoint': '52.863712008361716', 'inHumidity': '52', 'inTemp': '71.42', 'maxSolarRad': '326.89345093846424', 'outHumidity': '76', 'outTemp': '48.74', 'pressure': '28.372387999999997', 'rain': '0.0', 'rainRate': '0.0', 'usUnits': '1', 'UV': '0', 'windchill': '48.74', 'windDir': '123', 'windGust': '2.4233536729342466', 'windGustDir': 'None', 'windrun': 'None', 'windSpeed': '2.4233536729342466'
LOOP:   2022-02-13 10:32:29 CET (1644744749) 'altimeter': '30.122934097139712', 'appTemp': '44.055613403122315', 'barometer': '30.14900779712183', 'beaufort': '2', 'cloudbase': '3288.4136917377255', 'dateTime': '1644744749', 'dayRain': '0.0', 'dewpoint': '41.36', 'ET': 'None', 'extraHumid1': '65', 'extraHumid2': '91', 'extraHumid3': 'None', 'extraHumid4': 'None', 'extraHumid5': 'None', 'extraHumid6': 'None', 'extraHumid7': 'None', 'extraTemp1': '51.980000000000004', 'extraTemp2': '51.980000000000004', 'extraTemp3': 'None', 'extraTemp4': 'None', 'extraTemp5': 'None', 'extraTemp6': 'None', 'extraTemp7': 'None', 'heatindex': '46.88600000000001', 'humidex': '48.74', 'inDewpoint': '52.863712008361716', 'inHumidity': '52', 'inTemp': '71.42', 'maxSolarRad': '327.35927830102435', 'outHumidity': '76', 'outTemp': '48.74', 'pressure': '28.372387999999997', 'rain': '0.0', 'rainRate': '0.0', 'usUnits': '1', 'UV': '0', 'windchill': '46.79476438640812', 'windDir': '151', 'windGust': '7.33219829246772', 'windGustDir': 'None', 'windrun': 'None', 'windSpeed': '4.908844619533474'
LOOP:   2022-02-13 10:32:41 CET (1644744761) 'altimeter': '30.122934097139712', 'appTemp': '44.055613403122315', 'barometer': '30.14900779712183', 'beaufort': '2', 'cloudbase': '3288.4136917377255', 'dateTime': '1644744761', 'dayRain': '0.0', 'dewpoint': '41.36', 'ET': 'None', 'extraHumid1': '65', 'extraHumid2': '91', 'extraHumid3': 'None', 'extraHumid4': 'None', 'extraHumid5': 'None', 'extraHumid6': 'None', 'extraHumid7': 'None', 'extraTemp1': '51.980000000000004', 'extraTemp2': '51.980000000000004', 'extraTemp3': 'None', 'extraTemp4': 'None', 'extraTemp5': 'None', 'extraTemp6': 'None', 'extraTemp7': 'None', 'heatindex': '46.88600000000001', 'humidex': '48.74', 'inDewpoint': '52.863712008361716', 'inHumidity': '52', 'inTemp': '71.42', 'maxSolarRad': '327.8246605889759', 'outHumidity': '76', 'outTemp': '48.74', 'pressure': '28.372387999999997', 'rain': '0.0', 'rainRate': '0.0', 'usUnits': '1', 'UV': '0', 'windchill': '46.79476438640812', 'windDir': '109', 'windGust': '7.829296481787566', 'windGustDir': 'None', 'windrun': 'None', 'windSpeed': '4.908844619533474'

[Tom Keffer]

I was able to replicate the problem. Pre V4.6, if windGustDir was missing, windDir was substituted. That behavior went away with v4.6. This patch restores it.

Index: bin/weewx/wxxtypes.py
===================================================================
diff --git a/bin/weewx/wxxtypes.py b/bin/weewx/wxxtypes.py
--- a/bin/weewx/wxxtypes.py (revision 279bb450ab801c80a12b8ceadd0144e16c473bea)
+++ b/bin/weewx/wxxtypes.py (date 1644758479921)
@@ -98,11 +98,18 @@
         return ValueTuple(val, 'degree_compass', 'group_direction')
 
     def calc_windGustDir(self, key, data, db_manager):
-        # Return the current gust direction if windGust is non-zero, otherwise, None
-        if 'windGust' not in data or 'windGustDir' not in data:
+        """If windGustDir is missing, substitute windDir.
+        Set windGustDir to None if windGust is zero."""
+        if 'windGust' not in data:
             raise weewx.CannotCalculate
         if self.force_null and data['windGust'] == 0:
+            # windGust is zero. Force windGustDir to None
             val = None
+        elif 'windGustDir' not in data:
+            # windGustDir is missing. If available, substitute windDir.
+            if 'windDir' not in data:
+                raise weewx.CannotCalculate
+            val = data['windDir']
         else:
             val = data['windGustDir']
         return ValueTuple(val, 'degree_compass', 'group_direction')

Commit 9dc4da8. 

If you want this right away, you can replace weewx/wxxtypes.py with this version.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/70d09feb-27f0-40ce-909b-5d62c5757f6cn%40googlegroups.com.

[Remy Lavabre]

Yes !! Thank-you Tom it works great now :

[Aslak Vaa]

Brilliant, now it work just fine.

Thanks a lot.

[paul...@gmail.com]

I don't report wind gust values/direction in the Current Conditions section of my Seasons skin (version 4.6.2) but in the Statistics (previously High/Low) section I've always had a Max Wind entry with both speed and direction reported. Since 4.6.2 the direction is reported as N/A so I've followed this thread and replaced the wxxtypes.py file with the new one from Tom but it doesn't seem to have made any difference to my Max Wind entry. If you look at the Statistics report below it still shows N/A for Max Wind degrees for Day and Week but for Month and Year (presumably data from 4.5.1) a degree value is reported.

Any suggestions?

Paul

[Remy Lavabre]

Hi Paul.

Have you triedto stop weewx and restart it ?

You must do that if you modify python file.

Cordialy 

[paul...@gmail.com]

Hi Remy

Yes I did sudo /etc/init.d/weewx restart. Just in case I've done it again but no change.

Can I just check I swapped the correct wxxtypes.py file. I swapped the one in /usr/share/weewx/weewx

Paul

[Remy Lavabre]

Yes, it’s the good python file, same as me  (if standard installation...)

The Tom’s update/patch solves perfectly the problem and also min/max windGustDir data base.

Try stop and start weewx (logicaly same thing as restart... !?)

Try to look for all wxxtype.py on your system to see if there is another file ? Logicaly no...

[paul...@gmail.com]

Checked this morning and pleased to say Max Wind direction is now reporting and updating in the Day tab of the Statistics section. Still N/A in Week but I'm guessing it will update in due course.

Just out of interest I've noticed in the Statistics section the Year tab is blue and if you click on it you get the option of Year or Rain Year. What actually is the difference between these two? In my case the values reported are identical.

Paul

[Remy Lavabre]

A water year (also called hydrological year, discharge year or flow year) is a term commonly used in hydrology to describe a time period of 12 months for which precipitation totals are measured. Its beginning differs from the calendar year because part of the precipitation that falls in late autumn and winter accumulates as snow and does not drain until the following spring or summer's snowmelt.

Due to meteorological and geographical factors, the definition of the water years varies; the United States Geological Survey defines it as the period between October 1 of one year and September 30th of the next. The water year is designated by the calendar year in which it ends, so the 2010 water year (USGS) started on October 1, 2009 and ended on September 30, 2010.

One way to identify a water-year is to find that successive 12-month period that most consistently, year after year, gives the highest correlation between precipitation and streamflow and negligible changes in storage (i.e., soil water and snow).

Thank’s Wikipedia...

[Fische Namenlos]

I've similar failures for the WindGust, but only in the creation of "image"  for the stacked WindRose. I use the blue sofaskin on a renkforce WH2315, alias FineOffset.

It was working nicely with 4.5.1...

[Fische Namenlos]

[Tom Keffer]

Upon reflection, I really didn't like the solution I posted earlier. The reality is that for stations that do not supply windGustDir, such as the Fine Offset stations, weewx was calculating a substitute just fine: the vector average of the wind over an archive period was being extracted from the accumulators. The problem is that because of the peculiarities of how WXXTypes was set up, it was forcing windGustDir to None if windGustDir was missing, which caused the calculated value not to get substituted. Then I supplied a band-aid over this by substituting windDir instead.

A mess.

With commit 244a7f8, I've changed this to do nothing in WXXTypes if windGustDir is missing. This causes a value to be successfully extracted from the accumulators.

A long-winded way to say that I think the new solution is a better, more accurate one.

If you're comfortable using git, I'd appreciate it if one of you FO owners could give it a try.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/a2bfcae9-dd7f-43c7-a28c-40ac8a165d07n%40googlegroups.com.

[Aslak Vaa]

I have installed your last update, and I will keep an eye on the results for the next days. Thank you for following up the issue.

However, I performed queries from the accumulator tables for windSpeed, windDir, windGust and windGustDir. The mintime/maxtime for wind and direction are unequal both for windSpeed/windDir and windGust/windGustDir min/max values.

I expected that the min/max values of windDir/windGustDir should be the values at the same mintime/maxtime as the corresponding min/max for windSpeed/windGust  values. This is not the case.

Is this an error or a feature?

Please find queries and results in enclosed file.

Best Regards, Aslak

[Aslak Vaa]

The enclosed file was not correct in my last reply.

Please see this new file.

Best Regards, Aslak

[Tom Keffer]

LOL! Well, I guess you could say it's a feature.

max in archive_day_windSpeed is the maximum value seen for 'windSpeed'.

max in archive_day_windGust is the maximum value seen for 'windGust'.

They are not necessarily the same because archived windSpeed is typically the average wind seen over an archive interval. By contrast, archived windGust is the max wind seen over an archive interval.

Hope that's what you're looking for.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/432b2bea-e5a5-4c16-ab91-8d9e8e6097e5n%40googlegroups.com.

[Aslak Vaa]

Sorry, that was not my intended question.

My issue is that I see wind speed and direction as a pair of values for any given time. When we speak of max wind, I am interested of the speed of wind and its direction at that time. The speed is the main issue, and direction is an attribute to the speed.

This applies both to windSpeed and windGust.

This might be solved removing the accumulation tables for windDir and windGustDir, and adding directions to accumulation tables for windSpeed and windGust both for min and max values of wind speed.

I have not looked into the possibility that there may be other groups of information that may have a similar need of handling.

I see that this may break the accumulation scheme.

So, yes, for the moment I think you are correct in seeing the case as a feature.

[Tom Keffer]

The accumulators for windDir and windGustDir are useless. If the wind blows at 359° for an hour, then 1° for an hour, the windDir accumulator would return 180°. They are there only because it's easiest to just create an accumulator for everything.

Same with the daily summaries for windDir and windGustDir: worthless.

What you want is held in the accumulator for the vector quantity 'wind', where wind is treated as a true vector. It includes attributes max and max_dir, which holds the max wind speed seen and its direction, respectively. 

The equivalent in the database for daily summaries is held in archive_day_wind. There, you can calculate the average wind (from wsum and sumtime), average vector wind direction (from xsum and ysum), max gust (from max), as well as the direction of the max gust (from max_dir). This is all taken care of for you if you use the tag $wind.

Hope I got a little closer to answering your questions!

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/634b7352-be14-4d09-b555-d58702a09d24n%40googlegroups.com.

[Aslak Vaa]

Thank you for the clarification.

[f4n...@gmail.com]

How can I download the single files from github, or could anyone upload these here?

[Tom Keffer]

Several files are affected. I'll try to do a dot-dot release soon.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/c130f77d-c1f6-4573-853b-daf38431bcc2n%40googlegroups.com.