Weewx WD Import - Rain data units, values

[Václav Bittmann]

Hi, 

Due slow death of my old weather station ( Lacrosse WS2-550, monitored and saved for 14 years with Weather Display ) and death of dedicated laptop (very similar age) running Weather Display,  I'm trying to move to new weather station (DYI Based ) and to weewx. 

I made weewx 5.1 working over weekend (connected by MQTTSubscriber/Driver to my weather station), but I have failed with importing of historical data. WD Import in documentation (https://weewx.com/docs/5.1/utilities/weectl-import-wd/#mapping-data-to-archive-fields) maps log file field rainlastmin to weewx field rain. Unfortunatelly, this field is 0.0 in my log files and only accumulated dailyrain log field is properly populated.  I tried to map field dailyrain with units mm  and reimport (no errors during import), but it seems that weewx field rain expects really delta in particular minute. I have checked it on 1.1.2024, where 0.3mm rain happened on 19:57.  I checked base records, and between record 1187 and 1425 (last minute of day) is in database stored value 0.03, so import works, but NOAA report ( and archive_day_rain table)  shows total rain value for day 7.17. After some investigation, I found that 7.17 is close to (1425-1187)*0.03

I have changed all units to METRIC ( for me easier to calculate than US units) , generated new database before running import. 

My main questions/issues :

Any idea, how to fix import ? 

Is really weewx rain field expected to be rain delta from last period (I need to adjust code of my Weather station to provide this form of data)  ?

Thanks, Vaclav

[vince]

Yes.

In general the archive table is readings during that archive period. I did a little checking in Excel to verify sum of all the 'rain' elements in the archive table in a particular rainy day here indeed matches the 'sum' in the archive_day_rain table to verify this.  It did.

So if your 'rain' element is the rolling total of rain to date for a particular day, you should be able to calculate corrected input data with rain over each archive period. I have to think it's something python or the scripting language of your choice could do pretty quickly.

# example query that does the first 5 archive periods for Sept-13 localtime

echo "select datetime(dateTime,'unixepoch','localtime'),dateTime,rain from archive where dateTime < 1726383600 and dateTime >= 1726210800 limit 5;" | sqlite3 weewx.sdb

# example output

2024-09-13 00:00:00|1726210800|0.0
2024-09-13 00:05:00|1726211100|0.0
2024-09-13 00:10:00|1726211400|0.0
2024-09-13 00:15:00|1726211700|0.0
2024-09-13 00:20:00|1726212000|0.0

Check to see which dateTime your rolling total resets to zero.   It might be 00:00:00, it might be shortly before/after that.

[gjr80]

The WeeWX field rain holds the total rainfall that was recorded during the corresponding archive period, so it is a per-period value rather than a cumulative value. The default WD import config file obtains the WeeWX rain field value from the WD log rainlastmin field, as both are per-period values it is a simple import. If the rainlastmin field is not available any WD cumulative rainfall field can be used; however, as such fields are cumulative you need to tell weectl import that the source rain field is cumulative. This is done with the cumulative option in the field map.

Whilst a daily cumulative rain field can be used as the rain source, if multiple cumulative source fields are available the preferred approach is to use the source field with the longest period (eg monthly cumulative rain or yearly cumulative rain) to avoid possible issues at the cumulative field reset time. For WD I would suggest using (in order of preference)  yearlyrain, monthlyrain, dailyrain.

 Try editing your WD import config file and change the [[[rain]]] stanza under [[FieldMap]] as follows:

        [[[rain]]]
            source_field = yearlyrain
            unit = mm

            cumulative = True

Re-import the data and see how that goes. Note that you will need to delete the previously imported data before re-importing.

Gary

[Václav Bittmann]

Hi, Garry, 

thanks for your help, this is right solution of my issue, just small syntax correction ( for other users) 

        [[[rain]]]
            source_field = dailyrain
            unit = mm
            is_cumulative = True

With that option data was imported properly and summaries for imported months are matching my own database. 

I still  need to build some old data cleansing method, but this is not related to weewx itself. Just for curiosity, old weather station together with Wdisplay sometimes (I never found source of issue) generated random spike in rain data, where spike size was accumulated count of rain-gauge pulses for outdoor sensor runtime  ( simply some radio packet message had part of payload populated by 0  and next packet was fully populated again, most probably some missing CRC check on transferred data), in some days I was happy and corrected  data during day in WD, sometimes not. All this spikes are part of my WD Logs. I need to find way how to clean data either before or after import, I had some correction in my databases, but direct export will cover only errors, which I forgotten/missed during days, not these, which I manually corrected in WD, they are in dailyRain column as negative changes. 

Regards, Vaclav 

po 23. 9. 2024 v 9:39 odesílatel gjr80 <gjrod...@gmail.com> napsal:

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/Umc6L8NqXvc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/96cd8466-c330-4021-81ca-4bf89819f204n%40googlegroups.com.