Daily rain total and daylight saving time error.

[Russell Swan]

Hi, daylight saving time happened last night and my rain total ($day.rain.sum) for the day does not equal that on the Davis Vantage Pro2. The console reads 0.73" and weewx puts out 0.52". Interestingly, my Steel Gauges indicate the correct 0.73". I think an hour of rain was dropped.

weewx version 5.02

[vince]

I've learned many years ago to just ignore any days when daylight savings changes either way....

[michael.k...@gmx.at]

Just curious: where are you from that DST changed today?

[Russell Swan]

Massachusetts, USA

[gjr80]

If I understand correctly, clocks in Boston went forward one hour at 2am on 10 March. WeeWX should have handled this correctly; the net result being your database will have no data from 2am to 3am 10 March. This is expected behaviour and should not be the cause of any lost data. This is all predicated on the Vantage console clock and daylight saving settings being correctly set and the console changing time as/when it should. If the console behaved correctly I have no idea what the problem could be; you may find some clues by looking at the log from say 1am to 4am on 10 March (ie what records were added to archive) or you may find some clues by looking through the archive table in the database, but these are long shots. Tom or others may have more ideas.

WeeWX does have a known issue when daylight saving ceases in that one hour of data is lost when the clock is wound back by one hour. But I do not recall any issues such as this when daylight saving starts.

I'm guessing you are using the weewx-realtime-gauge_data extension to drive your SteelSeries weather gauges? If so, the reason you did not notice any error is that the weewx-realtime-gauge_data extension keeps its own running daily (midnight reset) rain total based on incoming loop packets. So it is impervious to (well non-midnight/non-1am) daylight saving changes. If you are using a template based gauge-data.txt it will suffer from the same issues as any other WeeWX generated web page as it uses the same $day.rain.sum tag.

Gary

[Tom Keffer]

Let's see what's in the database. I don't know what kind of install you did, but for a package install do this (you type bold text):

sqlite3 /var/lib/weewx/weewx.sdb

for a pip install

sqlite3 ~/weewx-data/archive/weewx.sdb

then see how much rain is in the archive records:

sqlite> select datetime(dateTime,'unixepoch','localtime'), rain from archive where date(dateTime,'unixepoch','localtime')='2024-03-10';

Do they look reasonable?

Also, check the total amount of rain in both the main archive table, and in the daily summaries:

sqlite> select sum(rain) from archive where date(dateTime,'unixepoch','localtime')='2024-03-10';

sqlite> select sum from archive_day_rain where date(dateTime,'unixepoch','localtime')='2024-03-10';

What are the values? Do they match?

On Sun, Mar 10, 2024 at 10:00 AM Russell Swan <Russe...@comcast.net> wrote:

Hi, daylight saving time happened last night and my rain total ($day.rain.sum) for the day does not equal that on the Davis Vantage Pro2. The console reads 0.73" and weewx puts out 0.52". Interestingly, my Steel Gauges indicate the correct 0.73". I think an hour of rain was dropped.

weewx version 5.02

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/2a443e7a-da18-4a65-9eb2-b73f0e4744fan%40googlegroups.com.

[Russell Swan]

Checked the database and I add up 0.52" present. The same as output by $day.rain.sum

I just recently converted to a new skin, Ken True's Saratoga weewx template. On checking the logs if find that at 3am (no 2am) errors are spit out by cheetah. I now suspect the origin for this behavior resides there? After a couple hours of these errors it all runs normal, but with 0.21" missing in the database. It was raining hard.

 Mar 10 03:00:15 weewx-server weewxd[4645]: INFO weewx.manager: Added record 2024-03-10 03:00:00 EDT (1710054000) to database 'weewx.sdb'
Mar 10 03:00:15 weewx-server weewxd[4645]: INFO weewx.manager: Added record 2024-03-10 03:00:00 EDT (1710054000) to daily summary in 'weewx.sdb'
Mar 10 03:00:15 weewx-server weewxd[4645]: INFO weewx.manager: Added record 2024-03-10 03:00:00 EDT (1710054000) to database 'weewxwd.sdb'
Mar 10 03:00:15 weewx-server weewxd[4645]: INFO weewx.manager: Added record 2024-03-10 03:00:00 EDT (1710054000) to daily summary in 'weewxwd.sdb'
Mar 10 03:00:15 weewx-server weewxd[4645]: INFO weewx.restx: Wunderground-PWS: Published record 2024-03-10 03:00:00 EDT (1710054000)
Mar 10 03:00:15 weewx-server weewxd[4645]: INFO weewx.restx: CWOP: Published record 2024-03-10 03:00:00 EDT (1710054000)
Mar 10 03:00:16 weewx-server weewxd[4645]: INFO weewx.cheetahgenerator: Generated 13 files for report LHWReport in 0.78 seconds
Mar 10 03:00:16 weewx-server weewxd[4645]: INFO weewx.imagegenerator: Generated 6 images for report LHWReport in 0.22 seconds
Mar 10 03:00:16 weewx-server weewxd[4645]: INFO weewx.reportengine: Copied 0 files to /var/www/html/weewx
Mar 10 03:00:16 weewx-server weewxd[4645]: INFO weewx.cheetahgenerator: Generated 1 files for report wxobs in 0.06 seconds
Mar 10 03:00:16 weewx-server weewxd[4645]: INFO weewx.reportengine: Copied 0 files to /var/www/html/wxobs
Mar 10 03:00:16 weewx-server weewxd[4645]: INFO weewx.reportengine: Copied 0 files to /var/www/html
Mar 10 03:00:16 weewx-server weewxd[4645]: INFO weewx.cheetahgenerator: Generated 1 files for report SteelSeries in 0.03 seconds
Mar 10 03:00:16 weewx-server weewxd[4645]: INFO weewx.imagegenerator: Generated 8 images for report SteelSeries in 0.15 seconds
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: Evaluation of template /etc/weewx/skins/WEEWXtags/WEEWXtags.php.tmpl failed with exception '<class 'ValueError'>'
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: **** Ignoring template /etc/weewx/skins/WEEWXtags/WEEWXtags.php.tmpl
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: **** Reason: start time (1710057000) is greater than stop time (1710054000)
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: ****  Traceback (most recent call last):
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/cheetahgenerator.py", line 334, in generate
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: ****      unicode_string = compiled_template.respond()
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: ****    File "_etc_weewx_skins_WEEWXtags_WEEWXtags_php_tmpl.py", line 808, in respond
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weewx/tags.py", line 129, in span
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: ****      weeutil.weeutil.archiveSpanSpan(self.report_time, time_delta=time_delta,
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weeutil/weeutil.py", line 414, in archiveSpanSpan
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: ****      return TimeSpan(start_ts, time_ts)
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: ****    File "/usr/share/weewx/weeutil/weeutil.py", line 171, in __new__
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: ****      raise ValueError("start time (%d) is greater than stop time (%d)" % (args[0], args[1]))
Mar 10 03:00:17 weewx-server weewxd[4645]: ERROR weewx.cheetahgenerator: ****  ValueError: start time (1710057000) is greater than stop time (1710054000)
Mar 10 03:00:17 weewx-server weewxd[4645]: INFO weewx.cheetahgenerator: Generated 0 files for report WEEWXtagsReport in 0.63 seconds
Mar 10 03:00:18 weewx-server weewxd[4645]: INFO weewx.imagegenerator: Generated 21 images for report WEEWXtagsReport in 1.02 seconds
Mar 10 03:00:18 weewx-server weewxd[4645]: INFO user.stackedwindrose: Generated 1 images for WEEWXtagsReport in 0.03 seconds
Mar 10 03:00:20 weewx-server weewxd[4645]: INFO weewx.restx: PWSWeather: Published record 2024-03-10 03:00:00 EDT (1710054000)
Mar 10 03:00:20 weewx-server weewxd[4645]: INFO weewx.cheetahgenerator: Generated 3 files for report ClientrawReport in 2.16 seconds
Mar 10 03:00:24 weewx-server weewxd[4645]: INFO weewx.restx: WeatherCloud: Published record 2024-03-10 03:00:00 EDT (1710054000)
Mar 10 03:00:34 weewx-server weewxd[4645]: INFO weewx.reportengine: ftpgenerator: Ftp'd 44 files in 13.57 seconds

[Tom Keffer]

I doubt the error is a problem: it's in the reporting thread, not the main thread. It's the main thread that puts things in the database. The reporting thread only reads from it. 

Make sure we're talking about the same thing: WeeWX reports "rain since midnight". It's possible the skin reports "rain over the last 24 hours."

The console also reports "rain since midnight," so I can't square that with what's in the database and what WeeWX is reporting.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/425b3e5c-1ddf-4b70-afc7-923fe668400dn%40googlegroups.com.

[Russell Swan]

The console reads 0.73" for the day and 0.97" for 24 hours.

Weewx 0.52" for the day and 0.24'" late yesterday.

Missing 0.21" in weewx database.

I've been running weewx since 2016 and never had experienced this with a daylight time change.

Anyway, I added the missing data to the database.

[Greg Troxel]

Why do you assume the console is correct? This is an edge case where
bugs continuing to exist is likely.

If daily rain is midnight to midnight, as is the US custom, then daily
rain on March 10 will be for 23 hours.

I suspect the difference is between

[0000 3/11] - [0000 3/10]

and

[0000 3/11] - [24 hours earlier]

I think the first expression is correct.

I would look at the hourly rain values and add them up and figure out
which is right.

Did you have 0.21" from 2300 on the 9th to 0000 on the 10th? I had
0.10" during that hour (also in MA, but rain is spatially quite
variabl). If so, that explains it. If not, more investigation is
warranted :-)

Greg

[Russell Swan]

The rain which fell during the missing 2 to 3 o’clock hour seems not to have been captured by the WeeWX database. If the console is not correct then nothing down stream of it can be correct either, so no need to be concerned with that. . The discrepancy between the console and database is .21”, and it happened on the day of the daylight saving time change. For the previous day of that rain event, there is no discrepancy.

That’s what I know. Why is the question. Thanks for your input. 

[Greg Troxel]

Well, then I suggesat you file a bug with Davis that the console should
be set to and logging everything in UTC, and only displaying in local
time, and could they please issue a firmware update!

But seriously, keeping logs/databases in local time is a mess.