Vantage Pro data logger corruption - Daylight Saving Time?

436 views
Skip to first unread message

Alastair L

unread,
Mar 28, 2021, 5:51:52 AM3/28/21
to weewx-user

Hi.  Looking back at previous posts memory corruption of the Davis WS has been discussed a fair bit.  My Pi 4 running Weewx 4.4.0 and connected to a Vantage Pro was working without any problems until 1am this morning when it stopped updating.  I could only get it to update again by clearing the Vantage memory (using wee_device --clear-memory).  My Pi is fitted with a RTC and there was no power outage last night. The only obvious thing that may be relevant is that daylight saving time 2021 in United Kingdom began at 01:00 on Sunday, 28 March,  the time weewx stopped updating.   I was wondering if anyone else with a Davis WS had a similar issue if so is there a fix?  Thanks. 

Mike Revitt

unread,
Mar 28, 2021, 5:57:03 AM3/28/21
to weewx-user
snap, just posted the same message and wee_device --clear-memory was the only way I could get it running again as well.

I want a cron job fix

vigilance wx

unread,
Mar 28, 2021, 6:35:54 AM3/28/21
to weewx-user
same issue
I have been using weewx around 7 years with a vantage and for the most part it runs flawlessly. i have version 3.9.1 running without any issues just this memory problem once every 6 months in march and October
Message has been deleted

Mike Revitt

unread,
Mar 28, 2021, 7:43:48 AM3/28/21
to weewx-user
I have just created a python script that can detect that the clock has changed, which can be used stop WeeWX and clean the Vantage memory, when daylight savings happens then 1 hour later restart WeeWX

I think this code will do what I want, but can only test it twice a year, would appreciate someone checking my logic, and you may need to use something other than UTC for North America.

Also if someone has the Python commands for starting and stopping WeeWX that would also help

import pytz

import datetime


gmt = pytz.utc

bst = pytz.timezone("Europe/London")


gmtStartTime = datetime.datetime.now(gmt).time().replace(microsecond=0)

bstStartTime = datetime.datetime.now(bst).time().replace(microsecond=0)


for utc_transition_time in bst._utc_transition_times:

  if datetime.date.today() == utc_transition_time.date():

    if utc_transition_time.time() <= gmtStartTime <= (utc_transition_time + datetime.timedelta(minutes=65)).time():

      if gmtStartTime < (utc_transition_time + datetime.timedelta(minutes=5)).time():

        print('Stop WeeWX')


      if gmtStartTime > (utc_transition_time + datetime.timedelta(hours=1)).time():

        print('Start WeeWX')


    print(datetime.date.today(), ' == ', utc_transition_time.date())

    print('  ', utc_transition_time.time(), ' <= ', gmtStartTime, ' <= ', (utc_transition_time + datetime.timedelta(minutes=65)).time())

    print('    ', gmtStartTime, ' <  ', (utc_transition_time + datetime.timedelta(minutes=5)).time())

    print('      Stop WeeWX\n')

    print('    ', gmtStartTime, ' > ', (utc_transition_time + datetime.timedelta(hours=1)).time())

    print('      Start WeeWX\n')


Devonian

unread,
Mar 28, 2021, 7:56:25 AM3/28/21
to weewx-user
Ditto here in the UK on my VP2 and Linux Buster on a re-purposed thin-client.
My 'fix' to this annual issue is:

service weewx stop

cd /home/weewx/bin

./wee_device --dump

./wee_device --clear

service weewx start

I don't lose data, but it is kind of annoying!

Nigel

Tom Keffer

unread,
Mar 28, 2021, 8:12:57 AM3/28/21
to weewx-user
Wish I could help you all, but this is a well-known problem with the Vantage logger: the records are stored in local time, with no indication of whether DST applies or not. 

One other solution: switch weewx over to software record generation for the transition. Then switch it back afterwards.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/b16e5a50-0da5-416b-bd59-064d804b2079n%40googlegroups.com.

mwall

unread,
Mar 28, 2021, 8:31:31 AM3/28/21
to weewx-user
would it help to run with the system clock set to UTC so the system and station never do a DST switch?

then display local time in the reports?

Nigel Head

unread,
Mar 28, 2021, 8:44:07 AM3/28/21
to weewx...@googlegroups.com
That is a possible solution Mathew, at least for purely weewx, BUT, the console clock would be 'wrong' and not display local (DST) time.
That wouldn't work for my wife!

Nigel

Tom Keffer

unread,
Mar 28, 2021, 8:48:11 AM3/28/21
to weewx-user
Unfortunately, WeeWX is constructed such that all reports use the system clock's time. So, if the system clock is set to UTC, your reports will be in UTC.

There are a few solutions to that problem, but they would either require a new pre-requisite (pytz), or restrict us to Python 3. 

On Sun, Mar 28, 2021 at 5:31 AM mwall <mw...@users.sourceforge.net> wrote:
would it help to run with the system clock set to UTC so the system and station never do a DST switch?

then display local time in the reports?

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

John Kline

unread,
Mar 28, 2021, 9:32:16 AM3/28/21
to weewx...@googlegroups.com
I have seen an issue when weewx calls set_time within the hour after falling back to standard time (because there is no way in the command to state the time is in DST and the console assumes standard time).  I got around this issue by forking the driver and adding code such that set_time does nothing when set_time is called during these transitions.  As such, I don’t see any issues during the clock changes.

The issue being reported here sounds different.  Are you guys saying you *always* have this problem going in and out of DST?  Do you have the correct DST setting in the console?  Even when I encountered the issue above, I didn’t have to clear memory in my logger.  Tom, do you see this issue?  Do you have to clear memory in your logger every six months?

On Mar 28, 2021, at 5:48 AM, Tom Keffer <tke...@gmail.com> wrote:



Graham Eddy

unread,
Mar 28, 2021, 10:04:11 AM3/28/21
to weewx...@googlegroups.com
i have never seen this problem viz having to clear logger memory to fix DST transition. heck, i have never had to clear logger memory

Greg Troxel

unread,
Mar 28, 2021, 10:11:25 AM3/28/21
to John Kline, weewx...@googlegroups.com

John Kline <jo...@johnkline.com> writes:

> I have seen an issue when weewx calls set_time within the hour after
> falling back to standard time (because there is no way in the command
> to state the time is in DST and the console assumes standard time). I
> got around this issue by forking the driver and adding code such that
> set_time does nothing when set_time is called during these
> transitions. As such, I don’t see any issues during the clock
> changes.

I would agree that weewx being reluctant to set the console time would
be good. Perhaps some notion of only doing so if there is a new offset
for 6 hours, and maybe even if it's consistent for 6 hours.

> The issue being reported here sounds different. Are you guys saying
> you *always* have this problem going in and out of DST? Do you have
> the correct DST setting in the console? Even when I encountered the
> issue above, I didn’t have to clear memory in my logger. Tom, do you
> see this issue? Do you have to clear memory in your logger every six
> months?

I have a VP2 with a Davis serial logger, and run the console and a RPI3
running weewx (NetBSD, python2) in "Eastern" time, which means EST5EDT.
My archive interval is 5 minutes.

I have been doing this since 2018-01 and had no clock-setting trouble
across DST changes.

I do see problems in graph generation on the day of moving to DST, but
<fuzzy_memory>I think that's about calling mktime with time values that
are in the DST gap. This is specified by POSIX to fail:

https://pubs.opengroup.org/onlinepubs/9699919799/functions/mktime.html

and I haven't gotten around to adjusting the code to either not do that
or deal gracefully with the -1.<fuzzy_memory>


(While it would be nice if weewx could collect and store data in UTC and
graph it in localtime, keeping the station in UTC also means that daily
highs/lows are in UTC which is not how weather services do things.)


signature.asc

John Kline

unread,
Mar 28, 2021, 10:11:30 AM3/28/21
to weewx...@googlegroups.com
heck, i have never had to clear logger memory
ditto

On Mar 28, 2021, at 7:04 AM, Graham Eddy <graha...@gmail.com> wrote:


i have never seen this problem viz having to clear logger memory to fix DST transition. heck, i have never had to clear logger memory

On 29 Mar 2021, at 12:32 am, John Kline <jo...@johnkline.com> wrote:

The issue being reported here sounds different.  Are you guys saying you *always* have this problem going in and out of DST?  Do you have the correct DST setting in the console?  Even when I encountered the issue above, I didn’t have to clear memory in my logger.  Tom, do you see this issue?  Do you have to clear memory in your logger every six months?

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

Greg Troxel

unread,
Mar 28, 2021, 10:14:14 AM3/28/21
to weewx...@googlegroups.com

I wonder if the people who are having trouble are in a situation where

the console's firmware idea of DST

and

their computer's idea of DST

are different. I think the OP said "0100" which suggests .eu to me and
I gather the rules have been changing, or maybe just there is a notion
that they will change.



signature.asc

John Kline

unread,
Mar 28, 2021, 10:17:06 AM3/28/21
to Greg Troxel, weewx...@googlegroups.com
> I have been doing this since 2018-01 and had no clock-setting trouble
across DST changes.

You have to be unlucky to see the issue. If WeeWX decided to set the time on the console within the hour of setting the clock back one hour when entering standard time, the clock setting would be one hour off.

Having said that, I believe this is a bit off topic as this issue doesn’t result in memory having to be cleared; and the issue being reported in this thread seems to require clearing memory.

> On Mar 28, 2021, at 7:11 AM, Greg Troxel <g...@lexort.com> wrote:
>
> 

Tom Keffer

unread,
Mar 28, 2021, 10:18:05 AM3/28/21
to weewx-user
I have never had to clear the console memory during DST transitions.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

John Kline

unread,
Mar 28, 2021, 10:18:27 AM3/28/21
to weewx...@googlegroups.com
Hi Greg,

It’s why I asked if they have the DST set correctly in the console.

> On Mar 28, 2021, at 7:14 AM, Greg Troxel <g...@lexort.com> wrote:
>
> 

Greg Troxel

unread,
Mar 28, 2021, 10:23:40 AM3/28/21
to John Kline, weewx...@googlegroups.com

John Kline <jo...@johnkline.com> writes:

> You have to be unlucky to see the issue. If WeeWX decided to set the
> time on the console within the hour of setting the clock back one hour
> when entering standard time, the clock setting would be one hour off.
>
> Having said that, I believe this is a bit off topic as this issue
> doesn’t result in memory having to be cleared; and the issue being
> reported in this thread seems to require clearing memory.

OK - but I still think it would be good if weewx never set the clock +/-
6 h from a DST change, or maybe just never set the clock from 2300 to
0500 daily. Once the console clock is ok this should only be slow drift
and rare. But maybe I am missing something.
signature.asc

John Kline

unread,
Mar 28, 2021, 10:30:45 AM3/28/21
to Greg Troxel, weewx...@googlegroups.com
> OK - but I still think it would be good if weewx never set the clock +/-
> 6 h from a DST change, or maybe just never set the clock from 2300 to
> 0500 daily.


I agree something like that would be good. For now, I’ve got DST enter/exit times specified in my weewx.conf files through to 2040 and drivers that ignore set_time around those transitions.

> On Mar 28, 2021, at 7:23 AM, Greg Troxel <g...@lexort.com> wrote:
>
> 

Greg Troxel

unread,
Mar 28, 2021, 10:31:01 AM3/28/21
to John Kline, weewx...@googlegroups.com

John Kline <jo...@johnkline.com> writes:

>> On Mar 28, 2021, at 7:14 AM, Greg Troxel <g...@lexort.com> wrote:
>>
>> 
>> I wonder if the people who are having trouble are in a situation where
>>
>> the console's firmware idea of DST
>>
>> and
>>
>> their computer's idea of DST
>>
>> are different.

> Hi Greg,
>
> It’s why I asked if they have the DST set correctly in the console.

Good point, but I (didn't say it well and) meant more like the console
and computer have different ideas of the DST rules. In the US, we
changed the rules in 2007 and there were devices with older firmware
following the older rules and other devices and computers following the
newer rules. A lot of people's thermostats in the spring were
e.g. still in standard time for couple of weeks while it was really
daylight time.

I should go look at my VP1 console (bought maybe 2002, ISS failure
~2012) and see if it is still in standard time.
signature.asc

John Kline

unread,
Mar 28, 2021, 10:33:52 AM3/28/21
to Greg Troxel, weewx...@googlegroups.com
Greg, that would be a problem. For firmware that doesn’t recognized current DST rules, setting the times manually would be required. I think the console (at least with current firmware) allows this.

> On Mar 28, 2021, at 7:30 AM, Greg Troxel <g...@lexort.com> wrote:
>
> 

wfs...@gmail.com

unread,
Mar 28, 2021, 12:19:11 PM3/28/21
to weewx-user
I have an original Vantage Pro (VP1) with a serial to USB converter cable attached to a RPi and Weewx does not crash at time changes.  To update the VP firmware for the 2007 DST rule change, Davis had to loan you a special piece of hardware to install the update, and it is indicated at VP console reboot as REV B.  After you patched your firmware, you returned the device to Davis.

Walt

Jon B

unread,
Mar 28, 2021, 3:51:14 PM3/28/21
to weewx-user
I also ran into the same problem when the clocks went forward last night. I've read elsewhere that doing wee_device --dump can solve the issue, but when I do that I get the error:

weewx.CRCError: Unable to pass CRC16 check while getting data

I also get a bunch of errors like this one:

ERROR weewx.manager: Unable to add record 2021-03-28 02:54:00 BST (1616896440) to database 'weewx.sdb': attempt to write a readonly database

which I don't understand, as I didn't have problems like that before the clock change.

Any ideas?

Tom Keffer

unread,
Mar 28, 2021, 3:57:43 PM3/28/21
to weewx-user
1. Make sure you stop weewx before running wee_device.

2. If you do not have write access to your database, you will have to precede the command with "sudo":

sudo wee_device --dump

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

Devonian

unread,
Mar 28, 2021, 4:53:17 PM3/28/21
to weewx-user
Not sure about the console clock and computer clocks not agreeing on DST changeover.  It does appear to (only) affect UK Davis stations?
I also have an old WS2300 running on a r-pi and it never has a DST problem.
Extract from my weewx/VP2 syslog below.
Simply stopped trying to get data at 02:05 and after (everything else runs though)

Mar 28 00:55:15 debian10 weewx[373] INFO weewx.manager: Added record 2021-03-28 00:55:00 GMT (1616892900) to database 'weewx.sdb'
Mar 28 00:55:15 debian10 weewx[373] INFO weewx.manager: Added record 2021-03-28 00:55:00 GMT (1616892900) to daily summary in 'weewx.sdb'
Mar 28 00:55:17 debian10 weewx[373] INFO weewx.cheetahgenerator: Generated 8 files for report StandardReport in 2.22 seconds
Mar 28 00:55:18 debian10 weewx[373] INFO weewx.imagegenerator: Generated 14 images for report StandardReport in 1.01 seconds
Mar 28 00:55:18 debian10 weewx[373] INFO weewx.reportengine: Copied 0 files to /home/weewx/public_html
Mar 28 00:55:21 debian10 weewx[373] INFO weewx.cheetahgenerator: Generated 8 files for report SeasonsReport in 2.17 seconds
Mar 28 00:55:22 debian10 weewx[373] INFO weewx.imagegenerator: Generated 14 images for report SeasonsReport in 1.07 seconds
Mar 28 00:55:22 debian10 weewx[373] INFO weewx.reportengine: Copied 0 files to /home/weewx/public_html
Mar 28 00:55:22 debian10 weewx[373] INFO weewx.reportengine: Copied 0 files to /home/weewx/public_html/ss
Mar 28 00:55:22 debian10 weewx[373] INFO weewx.cheetahgenerator: Generated 2 files for report SteelSeries in 0.11 seconds
Mar 28 00:55:22 debian10 weewx[373] INFO weewx.imagegenerator: Generated 11 images for report SteelSeries in 0.67 seconds
Mar 28 00:55:23 debian10 weewx[373] INFO weewx.cheetahgenerator: Generated 1 files for report wxobs in 0.22 seconds
Mar 28 00:55:23 debian10 weewx[373] INFO weewx.reportengine: Copied 0 files to /home/weewx/public_html/wxobs
Mar 28 02:00:15 debian10 weewx[373] INFO weewx.manager: Added record 2021-03-28 02:00:00 BST (1616893200) to database 'weewx.sdb'
Mar 28 02:00:15 debian10 weewx[373] INFO weewx.manager: Added record 2021-03-28 02:00:00 BST (1616893200) to daily summary in 'weewx.sdb'
Mar 28 02:00:17 debian10 weewx[373] INFO weewx.cheetahgenerator: Generated 8 files for report StandardReport in 2.25 seconds
Mar 28 02:00:22 debian10 weewx[373] INFO weewx.imagegenerator: Generated 28 images for report StandardReport in 4.85 seconds
Mar 28 02:00:22 debian10 weewx[373] INFO weewx.reportengine: Copied 0 files to /home/weewx/public_html
Mar 28 02:00:24 debian10 weewx[373] INFO weewx.cheetahgenerator: Generated 8 files for report SeasonsReport in 2.20 seconds
Mar 28 02:00:29 debian10 weewx[373] INFO weewx.imagegenerator: Generated 28 images for report SeasonsReport in 4.83 seconds
Mar 28 02:00:29 debian10 weewx[373] INFO weewx.reportengine: Copied 0 files to /home/weewx/public_html
Mar 28 02:00:29 debian10 weewx[373] INFO weewx.reportengine: Copied 0 files to /home/weewx/public_html/ss
Mar 28 02:00:29 debian10 weewx[373] INFO weewx.cheetahgenerator: Generated 2 files for report SteelSeries in 0.11 seconds
Mar 28 02:00:30 debian10 weewx[373] INFO weewx.imagegenerator: Generated 11 images for report SteelSeries in 0.67 seconds
Mar 28 02:00:30 debian10 weewx[373] INFO weewx.cheetahgenerator: Generated 1 files for report wxobs in 0.19 seconds
Mar 28 02:00:30 debian10 weewx[373] INFO weewx.reportengine: Copied 0 files to /home/weewx/public_html/wxobs
Mar 28 02:05:19 debian10 weewx[373] INFO weewx.cheetahgenerator: Generated 8 files for report StandardReport in 2.23 seconds
Mar 28 02:05:24 debian10 weewx[373] INFO weewx.imagegenerator: Generated 28 images for report StandardReport in 4.83 seconds
Mar 28 02:05:24 debian10 weewx[373] INFO weewx.reportengine: Copied 0 files to /home/weewx/public_html
Mar 28 02:05:26 debian10 weewx[373] INFO weewx.cheetahgenerator: Generated 8 files for report SeasonsReport in 2.20 seconds
Mar 28 02:05:31 debian10 weewx[373] INFO weewx.imagegenerator: Generated 28 images for report SeasonsReport in 4.90 seconds
Mar 28 02:05:31 debian10 weewx[373] INFO weewx.reportengine: Copied 0 files to /home/weewx/public_html
Mar 28 02:05:31 debian10 weewx[373] INFO weewx.reportengine: Copied 0 files to /home/weewx/public_html/ss
Mar 28 02:05:31 debian10 weewx[373] INFO weewx.cheetahgenerator: Generated 2 files for report SteelSeries in 0.11 seconds
Mar 28 02:05:32 debian10 weewx[373] INFO weewx.imagegenerator: Generated 11 images for report SteelSeries in 0.68 seconds
Mar 28 02:05:32 debian10 weewx[373] INFO weewx.cheetahgenerator: Generated 1 files for report wxobs in 0.20 seconds
Mar 28 02:05:32 debian10 weewx[373] INFO weewx.reportengine: Copied 0 files to /home/weewx/public_html/wxobs

Running ./bin/wee_device weewx.conf --info
returns...
    CONSOLE STATION INFO:
      Latitude (onboard):           +50.9
      Longitude (onboard):          -3.5
      Use manual or auto DST?       AUTO
      DST setting:                  N/A
      Use GMT offset or zone code?  ZONE_CODE
      Time zone code:               18
      GMT offset:                   N/A
      Temperature logging:          LAST
      Retransmit channel:           OFF (0)

Sure would be good to try to find a sensible automatic workaround/fix

Devonian

unread,
Mar 28, 2021, 5:21:24 PM3/28/21
to weewx-user
Extract of log during data dump shows weewx chokes on records >5 minutes after DST changeover (presumably the 5 minutes past DST record 'blocks' subsequent records) until a --dump and/or --clear

Mar 28 09:14:13 debian10 wee_device[23498] ERROR weewx.manager: Unable to add record 2021-03-28 00:50:00 GMT (1616892600) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Mar 28 09:14:13 debian10 wee_device[23498] ERROR weewx.manager: Unable to add record 2021-03-28 00:55:00 GMT (1616892900) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Mar 28 09:14:13 debian10 wee_device[23498] ERROR weewx.manager: Unable to add record 2021-03-28 02:00:00 BST (1616893200) to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime
Mar 28 09:14:13 debian10 wee_device[23498] INFO weewx.manager: Added record 2021-03-28 02:05:00 BST (1616893500) to database 'weewx.sdb'
Mar 28 09:14:13 debian10 wee_device[23498] INFO weewx.manager: Added record 2021-03-28 02:05:00 BST (1616893500) to daily summary in 'weewx.sdb'
Mar 28 09:14:13 debian10 wee_device[23498] INFO weewx.manager: Added record 2021-03-28 02:10:00 BST (1616893800) to database 'weewx.sdb'
Mar 28 09:14:13 debian10 wee_device[23498] INFO weewx.manager: Added record 2021-03-28 02:10:00 BST (1616893800) to daily summary in 'weewx.sdb'
Mar 28 09:14:14 debian10 wee_device[23498] INFO weewx.manager: Added record 2021-03-28 02:15:00 BST (1616894100) to database 'weewx.sdb'
Mar 28 09:14:14 debian10 wee_device[23498] INFO weewx.manager: Added record 2021-03-28 02:15:00 BST (1616894100) to daily summary in 'weewx.sdb'

Tom Keffer

unread,
Mar 28, 2021, 5:25:22 PM3/28/21
to weewx-user
Yup. That's what a corrupt logger looks like: the DMPAFT command stops working and data can no longer be downloaded.

It is a well-known flaw of the Davis logger.

If you want a failsafe solution, switch to software record generation.


--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

Jon B

unread,
Mar 28, 2021, 5:36:07 PM3/28/21
to weewx-user
Ah, thanks, that's done the trick! The data from when the clock changed now appear on my page. Out of interest - while the logs show that the records were added to the database file, I've just had a look and there seems to be a gap from the clock change until I just got it going again, so where is the data coming from/where has it been added? Also, there's no hour gap on the graphs for my station, which I thought there would be (there's data every minute between 01:00 and 02:00)?

Graham Eddy

unread,
Mar 28, 2021, 7:27:34 PM3/28/21
to weewx...@googlegroups.com
so *that* was what the upgrade was. they would only rent us that device (don’t recall how much but it was a lot) and couldn’t say what it did. as you would expect, i refused. glad i did: the change was not even relevant to my country. and the vp2 has ticked quite nicely since, without the update

Tom Keffer

unread,
Mar 28, 2021, 7:56:09 PM3/28/21
to weewx-user
On Sun, Mar 28, 2021 at 2:36 PM Jon B <jonbe...@gmail.com> wrote:
Ah, thanks, that's done the trick! The data from when the clock changed now appear on my page. Out of interest - while the logs show that the records were added to the database file, I've just had a look and there seems to be a gap from the clock change until I just got it going again, so where is the data coming from/where has it been added? Also, there's no hour gap on the graphs for my station, which I thought there would be (there's data every minute between 01:00 and 02:00)?
 
WeeWX uses time.mktime() to convert from local time to UTC. This function takes a "time tuple", one of whose fields is tm_isdst. You're supposed to fill this field out with whether DST is active or not but, of course, we don't know because the Vantage won't tell us. So, we use "-1" which indicates that the system should figure it out. Usually it gets the right answer. Here's what it looks like during the DST transition 14-Mar-2021 here in the USA:

>>> from time import mktime
>>> mktime((2021,3,14,1,0,0,0,0,-1))
1615712400.0
>>> mktime((2021,3,14,2,0,0,0,0,-1))
1615716000.0
>>> mktime((2021,3,14,3,0,0,0,0,-1))
1615716000.0
>>> mktime((2021,3,14,3,1,0,0,0,-1))
1615716060.0

You can see that both 0200 and 0300 give UTC time 261571600. No gap.

You didn't say how large a gap there was in your database. There shouldn't be one because, although local time is discontinuous, as we have seen, UTC is not. Perhaps the logger lost a record when it became corrupted? 

This is also why there is no hour gap in the plots --- time didn't stop or jump ahead, it just changed how it was labelled.. Indeed, if you look closely, you'll see that the vertical tick marks between midnight and 3am are a little closer together than they are between 3 and 6. 

Hope this helps.

-tk

Jon B

unread,
Mar 29, 2021, 7:27:15 AM3/29/21
to weewx-user
Thanks, that makes sense. I just checked the database again and the data are there so I must have been looking in the wrong place, doh!

When it comes to the clocks going back again in October is the same thing likely to happen again? If so I guess the same fix will work again, or is there something I should do to prevent it?

Greg Troxel

unread,
Mar 29, 2021, 8:45:11 AM3/29/21
to Tom Keffer, weewx-user

Tom Keffer <tke...@gmail.com> writes:

> WeeWX uses time.mktime()
> <https://docs.python.org/3/library/time.html#time.mktime> to convert from
> local time to UTC. This function takes a "time tuple", one of whose fields
> is tm_isdst. You're supposed to fill this field out with whether DST is
> active or not but, of course, we don't know because the Vantage won't tell
> us. So, we use "-1" which indicates that the system should figure it out.
> Usually it gets the right answer. Here's what it looks like during the DST
> transition 14-Mar-2021 here in the USA:
>
>>>> from time import mktime
>>>> mktime((2021,3,14,1,0,0,0,0,-1))
> 1615712400.0
>>>> mktime((2021,3,14,2,0,0,0,0,-1))
> 1615716000.0

There is no 2 AM, so mktime should be throwing an error. (It does
actually throw an error on NetBSD.) If you take the returned value from
the 0300 mktime call and convert it back with localtime, you get 3 (my
time_t numbers are different because of being in Boston).

On GNU/linux libc mktime appears to take the perhaps helpful but
non-standard position of returning a time_t anyway. At 0200 exactly
this is one thing, but at say 0230 it's something else entirely.

This is I am pretty sure what causes my graph generation to fail on
spring forward days.

I realize this email doesn't contain a patch to catch the exception and
skip the graph point :-)

signature.asc
Reply all
Reply to author
Forward
0 new messages