Wee-Import UTC Conversion

[Wayne]

I am looking into the process for converting my Davis weather archive files to be imported into weewx. If I'm not mistaken, Davis WeatherLink stores the timestamps in local time in the .wlk files. Accordingly, the application wlkReader creates a csv file also in the same local time format.

The import config file used by the utility wee-import provides an option raw_datetime_format for defining the text date/time format in the csv input file. Although I don't see it explicitly discussed in the Utilities Guide, is there a format code which can be use in this option which applies a UTC offset conversion to the supplied local time? If so can you give an example of its usage? Otherwise the only other option I see is to manually convert the times in the input csv files to UTC prior to importing. I'm sure I am not the first user to encounter this issue! My thanks in advance.

[gjr80]

Short answer is it depends. What is the format of the date-time data in your resulting CSV file? Is it straight date and time but in GMT, eg '24-Jan-2023 18:25' or does it have some sort of timezone indicator embedded eg '24-Jan-2023 18:25 GMT' or is an epoch timestamp eg '1674548700'? WeeWX does not recognise timezones but the python datetime calls used may support GMT. I will need to look at your data and the wee_import date-time parsing routines to see what is possible. 

Gary

[Wayne]

Here is a sample of a CSV format of my data:

Time,TempOut,TempIn,HumOut,HumIn,Baro,Rain,Rate,Wind,WindDir,Gust,GustDir
2020-03-26 10:20:00,56.3,71.8,36.0,31.0,29.742,0.00,-327.68,10.0,270.0,22.0,
2020-03-26 10:30:00,56.8,70.2,35.0,29.0,29.739,0.00,-327.68,12.0,270.0,23.0,
2020-03-26 10:40:00,57.1,69.1,34.0,28.0,29.739,0.00,-327.68,12.0,247.5,22.0,
2020-03-26 10:50:00,57.4,69.0,33.0,27.0,29.742,0.00,-327.68,11.0,247.5,21.0,
2020-03-26 11:00:00,57.5,68.5,33.0,27.0,29.736,0.00,-327.68,13.0,247.5,23.0,

My belief is that this is Local Time. There is no GMT offset nor timezone indicator. These are the first records that I captured at this particular location and I'm pretty sure I started this station around 10 AM and not 3 AM local (if the value shown were UTC). Maybe others could confirm the WeatherLink application behavior. This data came from a Davis Weather Monitor II. I hope this helps.

> --
> You received this message because you are subscribed to the Google
> Groups "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to weewx-user+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/15226f71-27a9-4444-a715-1d8fdc747e09n%40googlegroups.com
> <https://groups.google.com/d/msgid/weewx-user/15226f71-27a9-4444-a715-1d8fdc747e09n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[gjr80]

If that is local time then you are in luck and need do nothing special. If it's GMT/UTC then wee_import cannot handle it as is; it will be treated as local, there is no 'this_is_GMT' option in wee_import. Having looked at the wee_import date-time routines the only possible way to import a GMT/UTC type date-time is if the date-time data was formatted as an epoch timestamp. That being said, it should be fairly easy to implement a 'this_is_GMT' option to interpret the date-time string as GMT/UTC rather than local time, there is sufficient support for GMT/UTC in the python date time library without the need to include timezone data. I'll add it to the list of wee_import ToDos.

Gary

[Wayne]

Help me understand this. The weewx documentation says all data is stored in UTC time. If that is true (and I believe it is) how does my imported data in Local Time get converted to UTC? The only place where I can see that weewx knows my time zone is possibly the Longitude value in weewx.conf. Is there some other place where a GMT offset is stored? Does it look at the system clock offset? I'm really curious about this. Thanks.

> https://groups.google.com/d/msgid/weewx-user/6ba7290e-a0a4-4b08-a57a-c58cf1a60376n%40googlegroups.com
> <https://groups.google.com/d/msgid/weewx-user/6ba7290e-a0a4-4b08-a57a-c58cf1a60376n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[gjr80]

WeeWX stores date-times in unix time which is the number of seconds since the Unix Epoch which is 00:00:00 UTC on 1 January 1970. That is why you might say WeeWX stores date-time in UTC. The python datetime library used by WeeWX for manipulating dates/times knows the local computer time and timezone from the operating system (try typing date at a linux command prompt - you will get a date, time and timezone). The python date time library is thus able to convert between UTC and the local system time. You change the system timezone by using a utility such as timedatectl on systemd based systems. So whilst WeeWX knows the station lat/long, this has nothing to do with determining local time; it is all taken from the system. It is possible to programmatically determine the system offset from UTC through use of some of the methods in the python datetime library. 

WeeWX has no knowledge of any other timezones. Hence my comment that wee_import may have been able to handle UTC times. Any other timezones would be somewhat difficult to implement given the current WeeWX python dependencies.

Gary