Importing WLK files from Davis Instruments WeatherLink software

[Steve Preston]

This is just a FYI for the group.  After using WeatherLink for 15+ years, I have finally switched to WeeWx!

I wrote a basic Windows app for converting the WLK files into CSV format for importing the data into WeeWx.  This app is pretty basic - no real data checks and no attempt to adjust for differences in the archive interval (my station was already logging at 5 minute intervals).  And it is setup to import one month at a time: reads one *.wlk file and writes one *.csv file.  But I've moved over a few years of data to WeeWx without any problems (so far).  The app is written in C# with visual studio and I'm happy to share code and/or executables.

Steve

[Thomas Keffer]

That's great, Steve! The old way was to use a very out-of-date utility that came with wview. Nice to have something more modern.

Could you make an entry in the wiki about your app?

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/6d172d9f-d0f1-422c-9af2-31ed1c47d12f%40googlegroups.com.

[Steve Preston]

I have put the app on GitHub. And the readme has a fair amount of info:

https://github.com/stevepr/wlkReader

I'm happy to include a note on the wiki as well.  But I'm not sure of the proper method for adding info to the wiki. If someone can tell me how I will give it a try.

Steve

[Steve Preston]

Actually... I have two questions about the importing process with wee_import.

1. Is it OK to run the wee_import process during an archive interval "update"?

2. How does wee_import handle changes in the archive interval.  All of the data I have imported to this point has been from wlk files that had a five minute interval.  I may eventually find some earlier wlk files with a shorter, 1 min, interval.  Will this be a problem for wee_import?

Thanks

Steve

On Tuesday, April 14, 2020 at 3:24:16 PM UTC-7, Steve Preston wrote:

[gjr80]

Steve,

The wiki is editable by anyone. I would just create an entry under Data management and add the basic what, where and how.

Gary

[gjr80]

Steve,

Answers below.

Gary

On Thursday, 16 April 2020 02:55:42 UTC+10, Steve Preston wrote:

1. Is it OK to run the wee_import process during an archive interval "update"?

Not exactly sure what you mean here. Do you mean is it OK to run wee_import in between WeeWX adding records to the database during normal operation. If so there is not technical reason why wee_import can not be run while WeeWX is running though I guess you run the risk of contention, WeeWX (wee_import uses the WeeWX API to access the database) and the DBMS in use should handle this but I guess the risk remains that data to be imported may not be imported fully.

If I have missed your question please clarify.

2. How does wee_import handle changes in the archive interval.  All of the data I have imported to this point has been from wlk files that had a five minute interval.  I may eventually find some earlier wlk files with a shorter, 1 min, interval.  Will this be a problem for wee_import?

For a CSV import wee_import can import the interval value directly from the imported data if (1) there is a field in the imported data containing the interval data and (2) there is a suitable entry in the import config file field map (refer [[FieldMap]] in the wee_import section of the Utilities Guide). Otherwise wee_import determines the interval value in one of three ways. wee_import can derive the interval value from the timestamps of subsequent records, this works well if the data being imported in consistent and has not missing records. If the data to be imported is inconsistent and has missing records you can get some odd interval values which may or may not significantly skew the resulting imported data, really depends in the actual interval value and the number of missing records. Secondly, wee_import can use a fixed interval value as specified in the import config file. Finally, wee_import can use the archive_interval value specified under [StdArchive] in weewx.conf.

You will find basic guidance on the interval field for CSV imports here in the wee_import section of the Utilities Guide. Similar guidance is also provided for WU and Cumulus imports.

So back to your question, wee_import should handle changes to the interval just fine (from a data display viewpoint WeeWX has been able to handle a changing interval value since v3.7.0). Mind you I don't recall specifically testing changes to the interval value when developing wee_import so it is possible there may be an issue or two. I guess we will see when you come to import that data :)  

[Steve Preston]

Gary,

Your response has addressed my questions.  During my past imports I have avoided running wee_import while WeeWx is adding new data to the database (during the typical 5 minute intervals in my case).  I will add a comment to the ReadMe instructions on the GitHub page for wlkReader - suggesting that others do the same.

Thanks

Steve

On Tuesday, April 14, 2020 at 3:24:16 PM UTC-7, Steve Preston wrote:

[Steve Preston]

Opps!  I just found (and fixed) a bug in the wlkReader app.  It was swapping Inside / Outside humidity (wrong label on CSV header).

Steve

On Tuesday, April 14, 2020 at 3:24:16 PM UTC-7, Steve Preston wrote: