Import WeatherLinkLive exported CSV data into WeeWx

[Paul]

Hi I've now spent multiple hours to convert with python and hack a import config together but end at:
**** Invalid 'Time' field. Cannot convert '05 Dec 2024 00:00' to timestamp.
**** Nothing done, exiting.

So I hope someone in this forum has an idea how I can fix my issue.

A couple of weeks ago we lost a bit more than a day of recordings because the Ubuntu server hang and no one noticed. We use the WeatherLinkLive receiver for the Davis Vantage Pro 2.

I can see in the App that it continued to record. So I upgraded to Premium and tried to export the data in their web ui thing. This got me a csv sent to my inbox. I attached the file so you can have a look.

I now want to import this into the database of the running weewx instance and overwrite any existing data in the timeframe I import data for.

What is the recommended and simplest way of doing this for my data?

I already tried to do some clean-up and write a import.conf for it which did not work. I also attached this progress.

Many thanks in advance.

Paul

[Paul]

Also I'm currently running WeeWx 5.1.0-4 installed via apt.

[Chuck Rhode]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, 23 Dec 2024 12:04:09 -0800 (PST)
Paul <emili...@gmail.com> wrote:

> > **** Invalid 'Time' field. Cannot convert '05 Dec 2024 00:00' to
> > timestamp. **** Nothing done, exiting.

Just guessing here: Your *.csv timestamps are not what SQL expects.
Try something like this:

> python
>>> import datetime
>>> FMT_IN = '%d %b %Y %H:%S'
>>> time_stamp = '05 Dec 2024 00:00'
>>> datetime.datetime.strptime(time_stamp, FMT_IN).isoformat(sep=' ')
'2024-12-05 00:00:00'
>>> quit()

- --
.. Be Seeing You,
.. Chuck Rhode, Sheboygan, WI, USA
.. Weather: https://LacusVeris.com/Wx
.. 34° — Wind NW at 3 mph. Sky overcast; snow; fog.
-----BEGIN PGP SIGNATURE-----

iF0EARECAB0WIQT+MY/5I/LMPSswTbVg2/xipKOWUgUCZ2nX4gAKCRBg2/xipKOW
UmQMAJ4sm+CRzpSEpek+0UQ9jGo/MwUGhQCfbnaCkTc3rWUX0zPyemoWdWkt2jg=
=Nr6+
-----END PGP SIGNATURE-----

[gjr80]

Paul,

It would be useful to see the actual import config file and the actual CSV file (if the attached import config file is the the one you used then the CSV file would be /home/psemgeo/out.csv) that are actually used for the import along with the console output from weectl import.

Although, if the attached import config file is indeed the one you used then you need to correct the typo in the raw_datetime_format line; you have raw_detetime_format, it should be raw_datetime_format. If you did use that import config file weectl import would (rightfully) ignored raw_detetime_format and used the default %Y-%m-%d %H:%M:%S. This would certainly fail if your date-time data is of the format 05 Dec 2024 00:00.

As an aside I manually ran the %d %b %Y %H:%M format against '05 Dec 2024 00:00' and it was parsed correctly.

You will also find the import config file has a typo in the tranche config option as well; you have trache, it should be tranche. Avoiding typos is one of the reasons we provide example import config files, the recommended approach is to use the example import config files and modify the options (not option names!) as required.

 

I think you are making life more difficult trying to write some python to manipulate the data obtained from Weatherlink so it can be used with weectl import. Any capable text editor (not word processor) could be used to clean up the headings line to remove the odd ball characters. Your numeric data needs to have the quotes removed, but as your data uses a comma both for the field separator and decimal place you will first need to change one or the other - just a simple search an replace (you import config file will need to reflect what character is used for each - ie the delimiter and decimal config options. Then you can remove the quotes - again another simple search and replace. There is no need to alter the date-time data format; weectl import can handle date-time data in the format 5/12/24 00:00, you just need the right format string, in this case something like (untested) %d/%m/%Y %H:%M should work.

Gary