wee_import

[Willi Dennerlohr]

The csv-import does not work:

Type-Error: "delimitter" must be a 1-character string.

In the conv - file und csv - file is the delimitter a " , ".

What can I do?

[Rainer Lang]

convert " , " into "," (without "") e.g. by using find and replace in Microsoft Excel 😎 - and save as a text file

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/9b2690bc-593b-4cee-b438-00b551e50728n%40googlegroups.com.

[Willi Dennerlohr]

unfortunately no success

You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/YK090sHaud8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/962ba8ac-0911-2b03-b534-a0f7498afcd2%40gmail.com.

[Rainer Lang]

can you provide your .conf file and the first three lines of the csv file (header + 2 data lines) ?

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CALo2f9B4qBghh4akwsJmEp1RnfdGTqoVo6zaiBadLFdWRtxFdg%40mail.gmail.com.

[gjr80]

It seem's something has changed and the delimiter now needs to be enclosed in quotes in the import config file. In your import config file find the line:

    delimiter = ,

and change it to:

    delimiter = ","

Try running your import again. I will make the necessary changes to the example import config files and documentation.

Gary

[Graham Eddy]

isn’t this parsed as a list?

⊣GE⊢

On 12 Mar 2023, at 12:28 pm, gjr80 <gjrod...@gmail.com> wrote:

 delimiter = ,

[gjr80]

Correct, a zero length list.

[gjr80]

Apart from a change required to raw_datetime_format so that wee_import can correctly parse your data field I had no problem importing the data.csv file you emailed using the csv-example.conf import config file in your email:

gary@vm-weewx-expendable:~$ sudo /home/weewx/bin/wee_import --import-config=/var/tmp/csv-example.conf --dry-run
/home/weewx/bin/wee_import:719: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  from distutils.version import StrictVersion
Using WeeWX configuration file /home/weewx/weewx.conf
Starting wee_import...
A CSV import from source file '/var/tmp/data/data.csv' has been requested.
Using database binding 'wx_binding', which is bound to database 'weewx.sdb'
Destination table 'archive' unit system is '0x01' (US).
Missing derived observations will be calculated.
This is a dry run, imported data will not be saved to archive.
Starting dry run import ...
2 records identified for import.
Unique records processed: 2; Last timestamp: 2018-05-23 13:05:00 AEST (1527044700)
Finished dry run import
2 records were processed and 2 unique records would have been imported.
gary@vm-weewx-expendable:~$ cat /var/tmp/data/data.csv
date_and_time,temp,humid,wind,dir,rainfall,rad,river
23 May 2018 13:00,17.4,56,3.0,45,0.0,956,340
23 May 2018 13:05,17.6,56,1.0,22.5,0.4,746,341

I needed to set raw_datetime_format = %d %b %Y %H:%M, otherwise no changes were required. Note, the error in your email is nothing to do with the raw_datetime_format setting, rather it is indicative of an error in the header line of your data file. Is the data file that caused the error shown in your email the same as the data file you emailed me or is it an extract? If it is an extract check the original data file for non-alphanumeric characters in the header line, particularly at the start of the file. This will cause wee_import to be unable to recognise the first field name. If you cannot find anything please reply privately with a copy of the unedited original data file causing the problem.

And please, let's keep the conversation here rather than raising multiple Github issues, we keep GitHub issues for managing bugs, fixes etc.

Gary

 

On 12 Mar 2023, at 20:43, Willi Dennerlohr <willi.de...@gmail.com> wrote: 

the thing with the delimiter is ok, but a new error:

root@server1:~# wee_import --import-config=/etc/weewx/import/csv-example.conf --dry-run
Using WeeWX configuration file /etc/weewx/weewx.conf
Starting wee_import...
A CSV import from source file '/var/tmp/data/data.csv' has been requested.
Using database binding 'wx_binding', which is bound to database 'weewx.sdb'
Destination table 'archive' unit system is '0x01' (US).
Missing derived observations will be calculated.
This is a dry run, imported data will not be saved to archive.
Starting dry run import ...
**** Unable to map source data.
**** Field 'date_and_time' not found in source data.
**** Nothing done, exiting.