Difficulties of using wee_import

[P C]

Hello,

I come back to you with my English comprehension problem -> French for the Wee_import utility.

The facts: I have a hole of a good month in my data (from 13/05/21 to 19/06/21). As I am in the "Meteo Belgique" network, I was able to retrieve some data in CSV format: https://pastebin.com/pccbnzhs
I placed the configuration file (named "trou.conf") https://pastebin.com/essg6rbp and the CSV in the /home/pi file

The " wee_import --import-config=/home/pi/trou.conf --dry-run " returns:

Using WeeWX configuration file /etc/weewx/weewx.conf
Starting wee_import...
Traceback (most recent call last):
  File "/usr/share/weewx/weeimport/weeimport.py", line 302, in sourceFactory
    source = import_config_dict['source']
  File "/usr/lib/python3/dist-packages/configobj.py", line 554, in __getitem__
    val = dict.__getitem__(self, key)
KeyError: 'source'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/weewx/wee_import", line 903, in <module>
    main()
  File "/usr/share/weewx/wee_import", line 832, in main
    args)
  File "/usr/share/weewx/weeimport/weeimport.py", line 306, in sourceFactory
    _source_keys = [s for s in SUPPORTED_SOURCES if s in import_config_dict.keys]
  File "/usr/share/weewx/weeimport/weeimport.py", line 306, in <listcomp>
    _source_keys = [s for s in SUPPORTED_SOURCES if s in import_config_dict.keys]
TypeError: argument of type 'method' is not iterable

[gjr80]

Your import config file needs a source entry and it appears it does not exist. Neither of your pastebin links appear to work. If you can’t solve the source problem just paste a copy of trou.conf here, can’t provide any further advice without seeing trou.conf.

Gary

[PC]

OK, I move everything slowly:

I placed everything in the "/etc/weewx/import" directory

wee_import --import-config=/etc/weewx/import/trou.conf --dry-run

Using WeeWX configuration file /etc/weewx/weewx.conf
Starting wee_import...

A CSV import from source file '/etc/weewx/import/data.csv' has been requested.
Using database binding 'wx_binding', which is bound to database 'weewx'
Destination table 'archive' unit system is '0x10' (METRIC).
This is a dry run, imported data will not be saved to archive.
Starting dry run import ...

Traceback (most recent call last):
  File "/usr/share/weewx/wee_import", line 903, in <module>
    main()

  File "/usr/share/weewx/wee_import", line 833, in main
    source_obj.run()
  File "/usr/share/weewx/weeimport/weeimport.py", line 402, in run
    _mapped_data = self.mapRawData(_raw_data, self.archive_unit_sys)
  File "/usr/share/weewx/weeimport/weeimport.py", line 919, in mapRawData
    self.map[_field]['units'],
KeyError: 'units'

Here is the "trou.conf" configuration file

 source = CSV

[CSV]

    file = /etc/weewx/import/data.csv
    delimiter = ;
    decimal = ,
    interval = conf
    qc = True
    calc_missing = False
    ignore_invalid_data = True
    tranche = 250
    UV_sensor = True
    solar_sensor = True
    raw_datetime_format = %Y-%m-%d %H:%M:%S
    rain = discrete
    wind_direction = 0,360

    [[FieldMap]]
        dateTime    = date, unix_epoch
        barometer   = pression, hPa
        outTemp     = Temp, degree_C
        outHumidity = Hum, percent
        windSpeed   = Vent, km_per_hour
        windDir     = Direction, degree_compass
        rain        = Pluie, mm
        dewpoint    = Rose, degree_C
        radiation   = rayonnement, watt_per_meter_squared
        UV          = uv

et le fichier weewx.conf

[gjr80]

In this case you have not specified any units for the ‘uv’ field in your CSV data, try something like:

        UV          = uv, uv_index

Gary

[Blaise Cacramp]

Thank you Gary, it worked but I have a doubt about UV that are abnormally low. It is clearly seen on the graph https://cambier.eu/meteo/yearuv.png (around 06/01)
I do not know if it's the data that's not good
or if it's when importing there was a problem

Le 24-10-21 à 10:32, gjr80 a écrit :

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/b-5D9ZfqXDw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/5a096b18-d78b-4856-a346-e4b85666f217n%40googlegroups.com.

Garanti sans virus. www.avg.com

[gjr80]

Remember by default most year plots plot the day average value. You can always change the year uv plot to the day maximum instead. In the Seasons skin.conf you need to change:

        [[[yearuv]]]

#            yscale = 0, 16, 1

            [[[[UV]]]]

to:

        [[[yearuv]]]

#            yscale = 0, 16, 1

            [[[[UV]]]]

                aggregate_type = max

Remember also that year plots are produced every 24 hours so you will need to wait up to 24 hours for the change to take effect or delete the previously generated year uv plot.

Gary

[Blaise Cacramp]

Yes, I found that I had to wait 24 hours.
But if I understand you correctly, I have to start the import again?

Le 25-10-21 à 12:47, gjr80 a écrit :

> Remember also that year plots are produced every 24 hours so you will
> need to wait up to 24 hours for the change to take effect or delete
> the previously generated year uv plot.

--
Cet email a fait l'objet d'une analyse antivirus par AVG.
http://www.avg.com

[gjr80]

On Monday, 25 October 2021 at 22:12:16 UTC+10 blaise....@gmail.com wrote:

Yes, I found that I had to wait 24 hours.

If a plot includes an aggregate (for example, max or min or average over some interval known as the aggregate interval) then the plot is generated every ‘aggregate interval’. So if your plot is displaying a one hour average the plot is generated every one hour, if it’s displaying a three hour average it is generated every three hours. If there is no aggregate involved the plot is generated every report cycle. The year plots (by default) use a one day (or 24 hour) aggregate interval so are generated every 24 hours. You can of course change the aggregate interval used in the plot to something much less, but this will increase (among other things) the time taken to generate the plot (more correctly it increases the time taken to obtain the data from the database).

 

But if I understand you correctly, I have to start the import again?

No. I never said anything about re-importing data. Once the data is imported it is in the database and running reports and generating plots only reads the data from the database; it does not alter it. So re-importing achieves nothing. What I said is that the default settings used in the year plots will give you a uv plot that appears to have low values. This is because the default year plots mostly display the average daily value (some do not, eg rain). So for an observation like uv that typically peaks around midday and is zero at night the daily average is almost always substantially less than the daily maximum.

What I did say was that you can force WeeWX to regenerate plots on the next report cycle by deleting the previously generated plots on your WeeWX machine. This saves you having to wait for the aggregate interval to pass (saving you having wait 24 hours to pass for a year plot to regenerate).

 

Gary