Bug: Import CSV-File

84 views
Skip to first unread message

Carlo Zero

unread,
Dec 7, 2024, 3:01:22 PM12/7/24
to weewx-user
I am using WeeWX 5.1.0 on a Debian Server. I have installed WeeWX with apt.
I want to import CSV-files into WeeWX. I configured a import conf file and the import seems ok.
Each import line contains the data for one minute.
After checking the data, I noticed that certain rain data had not been transferred. No rain data is imported if the previous data line also contained rain with the same quantity. The remaining data in the line is being imported correctly. Only the rain is not.
The rain is only transferred again when the value in the import field for the rain has changed.
Everything looks like an initialization error in the programming.

gjr80

unread,
Dec 7, 2024, 5:42:54 PM12/7/24
to weewx-user
Sounds like expected operation to me. WeeWX records rainfall on a per-period basis; so an archive record (or for that matter a loop packet) should contain the total rainfall recorded for the period covered by that archive period (or since the last loop packet for loop packets). The field map in an import config file determines which source data is mapped to which WeeWX field and for each field the is_cumulative config option determines whether the source data is a cumulative value or not. If is_cumulative is set True the imported value is the difference between successive values. The default value for is_cumulative is False, but the example import config file has is_cumulative set True for the example rainfall entry (as experience shows most csv imports use a cumulative rainfall value).

Without knowing the nature of your source data, the field map config for your rainfall field and what is actually imported into your archive (you say 'no data is imported' - does the mean the value zero was the result or there is no data at all (the two are quite different)) it's impossible to say what is wrong, if anything at all. If the above does not explain what you are seeing please post an extract of your source data, your import config file and an extract of the imported data (covering the source data extract provided) and we will see what is going on.

Gary

Carlo Zero

unread,
Dec 7, 2024, 11:47:29 PM12/7/24
to weewx-user
Thx for helpinh me so fast.

My parameter for rain in the config file was:
       [[[rain]]]
           source_field = rain
           unit = mm
           is_cumulative = False
After I removed the line with is_cumulative, everything works as it should.
So it makes a difference whether I use 'is_cumulative = False' or the default value.

Carlo

gjr80

unread,
Dec 8, 2024, 7:43:20 PM12/8/24
to weewx-user
Thank you, that gave me some clues. I have found what I suspect is the cause; there appears to be a logic error in the weectl import code that causes is_cumulative to be always set True when an is_cumulative entry is used (omitting is_cumulative or setting is_cumulative = True would always work, but anything else would fail). When I have time I will test and confirm this logic error is the source of the problem and if so it should be fixed in the next WeeWX release.

In the meantime the workaround is to only include an is_cumulative option for source fields that are cumulative (ie do not use is_cumulative = False).

Gary
Reply all
Reply to author
Forward
0 new messages