weewx and Klimalogg Pro - looking for advice
Rainer Lang
I'm running weewx on Raspberry Pi with my Klimalogg Pro (KLP) and the KL extension/skin.
At the first start it reads all the existing
data records from the Klimalogg Pro (52,000+) and stores them in
the database.
Great!
Now I observed that when the connection for
some reason gets lost - or the RPi shut down longer than the
archive window -
and weewx restarts (or I restart weewx), it seems to to check
the records stored in the KLP, but skips them all and continues
to store new records only from the new startup date/time and
ignores the records collected by the KLP in between. Out of this
obviously data storage gaps occur.
Anyone has any idea how to make/force weewx continue storing the missing data, i.e. reading them from the KLP and store them based on the last timestamp in the database at restart before it continue storing new data ?
Feedback appreciated.
vince
Rainer Lang
not very long (except for the initial check - here are 51,330 records to be covered)
- the story is (I meanwhile checked the data
base content and the syslog) that weewx continues writing
(empty) records when it isn't connected to the console.
I think that's the reason for not reading the missed records as
the is a record (though empty) for the respective timestamps
(archiving window = 1 minute).
Weewx should stop writing (empty records)
into the database until it receives the signal from the console
again. Then check (like at the initial start)
which historical records are missing, i.e. exist in the console
memory but not yet in the weewx database, retrieve and store
them
and then continue with business as usual (archiving in the
interval, writing summary records, print charts ...)
below the relevant syslog excerpts:
I had weewx starting from scratch again -
empty database - it was then reading all available historical
records in the console and stored (archived) them
see below
........................
Jan 4 14:20:50 raspberrypi weewx[29188] INFO user.kl:
handleHistoryData: records cached=1766, records skipped=7,
next=38341.0
Jan 4 14:20:50 raspberrypi weewx[29188] INFO user.kl:
handleHistoryData: records cached=1772, records skipped=7,
next=38347.0
Jan 4 14:20:51 raspberrypi weewx[29188] INFO user.kl:
handleHistoryData: records cached=1778, records skipped=7,
next=38353.0
Jan 4 14:20:51 raspberrypi weewx[29188] INFO user.kl:
handleHistoryData: records cached=1784, records skipped=7,
next=38359.0
Jan 4 14:20:51 raspberrypi weewx[29188] INFO user.kl:
handleHistoryData: records cached=1790, records skipped=7,
next=38365.0
Jan 4 14:20:52 raspberrypi weewx[29188] INFO user.kl:
handleHistoryData: records cached=1796, records skipped=7,
next=38371.0
Jan 4 14:21:01 raspberrypi weewx[29188] INFO user.kl: Records
scanned: 1800
Jan 4 14:21:01 raspberrypi weewx[29188] INFO user.kl: Found 1799
historical records
Jan 4 14:21:01 raspberrypi weewx[29188] INFO user.kl: Handle
record 46801: 2021-01-01 11:00:00 CET (1609495200)
Jan 4 14:21:01 raspberrypi weewx[29188] INFO user.kl: Handle
record 46802: 2021-01-01 11:01:00 CET (1609495260)
Jan 4 14:21:01 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-01 11:01:00 CET (1609495260) to database
'weewx-kl.sdb'
Jan 4 14:21:01 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-01 11:01:00 CET (1609495260) to daily summary in
'weewx-kl.sdb'
Jan 4 14:21:01 raspberrypi weewx[29188] INFO user.kl: Handle
record 46803: 2021-01-01 11:02:00 CET (1609495320)
Jan 4 14:21:01 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-01 11:02:00 CET (1609495320) to database
'weewx-kl.sdb'
Jan 4 14:21:01 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-01 11:02:00 CET (1609495320) to daily summary in
'weewx-kl.sdb'
Jan 4 14:21:01 raspberrypi weewx[29188] INFO user.kl: Handle
record 46804: 2021-01-01 11:03:00 CET (1609495380)
.................
now the loading comes to an end and weewx runs in normal mode (i.e. reading not history but actual data)
.................
Jan 4 14:33:50 raspberrypi weewx[29188] INFO user.kl: Handle
record 51329: 2021-01-04 14:31:00 CET (1609767060)
Jan 4 14:33:50 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 14:31:00 CET (1609767060) to database
'weewx-kl.sdb'
Jan 4 14:33:50 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 14:31:00 CET (1609767060) to daily summary in
'weewx-kl.sdb'
Jan 4 14:33:50 raspberrypi weewx[29188] INFO user.kl: Handle
record 51330: 2021-01-04 14:32:00 CET (1609767120)
Jan 4 14:33:50 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 14:32:00 CET (1609767120) to database
'weewx-kl.sdb'
Jan 4 14:33:50 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 14:32:00 CET (1609767120) to daily summary in
'weewx-kl.sdb'
Jan 4 14:33:50 raspberrypi weewx[29188] INFO user.kl: Saved
929 historical records; ts last saved record 2021-01-04 14:32:00
CET (1609767120)
Jan 4 14:33:50 raspberrypi weewx[29188] INFO weewx.engine:
Starting main packet loop.
Jan 4 14:34:20 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 14:34:00 CET (1609767240) to database
'weewx-kl.sdb'
Jan 4 14:34:20 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 14:34:00 CET (1609767240) to daily summary in
'weewx-kl.sdb'
Jan 4 14:34:20 raspberrypi weewx[29188] INFO
weewx.cheetahgenerator: Generated 1 files for report kl in 0.28
seconds
Jan 4 14:34:29 raspberrypi weewx[29188] INFO
weewx.imagegenerator: Generated 33 images for report kl in 8.42
seconds
Jan 4 14:35:30 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 14:35:00 CET (1609767300) to database
'weewx-kl.sdb'
Jan 4 14:35:30 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 14:35:00 CET (1609767300) to daily summary in
'weewx-kl.sdb'
....................
Jan 4 14:58:25 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 14:58:00 CET (1609768680) to database
'weewx-kl.sdb'
Jan 4 14:58:25 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 14:58:00 CET (1609768680) to daily summary in
'weewx-kl.sdb'
Jan 4 14:58:25 raspberrypi weewx[29188] INFO
weewx.cheetahgenerator: Generated 1 files for report kl in 0.13
seconds
Jan 4 14:58:26 raspberrypi weewx[29188] INFO
weewx.imagegenerator: Generated 11 images for report kl in 0.70
seconds
Jan 4 14:59:15 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 14:59:00 CET (1609768740) to database
'weewx-kl.sdb'
Jan 4 14:59:15 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 14:59:00 CET (1609768740) to daily summary in
'weewx-kl.sdb'
Jan 4 14:59:15 raspberrypi weewx[29188] INFO
weewx.cheetahgenerator: Generated 1 files for report kl in 0.13
seconds
Jan 4 14:59:16 raspberrypi weewx[29188] INFO
weewx.imagegenerator: Generated 11 images for report kl in 0.73
seconds
Jan 4 15:00:15 raspberrypi weewx[29188] INFO weewx.manager: Added record 2021-01-04
15:00:00 CET (1609768800) to database 'weewx-kl.sdb'
***************** here the first empty record is written as
one see in the database **********
Jan 4 15:00:15 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 15:00:00 CET (1609768800) to daily summary in
'weewx-kl.sdb'
Jan 4 15:00:15 raspberrypi weewx[29188] INFO
weewx.cheetahgenerator: Generated 1 files for report kl in 0.16
seconds
Jan 4 15:00:24 raspberrypi weewx[29188] INFO
weewx.imagegenerator: Generated 33 images for report kl in 8.42
seconds
Jan 4 15:01:15 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 15:01:00 CET (1609768860) to database
'weewx-kl.sdb'
..................
Jan 4 15:17:18 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 15:17:00 CET (1609769820) to database
'weewx-kl.sdb'
Jan 4 15:17:18 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 15:17:00 CET (1609769820) to daily summary in
'weewx-kl.sdb'
Jan 4 15:17:19 raspberrypi weewx[29188] INFO
weewx.cheetahgenerator: Generated 1 files for report kl in 0.13
seconds
Jan 4 15:17:19 raspberrypi weewx[29188] INFO
weewx.imagegenerator: Generated 11 images for report kl in 0.64
seconds
Jan 4 15:18:18 raspberrypi weewx[29188] INFO user.kl: no
new weather data after 1209 seconds
Jan 4 15:18:18 raspberrypi weewx[29188] INFO user.kl: no
contact with console after 1209 seconds (press the USB button
to start communication)
************* of course it takes some time until you realize
that the connection is interrupted and you have to press this
USB button **********
Jan 4 15:18:18 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 15:18:00 CET (1609769880) to database
'weewx-kl.sdb'
Jan 4 15:18:18 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 15:18:00 CET (1609769880) to daily summary in
'weewx-kl.sdb'
...........................
Jan 4 21:19:28 raspberrypi weewx[29188] INFO user.kl: no
new weather data after 22879 seconds
Jan 4 21:19:28 raspberrypi weewx[29188] INFO user.kl: no contact
with console after 22879 seconds (press the USB button to start
communication)
Jan 4 21:20:18 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:20:00 CET (1609791600) to database
'weewx-kl.sdb'
Jan 4 21:20:18 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:20:00 CET (1609791600) to daily summary in
'weewx-kl.sdb'
Jan 4 21:20:19 raspberrypi weewx[29188] INFO
weewx.cheetahgenerator: Generated 1 files for report kl in 0.16
seconds
Jan 4 21:20:19 raspberrypi weewx[29188] INFO
weewx.imagegenerator: Generated 11 images for report kl in 0.63
seconds
Jan 4 21:21:19 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:21:00 CET (1609791660) to database
'weewx-kl.sdb'
Jan 4 21:21:19 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:21:00 CET (1609791660) to daily summary in
'weewx-kl.sdb'
Jan 4 21:21:19 raspberrypi weewx[29188] INFO
weewx.cheetahgenerator: Generated 1 files for report kl in 0.13
seconds
Jan 4 21:21:19 raspberrypi weewx[29188] INFO
weewx.imagegenerator: Generated 11 images for report kl in 0.63
seconds
Jan 4 21:22:19 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:22:00 CET (1609791720) to database
'weewx-kl.sdb'
Jan 4 21:22:19 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:22:00 CET (1609791720) to daily summary in
'weewx-kl.sdb'
Jan 4 21:22:19 raspberrypi weewx[29188] INFO
weewx.cheetahgenerator: Generated 1 files for report kl in 0.14
seconds
Jan 4 21:22:20 raspberrypi weewx[29188] INFO
weewx.imagegenerator: Generated 11 images for report kl in 0.67
seconds
Jan 4 21:23:19 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:23:00 CET (1609791780) to database
'weewx-kl.sdb'
Jan 4 21:23:19 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:23:00 CET (1609791780) to daily summary in
'weewx-kl.sdb'
Jan 4 21:23:19 raspberrypi weewx[29188] INFO
weewx.cheetahgenerator: Generated 1 files for report kl in 0.13
seconds
Jan 4 21:23:20 raspberrypi weewx[29188] INFO
weewx.imagegenerator: Generated 11 images for report kl in 0.62
seconds
Jan 4 21:24:19 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:24:00 CET (1609791840) to database
'weewx-kl.sdb'
Jan 4 21:24:19 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:24:00 CET (1609791840) to daily summary in
'weewx-kl.sdb'
Jan 4 21:24:19 raspberrypi weewx[29188] INFO
weewx.cheetahgenerator: Generated 1 files for report kl in 0.16
seconds
Jan 4 21:24:20 raspberrypi weewx[29188] INFO
weewx.imagegenerator: Generated 11 images for report kl in 0.62
seconds
Jan 4 21:25:19 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:25:00 CET (1609791900) to database
'weewx-kl.sdb'
Jan 4 21:25:19 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:25:00 CET (1609791900) to daily summary in
'weewx-kl.sdb'
Jan 4 21:25:20 raspberrypi weewx[29188] INFO
weewx.cheetahgenerator: Generated 1 files for report kl in 0.13
seconds
Jan 4 21:25:21 raspberrypi weewx[29188] INFO
weewx.imagegenerator: Generated 11 images for report kl in 0.67
seconds
Jan 4 21:25:30 raspberrypi vncserver-x11[472,root]: Connections:
connected: 192.168.1.88::60035 (TCP)
Jan 4 21:25:30 raspberrypi vncserver-x11[472,root]: Connections:
authenticated: 192.168.1.88::60035 (TCP), as pi (f permissions)
Jan 4 21:25:30 raspberrypi colord[495]: failed to get session
[pid 5510]: No data available
Jan 4 21:26:20 raspberrypi
weewx[29188] INFO weewx.manager: Added record 2021-01-04
21:26:00 CET (1609791960) to database 'weewx-kl.sdb'
********** missing connection discovered, USB button
pressed - first non-empty record written - however no
history since interruption (even though the
respective data resides in the console - the trouble is that weewx wrote these
empty records ***********
Jan 4 21:26:20 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:26:00 CET (1609791960) to daily summary in
'weewx-kl.sdb'
Jan 4 21:26:20 raspberrypi weewx[29188] INFO
weewx.cheetahgenerator: Generated 1 files for report kl in 0.13
seconds
Jan 4 21:26:21 raspberrypi weewx[29188] INFO
weewx.imagegenerator: Generated 11 images for report kl in 0.62
seconds
Jan 4 21:27:20 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:27:00 CET (1609792020) to database
'weewx-kl.sdb'
Jan 4 21:27:20 raspberrypi weewx[29188] INFO weewx.manager: Added
record 2021-01-04 21:27:00 CET (1609792020) to daily summary in
'weewx-kl.sdb'
How long does this check take when weewx restarts ? How much data do you think is being missed ?
We'd have to see some logs. Set debug=1 and restart weewx and let it run for a couple archive periods.
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/fe55c5e1-0b37-410f-94bb-9f68c3a2db5dn%40googlegroups.com.
vince
- periodically loses connection to the station
- yet when that happens it keeps writing out empty (your words) records every archive period
- and when you press the button to reconnect to the station, you expect weewx to grab any historical records ?
Rainer Lang
So you're saying:
- periodically loses connection to the station
- rather rarely - but at times (1-5 times in a year, usually due to maintenance - see below)
it also technically happens during a reboot or a short time shut down of the RPi
(before, when I had it run on Windows, reboots for update would interrupt the connection,
and once a power cut)
the hardware - a USB transceiver, is pretty reliable
- yet when that happens it keeps writing out empty (your words) records every archive period
- correct - I guess that's the main issue
- and when you press the button to reconnect to the station, you expect weewx to grab any historical records ?
- wouldn't that make sense ?? Or after a reboot or so ....
The first one sounds like a hardware issue. The second one is a possible bug. The third is a feature request.
If this could be handled by some simple options, entries in e.g. weewx.conf, that would be the advice I'm looking for - I don't know if such a thing exists,I think 2 and 3 are more or less interconnected - the KLP has this memory capacity to allow for disconnections of the software
But then the software should also do it i.e. pick up missed data - the Windows version of the software does this by the way, and does it properly
hence the question. If the developers of the extension still exist and were ready to fix that, this would be even greater.
That sound correct to you ? - with the addenda in bold italics, yes
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/2a8dfdea-8934-4080-96e5-79526d9d16fcn%40googlegroups.com.