partial packets and archive records

[matthew wall]

hey tom,

for some reason i am seeing heaps of loop packets, but only a few archive records are created from them. the loop packets are partial packets - each loop packet contains data for a single solar panel, while the database has columns for all solar panels. for exampler, panel 1 emits a loop packet with p1_current, p1_voltage, p1_rssi, panel 2 emits a loop packet p2_current, p2_voltage, p2_rssi, etc. the database has columns for 50 solar panels (p1_voltage, p2_voltage, etc). i am currently collecting data from 40 panels.

when i run weewxd directly, i see loop data for every panel, but i see only sporadic REC data. i would have expected to see each archive record populated with data from all of the panels, aggregated from all of the (partial) loop packets.

this is using the weewx-tigo driver:
    https://github.com/matthewwall/weewx-tigo

operating system is debian 12

weewx 5.1 installed from debian package

configuration file 'tigo.conf' is attached

log output 'tigo.log' is attached

this is one of 9 weewx instances running on a home monitor, feeding data into influx and mqtt, where they are used by home assistant and grafana. the other instances monitor per-circuit power consumption, water consumption, propane consumption, weather stations, solar production, and battery status.

gory details: so taking it from the bottom up, we have the following:

1) i am pulling data from a TIGO CCA (cloud connect advanced) monitor using the taptap binary to decode RS485 data from a RS485-to-usb device wired into the 'gateway' port of the CCA (a read-only tap). the taptap application works great - it is sending JSON data as fast as it is coming off the wire. for example, this is output from taptap:

{"gateway":{"id":4610},"node":{"id":37},"timestamp":"2025-10-08T10:38:01.942422696-04:00","voltage_in":31.7,"voltage_out":31.4,"current":0.645,"dc_dc_duty_cycle":1.0,"temperature":15.3,"rssi":126}

2) the weewx-tigo driver captures the JSON from taptap and converts it to weewx packets. for example, this is output from running the driver directly. it maps the TIGO gateway/node identifiers of the form xxxx.yy to weewx schema identifiers of the form p1, p2, p3, etc.

$ PYTHONPATH=/usr/share/weewx python3 /etc/weewx/bin/user/tigo.py --app /opt/taptap/bin/taptap --tap /dev/ttyUSB0 --config /etc/weewx/tigo.conf
'dateTime': '1760016266.812686', 'identifier': '4610.19', 'p19_current': '7.16', 'p19_dc_dc_duty_cycle': '1.0', 'p19_rssi': '123.0', 'p19_temperature': '21.9', 'p19_voltage_in': '33.45', 'p19_voltage_out': '33.1', 'usUnits': '16'
'dateTime': '1760016268.812686', 'identifier': '4610.19', 'p19_current': '7.075', 'p19_dc_dc_duty_cycle': '1.0', 'p19_rssi': '123.0', 'p19_temperature': '21.9', 'p19_voltage_in': '33.9', 'p19_voltage_out': '33.5', 'usUnits': '16'
'dateTime': '1760016268.742698', 'identifier': '4609.4', 'p4_current': '6.56', 'p4_dc_dc_duty_cycle': '1.0', 'p4_rssi': '216.0', 'p4_temperature': '14.1', 'p4_voltage_in': '38.5', 'p4_voltage_out': '38.2', 'usUnits': '16'
'dateTime': '1760016270.742698', 'identifier': '4609.4', 'p4_current': '6.635', 'p4_dc_dc_duty_cycle': '1.0', 'p4_rssi': '216.0', 'p4_temperature': '14.1', 'p4_voltage_in': '38.15', 'p4_voltage_out': '37.8', 'usUnits': '16'
'dateTime': '1760016270.742698', 'identifier': '4609.15', 'p15_current': '6.62', 'p15_dc_dc_duty_cycle': '1.0', 'p15_rssi': '192.0', 'p15_temperature': '19.1', 'p15_voltage_in': '37.9', 'p15_voltage_out': '37.5', 'usUnits': '16'

3) when i run weewxd directly with the driver, i see the loop packets:

$ weewxd /etc/weewx/tigo.conf
Using configuration file /etc/weewx/tigo.conf
LOOP: 2025-10-09 09:29:17 EDT (1760016557) 'dateTime': '1760016557.181576', 'identifier': '4610.22', 'p22_current': '7.435', 'p22_dc_dc_duty_cycle': '1.0', 'p22_rssi': '216.0', 'p22_temperature': '18.1', 'p22_voltage_in': '33.5', 'p22_voltage_out': '33.1', 'usUnits': '1'
LOOP: 2025-10-09 09:29:19 EDT (1760016559) 'dateTime': '1760016559.181576', 'identifier': '4610.22', 'p22_current': '7.39', 'p22_dc_dc_duty_cycle': '1.0', 'p22_rssi': '216.0', 'p22_temperature': '18.1', 'p22_voltage_in': '33.75', 'p22_voltage_out': '33.4', 'usUnits': '1'
LOOP: 2025-10-09 09:29:17 EDT (1760016557) 'dateTime': '1760016557.181576', 'identifier': '4610.40', 'p40_current': '7.32', 'p40_dc_dc_duty_cycle': '1.0', 'p40_rssi': '99.0', 'p40_temperature': '22.2', 'p40_voltage_in': '33.8', 'p40_voltage_out': '33.5', 'usUnits': '1'
LOOP: 2025-10-09 09:29:19 EDT (1760016559) 'dateTime': '1760016559.181576', 'identifier': '4610.40', 'p40_current': '7.425', 'p40_dc_dc_duty_cycle': '1.0', 'p40_rssi': '99.0', 'p40_temperature': '22.2', 'p40_voltage_in': '33.4', 'p40_voltage_out': '33.0', 'usUnits': '1'
LOOP: 2025-10-09 09:29:17 EDT (1760016557) 'dateTime': '1760016557.107579', 'identifier': '4609.8', 'p8_current': '4.155', 'p8_dc_dc_duty_cycle': '0.6078431372549019', 'p8_rssi': '198.0', 'p8_temperature': '19.6', 'p8_voltage_in': '41.0', 'p8_voltage_out': '24.2', 'usUnits': '1'

but you can see that a typical archive record does not include all of the data from loop packets:

$ weewxd /etc/weewx/tigo.conf | grep REC
REC: 2025-10-09 09:35:00 EDT (1760016900) 'dateTime': '1760016900', 'identifier': '4610.302000000001', 'interval': '5.0', 'p21_current': '7.67', 'p21_dc_dc_duty_cycle': '1.0', 'p21_rssi': '159.0', 'p21_temperature': '21.1', 'p21_voltage_in': '33.2', 'p21_voltage_out': '32.9', 'p31_current': '7.58', 'p31_dc_dc_duty_cycle': '1.0', 'p31_rssi': '105.0', 'p31_temperature': '19.2', 'p31_voltage_in': '33.775', 'p31_voltage_out': '33.349999999999994', 'p34_current': '7.577500000000001', 'p34_dc_dc_duty_cycle': '1.0', 'p34_rssi': '135.0', 'p34_temperature': '18.7', 'p34_voltage_in': '33.5', 'p34_voltage_out': '33.150000000000006', 'usUnits': '1'

REC:    2025-10-09 09:40:00 EDT (1760017200) 'dateTime': '1760017200', 'identifier': '4610.302000000001', 'interval': '5.0', 'p21_current': '7.845', 'p21_dc_dc_duty_cycle': '1.0', 'p21_rssi': '159.0', 'p21_temperature': '22.6', 'p21_voltage_in': '33.5', 'p21_voltage_out': '33.2', 'p31_current': '7.9025', 'p31_dc_dc_duty_cycle': '1.0', 'p31_rssi': '102.0', 'p31_temperature': '20.7', 'p31_voltage_in': '33.35', 'p31_voltage_out': '33.0', 'p34_current': '7.890000000000001', 'p34_dc_dc_duty_cycle': '1.0', 'p34_rssi': '135.0', 'p34_temperature': '20.2', 'p34_voltage_in': '33.05', 'p34_voltage_out': '32.6', 'usUnits': '1'

[Tom Keffer]

The only thing I can think of: are you sure that the archive records for 09:35 and 09:40 include LOOP packets for the missing observation types? For example, does it include a packet containing p22_current? You're only showing packets that would go into the previous record (09:30).

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/faa1332c-8cb3-4826-8116-5a539975c549n%40googlegroups.com.

[matthew wall]

On Thursday, October 9, 2025 at 11:31:04 AM UTC-4 tke...@gmail.com wrote:

The only thing I can think of: are you sure that the archive records for 09:35 and 09:40 include LOOP packets for the missing observation types? For example, does it include a packet containing p22_current? You're only showing packets that would go into the previous record (09:30).

capture from two archive intervals is attached.

these are the two archive records:

$ grep REC tigo-capture-251009
REC: 2025-10-09 11:45:00 EDT (1760024700) 'dateTime': '1760024700', 'identifier': '4610.25', 'interval': '5.0', 'p25_current': '0.55', 'p25_dc_dc_duty_cycle': '1.0', 'p25_rssi': '192.0', 'p25_temperature': '32.1', 'p25_voltage_in': '38.6', 'p25_voltage_out': '38.3', 'usUnits': '1'
REC: 2025-10-09 11:50:00 EDT (1760025000) 'dateTime': '1760025000', 'identifier': '4610.25', 'interval': '5.0', 'p25_current': '0.535', 'p25_dc_dc_duty_cycle': '1.0', 'p25_rssi': '192.0', 'p25_temperature': '32.1', 'p25_voltage_in': '37.75', 'p25_voltage_out': '37.5', 'usUnits': '1'

but you can see many other loop packets:

LOOP: 2025-10-09 11:44:45 EDT (1760024685) 'dateTime': '1760024685.50201', 'identifier': '4610.32', 'p32_current': '0.76', 'p32_dc_dc_duty_cycle': '0.9372549019607843', 'p32_rssi': '147.0', 'p32_temperature': '37.0', 'p32_voltage_in': '38.65', 'p32_voltage_out': '35.5', 'usUnits': '1'
LOOP: 2025-10-09 11:44:43 EDT (1760024683) 'dateTime': '1760024683.500156', 'identifier': '4610.31', 'p31_current': '0.6', 'p31_dc_dc_duty_cycle': '1.0', 'p31_rssi': '135.0', 'p31_temperature': '35.5', 'p31_voltage_in': '38.65', 'p31_voltage_out': '38.5', 'usUnits': '1'
LOOP: 2025-10-09 11:44:45 EDT (1760024685) 'dateTime': '1760024685.50201', 'identifier': '4610.31', 'p31_current': '0.82', 'p31_dc_dc_duty_cycle': '1.0', 'p31_rssi': '135.0', 'p31_temperature': '35.5', 'p31_voltage_in': '38.7', 'p31_voltage_out': '38.5', 'usUnits': '1'
LOOP: 2025-10-09 11:44:43 EDT (1760024683) 'dateTime': '1760024683.500156', 'identifier': '4610.38', 'p38_current': '0.435', 'p38_dc_dc_duty_cycle': '1.0', 'p38_rssi': '105.0', 'p38_temperature': '33.4', 'p38_voltage_in': '39.05', 'p38_voltage_out': '38.6', 'usUnits': '1'
LOOP: 2025-10-09 11:44:45 EDT (1760024685) 'dateTime': '1760024685.50201', 'identifier': '4610.38', 'p38_current': '0.59', 'p38_dc_dc_duty_cycle': '1.0', 'p38_rssi': '105.0', 'p38_temperature': '33.4', 'p38_voltage_in': '39.1', 'p38_voltage_out': '38.7', 'usUnits': '1'
LOOP: 2025-10-09 11:44:43 EDT (1760024683) 'dateTime': '1760024683.500156', 'identifier': '4610.30', 'p30_current': '0.595', 'p30_dc_dc_duty_cycle': '1.0', 'p30_rssi': '132.0', 'p30_temperature': '35.7', 'p30_voltage_in': '38.75', 'p30_voltage_out': '38.5', 'usUnits': '1'
LOOP: 2025-10-09 11:44:45 EDT (1760024685) 'dateTime': '1760024685.50201', 'identifier': '4610.30', 'p30_current': '0.82', 'p30_dc_dc_duty_cycle': '1.0', 'p30_rssi': '132.0', 'p30_temperature': '35.7', 'p30_voltage_in': '38.8', 'p30_voltage_out': '38.5', 'usUnits': '1'

[Tom Keffer]

Looking through the capture log, the LOOP packets are arriving out of sequence. This causes accumulators with good data to be thrown away.

Details: The way the engine works, it retains an accumulator for the immediate archive interval. As LOOP packets arrive, they are compared to the interval in the accumulator. If the timestamp of the packet is inside the interval, then it is added. If it is outside, an OutOfSpan exception is raised by the accumulator. The engine catches this and sets the accumulator aside and creates a new accumulator with an archive interval that includes the timestamp of the new LOOP packet. The packet is then added to the new accumulator. LOOP packets continue to arrive and get incorporated into the accumulator. Eventually, a packet arrives with a timestamp far enough into the future (the "archive delay", typically 15 seconds), that a record is extracted out of the old accumulator and put into the database. The old accumulator is discarded.

Now consider an out of sequence packet with a timestamp in the previous archive interval. Its timestamp will be outside what the present accumulator is prepared to accept, so an OutOfSpan exception will be raised. Note that while the intention is to detect when it's time to move on to a new archive interval, a timestamp in the past can have the same effect. The OutOfSpan exception will cause yet another new accumulator to be created, this time with a timestamp in the past. 

You can see how it would be easy to discard perfectly good accumulators to make space for the out of sequence packets.

What to do? I see 3 options:

1. The best solution is to make sure the timestamps of the LOOP packets arrive in monotonically increasing order. I thought we required this somewhere in the documentation, but if we did, I can't find it.
2. Right now, there are two choices when checking a LOOP packet timestamp: it's either in the accumulator's interval, or it is not. Instead, we would allow 3 states: if it's before the interval, discard it. If it's in the interval, incorporate it. If it's after the interval, create a new accumulator (as described above).
3. While the old accumulator is sitting around waiting for the archive delay passes, we could put it in play and allow the out of sequence packet to be added to it. This would give a 15 second grace for LOOP packets to catch up. I'm not keen on this one because it feels ripe for unintended consequences. 

Is solution #1 possible?

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/2636ac0e-994e-4782-861e-e1dcdfdef211n%40googlegroups.com.

[matthew wall]

i should be able to do some buffering in the driver. that gives me a chance to ensure the packets are in monotonically increasing order.

i should probably dump duplicate packets too.

also, the time resolution from taptap is nanoseconds. i trimmed that to microseconds since datetime does not handle nanoseconds (and i don't feel like introducing a pandas or numpy or other massive dependency). that trimming might be causing some of the duplicate packets.

so is it ok to post multiple LOOP packets with the same timestamp? i will end up with duplicate timestamps but different data, for example data from panel 1 might arrive with same timestamp as data from panel 5.

[Tom Keffer]

It's fine to have duplicate timestamps, but it will overweight the value extracted out of the accumulator.

One option would be to switch from the "avg" extractor to "last". Then it would just take the last value it received.

See the wiki article on accumulators.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/7F58A9A9-A349-4F87-95DB-47ABBF980F94%40gmail.com.