Broken decoding of compressed coordinates?

[Pavel Moravec (OK2MOP)]

Hello,

during approximately last week or two the compressed packets with a text comment sent through LoRa iGates started reporting "[Packet corrupted by iGate]". The iGate adds SNR information in this case, however it should be part of the comment section and I do not see it being the reason. Also, displaying decoded packet works.

Was there any breaking change on aprs.fi in last month? The decoding normally displays the values. Previously these messages have been accepted by aprs.fi

My suspicion is that it is because of compression type "Q" which is used by these messages, generated by "Arduino"/ESP32 APRS library, and is probably incorrect based on the documentation (bit 6 should be 0)? Can somebody confirm or deny this? The library is unfortunately being used in multiple projects.

Example of corruption: https://aprs.fi/?c=raw&call=OK2ULQ-7&limit=5&view=decoded

(on OK0DSK-12):

Full message:

2024-06-17 16:24:16 CEST: OK2ULQ-7>APLC13,qAO,OK0DSK-12:!\5IE!S){FkH8QLoRa-APRScube 1Watt SNR=-0dB RSSI=-91db [Packet corrupted by iGate]
   type: location
   format: compressed
   srccallsign: OK2ULQ-7
   dstcallsign: APLC13
   latitude: 49.556617295748 °
   longitude: 18.217107784714 °
   altitude: 383.2801009267 m
   symboltable: \
   symbolcode: k
   messaging: 0
   posresolution: 0.291 m
   comment: LoRa-APRScube 1Watt SNR=-0dB RSSI=-91db

Comment length is not the reason as even following message is considered corrupted:

2024-06-16 11:42:42 CEST: OK2MOP-9>APLRT1,WIDE1-1,qAO,OK2MOP-16:!/5IU:S-nG>I$Q SNR=13dB RSSI=-62dB [Packet corrupted by iGate]

73, Pavel, OK2MOP on behalf of OK2ULQ and others.

[Heikki Hannikainen]

Hi,

On Mon, 17 Jun 2024, Pavel Moravec (OK2MOP) wrote:

> during approximately last week or two the compressed packets with a text
> comment sent through LoRa iGates started reporting "[Packet corrupted by
> iGate]". The iGate adds SNR information in this case, however it should
> be part of the comment section and I do not see it being the reason.
> Also, displaying decoded packet works.

It's precisely that - the corruption is done by the iGate which modifies
the comment field by adding SNR information in the original packet. By
modifying the packet data, the iGate creates a modified duplicate packet.

When the packet goes through multiple igates, and the igates add different
SNR information, the original packet gets multiplied into a number of
different packets with different data content. This breaks the duplicate
packet filtering mechanism of the APRS-IS network. A single original
packet may appear as 3 or 6 or 10 different packets, depending on the
number of digipeaters and igates the packet traverses through. This
increases the load on the APRS-IS network, and its clients (such as
aprs.fi).

aprs.fi generally tries to drop corrupted and duplicated packets, and this
is just a new variation of the filtering of corrupted data. This way
aprs.fi only needs to process and archive the data in the original
unmodified packets.

Please see the iGate developer hints document I've added in the aprsc
documentation a few years back:

https://github.com/hessu/aprsc/blob/main/doc/IGATE-HINTS.md

The APRS-IS servers (javAPRSSrvr and aprsc) contain code to catch some
similar behaviours by buggy iGates (the trimming of whitespace in the end
of packets, by the unmaintained UI-View32 software, and a few others), but
they don't handle the LoRa iGates yet. I'm hoping the LoRa iGates will be
fixed, as they're still being actively developed.

If the developers of the LoRa iGates wish to provide SNR or RSSI
information, it needs to be provided through a different mechanism, but it
needs to be discussed first. I'm happy to discuss options which do not
break current APRS-IS duplicate filtering mechanisms or corrupt packet
data sent by the originator.

- Hessu

[Ricardo Guzman]

Pavel,

to avoid modding the firmware code , modding received LoRa Packets and creating also duplicated packets I've created the "?APRSSR" query on igate firmware (available from 2024.06.15 firmware)

this will answer with a message to the tracker/station a single Signal Report for the Query Packet.

[Lynn W Deffenbaugh (Mr)]

Just to plant one possible idea for IGates to provide access to this
signal strength information while still remaining 100% compliant with
current specifications.

The IGate could maintain a local "heard" list which includes the signal
strength and time information.   An APRS message based query could be
implemented which returns this information in a response to the
requesting station.  That way the original packet is gated, and the
meta-data could be queried on demand and only when needed.

But this discussion should take place on the aprssig or the
ap...@groups.io groups as it has nothing to do with aprs.fi and how it
chooses to process packets!

Lynn (D) - KJ4ERJ - APRS Foundation - Ensuring the Future of APRS

PS.  Or you could join the APRS Foundation Discord and discuss ideas there.

https://discord.gg/JHtAA4nv

[Pavel Moravec (OK2MOP)]

Hello,

Richardo Guzman has already  pointed me in correct direction in (big thanks), but I was asking because I did not see anything in aprs.fi changelog https://aprs.fi/page/changelog to indicate this being intentional and as I do not have Facebook, and did not see any announcement here, I have made the inquiry.

Now the main problem is, that this concerns iot4pi gateways running without issues several years, where the latest release (even on their web) is from 2019. I was able to find repo on github containing probable code of the gateway, but it was distributed only in binary form with gateways, and the historic gcc 4.9  is so old, it may be a problem to compile it from sources for the image they are using. 

I have also patched the binary to stop sending the SNR report for the friend who asked about fixing this on gateway, but we will have to test the fix first.

Regards,

Pavel Moravec, OK2MOP

[Heikki Hannikainen]

Hi,

The changelog is a bit out of date, with the last update being from 2019.
Quite a few small changes have happened since then. Sorry for that.

Facebook has a strong habit of not showing updates to followers unless I
pay to Facebook, so sometimes I skip it too. This update went to Mastodon
and Twitter (X these days... I'll probably abandon it at some point as
it's more and more awful):

https://mastodon.radio/deck/@aprsfi/112520812502849554
https://twitter.com/aprsfi/status/1795560326781358153

But based on my experience, these messages are mostly being ignored, no
matter where I post them. Bug tickets on github projects sometimes work,
but not always.

Unmaintained software is quite a big problem, it'd be great if software
which doesn't get bug fixes any more would be simply replaced with
software which does get fixed. UI-View32 is a good example - it has been
very popular, but it corrupts packets when operating as an iGate, and it
has been unmaintained after the author passed away.

The iGate developer hints document has been there for about 7 years, and
I've advertised it on all possible forums quite a few times. It has
certainly been available in 2019. The first item on the list is "Packets
modified by iGates":

https://github.com/hessu/aprsc/blob/main/doc/IGATE-HINTS.md

> --
> You received this message because you are subscribed to the Google Groups "aprs.fi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> aprsfi+un...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/aprsfi/2c67646e-3e3f-4c16-bf0c-43ffea5e4900n%40googlegroups.com.
>
>

- Hessu

[Nosey Nick VA3NNW]

This thread talks about *duplicates*, which are bad enough, but I think I have to assume this behaviour could also cause *loops*.

It would be perfectly legitimate, perhaps even encouraged, for igates to gate (appropriately-selected) traffic back in the other direction too. If they do so, they should be sending the packets unmodified in that direction too, NOT adding (or indeed stripping) any comments or any other parts of the packet (caveats apply - see Hessu's / other igate guide(s))...

If there were "only" 2 igates, 1 in each direction, I assume you could therefore end up with:

OK2ULQ-7>APLC13,qAO,OK0DSK-12:!\5IE!S){FkH8QLoRa-APRScube 1Watt SNR=-0dB RSSI=-91db

OK2ULQ-7>APLC13,qAO,OK0DSK-12:!\5IE!S){FkH8QLoRa-APRScube 1Watt SNR=-0dB RSSI=-91db SNR=-AdB RSSI=-BBdb

OK2ULQ-7>APLC13,qAO,OK0DSK-12:!\5IE!S){FkH8QLoRa-APRScube 1Watt SNR=-0dB RSSI=-91db SNR=-AdB RSSI=-BBdb SNR=-CdB RSSI=-DDdb

OK2ULQ-7>APLC13,qAO,OK0DSK-12:!\5IE!S){FkH8QLoRa-APRScube 1Watt SNR=-0dB RSSI=-91db SNR=-AdB RSSI=-BBdb SNR=-CdB RSSI=-DDdb SNR=-EdB RSSI=-FFdb ....

These are also not de-duped, because they are DIFFERENT, so if there were multiple igates in different directions, with (presumably) DIFFERENT SNRs, you might have 5, then 25, then 125, then 625 "different" copies of the SAME source packet   :-O

[Heikki Hannikainen]

On Fri, 21 Jun 2024, Nosey Nick VA3NNW wrote:

> This thread talks about *duplicates*, which are bad enough, but I think I have to assume this
> behaviour could also cause *loops*.
>
> It would be perfectly legitimate, perhaps even encouraged, for igates to gate
> (appropriately-selected) traffic back in the other direction too. If they do so, they should be
> sending the packets unmodified in that direction too, NOT adding (or indeed stripping) any comments
> or any other parts of the packet (caveats apply - see Hessu's / other igate guide(s))...

Actually, TX igates must send all frames in the 3rd party format with
"TCPIP" in the inner packet path, and RX igates must not gate such packets
back from RF to APRS-IS. The APRS-IS servers will drop the packet if the
RX igate fails to drop it.

This is documented here:

https://aprs-is.net/IGating.aspx

"""
* An IGate should not gate third-party packets (data type }) with TCPIP or
TCPXX in the third-party header to APRS-IS.
* Third-party packets without TCPIP in the header are to be gated to
APRS-IS AFTER stripping the RF header and third-party data type.
"""

https://aprs-is.net/IGateDetails.aspx

"""
IGates must use the 3rd-party format on RF of

IGATECALL>APRS,GATEPATH:}FROMCALL>TOCALL,TCPIP,IGATECALL*:original packet data

where GATEPATH is the path that the gated packet is to follow on RF. This
format will allow IGates to prevent gating the packet back to APRS-IS. The
format of the 3rd party path (TCPIP,IGATECALL*) is mandatory; APRS-IS
paths MUST be removed before gating to RF.
"""

There have been a few igate implementations out there which fail to do the
3rd party formatting on transmit. When the packets were delayed a bit on
RF (due to digipeaters and congestion), they were re-injected to APRS-IS
by other igates outside the 30-second dupe checking window, and we got
loops. It caused part of the mess on APRS Thursday last year.

The non-third-party packets also caused all other igates and their
upstream APRS-IS servers to think that the originators of the packets were
locally reachable on RF, and all APRS text messages to those stations were
then transmitted to RF by all those other (correctly-working) iGates.

Full story:
https://blog.aprs.fi/2023/06/cg-antenna-gw-1000-aprs-igate-has.html

This 3rd-party packet format, when correctly implemented, should prevent
loops going APRS-IS -> RF -> APRS-IS completely, even if the packet data
is modified.

But yes, modifying the packet does open up other possibilities of packet
loops, if modifications are stacked. It's one good reason to not do it
when duplicate packet suppression is based on packet content.

- Hessu