erratic dupe filtering based on time stamps? [Location changes too fast (>500 km/h)]
Mike, OE3MZC
hi all,
recently I have observed following problem:
since about Easter, there is seems to be some problems with either the way that dupes are filtered in the tier3 network or at aprs.fi.
If posit does contain time stamp along with position, but being received with some seconds delay, it seems to be be discarded?
(different error messages: changes too fast, duplicate, old, out of order, unsupported format, rate limited, etc)
This results in low number of posits beeing displayed on map of aprs.fi. In raw data a high number of packets are marked red.
If you travel some route on highway, your track will jump backwards frequently,new positions are displayed very late or erratic, etc...
For example see track of oe3mzc-9 on 10.April.
Overall it seems as if the entire APRS infrastructure (digis&i-Gates) are not working, although nothing has been changed- neither on
equipment and setup used in car-nor on setup of Igates...
Any glue or correction?
see example raw data below
2012-04-10 16:22:12 UTC: OE3MZC-9>TX1P28,OE3XLR*,WIDE1*,OK2RA-2*,WIDE2-1,qAS,OK2RKB-1:`+8MsS.>/"6a} [Location changes too fast (>500 km/h)]
2012-04-10 16:22:50 UTC: OE3MZC-9>TX1P50,WIDE1-1,WIDE2-2,qAR,OE3XAR-11:`+;~r*c>/"6k}
2012-04-10 16:22:50 UTC: OE3MZC-9>TX0X17,OE3XDA-10*,qAS,OE3XDA-10:`*Vcs? >/"6t} [Rate limited (< 5 sec)]
2012-04-10 16:23:34 UTC: OE3MZC-9>TX1P65,WIDE1-1,WIDE2-2,qAR,OE3XAR-11:`+<~og$>/"7%} [Location changes too fast (>500 km/h)]
2012-04-10 16:23:40 UTC: OE3MZC-9>TX0W85,OE3XDA-10*,qAS,OE3XDA-10:`*W{s?*>/"6k} [Location changes too fast (>500 km/h)]
2012-04-10 16:23:54 UTC: OE3MZC-9>TX1P63,WIDE1-1,WIDE2-2,qAR,OE3XAR-11:`+=+lSo>/"7/} [Location changes too fast (>500 km/h)]
2012-04-10 16:24:03 UTC: OE3MZC-9>TX1P63,WIDE1-1,WIDE2-2,qAR,OE3XAR-11:`+=,l46>/"75}
2012-04-10 16:26:51 UTC: OE3MZC-9>TX0X65,OE3XDA-10*,qAS,OE3XDA-10:`+\Ys\T>/"6M}
2012-04-10 16:28:32 UTC: OE3MZC-9>TX1P63,WIDE1-1,WIDE2-2,qAR,OE3XAR-11:`+=,l >/"7)}
2012-04-10 16:28:42 UTC: OE3MZC-9>TX1P63,WIDE1-1,WIDE2-2,qAR,OE3XAR-11:`+=-l] >/"6v}
2012-04-10 16:29:09 UTC: OE3MZC-9>TX0Y59,OE3XDA-10*,qAS,OE3XDA-10:`+_0q4p>/"6y} [Location changes too fast (>500 km/h)]
2012-04-10 16:29:21 UTC: OE3MZC-9>TX1P63,WIDE1-1,WIDE2-2,qAR,OE3XAR-11:`+=Bm5U>/"76} [Location changes too fast (>500 km/h)]
2012-04-10 16:30:33 UTC: OE3MZC-9>APT311,WIDE1-1,WIDE2-2,qAR,OE3XAR-11:>mobile fighting PLC
2012-04-10 16:30:35 UTC: OE3MZC-9>TX0Y78,OE3XDA-10*,qAS,OE3XDA-10:`+a=rRd>/"6d} [Location changes too fast (>500 km/h)]
2012-04-10 16:31:21 UTC: OE3MZC-9>TX1P15,OE3XDA-10*,qAS,OE3XDA-10:`+&DrHW>/"6V} [Duplicate position packet]
2012-04-10 16:39:40 UTC: OE3MZC-9>TX1Q88,OE3XDA-10*,qAS,OE3XDA-10:`+2;s?4>/"6O}
2012-04-10 16:40:48 UTC: OE3MZC-9>TX1P26,WIDE1-1,WIDE2-2,qAS,OE3ACS-4:`+P]s?/>/"7<} [Location changes too fast (>500 km/h)]
2012-04-10 16:41:33 UTC: OE3MZC-9>APT311,WIDE1-1,WIDE2-2,qAS,OE3ACS-4:>mobile fighting PLC
2012-04-10 16:43:20 UTC: OE3MZC-9>TX1P28,OE3XLR,WIDE1*,WIDE2-2,qAU,OE5XDO-10:`+8MsS.>/"6a}
2012-04-10 16:45:07 UTC: OE3MZC-9>TX0Y14,WIDE1-1,WIDE2-2,qAS,OE3ACS-4:`+Vir\q>/"82} [Location changes too fast (>500 km/h)]
2012-04-10 16:45:55 UTC: OE3MZC-9>TX0Y68,WIDE1-1,WIDE2-2,qAR,OE3XIA-10:`+WerRV>/"8L}
2012-04-10 16:48:41 UTC: OE3MZC-9>TX1P54,WIDE1-1,WIDE2-2,qAR,OE3XIA-10:`,[\op$>/"7p}
2012-04-10 16:49:12 UTC: OE3MZC-9>TX1P68,WIDE1-1,WIDE2-2,qAR,OE3XIA-10:`,[uo n>/"7K}
2012-04-10 16:49:13 UTC: OE3MZC-9>APT311,WIDE1-1,WIDE2-2,qAR,OE3XIA-10:>mobile fighting PLC
2012-04-10 16:50:55 UTC: OE3MZC-9>TX1P63,OE3XDA-10*,qAS,OE3XDA-10:`+=-l] >/"6v} [Location changes too fast (>500 km/h)]
2012-04-10 16:51:32 UTC: OE3MZC-9>TX1P63,OE3XDA-10*,qAS,OE3XDA-10:`+=Bm5U>/"76}
2012-04-10 16:56:06 UTC: OE3MZC-9>TX1Q64,WIDE1-1,WIDE2-2,qAR,OE3XOC-5:`,`2o y>/"6l} [Location changes too fast (>500 km/h)]
2012-04-10 16:57:01 UTC: OE3MZC-9>TX1Q64,OE3XOC-10,OE3XHR,OK1FRN-2,OK1RQ-2*,ECHO,qAR,OK1DWW,T2CZECH:`,`2o y>/"6l}
2012-04-10 16:58:14 UTC: OE3MZC-9>TX1R04,WIDE1-1,WIDE2-2,qAR,OE3XOC-5:`,aknzk>/"6]}
2012-04-10 16:59:05 UTC: OE3MZC-9>TX1R10,WIDE1-1,WIDE2-2,qAR,OE3XOC-5:`,&+m+?>/"6\}
2012-04-10 16:59:22 UTC: OE3MZC-9>TX1R03,WIDE1-1,WIDE2-2,qAR,OE3XOC-5:`,&+m?l>/"6]}
2012-04-10 16:59:52 UTC: OE3MZC-9>TX1Q88,WIDE1-1,WIDE2-2,qAR,OE3XIA-10:`,& ngn>/"6_}
2012-04-10 17:00:37 UTC: OE3MZC-9>TX1Q59,WIDE1-1,WIDE2-2,qAR,OE3XIA-10:`,&-nSW>/"6p}
2012-04-10 17:00:38 UTC: OE3MZC-9>APT311,OE3XOC-10*,WIDE2-2,qAS,OE1SSU-10:>mobile fighting PLC
2012-04-10 17:02:12 UTC: OE3MZC-9>TX1Q03,WIDE1-1,WIDE2-2,qAR,OE3XIA-10:`,&Vm,4>/"7;}
2012-04-10 17:02:25 UTC: OE3MZC-9>TX1Q01,WIDE1-1,WIDE2-2,qAR,OE3XIA-10:`,&Qm!d>/"7B}
2012-04-10 17:19:37 UTC: OE3MZC-9>TX1Q64,OE3XOC-10*,OE6XTR*,WIDE2,qAS,OE6WSF-10:`,`2o y>/"6l} [Duplicate position packet]
2012-04-10 17:21:44 UTC: OE3MZC-9>TX1R04,OE3XDA-10*,qAS,OE3XDA-10:`,aknzk>/"6]} [Duplicate position packet]
2012-04-10 17:24:26 UTC: OE3MZC-9>TX1Q59,OE3XDA-10*,qAS,OE3XDA-10:`,&-nSW>/"6p} [Duplicate position packet]
2012-04-11 06:52:39 UTC: OE3MZC-9>APT311,WIDE1-1,WIDE2-2,qAR,OE3XOC-5:>mobile fighting PLC
Tom Hayward
> Hi Hessu,
> hi all,
> recently I have observed following problem:
> since about Easter, there is seems to be some problems with either the way
> that dupes are filtered in the tier3 network or at aprs.fi.
> If posit does contain time stamp along with position, but being received
> with some seconds delay, it seems to be be discarded?
> (different error messages: changes too fast, duplicate, old, out of order,
> unsupported format, rate limited, etc)
2012-04-10 10:58:14 MDT: OE3MZC-9>TX1R04,WIDE1-1,WIDE2-2,qAR,OE3XOC-5:`,aknzk>/"6]}
2012-04-10 10:59:05 MDT: OE3MZC-9>TX1R10,WIDE1-1,WIDE2-2,qAR,OE3XOC-5:`,&+m+?>/"6\}
2012-04-10 10:59:22 MDT: OE3MZC-9>TX1R03,WIDE1-1,WIDE2-2,qAR,OE3XOC-5:`,&+m?l>/"6]}
2012-04-10 10:59:52 MDT: OE3MZC-9>TX1Q88,WIDE1-1,WIDE2-2,qAR,OE3XIA-10:`,& ngn>/"6_}
2012-04-10 11:00:37 MDT: OE3MZC-9>TX1Q59,WIDE1-1,WIDE2-2,qAR,OE3XIA-10:`,&-nSW>/"6p}
2012-04-10 11:00:38 MDT: OE3MZC-9>APT311,OE3XOC-10*,WIDE2-2,qAS,OE1SSU-10:>mobile fighting PLC
2012-04-10 11:02:12 MDT: OE3MZC-9>TX1Q03,WIDE1-1,WIDE2-2,qAR,OE3XIA-10:`,&Vm,4>/"7;}
2012-04-10 11:02:25 MDT: OE3MZC-9>TX1Q01,WIDE1-1,WIDE2-2,qAR,OE3XIA-10:`,&Qm!d>/"7B}
2012-04-10 11:19:37 MDT: OE3MZC-9>TX1Q64,OE3XOC-10*,OE6XTR*,WIDE2,qAS,OE6WSF-10:`,`2o y>/"6l} [Duplicate position packet]
2012-04-10 11:21:44 MDT: OE3MZC-9>TX1R04,OE3XDA-10*,qAS,OE3XDA-10:`,aknzk>/"6]} [Duplicate position packet]
2012-04-10 11:24:26 MDT: OE3MZC-9>TX1Q59,OE3XDA-10*,qAS,OE3XDA-10:`,&-nSW>/"6p} [Duplicate position packet]
Mike, OE3MZC
I am not saying that aprs.fi is causing the problem,
but something is going severely wrong here! and it is not a single I-Gate to be blamed or my general understanding of APRS....
Look at this guy, as he is starting his day in the morning and all of a sudden the system goes crazy:
2012-04-12 05:58:06 UTC: OE1PDB-5>TX1R37,WIDE2-2,qAR,OE1PDB-10,T2CZECH:`,'7n"o>/'"6`}|!/&-'\|!wEw!|3
2012-04-12 05:58:36 UTC: OE1PDB-5>TX1R42,WIDE2-2,qAR,OE1PDB-10,T2CZECH:`,&wn|d>/'"6`}|!0&-'\|!w$k!|3
2012-04-12 05:58:54 UTC: OE1PDB-5>TX1R45,WIDE2-2,qAR,OE1PDB-10,T2CZECH:`,&do#&>/'"6[}|!1&,']|!w*2!|3
2012-04-12 05:59:05 UTC: OE1PDB-5>TX1R48,WIDE2-2,qAR,OE1PDB-10:`,&ZnTf>/'"6f}|!2&-']|!wPG!|3
2012-04-12 05:59:11 UTC: OE1PDB-5>TX1R49,WIDE2-2,qAR,OE1PDB-10:`,&TnA(>/'"6g}|!3&,']|!wYb!|3
2012-04-12 05:59:43 UTC: OE1PDB-5>TX1R70,WIDE2-2,qAR,OE1PDB-10:`,&Clf1>/'"6g}|!4&.'^|!wc&!|3
2012-04-12 05:59:48 UTC: OE1PDB-5>TX1R71,WIDE2-2,qAR,OE1PDB-10:`,&Dlq<0x1c>>/'"6h}|!5&-'^|!w8:!|3
2012-04-12 06:09:36 UTC: OE1PDB-5>TX1R70,WIDE2-2,qAR,OE1PDB-10:`,&Cl b>/'"75}144,800MHz|!"&0'Q|!w\@!|3 [Duplicate telemetry sequence]
2012-04-12 06:09:45 UTC: OE1PDB-5>TX1S01,WIDE2-2,qAR,OE1PDB-10:`,a1m,@>/'"?Y}|!$&%'Q|!ww-!|3 [Duplicate telemetry sequence, Location changes too fast (>500 km/h)]
2012-04-12 06:10:56 UTC: OE1PDB-5>TX1R83,WIDE2-2,qAR,OE1PDB-10,T2CZECH:`,&6nK5>/'"6k}|!&&-'U|!w:T!|3 [Duplicate telemetry sequence]
2012-04-12 06:11:27 UTC: OE1PDB-5>TX1S01,WIDE2-2,qAR,OE3XOC-5:`,&!ns4>/'"6v}|!'&+'V|!wV{!|3 [Duplicate telemetry sequence]
2012-04-12 06:13:02 UTC: OE1PDB-5>TX1S48,WIDE2-2,qAR,OE1PDB-10,T2CZECH:`,aWm_5>/'"7!}|!)&*'X|!wvO!|3 [Duplicate telemetry sequence]
2012-04-12 06:13:34 UTC: OE1PDB-5>TX1S66,WIDE2-2,qAR,OE1PDB-10:`,aBni">/'"7#}|!*&,'Y|!w0Y!|3 [Duplicate telemetry sequence]
2012-04-12 06:14:44 UTC: OE1PDB-5>TX1S93,WIDE2-2,qAR,OE1PDB-10,T2CZECH:`,`ko-$>/'"7(}|!,&-'Z|!w\6!|3 [Duplicate telemetry sequence]
2012-04-12 06:15:43 UTC: OE1PDB-5>TX1T31,WIDE2-2,qAR,OE1PDB-10,T2CZECH:`,`4oA->/'"7-}|!.&,'[|!wqc!|3 [Duplicate telemetry sequence]
Heikki Hannikainen
Hi,
Let's look at these two cases as separate individual cases, as they are
a bit different.
OE1PDB-5's *telemetry* is discarded due to the telemetry sequence number
suddenly jumping back. Please look at the map for OE1PDB-5 - his positions
contained in those same packets are not discarded!
To see the decoded telemetry sequence counter switch from "Normal" to
"Decoded" view in the raw packets display. You can also press CTRL-F (or
CMD-F on a mac) and search for the beginning of the telemetry section (for
example, |!" in the first "Duplicate telemetry sequence" packet).
The system is not crazy, it's working as designed. The sequence number
should increase and only wrap to 0 much later. aprs.fi allows the same
sequence numbers to be used after 30 minutes, so if a
telemetry-transmitting device reboots quicker than that, the telemetry is
lost.
This is good, since there are a good bunch of broken igates and
digipeaters occacionally transmitting pretty old packets.
> 2012-04-10 10:57:01MDT: OE3MZC-9>TX1Q64,OE3XOC-10,OE3XHR,OK1FRN-2,OK1RQ-2*,ECHO,qAR,OK1DWW,T2CZECH:`,`2o y>/"
> --
> You received this message because you are subscribed to the Google Groups "aprs.fi"
> group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/aprsfi/-/gPzV25GmyPQJ.
> To post to this group, send email to apr...@googlegroups.com.
> To unsubscribe from this group, send email to aprsfi+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/aprsfi?hl=en.
>
>
- Hessu
Heikki Hannikainen
> On Wed, Apr 11, 2012 at 08:33, Mike, OE3MZC <oe3...@gmail.com> wrote:
> > recently I have observed following problem:
> > since about Easter, there is seems to be some problems with either the way
> > that dupes are filtered in the tier3 network or at aprs.fi.
> > If posit does contain time stamp along with position, but being received
> > with some seconds delay, it seems to be be discarded?
This is not true. The absolute value of the timestamp is not trusted or
used for anything as such - there are so many stations transmitting
completely invalid timestamps that if I were to discard all, there would
be quite a flood of complaints for sure. :)
If a second packet arrives with an older timestamp than a recently
received previous packet, then the packet having an older timestamp is
discarded, since it's surely delayed and arriving in the wrong order.
OE3MZC-9 displayed here does not currently transmit timestamps along with
position.
> > (different error messages: changes too fast, duplicate, old, out of order,
> > unsupported format, rate limited, etc)
>
> It appears the OE3XDA-10 igate is behaving oddly. It seem to be waiting 24 minutes
> to forward your position to APRS-IS, and it's also marking the packet wrong (if
> you're heard on RF, you should see qAR):
>
> I recommend you put a timestamp in your packets so the delay can be proved and
> measured. Then watch your raw packets closely to improve your understanding of the
> problem.
>
> By the way, aprs.fi seems to be working just fine--it's detecting these delayed
> packets very accurately!
I agree, using browser's CTRL-F/CMD-F (find text in page, and then
copy-pasting the content of the claimed duplicated packet) to find the
previous copies of the duplicate packets seems to display that the
duplicate filtering is working pretty well in this case. Adding timestamps
to the packets can help in diagnosing and proving the case if needed.
- Hessu
Ian Wade G3NRW
From: Heikki Hannikainen <he...@hes.iki.fi>
Date: Thu, 12 Apr 2012 Time: 10:52:50
>The absolute value of the timestamp is not trusted or used for anything
>as such - there are so many stations transmitting completely invalid
>timestamps that if I were to discard all, there would be quite a flood
>of complaints for sure. :)
>
But surely that is what you want to happen? There is no point in
transmitting timestamps if they don't mean anything.
Take a hard line. Let the people complain. Tell them their timestamps
don't make sense. With luck they will fix them, and then everyone will
be happy. If the timestamps are not fixed, they won't appear in your
database. End of story.
If people don't realize they are making mistakes, they will never learn.
--
73
Ian, G3NRW
Heikki Hannikainen
> ___Original Message_________________________________________
>> The absolute value of the timestamp is not trusted or used for anything as
>> such - there are so many stations transmitting completely invalid
>> timestamps that if I were to discard all, there would be quite a flood of
>> complaints for sure. :)
>
> But surely that is what you want to happen? There is no point in transmitting
> timestamps if they don't mean anything.
>
> Take a hard line. Let the people complain. Tell them their timestamps don't
> make sense. With luck they will fix them, and then everyone will be happy. If
> the timestamps are not fixed, they won't appear in your database. End of
> story.
I agree somewhat. I would like to take a harder line at broken
configurations, but maybe I'll do it in small steps, for reasons of
personal comfort and workload. When I make larger changes (like recently)
it always causes some new real bugs, and invariably also some complaints
that the new features are bugs, all of which add to the "customer support"
workload which I try to keep at a manageable level. It takes time to tell
them. :)
Invalid timestamps are not too bad, after all! They're still useful for
duplicate filtering and can be used as a good substitute for the sequence
counter so badly missing from APRS packets - as long as the timestamp
keeps incrementing and eventually wraps over, even if it doesn't resemble
a real wall-clock time.
I'd rather start with filtering everyone using WIDE7-7 paths or worse,
they're a far worse thing for the health of the network. How about
filtering packets which have a path requesting more than
(7-(CURRENT_YEAR-2012)) hops? This year, cap at 7 hops, next year 6 hops,
and let it run until 2015 and stop at 4 hops for the future. :) I'm sure
someone thinks that would be unacceptable.
- Hessu
Lynn W Deffenbaugh (Mr)
>
> I'd rather start with filtering everyone using WIDE7-7 paths or worse,
> they're a far worse thing for the health of the network. How about
> filtering packets which have a path requesting more than
> (7-(CURRENT_YEAR-2012)) hops? This year, cap at 7 hops, next year 6
> hops, and let it run until 2015 and stop at 4 hops for the future. :)
> I'm sure someone thinks that would be unacceptable.
Yep, I'll vote first for it being unacceptable! Some folks might run
seemingly-abusive paths just as a test and they shouldn't be penalized,
IMHO. But my real objection is the expected response of the folks
routinely transmitting such paths when they finally start getting
dropped. They're not going to check into why, because if they were wont
to do that, they'd no longer be transmitting the path to start with.
Nope, their response is likely to be "I'm not getting through so I must
need more power or to transmit more often". Dropping or making such
packets invisible is likely to cause worse degradation reactions, not
encourage more proper behaviors.
Now, if you started detecting such paths in packets and making them
appear ugly (maybe big bold skull and crossbones balloon outlines?) on
everyone's map, you could elicit the kind of corrective response you
were after.
Come to think of it, I might do such a thing with my Paths display in
APRSISCE/32... I've had requests from users to detect and flag "bad"
paths from stations.
Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32
>
> - Hessu
>
Heikki Hannikainen
> transmit more often". Dropping or making such packets invisible is likely to
> cause worse degradation reactions, not encourage more proper behaviors.
>
> Now, if you started detecting such paths in packets and making them appear
> ugly (maybe big bold skull and crossbones balloon outlines?) on everyone's
> map, you could elicit the kind of corrective response you were after.
Right, making badness visible is better than hiding it, agreed (and I
didn't actually mean making them completely invisible, I would have shown
descriptive error messages instead of "not found"). I've been trying the
education part a bit on the /info/ pages but maybe it would be better to
make it more prominently visible.
Maybe a piece of red tape saying "HARMFUL CONFIGURATION" on the station's
info balloon too, covering the actual info, requiring a click on an 'X'
button to see the data. Or something. :) And ignore the symbol
configuration, use a separate "bad config" symbol instead.
- Hessu
Mike, OE3MZC
But please let us look at this example:
I use my smartphone (Android) to send aprs-posits via UMTS.
See result below:
So how come that many of these posits are classified as DUPE ?
For sure they are not duplicates in my understanding. Often even the position is different, but for sure it is the time of origin.
..and even if I have not moved much, I would like to indicate that I am still there.
Since it goes directly to internet, there is no effect from "oddly configured" Igates or such.
So again I am not convinced that everything is working as intended. sorry.
73 de Mike, OE3MZC
2012-04-12 07:51:27 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.06E$ Mike oe3...@qsl.net
2012-04-12 07:52:29 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.06E$ Mike oe3...@qsl.net
2012-04-12 07:54:27 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.20N/01620.10E$ Mike oe3...@qsl.net
2012-04-12 07:55:27 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.06E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 07:56:27 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.06E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 07:57:27 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.20N/01620.10E$ Mike oe3...@qsl.net
2012-04-12 07:58:27 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.07E$ Mike oe3...@qsl.net
2012-04-12 07:59:28 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.20N/01620.10E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 08:00:27 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.20N/01620.10E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 08:01:30 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.06E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 08:02:32 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.06E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 08:04:29 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.06E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 08:05:28 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.07E$ Mike oe3...@qsl.net
2012-04-12 08:06:28 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.06E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 08:07:28 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.07E$ Mike oe3...@qsl.net
2012-04-12 08:08:29 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.06E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 08:09:29 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.06E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 08:10:29 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.06E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 08:11:29 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.21N/01620.06E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 08:12:28 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.20N/01620.10E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 08:13:29 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.20N/01620.10E$ Mike oe3...@qsl.net [Duplicate position packet]
2012-04-12 08:14:29 UTC: OE3MZC-7>APAND1,TCPIP*,qAC,AE5PL-JF:!4812.20N/01620.10E$ Mike oe3...@qsl.net [Duplicate position packet]
> To unsubscribe from this group, send email to aprsfi+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/aprsfi?hl=en.
>
>
- Hessu
Lynn W Deffenbaugh (Mr)
Also, you seem to be using an older version of APRSdroid which is using the obsolete APAND1 application identifier. According to http://www.aprs.org/aprs11/tocalls.txt, this has been replaced by APDRxx, probably at the request of the author.
Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32
To view this discussion on the web visit https://groups.google.com/d/msg/aprsfi/-/zyEV07DPIj4J.
To post to this group, send email to apr...@googlegroups.com.
To unsubscribe from this group, send email to aprsfi+un...@googlegroups.com.
Tom Hayward
<ldef...@gmail.com> wrote:
> If you haven't moved (at all or not much) why are you beaconing the same
> position every minute? Even on the APRS-IS, moderation is reasonable.
> Seems like you might want to encourage the APRSdroid author to support
> SmartBeaconing (http://www.hamhud.net/hh2/smartbeacon.html) or
> GeniusBeaconing (http://aprsisce.wikidot.com/menu:configure-genius) or find
> some other way to throttle back the beacon frequency when not moving.
The problem is not the application. APRSdroid support SmartBeaconing,
minimum update distance, and minimum update time.
Tom KD7LXL
Mike, OE3MZC
would you please focus on the original problem of my posting,
which relates to the high number of packets that recently -in my opinion- seem to get discarded or sorted out as DUPE, UNSUPPORTED TYPE, WRONG ORDER, WRONG SEQUENCE, etc...
So the question is: if I send every minute a new position directly to APRS-IS server, why is it marked "DUPLICATE" ?
I think we should have a very transparent APRS system to make experiments with.
So please do not police others in your postings or ask questions why I would like to make certain things or not using latest software, etc.
If you do not have to contribute to the above toppic, please refrain from replying to my posting.
I also have not started a discussion about WIDE7-7, as limiting the traffic on air is a task for the DIGI-SysOps and not for aprs.fi
thank you.
73 de Mike
OE3MZC
Heikki Hannikainen
> But please let us look at this example:
Ok - this example is again different and not related to the others!
Different symptoms.
> I use my smartphone (Android) to send aprs-posits via UMTS.
> See result below:
> So how come that many of these posits are classified as DUPE ?
> For sure they are not duplicates in my understanding. Often even the position is
> different, but for sure it is the time of origin.
> ..and even if I have not moved much, I would like to indicate that I am still
> there.
It's, again, working as designed. The packets look like duplicates, since
they are exact copies of a very recent previous packet (but not exactly
the same as *the* previous packet, which would cause the previous
position's "last seen here" timestamp to be updated instead of inserting
a new database row).
There is nothing in the packets that would indicate that they are a new
unique position, and they look exactly the same as a recent packet, so I
throw them away.
In addition to dropping actual duplicate packets the dupe filters help me
to reduce the amount of trash in the database. It might seem like a small
thing to you, but the database at aprs.fi is large - I have actually
purchased 10 1TB disks for the purpose - and to keep the system running
fast a certain subset of that data needs to stay in memory, so the servers
have 8 to 32 GB of RAM installed. To reduce the amount of hardware needed
to some sensible limit I have to do some filtering of relatively useless
packets (such as "I am still here" transmitted once per minute!).
There are a lot of stations transmitting such useless data at a high rate.
Some are like this - they have a GPS which wanders around like this, even
though the station is in fact not moving. If they simply removed the GPS
and transmitted a manually fixed position they wouldn't eat up aprs.fi's
hardware resources for no reason.
If you configure the application in question to transmit less often when
it's not moving (say, once per 20 minutes), and more often when it's
actually moving, this filtering will stop hitting you. By default aprs.fi
will show you on the map if you've transmitted a position within the past
60 minutes so I don't see any reason to transmit it once per minute.
> > 2012-04-1010:57:01MDT: OE3MZC-9>TX1Q64,OE3XOC-10,OE3XHR,OK1FRN-2,OK1RQ-2*,ECHO,qAR,OK1DWW,T2CZECH:`,
> aprsfi+un...@googlegroups.com.
> > For more options, visit this group at
> http://groups.google.com/group/aprsfi?hl=en.
> >
> >
>
> - Hessu
>
> --
> You received this message because you are subscribed to the Google Groups "aprs.fi"
> group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/aprsfi/-/zyEV07DPIj4J.
> To post to this group, send email to apr...@googlegroups.com.
> To unsubscribe from this group, send email to aprsfi+un...@googlegroups.com.
Heikki Hannikainen
> would you please focus on the original problem of my posting,
> which relates to the high number of packets that recently -in my opinion- seem to
> get discarded or sorted out as DUPE, UNSUPPORTED TYPE, WRONG ORDER, WRONG SEQUENCE,
> etc...
Lynn's instructions are pretty good, if you follow them, I believe the
filtering will stop bothering you.
> I think we should have a very transparent APRS system to make experiments with.
The APRS-IS is quite transparent. I cannot afford to store everything
forever at no cost for you, and I have to filter the obvious redundant and
erroneous data to keep aprs.fi running quite smoothly like it does.
aprs.fi's transparency is in that it reports what it drops and why (and
I'll try to do my best to explain the errors in detail).
> So please do not police others in your postings or ask questions why I would like
> to make certain things or not using latest software, etc.
> If you do not have to contribute to the above toppic, please refrain from replying
> to my posting.
They were valid comments - using smart beaconing and other decaying
algorithms can help you to transmit packets at reduced intervals when
you're not moving. That'll leave less duplicate-like packets for aprs.fi
to drop.
> I also have not started a discussion about WIDE7-7, as limiting the traffic on air
> is a task for the DIGI-SysOps and not for aprs.fi
Yeah, but we started one - I hope you don't mind if we have some
additional discussions which relate to the development of our
applications. I changed the subject line, too. I think aprs.fi can
certainly play a role in educating users about good network practices,
since it gets more visibility from the eyeballs of the actual users than
the digipeaters out there!
- Hessu
Lynn W Deffenbaugh (Mr)
an excellent one at that) of APRS-sourced data.
On 4/12/2012 6:09 PM, Heikki Hannikainen wrote:
> On Thu, 12 Apr 2012, Mike, OE3MZC wrote:
>
>> I think we should have a very transparent APRS system to make
>> experiments with.
>
> The APRS-IS is quite transparent. I cannot afford to store everything
> forever at no cost for you, and I have to filter the obvious redundant
> and erroneous data to keep aprs.fi running quite smoothly like it does.
>
> aprs.fi's transparency is in that it reports what it drops and why
> (and I'll try to do my best to explain the errors in detail).
You DO have a very transparent "APRS system" available.
If you transmit on RF, you can use any number of devices and software
packages to both perform and observe your experiments. Whatever you can
transmit and decode, you can see and deal with.
If you are gated to APRS-IS or are injecting packets directly to
APRS-IS, you also have a well documented system with some restrictions.
The APRS-IS includes a dupe filter that will drop any identical packet
received within the dupe-detect period, typically 30 seconds. If you
inject duplicate APRS-IS packets more frequently than 30 seconds, you
won't see them show up anywhere. They are completely and totally
dropped. And in some informal testing of my own, that's a rolling 30
second window. So if you transmit a duplicate packet every 28 seconds,
you can expect to see the first be delivered through APRS-IS and none
following, not even the one at 56 seconds because that'd be only 28
seconds since the last one.
Now, once your packets enter the APRS-IS as non-duplicates, there are
lots of ways you can observe them. You can telnet directly to an
APRS-IS server, validate yourself, enter a filter, and watch the raw
packets go by (yes, I do this on a semi-regular basis, see ). You
probably want a copy of aprs101.pdf handy so that you can understand
what you're seeing. Not the most user-friendly way to go, but certainly
transparent.
Or, if you're looking for something more friendly, you can use one of
the many APRS-IS-viewing clients out there. Each of these will provide
basic packet interpretation and display and as a result will hide some
details from you. You trade transparency for convenience.
Or, if you don't want to run any local software drawing directly from
the APRS-IS stream, you can use one of the web-based APRS-IS viewing
sites like aprs.fi, findu.com or ... whatever. When you make this
choice, you're again trading convenience (you don't have to install any
particular program) for transparency (you're at the mercy of the site
operator's choice of how things work).
And if none of these appeal to you, then you can always write your own
aprs.fi work-alike and make it do what YOU want it to do. But demanding
that someone else, who is offering services and features for free,
change something because you don't particularly like their choices is
likely to be counter-productive.
Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32
PS. I will not be responding to any comments to this posting as the
aprsfi google group is for the support of the aprs.fi site by Hessu and
not a generic APRS discussion forum.