Distance measurement display wrong?

[Tim Passingham]

Recently I found the displayed distance measurement from the GPS plugin whilst travelling and recording a track was under-reading.  If I later look at the data for a completed saved track the distance is correct.

I've tried changed units (KM, Miles/Yards etc) to no effect.  All the other display data, including current speed and time are correct.

The only two things that I am aware of that have changed recently are that:

a) I changed the 'General logging interval' and 'Logging interval during navigation' to 60 seconds (so as to save battery)

b) the 'GPS Status & Toolbox' app was updated

On my last measurement, walking, the display said 4.96 miles when I stopped.  Having saved the track and then inspected it, it said 6.07 miles (which is correct).

So this seems to be a display issue.  I think it may be related to the logging interval.  On my last short walk I changed the interval to 30 seconds half way though my walk and although the display was still wrong, the ratio to the correct measurement was less.

Has anyone else had similar problems?

[Poutnik]

Do the both methods use the same data AND the same data filtering algorithm ? If not, distances will differ.

Longer log interval = shorter distance
Stronger filtering  = shorter distance

Poutnik

[Tim Passingham]

Sorry, I don't understand what you mean.  What 2 methods are you referring to?  All the data is in OSMAND+.  The display when travelling is wrong, but when I inspect the data using OSMAND itself to show the track details when I have stopped the distance is correct.

What data filtering are you referring to?

How can a longer log interval corrupt the displayed distance travelled but only on the real time display?

[Poutnik]

Perhaps neither I understand well what you exactly mean.
Some screenshots would help.

The question is , if the 2 distances are supposed to be the same, even if along the same way.

Planned one is always shorter than GPS measured,
point-by-point one always shorter than one precalculated from OSM data.

Dne čtvrtek 17. března 2016 14:13:43 UTC+1 Tim Passingham napsal(a):

[Tim Passingham]

I don't know to get a screenshot on an Android phone, sorry.

I am running OSMAND+ with the Trip Recording plugin.  I am not using 'navigation', just recording a track of where I have been.

When I am travelling I have started recording, and the screen shows me the current speed, height and distance travelled so far.  The distance travelled is wrong - it is too low.

When I have saved a track and then take a look at it using the 'Dashboard', 'My Tracks' I get a summary of all the data, including Distance, average speed, max speed, start and end time, and so on.  All that data is 100% correct.

The only thing that is wrong is the display of distance travelled whilst recording a new track.

If you haven't used this plugin I can understand that this may not be very clear, but to people who use it I hope it is.

[Poutnik]

If I press and hold the OFF device button in my androind 4.3, I will get an menu with an Item to make a screenshot.

I do use trip recording plugin time by time,
but did not use it for some time, as I use mostly LocusMap for autdoor activities now.

My guess is the program module involved in displaying the passed distance ( active AFAIK even without trip recording ON )  - processes the data differently than the Dashboard, and therefore produce different result.

Its sampling rate may be much lower than samling rate of recorder,
implicitly filtering all the way curvature between sampling points.

Another factor is realtime distance module measuring may use  explicit filtering of GPS data for its purpose
( otherwise GPS distance would be longer then real one due position noise )

Generally, raw GPS data based distance is ALWAYS longer than real distance due zig zag noisy GPs position data.
With proper combination of sampling rate and filtering level the distance is equal to real one.
But nobody knows what such a combination looks like, unless one knows the real one.
And it is valid just for this case. Other trips can be different.


It is closely related to a math problem : How long is the Norwegian coast. The answer does not exist, as the distance depends on how long is your least measuring distance ( sampling rate time your speed ). The cost sampled by 2 m stickj is shorter than measured by 1 m stick is shorter than measured by 50cm stick......

-


Dne čtvrtek 17. března 2016 15:14:43 UTC+1 Tim Passingham napsal(a):

[Tim Passingham]

All I know is that the measurement used to be exactly consistent, and now it is not, doing identical walks.

It seems strange to me that a different mechanism would be used to work out real-time distance than the one used to analyse a saved track - why not use the data already recorded?

So either something has changed, possibly due to the new release of GPS Status app, or there is a problem when the interval recording is 60 seconds rather than shorter.  I can understand some difference, but the error is quite large, over 20% out.

If I extract the gpx track and use another tool to analyse it (eg gpsprune) the result is exactly the same as shown via the OSMAND+ dashboard, so it's not that.

[Poutnik]

Real time distance does not follow the trip recording plugin setting, AFAIK.

They could agree in past by former coincidence of settings
leading to the same final value. another possibility may be
the dashboard GPX data  filtering changed.

Just for fun, try to compare distances for the identical way
( best if frequently and irregularly winding ),
but with different trip recorder sampling rates e.g. 10s, 30s, 60 s.

They will differ.

I do not remember what is sampling rate of realtime odometer.
It is possible its sampling rate was prolonged wrt former OSMAnd releases.

Dashboard and the other tool use the same data, Dashboard and the realtime odometer do not, as their sampling rate generally differ.

[Poutnik]

Or,
analyze the distance of a GPX file from trip recording log,
   best with high sampling rate.
Edit it and remove every other trackpoint
analyze the distance the 2nd time
Edit it and remove every other trackpoint
analyze the distance the 3rd time
Edit it and remove every other trackpoint
analyze the distance the 4th time

Compare the distances.

Dne ctvrtek 17. brezna 2016 16:49:29 UTC+1 Poutnik napsal(a):

[Tim Passingham]

I was trying to report what I saw as a possible fault, recently introduced, since it's always (and I do mean always - for several years) been OK in the past.

I am a simple user, not a gpx expert, so I'm not going to attempt editing the data. 

I won't get any time to test with different intervals for some time now.  But will try to at some point.

Can you explain the difference between the two logging intervals -  'General logging interval' and 'Logging interval during navigation' ?  

Thanks for your help.

[Poutnik]

I would say.
"General logging interval" is be used if navigation IS NOT not active.
"Logging interval during navigation" is used if navigation IS active.

But I have found just the latter in Trip Recording plugin settings.

Dne ctvrtek 17. brezna 2016 17:09:19 UTC+1 Tim Passingham napsal(a):

[Andrew Davie]

I haven’t been following the conversation closely, but…

GPS locations are inaccurate. There is an uncertainty in actual position, so think of this as ‘noise’ giving position along your route. Plus or minus a few metres here and there, which errors add up to a longer path via the GPS-position points compared to the calculated path points. Perhaps that is what is happening? Are you assuming the points on two paths being compared are in exactly the same spots?  If they are NOT, then summing the distances between the points is always going to be different. 

--
You received this message because you are subscribed to the Google Groups "Osmand" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osmand+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Poutnik]

I am quite well aware of this.

What OP had in mind was
realtime distance odometer - reportedly too low
versus
distance backward evaluated from GPX tracklog by Trip recording plugin.

While they can theoretically provide the same result, evaluating the same path of the same way,
due different internal sampling and filtering can and do provide different results.


Dne čtvrtek 17. března 2016 17:45:15 UTC+1 Andrew Davie napsal(a):

[Tim Passingham]

Sorry, I must be stupid.  I still don't see an answer to my original question.

Originally I was logging more frequently, from 5 to 30 seconds, and the display during recording matched the recorded track details exactly (not almost - but exactly).

I lengthened the 2 logging intervals to 60 seconds, which one would imagine would cut corners and log a shorter distance than actually made.  But the reverse happened with respect to the realtime display.  The display showed a shorter distance versus the recorded track.  I know for sure that the latter is closer to the truth.

I repeat that I previously always had the same results, so I still believe something else is going on.

[Poutnik]

Does it mean that at the same track the realtime distance is shorter than before ?

If yes, then authors may implemented lower sampling rate a/o stronger data filtering for it.

Another possibility is that realtime measuring is affected by logging sampling rate as shortening the distance,
but is affected more then trip recording, in case realtime one does filter and dashboard does not.

[Tim Passingham]

I have a hypothesis.

If the real-time Display when recording is based on a frequency that is a significant multiple (say 5) of the logging interval, and the logging interval itself is quite long (eg 60 seconds), then even at walking speed the Displayed distance could be cutting major corners off my circular routes, since in 5 minutes I have may have walked round quite a few corners.  

If I reduce the logging interval down to lower numbers (say 20 seconds), and if the Display recording frequency is still at 5 times the logging interval, then it will be 100 seconds, which is not so short as to be completely inaccurate.

This hypothesis only works if the frequencies are linked in the way I have attempted to describe.  If they are not linked, then the Displayed frequency ought to have always been wrong, which it wasn't.

[Tim Passingham]

The same track, realtime display is much shorter than it used to be (over 20%).  See my hypothesis just posted.

[poutnik]

Is it now 20% shorter then should be, or was it 25% longer ( and gpx log still is)
than should be before?
Can you compare it to e.g. bicycle odometer?

17. března 2016 19:14:12 CET, 'Tim Passingham' via Osmand <osm...@googlegroups.com> napsal:

--
Sent from my phone via Android email client K-9.
Please, forgive my brevity.

[Tim Passingham]

I don't quite understand your question.

The realtime display when I have logging at 60 seconds is around 20-25% too low, to, as I said at the start, the display said 4.96 miles when I stopped.  Having saved the track and then inspected it, it said 6.07 miles (which is correct).

I know the 6.07 miles is correct because I have done this walk many, many time, and it's always the same length (a little over 6 miles).  I have also measured it on google maps and got the same figure to within 2/10 of a mile.

[Poutnik]

It does not matter, as you have answered it.
But it is not correct because you have done it many times, but because
you checked it with google maps.

I have checked that rate of realtime distance update is equal to rate
set for the logging.

I have hypothesis that with high logging rate
the shortening by filtering at realtime displaying may be negligible,
wrt the log,
but with low logging rate such a shortening can be significant.

Try to make the logging at the higher rate as in past,
is it returns to previous behaviour ( than the rate is the cause )
or not ( than some change in OSMAnd is the cause )

Dne 17/03/2016 v 21:23 'Tim Passingham' via Osmand napsal(a):

> I don't quite understand your question.
>
> The realtime display when I have logging at 60 seconds is around
> 20-25% too low, to, as I said at the start, the display said 4.96
> miles when I stopped. Having saved the track and then inspected it,
> it said 6.07 miles (which is correct).
>
> I know the 6.07 miles is correct because I have done this walk many,
> many time, and it's always the same length (a little over 6 miles). I
> have also measured it on google maps and got the same figure to within
> 2/10 of a mile.
>

--
Poutnik ( the Pilgrim, Der Wanderer )

My Brouter profiles
https://github.com/poutnikl/Brouter-profiles/wiki

[Nico W]

Tim is correct. When you start recording one of the little boxes in top right corner shows distance. It should be updated every 5, 10, 30, whatever seconds you have specified. Your Gps track is updated at the same time.
Now why at the end is there a significant distamce difference between the distance in the box as the distance of the gpx track?

Only way I can see that happening is if the distance in the box has a rounding error and these add up. For example: in 5 secs you walk 10.5 meters, but is shows as 10. Next 5 seconds another 10.5 but instead showing as 21 it shows 20, creating the error. The Gpx track uses coordinates and at the end makes a total that is correct.

[poutnik]

Hmmmm, but that would mean higher the sampling rate, higher the difference. But the opposite is reportedly observed. the difference 20% is noticed at low sampling rate 60 s.

18. března 2016 4:16:26 CET, Nico W <wiss...@gmail.com> napsal:

Tim is correct. When you start recording one of the little boxes in top right corner shows distance. It should be updated every 5, 10, 30, whatever seconds you have specified. Your Gps track is updated at the same time.
Now why at the end is there a significant distamce difference between the distance in the box as the distance of the gpx track?

Only way I can see that happening is if the distance in the box has a rounding error and these add up. For example: in 5 secs you walk 10.5 meters, but is shows as 10. Next 5 seconds another 10.5 but instead showing as 21 it shows 20, creating the error. The Gpx track uses coordinates and at the end makes a total that is correct.

[Reinhard]

I have a hypothesis too.

Could it be that the final distance, calculated by the GPX-Plugin, is different, because the displayed distance during measurement is a "linear distance"/beeline?

[poutnik]

Hmm it is similar as for previous hypothesis. The relative difference would increase with increased sampling rate, and not decrease as observed.

Also, the displayed distance sampling rate would have no reason to follow recording sampling rate.

18. března 2016 8:16:01 CET, Reinhard <reinhard....@gmail.com> napsal:

I have a hypothesis too.

Could it be that the final distance, calculated by the GPX-Plugin, is different, because the displayed distance during measurement is a "linear distance"/beeline?

[Poutnik]

Hmmm, I made a test distance trip recoding this morning during public city transportation to work, with sampling rate 60 s.
Displayed distance was 7.5 km, consisting of  2 rides of total of cca 6.7 km, 800 m walking and 2 watings walking around.
Therefore sampling points were effectively more sparse then with 60s rate and walking.

After stopping and saving the recording I reviewed in Daskboard the recorded track. the distance - 7.5 km as was displayed.

Note that I had the OSMAnd+ always on foreground with display ON.
The things may change if GPS system gets periodic wakings up, as 60s sampling rate with device put asleep is about being threshold for putting asleep the GPS.


Dne pátek 18. března 2016 8:16:01 UTC+1 Reinhard napsal(a):

[Tim Passingham]

Interesting - thanks.  I always have the display off on mine to save power.  I don't know what the threshold is for putting GPS to sleep - where is that set?

You also said before that you only have one interval setting for the plugin, which puzzles me. On mine the 'General logging interval' is the 2nd one down on the Trip Recording settings, and the 'Logging interval during navigation' is the 5th one.  

[Poutnik]

It is not set anywhere, AFAIK. But it was written somewhere in context of saving power and GPS logging, that GPS logging interval has to be at least 60s to safe extra power. GPS subsystem then reportedly switches itself of by timeout - if GPS is not otherwise used, and wakes up again by a timer. But it may be application dependent.
If the only purpose of OSMAnd usage is the logging, than a dedicated GPS logger could be used instead of it.

about setting - I later realized the general rate dialog occurs when I start recording.
But in the plugin settings I have only navigation related interval, no settings for General Logging interval.

Dne pátek 18. března 2016 10:52:21 UTC+1 Tim Passingham napsal(a):

Interesting - thanks.  I always have the display off on mine to save power.  I don't know what the threshold is for putting GPS to sleep - where is that set?

You also said before that you only have one interval setting for the plugin, which puzzles me. On mine the 'General logging interval' is the 2nd one down on the Trip Recording settings, and the 'Logging interval during navigation' is the 5th one.  

On Friday, 18 March 2016 09:22:02 UTC, Poutnik wrote:

Hmmm, I made a test distance trip recoding this morning during public city transportation to work, with sampling rate 60 s.
Displayed distance was 7.5 km, consisting of  2 rides of total of cca 6.7 km, 800 m walking and 2 watings walking around.
Therefore sampling points were effectively more sparse then with 60s rate and walking.

After stopping and saving the recording I reviewed in Daskboard the recorded track. the distance - 7.5 km as was displayed.

Note that I had the OSMAnd+ always on foreground with display ON.
The things may change if GPS system gets periodic wakings up, as 60s sampling rate with device put asleep is about being threshold for putting asleep the GPS.

[Tim Passingham]

I got time to do a test walk today.  This time instead of setting both logging intervals to 60 seconds I set both to 30 seconds. Everything else was the same (including walking with the display off).

The real-time Display and the detailed track data as analysed with the Dashboard read exactly the same (6.9 miles).

So, in my opinion, there is some sort of error in the system when both are set to 60 seconds.  But never mind, I'll just use 30 seconds instead.

:-)

[Poutnik]

Rather consequence of system timeout behaviour then error.

Due timeouts the Locus may not to be able to properly count the passed
distance,
if realtime distance is based not on evaluation of coordinate serie,
but on integration of speed value provided by GPS NMEA protocol.

As the latter is easier.

Dne 19/03/2016 v 12:52 'Tim Passingham' via Osmand napsal(a):
> ...

> So, in my opinion, there is some sort of error in the system when both
> are set to 60 seconds. But never mind, I'll just use 30 seconds instead.
>

[Tim Passingham]

In my view whatever the cause that's a bug in the design.  

But I can avoid the problem so never mind.

[Poutnik]

Not of design, but of understanding of metrology,
as both methods measure passed distance during different time intervals.

Dne 19/03/2016 v 13:24 'Tim Passingham' via Osmand napsal(a):

[Andrew Davie]

I have been playing with OsmAnd source code and learning how it all works.
I thought I’d have a stab at improving the GPX display, and now it’s all working I thought I’d share a video…

https://youtu.be/xD18z4EsRMQ

What you’re looking at in this is the behaviour of the GPX tracks - how they look at various zoom levels, and visuals. This is a screen grab from an emulator, so it’s about 10x slower than on my basic Android phone. Not sure why the emulation is so slow on my machine, but that’s another issue...

New stuff;
1. GPX track loading is heaps faster - about 3x speed of current release.
2. Track width is now automatically adjusted based on zoom level, so you don’t have mega-messes when zoomed out. Previously width was constant so when you zoomed out you got a confusing block of colour. Now it’s very clear what a track looks like even zoomed out.
3. Track colours and widths are customisable - down to a segment-level. The video shows this with the red/yellow track, which has two segments. One is transparent red, thick track, and the other is transparent yellow, thin track. The other GPX shown in the video (the blue one) is default with no extra colouring. The colours come from (already supported) “extensions” capability in the GPX reader. I added “zoom” to the keys to specify the track (or segment) width.
4. Tracks are point-reduced based on zoom level. As you zoom out, less points, so the UI remains faster even with super-huge GPX tracks. I tried to show this clearly in the video. I may tweak this so it’s less obvious, but it’s reasonably good now, IMHO. The huge advantage of this is that you no longer get those awful “shimmering pointy triangles” that the previous version displayed. It’s much more pleasing now.

I’m such a dunce though - this is my first foray into git, and android programming, android studio and all that... and I killed the association between my local copy and the git copy of the code. So I’ve yet to figure out how to properly check-back in the source files I’ve modified. I will get to that soon.I thought I could just re-grab everything and copy my changes over the top, but i can no longer get a fresh grab of the code to compile. Strange.

Cheers
A

[Tim Passingham]

One measurement is quite accurate, one is not.  The one that is not is wrong.  That's an error in design.

Never mind. 

[Andrew Davie]

I have finished my code updates locally - GPX track updates.

I’d like to get these into the hands of the OsmAnd authors - so I tried a git push… and it failed thus:

remote: Permission to osmandapp/Osmand.git denied to andrew-davie.

fatal: unable to access 'https://github.com/osmandapp/Osmand.git/': The requested URL returned error: 403

So, I’m missing a step - do I need to get permission to push my code to the project?  Can anyone help?

Thanks

A

[Andrew Davie]

I think I might answer my own question. I don’t get write access to the project.

What I need to do is fork the project, write to my own fork, and then send a pull request to the original project.

OK, will try that :)  Getting there.

[Andrew Davie]

Some more GPX experimentation.

https://youtu.be/U1yg5Msm_N8

I have coloured tracks, and auto-zooming. Custom track widths.

The UI is now asynchronously reducing the number of points/track based on the zoom level.

Some of this will hopefully make it into the official release.

If anyone has any specific request around GPX functionality let me know, because I’m deeply embedded in that source code right now.

[Andrew Davie]

I got the display of GPX tracks, and direction indication working how I like them. 

https://youtu.be/Qb1mO45vdzg

I also added distance markers along the GPX track, just to show that it can be done.

None of this is currently slated for OsmAnd official, but I’d like to see what can be done about that.

[Andrew Davie]

Another video…This one includes...

1. original base GPX track

2. altitude colour-coded band showing relative altitude of where you are (red=highest, green=lowest)

3. moving “conveyor points" showing the direction of travel along the track.

4. different size tracks from .gpx configuration

5. different colour tracks from .gpx configuration

6. waypoint "1km" markers

7. end of track length displayed

Any/all of these are optional and sit on top of the existing colour/size calculations.

Have a peek at https://youtu.be/2dX9Uhr0R1s

This is an experimental code fork. I’m mostly learning how to program Android and OsmAnd.

Hopefully some of these can make it into the official release sometime down the track.

[Andrew Davie]

Here’s just a bit of fun - using the point ordering on a GPX track to show the paths taken when exploring a track on my motorbike.

Pretty cool, huh?  See it here —>  https://youtu.be/QQP2-vVFd5U

[Andrew Davie]

All the bells and whistles, and every optimisation I could think of - here’s a short video showing the GPX rendering code I’ve been working on in action. It includes (and these are options only, I’m not suggesting it should always look like this)…

rainbow colours around the track indicating relative altitude along the track

 - calculated automatically from the GPX data ‘ele’ tag.

 - displays as green if there’s no elevation data

rainbow colours for speed

 - calculated if there are time tags in the GPX data

‘conveyor’ effect - line marching along the track

arrows - at higher zoom levels, the arrows follow along the track

 - this code is intended for incorporation into route-following visual/UI

distance numbers along track

 - visible at higher zooms

Additionally, many of these (speed/altitude/regular track/arrows/conveyor) resize themselves to suit the current zoom level, so they look “cleaner” rather than being a mess of points. It also means that the screen refresh is much quicker at lower zoom levels where track size can be significantly culled to fewer lines. The resizing is asynchronous (happens in the background) and only happens on-demand (when the track is actually visible on the screen).

The video link:  https://youtu.be/2kTnU-4-rhU

[Ben Boeckel]

On Sun, Apr 03, 2016 at 04:16:21 +1000, Andrew Davie wrote:
> The video link: https://youtu.be/2kTnU-4-rhU

This looks *amazing*.

<pie-in-the-sky>
Any chance for things like this for routes as well?
</pie-in-the-sky>

Thanks,

--Ben

[Andrew Davie]

HI Ben
It is my plan to improve the routing display (particularly the arrows) with some of the functionality you see in this video.
The decision as to inclusion of my code into OsmAnd is not mine, but I am pleased to say that the process has started satisfactorily.

[Ben Boeckel]

On Wed, Apr 06, 2016 at 21:40:01 +1000, Andrew Davie wrote:
> It is my plan to improve the routing display (particularly the arrows)
> with some of the functionality you see in this video.

Sounds great.

> The decision as to inclusion of my code into OsmAnd is not mine, but I
> am pleased to say that the process has started satisfactorily.

I know that story :) .

Thanks,

--Ben

[Andrew Davie]

Just thought I'd share a video of 100MB of GPX tracks loaded and displayed at the same time in OsmAnd. These are over
about a 300 km x 300 km area (Tasmania) with several hundreds of thousands of points total. There's no slowdown in the
UI, even when all tracks are displayed. There's a bit of noticeable "resampling" delays (but not to the UI) as this is running
on a slow-ish emulator. On my phone it's pretty good.
https://www.youtube.com/watch?v=L1MXf20mqSw
These tracks are all using the rainbow altitude band around the track itself, and the distance-along-track kilometre markings
when you zoom in a bit. Of course with just the plain track it's many times faster. Code is submitted - hopefully it will be
reviewed for inclusion soon.