How is distance calculated?

[isw]

I walk a lot, and tote along a GPS receiver that creates a GPX file on
my phone (using GPSMid) as I go. When I get home, I copy the file to my
Mac, and push it through gpsBabel to create a KML.

As I travel, I can look at a data page created by GPSMid that tells me
the distance I've walked. When I get home, I can open the KML file
created from the GPX file and see the distance I traveled (in a summary
along with some other items). They are always the same, or very close.

Now, I know that the GPX file is just an ordered list of locations and
times, and does not contain any information concerning the accumulated
distance. That tells me -- I think -- that gpsBabel must be calculating
the path length from the coordinate data. GPSMid, on my phone, is
presumably doing the same thing, because I don't think the GPS unit
emits messages containing data about distance traveled.

So the question is, how is the length of a path calculated, so that
identical answers are produced by both methods?

Isaac

[Mike Coon]

But surely "both methods" are just calculating distance from the
co-ordinates? However there are a range of different algorithms giving
different levels of accuracy. The simplest assumes the area concerned is
flat so that Pythagoras for a plane surface applies. More complex
calculations take the real shape of the Earth into account.

Mike.
--
If reply address is Mike@@mjcoon.+.com (invalid), remove spurious "@"
and substitute "plus" for +.

[Terje Mathisen]

Isaac, you should listen to Mike here!

Almost by definition, the two devices _have_ to generate the same total
distance.

The "perfect" solution is to calculate each track point to track point
distance using Vincenty, this is an algorithm which is accurate to a few
mm between any two points on the Earth's surface.

For a relatively short trip, like a hike or bike ride (or when you have
track points close together) you will get cm-level accuracy by simply
taking the "flat" distance from track point to track point:

meter_pr_degree = 20e6 / 180;
degree_2_radian = 180 / 3.1415926535;
dx = (lon2-lon1)*cos((lat1+lat2)/2 * degree_2_radian);
dy = (lat2-lat1);
dist = sqrt(dx*dx + dy*dy) * meter_pr_degree;

As long as the distance between individual track points is within
several km this will be effectively identical to the "perfect" distance.

Within 100 km or so you can do the same even easier using UTM
coordinates, in which case you are effectively projecting all the track
points onto a single flat surface, and you get all measurements directly
in meters.

Terje
--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

[Alan Browne]

On 2012-02-05 00:10 , isw wrote:
> I walk a lot, and tote along a GPS receiver that creates a GPX file on
> my phone (using GPSMid) as I go. When I get home, I copy the file to my
> Mac, and push it through gpsBabel to create a KML.
>
> As I travel, I can look at a data page created by GPSMid that tells me
> the distance I've walked. When I get home, I can open the KML file
> created from the GPX file and see the distance I traveled (in a summary
> along with some other items). They are always the same, or very close.
>
> Now, I know that the GPX file is just an ordered list of locations and
> times, and does not contain any information concerning the accumulated
> distance. That tells me -- I think -- that gpsBabel must be calculating
> the path length from the coordinate data. GPSMid, on my phone, is
> presumably doing the same thing, because I don't think the GPS unit
> emits messages containing data about distance traveled.

Typically not.

> So the question is, how is the length of a path calculated, so that
> identical answers are produced by both methods?

I'd be surprised if you got identical results from two different methods
unless both do the simple integration of delta-pos for each recorded
point and both have the same resolution in calculation (or otherwise run
the same method). But they should be close.

Most handheld GPS' compute position at 1 Hz. If you're walking (about 1
to 1.5 metres per second is typical) the position recording tends to be
noisy.

GPS makers know this and use various strategies to reduce the noise. My
Garmin, for example, is set to record position every 5 seconds. (I
could also set it to record only every x metres of change). But I find
the Garmin to record very poor tracks compared to my other GPS. (An AMOD
3080 photo tracker). (I do need to test the Garmin in recording
position changes)

OTOH, the same Garmin has an "odometer" feature, and it is quite
accurate as far as I've checked. So it seems while Garmin tracks are
not great their algorithm for distance is quite good. Note that Garmin
make a lot of products for athletes, so they've probably honed this
feature well and the same is integrated into their handheld GPS'.

If I integrate (from the AMOD data) on Google Earth (open the track and
right-click the elevation profile - that also gives you distance) then I
get hiking distance results that are about 10% longer than the actual hike.

I wrote my own integrator/filter to take track logs, do a moving average
delta-pos and then integrate only intervals of x metres or more (5 or 10
being the usual value). It also writes a new log with the filtered
track. Looks smoother on Google Earth, etc. That has given me accurate
(better than 1%) distance on courses that I could measure using Google
Earth "path" (clear view of the path on the ground in GE).

You can be sure GPSBabel has its own assumptions and algorithm. I
didn't even know it had a distance integration feature. (I only use it
to convert my AMOD tracks from NMEA to GPX so I can upload them to
Google Maps when needed).

As to phones, I have the iPhone 4 and its tracks are not that great,
esp. in the woods. The iPhone is great for finding restaurants and
navigating there, it's not a great hiking GPS - though the display
quality is fantastic, the "compass" is nice, and I have several paid for
GPS Apps that are very good. Too bad the GPS in the iPhone is a bit
crippled.

It would be nice if the iPhone GPS could be set by the user to run at a
higher rate (at the expense of battery power) such that the GPS solution
would be better.

--
"We demand rigidly defined areas of doubt and uncertainty."
Douglas Adams - (Could have been a GPS engineer).

[isw]

In article <ho5209-...@ntp6.tmsw.no>,

Thanks to both you and Mike. That's the answer I was looking for. I was
unaware of the existence of the algorithm you mentioned.

Isaac

[Alan Browne]

Except that answer is wrong. The "code" above is fine (it's about
identical to my own) for calculating the distance between two points at
a small distance apart.

But, because small errors over short distances and time will accumulate
quickly the error will grow quickly as well. A degree of filtering of
the distances (that takes account of the kind of movement - eg: hiking
v. cycling) is required to reduce that error.

[Alan Browne]

On 2012-02-05 06:28 , Terje Mathisen wrote:

> For a relatively short trip, like a hike or bike ride (or when you have
> track points close together) you will get cm-level accuracy by simply
> taking the "flat" distance from track point to track point:
>
> meter_pr_degree = 20e6 / 180;
> degree_2_radian = 180 / 3.1415926535;
> dx = (lon2-lon1)*cos((lat1+lat2)/2 * degree_2_radian);
> dy = (lat2-lat1);
> dist = sqrt(dx*dx + dy*dy) * meter_pr_degree;
>
> As long as the distance between individual track points is within
> several km this will be effectively identical to the "perfect" distance.

Except that accumulating differences between 2 GPS points over short
distances and time will result in an error accumulation. Those short
distances need to be filtered (appropriate to the speed of movement) to
reduce those errors.

[Mike Coon]

Alan Browne wrote:
> Except that accumulating differences between 2 GPS points over short
> distances and time will result in an error accumulation. Those short
> distances need to be filtered (appropriate to the speed of movement)
> to reduce those errors.

But not an error *percentage* accumulation.

Another way of getting distance would be to integrate the speed
measurements. But this is not likely to be easier of more accurate.

[HIPAR]

Using a typical consumer class receiver, iff you look at a scatter
plot of the fixes, you will typically see a grouping where most of
the points are within a 80 foot diameter circle. Most of them will
actually fall within a 20 foot diameter circle providing a good
probability that your position is fixed within about 10 feet of true.

So I'd suspect your travel path would need to exceed 40 feet by a
great margin to cancel out the precision error at the beginning and
end points of your route. If you walked around a 5 foot radius
circle, could you accurately measure the total distance by summing the
distance between successive fixes?

Of course if your route is long enough you can use flat earth
Pythagoras for distances of 30 miles (or so). Round earth Haversine
type solutions can provide solutions accurate to a percent or so. If
you need a 'squashed earth' solution, the method of Andoyer-Lambert is
excellent providing an error of just a few meters.

Other iterative type solutions can be employed for ultimate accuracy
on the reference ellipsoid.

--- CHAS

[Terje Mathisen]

Alan Browne wrote:
> On 2012-02-05 06:28 , Terje Mathisen wrote:
>> As long as the distance between individual track points is within
>> several km this will be effectively identical to the "perfect" distance.
>
> Except that accumulating differences between 2 GPS points over short
> distances and time will result in an error accumulation. Those short
> distances need to be filtered (appropriate to the speed of movement) to
> reduce those errors.

Only true if you have transverse noise.

I.e. errors along the direction of travel don't matter for the final
distance accumulator.

[Alan Browne]

On 2012-02-06 13:08 , Terje Mathisen wrote:
> Alan Browne wrote:
>> On 2012-02-05 06:28 , Terje Mathisen wrote:
>>> As long as the distance between individual track points is within
>>> several km this will be effectively identical to the "perfect" distance.
>>
>> Except that accumulating differences between 2 GPS points over short
>> distances and time will result in an error accumulation. Those short
>> distances need to be filtered (appropriate to the speed of movement) to
>> reduce those errors.
>
> Only true if you have transverse noise.

Which you always have. And the lower the speed, the more noise per
along track travel. The more and thicker the tree cover, the more
"transverse noise".

> I.e. errors along the direction of travel don't matter for the final
> distance accumulator.

Back to the "transverse noise" and that continues to integrate.
I've proven this with simple integrators like that in Google Earth and
my own code - which is why I've implemented filters that reduce the
error significantly, esp. for hiking. For higher speeds, less filtering
is required.

[Alan Browne]

On 2012-02-06 10:02 , Mike Coon wrote:
> Alan Browne wrote:
>> Except that accumulating differences between 2 GPS points over short
>> distances and time will result in an error accumulation. Those short
>> distances need to be filtered (appropriate to the speed of movement)
>> to reduce those errors.
>
> But not an error *percentage* accumulation.

If you mean "the percentage changing" yes. However, it will be
different for hiking (worse case) than cycling as the "noise" is highest
v. distance traveled at low speeds.

Recall the original post. He's talking about what his GPS reports as
distance traveled while walking. To do so, the receiver integrates
small changes in position. Every time it does so, it integrates the
error into the sum.

Receivers like Garmin, appear to use a smarter algorithm that filters
appropriately for walkers, bikers, automobile users. Some form of
filtering is occurring.

Using the 'code' presented earlier, and integrating the difference, the
error will pile up.

In the earlier days of GPS, handheld receivers performed very poorly at
integrating distance (in large part because of lower satellite coverage
and SA, but also the algorithms were simply too simple).

If I use Google Earth to calculate the distance traveled (for a hike),
the error is always positive and generally around 10%. I suspect that
for a bike ride its error will be less - I'll try to validate that this
spring.

My own filter both smooths the track (reducing what Terje calls
"transverse noise") and integrates on a minimum distance moved setting
(usually set to 5 metres for hikes). With that I get errors of better
than 1% - similar to the error with the Garmin.

(I've checked these on easy to measure courses on Google Earth using the
distance/path feature).

[oriel36]

On Feb 6, 6:35 pm, Alan Browne <alan.bro...@FreelunchVideotron.ca>
wrote:

You are such strange people,entirely oblivious to the Earth's motions
as the concentration here is between a moving device and the Earth's
surface and has noting to say about the surface of the Earth or its
motion.

To the nearest mile,the Earth turns 1037.5 miles an hour at the
equator and a full 24901 mile circumference in 24 hours yet you
dummies follow a system where 15 degrees of geographical separation
does not equate to 1 hour time difference as you insist on linking
daily rotation directly to stellar circumpolar motion and come up with
an imbalance of 1465 rotations in 1461 days.

Engineers are creating havoc and they don't know it for when you are
4 rotations adrift of the 1461 rotations in 1461 days I wouldn't
consider a person in any way brilliant.Do you do this on purpose or is
it some kind of protective stupidity that prevents you from
distinguishing a moving device and distance on a planet and the motion
of the planet itself ?.

You all actually know that the Earth is round and rotating ?,maybe
even that Feb 29th is the final 1461 st rotation enclosing 4 orbital
circuits of the Earth.Ah,I am wasting my time with a bunch of
astronomical illiterates and programming riff-raff.

[Mike Coon]

Alan Browne wrote:
> On 2012-02-06 10:02 , Mike Coon wrote:
>> Alan Browne wrote:
>>> Except that accumulating differences between 2 GPS points over short
>>> distances and time will result in an error accumulation. Those
>>> short distances need to be filtered (appropriate to the speed of
>>> movement) to reduce those errors.
>>
>> But not an error *percentage* accumulation.
>
> If you mean "the percentage changing" yes. However, it will be
> different for hiking (worse case) than cycling as the "noise" is
> highest v. distance traveled at low speeds.

I have no idea what "the percentage changing" means, so I cannot comment on
that. All I actually meant is that an accumulation of 10% errors is still no
more than a 10% error in the total however many items are added.

[Alan Browne]

On 2012-02-06 17:42 , Mike Coon wrote:
> Alan Browne wrote:
>> On 2012-02-06 10:02 , Mike Coon wrote:
>>> Alan Browne wrote:
>>>> Except that accumulating differences between 2 GPS points over short
>>>> distances and time will result in an error accumulation. Those
>>>> short distances need to be filtered (appropriate to the speed of
>>>> movement) to reduce those errors.
>>>
>>> But not an error *percentage* accumulation.
>>
>> If you mean "the percentage changing" yes. However, it will be
>> different for hiking (worse case) than cycling as the "noise" is
>> highest v. distance traveled at low speeds.
>
> I have no idea what "the percentage changing" means, so I cannot comment on
> that. All I actually meant is that an accumulation of 10% errors is still no
> more than a 10% error in the total however many items are added.

That's what I meant and yes I wrote that poorly.

But the objective is to get that 10% down as much as possible. Garmin
does that well in the etrex that I have, and my filter does it well too
... but Google Earth's "Elevation Profile" integration shows about 10%
positive error.

[Terje Mathisen]

My "hikes" are in the form of orienteering competitions, running in both
regular and very dense forests: Under these conditions the actual track
length error is almost always in the other direction, i.e. the track log
is showing _less_ sidewise movement than what I've really done, in order
to run around individual trees, rocks and other obstacles!
:-)

OTOH when I'm sailing, the actual/real track is usually less wiggly than
the raw GPS fixes, so some filtering will improve it.

About 10 years ago I wrote a filter which would optimize a given track
log, i.e. either minimizing the number of track points needed to
maintain a maximum allowable sidewise offset from the original track
points, or doing the same with a given budget of track points.

The latter was most useful when converting a track log into a route for
a GPS with relatively low limits on the number of route points.

I gave this code to the author of OziExplorer, he used it as the basis
for the filter in that application.

[claudegps]

On 6 Feb, 16:02, "Mike Coon" <Mike@@mjcoon.+.com> wrote:
> Alan Browne wrote:
> > Except that accumulating differences between 2 GPS points over short
> > distances and time will result in an error accumulation.  Those short
> > distances need to be filtered (appropriate to the speed of movement)
> > to reduce those errors.
>
> But not an error *percentage* accumulation.
>
> Another way of getting distance would be to integrate the speed
> measurements. But this is not likely to be easier of more accurate.

This should be usually worst, because if loose the fix you loose also
the speed you would like to integrate.
Using position delta instead, unless you make turns while you son't
have the fix, you can recover the distance lost when in no-fix.

[Alan Browne]

If the number of speed samples = number of position samples, and the
resolution of the speed sample is fine enough, there would be no
difference.

Note that hiking rarely allows straight line travel, so if you lose the
GPS, then your last velocity (or average v) will integrate well whereas
the last position to new position when GPS comes back will be a straight
line that does not represent the line of travel.

[Alan Browne]

On 2012-02-07 06:36 , Terje Mathisen wrote:
> Alan Browne wrote:
>> On 2012-02-06 13:08 , Terje Mathisen wrote:
>>> I.e. errors along the direction of travel don't matter for the final
>>> distance accumulator.
>>
>> Back to the "transverse noise" and that continues to integrate.
>> I've proven this with simple integrators like that in Google Earth and
>> my own code - which is why I've implemented filters that reduce the
>> error significantly, esp. for hiking. For higher speeds, less filtering
>> is required.
>>
>
> My "hikes" are in the form of orienteering competitions, running in both
> regular and very dense forests: Under these conditions the actual track
> length error is almost always in the other direction, i.e. the track log
> is showing _less_ sidewise movement than what I've really done, in order
> to run around individual trees, rocks and other obstacles!
> :-)

Then you need a GPS with a higher sampling rate.

> OTOH when I'm sailing, the actual/real track is usually less wiggly than
> the raw GPS fixes, so some filtering will improve it.
>
> About 10 years ago I wrote a filter which would optimize a given track
> log, i.e. either minimizing the number of track points needed to
> maintain a maximum allowable sidewise offset from the original track
> points, or doing the same with a given budget of track points.

How does it select?

> The latter was most useful when converting a track log into a route for
> a GPS with relatively low limits on the number of route points.
>
> I gave this code to the author of OziExplorer, he used it as the basis
> for the filter in that application.
>
> Terje


--

[Terje Mathisen]

(This is from decade-old memory)

Reduce a track to N points:

The first iteration would start with a straight line from start to end,
then locate the track point with the largest perpendicular offset.

The track would then be split into two parts, repeating the previous
process while looking for offsets from the two straight segments.

I did this until I had something like twice as many points as the target
(N) I was looking for.

Next I went through the 2N selected points and deleted the one which had
the least offset from a straight line between its two neighbors,
repeating this process until I was back at N points.

To further improve the fit I would also try some genetic permutations,
randomly selecting a track point to be moved to the next/previous point.

The final result is/was pretty good; even though it probably isn't
optimal it will be quite close.

[Alan Browne]

Interesting.

This would suit a B-tree sort of ever shorter segments until a threshold
was met. From there, a lot of possibilities open up to reduce "stop
noise" (when stopped, the random walk adds a lot to the distance with my
AMOD GPS).

> Next I went through the 2N selected points and deleted the one which had
> the least offset from a straight line between its two neighbors,
> repeating this process until I was back at N points.
>
> To further improve the fit I would also try some genetic permutations,
> randomly selecting a track point to be moved to the next/previous point.
>
> The final result is/was pretty good; even though it probably isn't
> optimal it will be quite close.

[claudegps]

> > On 6 Feb, 16:02, "Mike Coon"<Mike@@mjcoon.+.com> wrote:
> >> Alan Browne wrote:
> >>> Except that accumulating differences between 2 GPS points over short
> >>> distances and time will result in an error accumulation. Those short
> >>> distances need to be filtered (appropriate to the speed of movement)
> >>> to reduce those errors.
>
> >> But not an error *percentage* accumulation.
>
> >> Another way of getting distance would be to integrate the speed
> >> measurements. But this is not likely to be easier of more accurate.
>
> > This should be usually worst, because if loose the fix you loose also
> > the speed you would like to integrate.
> > Using position delta instead, unless you make turns while you son't
> > have the fix, you can recover the distance lost when in no-fix.
>
> If the number of speed samples = number of position samples, and the
> resolution of the speed sample is fine enough, there would be no
> difference.

This not what generally happens.
If I compute the distance by delta-pos and I compare it with speed
integration, the result is different.

Distance computed by delta-pos (no filtering), 1Hz: 3858 meters
Distance computed by speed integration, 1Hz: 3563 meters
If you look at the speed of a recorded track, it does not generally
represents exactly the distance/time
In fact, the speed is generally lower than the delta_pos/time and that
reflects that the distance computed integrating the speed is lower
than the delta_pos method.

(I can share the track, if you want)

> Note that hiking rarely allows straight line travel, so if you lose the
> GPS, then your last velocity (or average v) will integrate well whereas
> the last position to new position when GPS comes back will be a straight
> line that does not represent the line of travel.

Ok, get it. You presume that your speed will be constant while in no-
fix.
Makes sense, especially when hiking.
May not work very well with a vehicle (that may be better approximated
by a straigth line), but this depends on the case.
Anyway, the problem that the speed is generally lower than the
delta_pos/time will make you underestimate the total distance.

[claudegps]

On 6 Feb, 19:08, Terje Mathisen <"terje.mathisen at tmsw.no"> wrote:
> Alan Browne wrote:
> > On 2012-02-05 06:28 , Terje Mathisen wrote:
> >> As long as the distance between individual track points is within
> >> several km this will be effectively identical to the "perfect" distance.
>
> > Except that accumulating differences between 2 GPS points over short
> > distances and time will result in an error accumulation. Those short
> > distances need to be filtered (appropriate to the speed of movement) to
> > reduce those errors.
>
> Only true if you have transverse noise.
>
> I.e. errors along the direction of travel don't matter for the final
> distance accumulator.

Hi Terje, I'm not getting why you say that the errors along the
direction of travel don't matter for the final distance computation.

If my car is stopped, and I accumulate the delta-pos, I will
accumulate all the distances due to noise (not only the transeverse).
This will result in an overestimation of the distance travelled,
because every time the car is stopped I go on accumulating several
meters(of course it depends how much noise is present), no matter the
direction.

[Alan Browne]

On 2012-02-08 10:20 , claudegps wrote:
>
>>> On 6 Feb, 16:02, "Mike Coon"<Mike@@mjcoon.+.com> wrote:
>>>> Alan Browne wrote:
>>>>> Except that accumulating differences between 2 GPS points over short
>>>>> distances and time will result in an error accumulation. Those short
>>>>> distances need to be filtered (appropriate to the speed of movement)
>>>>> to reduce those errors.
>>
>>>> But not an error *percentage* accumulation.
>>
>>>> Another way of getting distance would be to integrate the speed
>>>> measurements. But this is not likely to be easier of more accurate.
>>
>>> This should be usually worst, because if loose the fix you loose also
>>> the speed you would like to integrate.
>>> Using position delta instead, unless you make turns while you son't
>>> have the fix, you can recover the distance lost when in no-fix.
>>
>> If the number of speed samples = number of position samples, and the
>> resolution of the speed sample is fine enough, there would be no
>> difference.
>
> This not what generally happens.
> If I compute the distance by delta-pos and I compare it with speed
> integration, the result is different.
>
> Distance computed by delta-pos (no filtering), 1Hz: 3858 meters
> Distance computed by speed integration, 1Hz: 3563 meters

Was there a period where GPS was not tracking?
Is the velocity resolution fine enough?

> If you look at the speed of a recorded track, it does not generally
> represents exactly the distance/time
> In fact, the speed is generally lower than the delta_pos/time and that
> reflects that the distance computed integrating the speed is lower
> than the delta_pos method.

It is true, of course, that GPS velocity is not computed by the GPS
based on position but with the Doppler shifts from the various
satellites. I did do a comparison (a couple years ago) and over long
enough periods (about an hour) of travel in a car and the difference
between the two sets tended to 0. (And that was with a seemingly coarse
velocity output). Tracked all of the time, however.

It's a spreadsheet (Excel) that I could send you. (Note the data is
processed out of a Pascal program running under windows - so one column
of data is the GPS velocity and the other is the dx/dt computed velocity.

> (I can share the track, if you want)

Sure. Take the "Freelunch" out of the e-meyl address above.

>> Note that hiking rarely allows straight line travel, so if you lose the
>> GPS, then your last velocity (or average v) will integrate well whereas
>> the last position to new position when GPS comes back will be a straight
>> line that does not represent the line of travel.
>
> Ok, get it. You presume that your speed will be constant while in no-
> fix.

Not sure. Different devices do different things. For example my TomTom
in the car at the _nav_ level (nothing to do with GPS) does coast the
velocity when in tunnels. I have no idea what the GPS module in the
TomTom is outputting. How systems are integrated is what makes the
difference.

> Makes sense, especially when hiking.
> May not work very well with a vehicle (that may be better approximated
> by a straigth line), but this depends on the case.
> Anyway, the problem that the speed is generally lower than the
> delta_pos/time will make you underestimate the total distance.

Shouldn't be too bad if the GPS tracked the entire time.

[claudegps]

Very interesting.
I did something similar, but (old-time memory for me also) with a
different method.
My computatio is sequential.
Start from point N, draw a line to point N+2 and determine the
transverse error of point N+1.
If the error is lower than the thresold, point N+1 is discarded.
The compute the N to N+3 and discard N+2 is the error is lower than
the threshold. If the error is above the threshold, then point N+2 is
kept and used as initial reference.
And so on... (hope that this bad explanation wil lresult clear :) )

From what I can see from you method is that allows you to select the
maximum number of points and have the best possible fit for that
number of points, not possible for my method.
Mine can only ensure that the error is lower than the threshold, but
cannot predict(or let the user decide) how many points will be used.

Can I steal the idea to implement it in my GPS 2 Google earth
converter? :)

[Alan Browne]

On 2012-02-08 10:26 , claudegps wrote:
> On 6 Feb, 19:08, Terje Mathisen<"terje.mathisen at tmsw.no"> wrote:
>> Alan Browne wrote:
>>> On 2012-02-05 06:28 , Terje Mathisen wrote:
>>>> As long as the distance between individual track points is within
>>>> several km this will be effectively identical to the "perfect" distance.
>>
>>> Except that accumulating differences between 2 GPS points over short
>>> distances and time will result in an error accumulation. Those short
>>> distances need to be filtered (appropriate to the speed of movement) to
>>> reduce those errors.
>>
>> Only true if you have transverse noise.
>>
>> I.e. errors along the direction of travel don't matter for the final
>> distance accumulator.
>
> Hi Terje, I'm not getting why you say that the errors along the
> direction of travel don't matter for the final distance computation.

They are pretty gaussian along track so they average to 0 over 100's of
points. Whereas x-track accumulate like the path of a drunken sailor.

>
> If my car is stopped, and I accumulate the delta-pos, I will
> accumulate all the distances due to noise (not only the transeverse).
> This will result in an overestimation of the distance travelled,
> because every time the car is stopped I go on accumulating several
> meters(of course it depends how much noise is present), no matter the
> direction.

He's "in motion" most of the time in his case.

[claudegps]

[cut]

> >> If the number of speed samples = number of position samples, and the
> >> resolution of the speed sample is fine enough, there would be no
> >> difference.
>
> > This not what generally happens.
> > If I compute the distance by delta-pos and I compare it with speed
> > integration, the result is different.
>
> > Distance computed by delta-pos (no filtering), 1Hz: 3858 meters
> > Distance computed by speed integration, 1Hz:       3563 meters
>
> Was there a period where GPS was not tracking?
> Is the velocity resolution fine enough?

100% fix valid, 1Hz... and... Speed from RMC, so not really the best
resolution available, but if you look at the track you will notice
that
the speed is always lower than the delta_pos/time by a large amount in
certain cases (far more that the resolution of the speed)

> > If you look at the speed of a recorded track, it does not generally
> > represents exactly the distance/time
> > In fact, the speed is generally lower than the delta_pos/time and that
> > reflects that the distance computed integrating the speed is lower
> > than the delta_pos method.
>
> It is true, of course, that GPS velocity is not computed by the GPS
> based on position but with the Doppler shifts from the various
> satellites.  I did do a comparison (a couple years ago) and over long
> enough periods (about an hour) of travel in a car and the difference
> between the two sets tended to 0.  (And that was with a seemingly coarse
> velocity output).  Tracked all of the time, however.

> It's a spreadsheet (Excel) that I could send you.  (Note the data is
> processed out of a Pascal program running under windows - so one column
> of data is the GPS velocity and the other is the dx/dt computed velocity.
>
> > (I can share the track, if you want)
>
> Sure.  Take the "Freelunch" out of the e-meyl address above.
>

I uploaded the file here: http://goo.gl/GeGPq
so that anyone following the discussion can have a look at it.
You will find teh NMEA log, the distance computation in the excel file
and the KMZ for google earth.
You can see, especially during turns, how the speed is much smaller
than the delta_pos/time

The GPS is a SSIII

[cut]

> > Makes sense, especially when hiking.
> > May not work very well with a vehicle (that may be better approximated
> > by a straigth line), but this depends on the case.
> > Anyway, the problem that the speed is generally lower than the
> > delta_pos/time will make you underestimate the total distance.
>
> Shouldn't be too bad if the GPS tracked the entire time.

I find 5-10% less distance if computing with speed integration than
using delta-pos

[claudegps]

On 8 Feb, 16:49, Alan Browne <alan.bro...@FreelunchVideotron.ca>
wrote:

> On 2012-02-08 10:26 , claudegps wrote:
>
>
>
>
>
>
>
>
>
> > On 6 Feb, 19:08, Terje Mathisen<"terje.mathisen at tmsw.no">  wrote:
> >> Alan Browne wrote:
> >>> On 2012-02-05 06:28 , Terje Mathisen wrote:
> >>>> As long as the distance between individual track points is within
> >>>> several km this will be effectively identical to the "perfect" distance.
>
> >>> Except that accumulating differences between 2 GPS points over short
> >>> distances and time will result in an error accumulation. Those short
> >>> distances need to be filtered (appropriate to the speed of movement) to
> >>> reduce those errors.
>
> >> Only true if you have transverse noise.
>
> >> I.e. errors along the direction of travel don't matter for the final
> >> distance accumulator.
>
> > Hi Terje, I'm not getting why you say that the errors along the
> > direction of travel don't matter for the final distance computation.
>
> They are pretty gaussian along track so they average to 0 over 100's of
> points.  Whereas x-track accumulate like the path of a drunken sailor.

Ok, but accumulating the delta_pos is not 0 average, because you sum
all delta_pos in absolute value.
If you discard along track noise, then you can also loose the case
where you go straigth, go back for some meters, and then head back to
the destination.
Of course A to B is the same, but not the distance you travelled.
(Of course also a filter may loose this, because if you go back by a
distance lower than the filter threshold, you filter that out)

>
> > If my car is stopped, and I accumulate the delta-pos, I will
> > accumulate all the distances due to noise (not only the transeverse).
> > This will result in an overestimation of the distance travelled,
> > because every time the car is stopped I go on accumulating several
> > meters(of course it depends how much noise is present), no matter the
> > direction.
>
> He's "in motion" most of the time in his case.

In that case, no problem

[Alan Browne]

On 2012-02-08 11:46 , claudegps wrote:
> On 8 Feb, 16:49, Alan Browne<alan.bro...@FreelunchVideotron.ca>
> wrote:
>> On 2012-02-08 10:26 , claudegps wrote:
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>> On 6 Feb, 19:08, Terje Mathisen<"terje.mathisen at tmsw.no"> wrote:
>>>> Alan Browne wrote:
>>>>> On 2012-02-05 06:28 , Terje Mathisen wrote:
>>>>>> As long as the distance between individual track points is within
>>>>>> several km this will be effectively identical to the "perfect" distance.
>>
>>>>> Except that accumulating differences between 2 GPS points over short
>>>>> distances and time will result in an error accumulation. Those short
>>>>> distances need to be filtered (appropriate to the speed of movement) to
>>>>> reduce those errors.
>>
>>>> Only true if you have transverse noise.
>>
>>>> I.e. errors along the direction of travel don't matter for the final
>>>> distance accumulator.
>>
>>> Hi Terje, I'm not getting why you say that the errors along the
>>> direction of travel don't matter for the final distance computation.
>>
>> They are pretty gaussian along track so they average to 0 over 100's of
>> points. Whereas x-track accumulate like the path of a drunken sailor.
>
> Ok, but accumulating the delta_pos is not 0 average, because you sum
> all delta_pos in absolute value.

It depends on the method.

> If you discard along track noise, then you can also loose the case
> where you go straigth, go back for some meters, and then head back to
> the destination.
> Of course A to B is the same, but not the distance you travelled.
> (Of course also a filter may loose this, because if you go back by a
> distance lower than the filter threshold, you filter that out)
>
>>
>>> If my car is stopped, and I accumulate the delta-pos, I will
>>> accumulate all the distances due to noise (not only the transeverse).
>>> This will result in an overestimation of the distance travelled,
>>> because every time the car is stopped I go on accumulating several
>>> meters(of course it depends how much noise is present), no matter the
>>> direction.
>>
>> He's "in motion" most of the time in his case.
>
> In that case, no problem
>

[Alan Browne]

I looked at it.
Did you account for cos(lat) in your delta-easts?

> You can see, especially during turns, how the speed is much smaller
> than the delta_pos/time
>
> The GPS is a SSIII
>
> [cut]
>>> Makes sense, especially when hiking.
>>> May not work very well with a vehicle (that may be better approximated
>>> by a straigth line), but this depends on the case.
>>> Anyway, the problem that the speed is generally lower than the
>>> delta_pos/time will make you underestimate the total distance.
>>
>> Shouldn't be too bad if the GPS tracked the entire time.
>
> I find 5-10% less distance if computing with speed integration than
> using delta-pos

I can send you my Excel for a 1 hour trip and you'll see the error
pretty close to 0.

[Alan Browne]

On 2012-02-08 11:00 , claudegps wrote:

> I uploaded the file here: http://goo.gl/GeGPq
> so that anyone following the discussion can have a look at it.
> You will find teh NMEA log, the distance computation in the excel file
> and the KMZ for google earth.
> You can see, especially during turns, how the speed is much smaller
> than the delta_pos/time

By the way: Google Earth puts that run at 3.87 km.

Ignore my cos(lat) * delta east comment. That would go the opposite way...

[Terje Mathisen]

claudegps wrote:
> Hi Terje, I'm not getting why you say that the errors along the
> direction of travel don't matter for the final distance computation.
>
> If my car is stopped, and I accumulate the delta-pos, I will
> accumulate all the distances due to noise (not only the transeverse).

OK, that is an issue: I was assuming continuous forward movement, at
sufficient speed so that GPS noise will never cause a backwards step in
position.

> This will result in an overestimation of the distance travelled,
> because every time the car is stopped I go on accumulating several
> meters(of course it depends how much noise is present), no matter the
> direction.

Sure, and this is why pretty much all GPSs have some sort of "I'm
stopped" logic where they freeze the position. I.e. a truncate to zero
for any sufficiently low speed value.

[Terje Mathisen]

claudegps wrote:
> Very interesting.
> I did something similar, but (old-time memory for me also) with a
> different method.
> My computatio is sequential.
> Start from point N, draw a line to point N+2 and determine the
> transverse error of point N+1.
> If the error is lower than the thresold, point N+1 is discarded.
> The compute the N to N+3 and discard N+2 is the error is lower than
> the threshold. If the error is above the threshold, then point N+2 is
> kept and used as initial reference.

You have to go back and check the N+1 error as well, right?

> And so on... (hope that this bad explanation wil lresult clear :) )
>
> From what I can see from you method is that allows you to select the
> maximum number of points and have the best possible fit for that
> number of points, not possible for my method.

Yes, that was crucial for a track-to-route conversion.

> Mine can only ensure that the error is lower than the threshold, but
> cannot predict(or let the user decide) how many points will be used.

What you really want is a Chebychev-type min/max filter, where you pick
points such that you have the same maximum error on both sides of
each/every line segment, this means that you should not end up with the
extremist points my algorithm picks during the first iteration.

This is also the reason that the final genetic part tend to find a
better solution by looking for track points nearby the maximum offset
points.

>
> Can I steal the idea to implement it in my GPS 2 Google earth
> converter? :)

Of course! That's why I publish idea, so that they will (hopefully!) be
of use to someone. :-)

[claudegps]

On 8 Feb, 20:35, Alan Browne <alan.bro...@FreelunchVideotron.ca>
wrote:

Ok, no need to send the file :) I believe it's close to 0 in your
case.
It may depend on the receiver behaviour, speed, signal conditions
ecc...

[claudegps]

On 9 Feb, 08:33, Terje Mathisen <"terje.mathisen at tmsw.no"> wrote:
> claudegps wrote:
> > Very interesting.
> > I did something similar, but (old-time memory for me also) with a
> > different method.
> > My computatio is sequential.
> > Start from point N, draw a line to point N+2 and determine the
> > transverse error of point N+1.
> > If the error is lower than the thresold, point N+1 is discarded.
> > The compute the N to N+3 and discard N+2 is the error is lower than
> > the threshold. If the error is above the threshold, then point N+2 is
> > kept and used as initial reference.
>
> You have to go back and check the N+1 error as well, right?

Well... no. Once a point is discarded because the next one is inside
the threshold, it's basically lost forever :)
The target was to be able to simplify the path stored in with points
coming in real-time and reducing as much as possible memory
requirements and complexity.
So, no way to go back to a point you decided to discard.

> > And so on...   (hope that this bad explanation wil lresult clear :) )
>
> >  From what I can see from you method is that allows you to select the
> > maximum number of points and have the best possible fit for that
> > number of points, not possible for my method.
>
> Yes, that was crucial for a track-to-route conversion.
>
> > Mine can only ensure that the error is lower than the threshold, but
> > cannot predict(or let the user decide) how many points will be used.
>
> What you really want is a Chebychev-type min/max filter, where you pick
> points such that you have the same maximum error on both sides of
> each/every line segment, this means that you should not end up with the
> extremist points my algorithm picks during the first iteration.
>
> This is also the reason that the final genetic part tend to find a
> better solution by looking for track points nearby the maximum offset
> points.
>
>
>
> > Can I steal the idea to implement it in my GPS 2 Google earth
> > converter? :)
>
> Of course! That's why I publish idea, so that they will (hopefully!) be
> of use to someone. :-)

Ok, Thanks. I'll give you the credit of the method anyway ;)

[dalede]

Alan Browne wrote:

>
> As to phones, I have the iPhone 4 and its tracks are not that great,
> esp. in the woods. The iPhone is great for finding restaurants and
> navigating there, it's not a great hiking GPS - though the display
> quality is fantastic, the "compass" is nice, and I have several paid for
> GPS Apps that are very good. Too bad the GPS in the iPhone is a bit
> crippled.
>
> It would be nice if the iPhone GPS could be set by the user to run at a
> higher rate (at the expense of battery power) such that the GPS solution
> would be better.
>

You could replace the GPS on the phone with a better one. Apple provides
two solutions. One uses a proprietary modified Bluetooth GPS device to
provide the GPS solution using Bluetooth frequencies. The second uses
the Apple connector to hook up GPS hardware.

Dale

[Alan Browne]

I know. But I have so many GPS devices that getting anymore would be a
legally actionable sign of insanity as far as my SO is concerned. (If I
spot one at a good price though...).

--
"I was gratified to be able to answer promptly, and I did.
I said I didn't know."
-Samuel Clemens.

[Terje Mathisen]

Alan Browne wrote:
> I know. But I have so many GPS devices that getting anymore would be a
> legally actionable sign of insanity as far as my SO is concerned. (If I
> spot one at a good price though...).
>

How many is too many?

A quick count indicates that I have 7-8 currently operational GPS'es,
plus another 2-3 (timing) GPS boards, all at home except for the Sure
box which is currently connected to an old FreeBSD laptop at my mountain
cabin.

I also operate a bunch of gps-based NTP servers based on FreeBSD which I
have installed at work.

[Alan Browne]

On 2012-03-04 16:47 , Terje Mathisen wrote:
> Alan Browne wrote:
>> I know. But I have so many GPS devices that getting anymore would be a
>> legally actionable sign of insanity as far as my SO is concerned. (If I
>> spot one at a good price though...).
>>
>
> How many is too many?
>
> A quick count indicates that I have 7-8 currently operational GPS'es,
> plus another 2-3 (timing) GPS boards, all at home except for the Sure
> box which is currently connected to an old FreeBSD laptop at my mountain
> cabin.
>
> I also operate a bunch of gps-based NTP servers based on FreeBSD which I
> have installed at work.

I have about that many. No mood to make an inventory. But there's
always room for another.

How do I hide them from my SO?

[Mike Coon]

Alan Browne wrote:
> I have about that many. No mood to make an inventory. But there's
> always room for another.
>
> How do I hide them from my SO?

Disguise them in plain sight as stud finders? On 2nd thoughts...

Mike.
--
If reply address is Mike@@mjcoon.+.com (invalid), remove spurious "@"
and substitute "plus" for +.