Survey work for Digital Elevation Model DEM attributing of OSM roads (Non-GPS elevations) ele + incline tags

[Chris Cooper]

Hi all. 

Am a newbie to OJ, but already feel at home - and think OJ is a fabulous project.

My personal project involves capturing elevation values along roads, to build a road elevation profile.

I am seeking anyone who has a similar interest, or knows of any tools to assist this project.

If none exist, I may need to get a plugin written.

My project needs very accurate elevation. GPS elevation is far too coarse, as is STRM data, or any other source.

Google and other sources have free API's for providing elevation for any point or track requested - but these values are also too coarse for me. Good, and close, but still dirty data, and not good for determining local incline values accurately.

I am currently bike riding and driving to survey my local area, using an iPhone 4S GPS logger, with a barometric pressure logger to deduce local elevation at 1 Hz. 

Capture elevation resolution is <50cm and I want to store 1m ele resolution or better, with nodes at each change in slope (of above a set change in slope threshold)

The iPhone 4S GPS is good enough for my needs, and is reportedly now with GLONASS support for better 'GPS' accuracy, but I am suspicious this is more just a marketing ploy to evade the high Russian import taxes on non-GLONASS supporting devices than actually 'working' - but nobody seems to have been able to verify this question yet. :-)

I merge 'corrected' pressure based elevation values with the GPS stream, to generate a GPX file with ele tags to store the 'much more accurate and stable than GPS' elevation values along the route.

Ele values are corrected by better than 'published' local pressure values, as I correct off known locations to derive a more accurate and more localised pressure offset, which is then prorata averaged across my survey data for each of a very few number of ele reference points - i.e. the ele correction is variable for all points on the track, thus catering to changes in pressure during the recordings, or changes in position. (This bit is working now)

Then, I need to use OJ RoadMatcher to align the GPX path to the road network, to correct for GPS errors (hence no real need for DGPS etc) - or occasionally to 'correct' the road lines in OSM if the recorded tracks show it needs to be adjusted (This is my current area of work and is not yet done)

Then I want to attribute the road network with the survey elevation values, noting this is not just attributing existing nodes, but generating new nodes for elevation (or incline) changes - e.g. a big hill in the middle of an otherwise straight link.

Not sure if OSM will appreciate a large increase in new nodes, solely to capture elevation changes that are not aligned to a bend or intersection in the road - but I would hope to upload ele and incline values to existing OSM nodes at the very least.

Questions:

1. Who out there has similar interests or experience in this area of road survey + DEM?

2. Any idea on how well supported OJ and RoadMatcher plugin are for handling ele elevation values? or Incline? E.g. as GPX tags

3. As above, if ele and incline are supported in OJ, then how easy is it to populate road links with new 'elevation' nodes?

4. Who may have interest in developing a DEM survey OJ plugin (if needed) for this project - it is a personal project, but I could fund some development work. Perhaps to merely 'improve' ele support for the existing GPX and RoadMatcher plugins.

Thanks,

Chris

[Stefan Steiniger]

Hi Chris,

uiii - thats some very specific questions.
So, from my side rather some notes:

a) don't know if Roadmatcher supports elevation. The OJ underlying
geometry library is called JTS (JTS Topology Suite). It is not a 3D
model but it allows you to carry a z-coordinate with you.

Michael recently implemented that z-coordinates are observed for
intersection calculations, I think.

b) When I did read that you wanted to measure along the road I was
worried that you need "along-the-road" indexing. Because OJ can't do
that (or: nobody ever cared about developing this).

c) instead of using the GPX plugin, Jukka (maybe he has time to respond
you) uses the GPS plugin from Ede:

http://sourceforge.net/apps/mediawiki/jump-pilot/index.php?title=GPS_plugin

not sure if that is an alternative if you have a GPS mouse? I think it
is even realtime.. though I never tested it and can't say if what it
does with the elevation value.

d) I am not sure if ever someone had really made some DEM stuff with
OpenJUMP. Though, there was a TIN plugin, to create TINs, but we never
really integrated this Google Summer of Code work into OpenJUMP :(
sadly... because it looked very promising. On the other hand the
Sextante Toolbox of OJs Plus Edition has some interpolation tools and
DEM tools - maybe they are helpful?

And yes, as you noted - we are a fairly small project, and sometimes
this is an advantage, as we more or less know each other after some time
- even that we are really distributed all over the world :)
Though on the downside we have many user (well.. downloads), but just
very few are subscribed to and listen to the user list. Hence, you may
not get many responses.. but lets see...

cheers,
stefan

Am 17.01.12 16:21, schrieb Chris Cooper:

> to generate a GPX file with ele tags to store the '/much more accurate
> and stable than GPS'/ elevation values along the route.

> Ele values are corrected by better than 'published' local pressure
> values, as I correct off known locations to derive a more accurate and
> more localised pressure offset, which is then prorata averaged across my
> survey data for each of a very few number of ele reference points - i.e.
> the ele correction is variable for all points on the track, thus
> catering to changes in pressure during the recordings, or changes in
> position. (This bit is working now)
>
> Then, I need to use OJ RoadMatcher to align the GPX path to the road
> network, to correct for GPS errors (hence no real need for DGPS etc) -
> or occasionally to 'correct' the road lines in OSM if the recorded
> tracks show it needs to be adjusted (This is my current area of work and
> is not yet done)
>
> Then I want to attribute the road network with the survey elevation
> values, noting this is not just attributing existing nodes, but
> generating new nodes for elevation (or incline) changes - e.g. a big
> hill in the middle of an otherwise straight link.
>
> Not sure if OSM will appreciate a large increase in new nodes, solely to
> capture elevation changes that are not aligned to a bend or intersection
> in the road - but I would hope to upload ele and incline values to

> /existing /OSM nodes at the very least.

[sunburned...@gmail.com]

Chris:

Let me chew on your email. I will try to respond this weekend.

Landon

Sent from myTouch 4G

----- Reply message -----
From: "Chris Cooper" <chris...@gmail.com>
To: <openjum...@googlegroups.com>
Subject: [openjump-users] Survey work for Digital Elevation Model DEM attributing of OSM roads (Non-GPS elevations) ele + incline tags
Date: Tue, Jan 17, 2012 3:21 pm

Hi all. 

Am a newbie to OJ, but already feel at home - and think OJ is a fabulous project.

My personal project involves capturing elevation values along roads, to build a road elevation profile.

I am seeking anyone who has a similar interest, or knows of any tools to assist this project.

If none exist, I may need to get a plugin written.

My project needs very accurate elevation. GPS elevation is far too coarse, as is STRM data, or any other source.

Google and other sources have free API's for providing elevation for any point or track requested - but these values are also too coarse for me. Good, and close, but still dirty data, and not good for determining local incline values accurately.

I am currently bike riding and driving to survey my local area, using an iPhone 4S GPS logger, with a barometric pressure logger to deduce local elevation at 1 Hz. 

Capture elevation resolution is <50cm and I want to store 1m ele resolution or better, with nodes at each change in slope (of above a set change in slope threshold)

The iPhone 4S GPS is good enough for my needs, and is reportedly now with GLONASS support for better 'GPS' accuracy, but I am suspicious this is more just a marketing ploy to evade the high Russian import taxes on non-GLONASS supporting devices than actually 'working' - but nobody seems to have been able to verify this question yet. :-)

I merge 'corrected' pressure based elevation values with the GPS stream, to generate a GPX file with ele tags to store the 'much more accurate and stable than GPS' elevation values along the route.

Ele values are corrected by better than 'published' local pressure values, as I correct off known locations to derive a more accurate and more localised pressure offset, which is then prorata averaged across my survey data for each of a very few number of ele reference points - i.e. the ele correction is variable for all points on the track, thus catering to changes in pressure during the recordings, or changes in position. (This bit is working now)

Then, I need to use OJ RoadMatcher to align the GPX path to the road network, to correct for GPS errors (hence no real need for DGPS etc) - or occasionally to 'correct' the road lines in OSM if the recorded tracks show it needs to be adjusted (This is my current area of work and is not yet done)

Then I want to attribute the road network with the survey elevation values, noting this is not just attributing existing nodes, but generating new nodes for elevation (or incline) changes - e.g. a big hill in the middle of an otherwise straight link.

Not sure if OSM will appreciate a large increase in new nodes, solely to capture elevation changes that are not aligned to a bend or intersection in the road - but I would hope to upload ele and incline values to existing OSM nodes at the very least.

[Chris Cooper]

My first attempt at using the RoadMather plugin to match GPX to OSM resulted in errors:

EdgeEnd with identical endpoints found. (Had to get this from the DOS console, not the App reported error)
java.lang.IllegalArgumentException: Cannot compute the quadrant for point ( 0.0, 0.0 )

I used the excellent Tools > QA to validate OSM. This allowed me to track matching errors down to problems in the OSM source data, where there were linestrings that were 'Not simple' - meaning they intersect themselves.

So it seems before I start GPX to road matching, I need to resolve all the OSM linestring errors at least (14 instances at least, in about a 5km radius).

There are more errors here than I was expecting for OSM. Is this normal? Perhaps not many care about this, as fewer use OSM data for routing and matching than other cartographic work where not-simple linestrings are less of a problem?

Anyone know comparison of OSM's JOSM vs. OJ for this work? I would like to 'fix' the Linestring errors in OSM, at least for my small survey area, or to better understand what I am doing (never fixed a linestring error before!). I suspect OJ may be better at detecting the errors, though JOSM better at fixing them with direct uploads etc?

I also see many other OSM errors like extremely tiny, unrealistic segment lengths that probably need node reducing?

Any comments on this area - or the comparison of JOSM vs OJ for OSM editing? (E.g. of linestring errors)

[Jukka Rahkonen]

Hi,

Data model of OSM has some brilliant features but unfortunaly it has such
fundamental differences with the Simple Feature based GIS world that it is
very hard to use for example OpenJUMP as an OSM editor. OSM data model is
a topological one where features may share common nodes but it is not
totally a planar graph either, I think.

OSM database itself accepts just everything and therefore it is not at all
uncommon to see errors that you described in the data. The main quality
control tool is the Mapnik renderer. If some feature has such an error
that Mapnin does not render it then probably someone goes and fixes it.
But Mapnik is rather tolarable and it accepts some errors which are not
accepted by OpenJUMP and other tools like it. I suppose that those who
have developed routing engines with OSM data have similarly made their
programs to handle or tolerate the typical data errors.

In the past I have used OpenJUMP rather a lot for OSM work. I was working
then on a white area without much existing OSM data. Then it was rather
straigh forward because there was not much work to do with connecting new
data with the old. Still I had to do the final step with JOSM. At minimum
it was combining the end and start nodes of ways and joining new ways into
existing ones. And naturally I had to add all the tags manually because I
was using a GPX format for transforming OpenJUMP edits into JOSM.

In conclusion, it is possible to create high quality data with OpenJUMP
that is for sure good for OSM after combining dublicate nodes with OSM.
However, combining data with existing data is rather hard. And in would
extremely hard to take the existing OSM data, run it through QA tools and
write it back which is a pity because it would improve the quality of
data.

-Jukka Rahkonen-

Chris Cooper wrote:
> My first attempt at using the RoadMather plugin to match GPX to OSM
> resulted in errors:
>

> *EdgeEnd with identical endpoints found. *(Had to get this from the DOS

> console, not the App reported error)

> *java.lang.IllegalArgumentException: Cannot compute the quadrant for point
> ( 0.0,** 0.0 )*

[Landon Blake]

Chris:

I'll try to interpret your e-mails and offer some suggestions.

Chris wrote: "My first attempt at using the RoadMather plugin to match

GPX to OSM resulted in errors:

EdgeEnd withidentical endpoints found. (Had to get this from the DOS

console, not the App reported error)

java.lang.IllegalArgumentException:Cannot compute the quadrant for
point ( 0.0, 0.0 )"

It sounds like the RoadMatcher could use some better error reporting. :]

Chris wrote: "This allowed me to track matching errors down to

problems in the OSM source data, where there were linestrings that
were 'Not simple' - meaning they intersect themselves.

So it seems before I start GPX to road matching, I need to resolve all
the OSM linestring errors at least (14 instances at least, in about a
5km radius)."

I would also suspect you need to clean-up the OSM data first. OpenJUMP
doesn't like self-intersecting LineStrings. The important question is:
How will you fix the self-intersections?

If you are going to use the same solution to fix all of the
self-intersections, you could automate this clean-up with a plug-in.
However, it sounds like you might have to deal with the
self-intersections on a case-by-case basis.

Chris wrote: "I also see many other OSM errors like extremely tiny,

unrealistic segment lengths that probably need node reducing?"

I know we have to have a LineString simplification plug-in in OJ. I
bet Michael or Stefan can tell you right where it is at.

Landon