USGS Streams

[Michael Medina]

Recently I was discussing OSM with someone and they mentioned how few of our waterways are on the map.  Looking at the shapefiles for our streams, there are a ton of them and the density is quite high.  There's no data on their seasonal nature but I'm willing to occasionally add swaths of USGS waterways to the map.  Attached is the density level I'm seeing.  I have no way to sort out what "should" or "shouldn't" be on the map, but "should" this be something to work on or is this too much?

[Jack Newlevant]

I have noticed how much waterway coverage varies by location, do love having them included.  So I vote yes for more effort locally!

On Thu, Nov 16, 2017 at 2:18 AM, Michael Medina <802re...@gmail.com> wrote:

Recently I was discussing OSM with someone and they mentioned how few of our waterways are on the map.  Looking at the shapefiles for our streams, there are a ton of them and the density is quite high.  There's no data on their seasonal nature but I'm willing to occasionally add swaths of USGS waterways to the map.  Attached is the density level I'm seeing.  I have no way to sort out what "should" or "shouldn't" be on the map, but "should" this be something to work on or is this too much?

--
You received this message because you are subscribed to the Google Groups "OSM PDX" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osm-pdx+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

   Jack Newlevant
   __o         Bi(cycle) Ro(ute) Te(chnology)
 _'\<,_        home: 503.236.4920, cell: 503.806.3771
(_)/ (_)____ txt: 50380...@messaging.sprintpcs.com

[Alan Millar]

I was used to be a big advocate of "import now; fix later".  I did a lot of points from USGS like schools and libraries, and airports from FAA and IATA public data.  And most relevant, I did the NHD waterways import for Washington County. 

They all contained a LOT of bad or obsolete data.  I spent a lot of time reconciling streams to aerial photos. For example, we have many, many places where original stream beds have been replaced by culverts and storm drains from suburban construction. Sometimes it's clear the stream is no longer in the middle of those two houses in the cul-de-sac, but it's not obvious where it went.. Etc.

Most of my work I think was perhaps about 8 years ago.  Even though I vetted the data as much as I could, shoddy data still got through. I still get messages about twice a year asking me why I put something clearly wrong on the map in this place or that.

Personally I'm no longer a big fan of imports, unless there is an ongoing sync process like Trimet (which is awesome; go team!). Data changes too much.

I'm not saying you can't do it. But I am saying prepare yourself for a long investment in time and effort. Good luck!

- Alan

On Nov 16, 2017, at 12:18 AM, Michael Medina <802re...@gmail.com> wrote:

Recently I was discussing OSM with someone and they mentioned how few of our waterways are on the map.  Looking at the shapefiles for our streams, there are a ton of them and the density is quite high.  There's no data on their seasonal nature but I'm willing to occasionally add swaths of USGS waterways to the map.  Attached is the density level I'm seeing.  I have no way to sort out what "should" or "shouldn't" be on the map, but "should" this be something to work on or is this too much?

--
You received this message because you are subscribed to the Google Groups "OSM PDX" group.

To unsubscribe from this group and stop receiving emails from it, send an email to osm-pdx+u...@googlegroups.com.

[Grant Humphries]

@Michael here are the import guidelines from the OSM wiki FYI

To unsubscribe from this group and stop receiving emails from it, send an email to osm-pdx+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "OSM PDX" group.

To unsubscribe from this group and stop receiving emails from it, send an email to osm-pdx+unsubscribe@googlegroups.com.

[Darrell Fuhriman]

Yah, I’d be really skeptical of doing NHD for all the reasons that Alan mentions.

Or if I did, I’d limit it to streams above a certain size that would be relatively easy to verify via satellite imagery.

And then, of course, you have to deal with the conflation aspect, which is non-trivial.

I think doing a straight up import would be a hard sell — it would have to be have to a pretty manual process to do it properly, IMO.

Darrell

[Alan Millar]

> On Nov 17, 2017, at 8:53 AM, Darrell Fuhriman <dar...@garnix.org> wrote:
>
> And then, of course, you have to deal with the conflation aspect, which is non-trivial.

Yes, it was a lot of work to untangle the overlap with existing rivers and streams already in there.

[Michael Medina]

I certainly would not do a mass import, already I've been playing around with a small area and it's a bit more work than I anticipated.  I have at least determined to query out the named streams for the conflation part.  Unnamed streams could almost be bulk imported, but I'd still do smaller areas at a time, that's just how I usually work.

Does anyone follow those import guidelines?  As far as I can tell having 20 steps is really just a method to say "we don't want you to do this" yet it's ok for me to trace out every stream directly over the National Map.  This makes no sense to me to have such a convoluted process and if I'm not dropping a whole region in, I'm not sure it's needed.

[Peter Dobratz]

Yes, data for streams should be part of what's in OpenStreetMap.  It's certainly one of the things that I attempt to pay attention to when I'm out and about collecting data for OSM.

I've noticed that when attempting to manually add a stream to OSM, there are often trees in the way which makes tracing from aerial photos difficult.  It's also often not practical to walk in the stream itself to collect a GPS trace.

Here are some things to consider when adding the data:

• Be careful of intersections with existing OSM data.  You'll probably find that you want to split portion of the stream out that goes under a road and add layer=-1 and tunnel=culvert.  If there is a bridge, instead split the road and add bridge=yes, and layer=1.
• Manually inspect any intersections of streams with buildings.  I imagine that any intersections are due to the stream or the building being in the wrong spot.
• Sometimes streams or rivers form administrative boundaries.  It's possibly to add the water feature as a member of the administrative boundary relation.  Administrative boundaries are another area that is difficult to verify "on the ground" and legally the boundary may not actually be defined by the water feature.  The path of the river may change over time, but the administrative boundary may not change.
• Note that the direction of the Way object is OSM is significant and streams are expected to point downstream.  Also, streams should connect to other streams.
• Think about where you want to just use a Way (waterway=stream or waterway=river) or whether you want an Area (waterway=riverbank on a multipolygon or closed Way).
• Think about Node density.  There's no hard and fast rule here, but I've seen problems where imported data uses way too many Nodes to represent things and it just makes future editing of the data more complicated.  I recently noticed that a lot of small traffic circles in Portland were created using 52 Nodes, which is probably too dense (maybe 8 would be a better number).
• Use the source on your changesets to indicate where the data comes from.

Looking at JOSM warnings before you upload data is usually a good starting point.  Download existing OSM data into JOSM before merging in your new data and uploading (don't just upload and hope for the best).

If you'd like to provide more information on the data set that you are using and your current methodology for incorporating portions of that data set into OSM, I'd be happy to help with the task of manually conflating the data.

Peter

--
You received this message because you are subscribed to the Google Groups "OSM PDX" group.

To unsubscribe from this group and stop receiving emails from it, send an email to osm-pdx+unsubscribe@googlegroups.com.

[Alan Millar]

> On Nov 17, 2017, at 11:10 AM, Michael Medina <802re...@gmail.com> wrote:
> Unnamed streams could almost be bulk imported

From my prior experience, the smaller unnamed streams had the largest variance from the current aerial photos. Those are the spots most likely to have been changed with construction and housing development. As bulk operations go, they are the best candidates for excluding.

> Does anyone follow those import guidelines? As far as I can tell having 20 steps is really just a method to say "we don't want you to do this" yet it's ok for me to trace out every stream directly over the National Map. This makes no sense to me to have such a convoluted process and if I'm not dropping a whole region in, I'm not sure it's needed.

Hoo boy. Yes, absolutely people follow the guidelines, and you should also. Yes, it is indeed there to put a damper on imports. That's to discourage dumping garbage into OSM. Nobody starts out intending to dump garbage in. But that has ended up as the result too many times.

Tracing streams off the National Map is a bad idea also. In my experience, it is naive to think that data is accurate. It's going to be the same out-of-date NHD data.

Tracing from aerial photos would be more acceptable, because it forces you to validate every point against recent photos. Much more work, for much more up-to-date results.

I hate to be the naysayer to rain on the parade. But I was the eager importer in your shoes previously, and I can vouch that all those import guidelines are there from hard-learned lessons by the folks that preceded you. Please take their advice to heart. Thanks!

- Alan

[Michael Medina]

I was always just going to do very localized imports that I could take the time to individually verify so this doesn't change my ambitions much.  Probably limited to the area that I could download in JOSM.  The discussion I suppose seemed like I might just drop that whole shape file in there, but that would never be the case, I don't work like that.  I did some stuff around Beacon Rock last night and it was, like I said, a little more involved than I had thought it would be.  The NHD data was better than the OSM on 4 of the 5 streams so I merged the best data, still faster than tracing aerials one click at a time.  Most of the places I'm looking at doing are remote and forested so I don't foresee many conflation issues.

--
You received this message because you are subscribed to a topic in the Google Groups "OSM PDX" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/osm-pdx/KS959lwY_Ew/unsubscribe.
To unsubscribe from this group and all its topics, send an email to osm-pdx+u...@googlegroups.com.