[OSM-talk] ODbL-clean Coastlines
Paul Norman
latest data from my jxapi server. Tonight I switched it over to filter out
data that WTFE reports as dirty. This is somewhat more aggressive than the
rebuild will be, but the results are worrisome.
A PNG showing the OSM coastline post-transition along with red dots for
where the coastcheck program found errors is at
http://maps.paulnorman.ca/coastlines24b.png
The red dots represent places where coastcheck found an error
A few things can be seen:
No continent will generate a satisfactory rendering. Australia, the US, most
of Africa and most of Eurasia from 50 degrees south will be completely
flooded.
There are 3471 points where coastcheck found errors. Normally I would expect
about 10.
The following areas are particularly bad:
The west coast of the US
The US coast of the Great Lakes
Australia
Antarctica (which has significant parts completely missing with no error
points since there's nothing left to even try to close)
When examining the data, I found most of the ODbL-dirty status was mainly
from the following accounts:
hasse_osm_korinthenkacker (Antarctica)
Kin (Antarctica)
Blars (US West coast)
I see a few ways for dealing with the coastlines:
- Remapping. Obviously this is the only option where the coastline was
actually created by the non-acceptor.
- Adopting changesets. Many of the dirty ways and nodes appear to be
imported. If the imported just imported PD data than they have no IP in the
ways and they can be retained.
- odbl=clean. It's generally easy to verify that a way is a coastline from
imagery
My coastline files are available in the usual location of
http://pnorman.dev.openstreetmap.org/coastlines/ but are named coastlinec_p
and processedc_p. processedc_p has not yet uploaded but will be complete in
about an hour.
Additional detail of some regions can be found at
http://maps.paulnorman.ca/coastlines-europe.png
http://maps.paulnorman.ca/coastlines-na-lakes.png
http://maps.paulnorman.ca/coastlines-na-west.png
http://maps.paulnorman.ca/coastlines-nz.png
http://maps.paulnorman.ca/coastlines-au.png
If opening processedc_p in QGIS be sure to make a spatial index or it will
crawl.
A refresher on coastlines for those who haven't tagged them in awhile:
Direction *matters*. Land on the left, water on the right.
Each end of each way should join with exactly one other natural=coastline
way
I am not sure when each processedc_p file will be completed. Cron starts the
job at 5 AM PST and I'd expect it to take about 5 hours, mainly depending on
WTFE server speed and my upload speed to errol. The coastlinec_p files will
be uploaded first so arrive at the server earlier.
Technical details:
The results of the jxapi way[natural=coastline] is filtered by cleanway to
remove any objects reported as severity=normal, leaving behind 0 or 1 node
ways if necessary. It then is passed to the coastcheck programs and the
results uploaded.
A CT-clean way to which a decliner added a tag other than natural=coastline
would be removed by this algorithm, but not by the rebuild.
_______________________________________________
talk mailing list
ta...@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk
Simon Poole
I was just going to suggest this, while it limits you to CC_by-SA 2.0
as a licence for tiles (or similar), there is no reason that you
couldn't use a pre-licence change coastline for rendering (in the case
of the main OSM site, the tiles are going to remain CC-by-SA 2.0 anyway).
Simon
Am 24.03.2012 12:13, schrieb David Groom:
>
>
> There is I think one other option. Since a pretty much error free
> processed_p.shp file will be available just up to the licence switch
> over, then is it not legally possible to continue use this file in
> Mapnik combined with post switch over OSM data to create the maps.
> The missing coastline in OSM can then be re-mapped on a more leisurely
> basis.
>
> David
Thomas Davie
> One advantage of remapping is the possibility that there are now better data sets available for import than the PGS data which was originally used. I'm particularly thinking here of the US & Canada.
This is exactly what I've been doing – non-safe coastline in the UK using the OS OpenData StreetView MHWS to go by, it's a massive amount more detailed than the PGS stuff as an added bonus.
Bob
Russ Nelson
> I have been running a nightly coastline generation on my server, using the
> latest data from my jxapi server. Tonight I switched it over to filter out
> data that WTFE reports as dirty. This is somewhat more aggressive than the
> rebuild will be, but the results are worrisome.
Nahhhh, not particularly. If you've looked at the PGS, you'll see that
it is 99% crap. My problem with it is that 1) it's public domain, 2)
imported by an anonymous user, 3) I fixed it in my region, leaving
nothing remaining from the original import except the node and way
existence, BUT (you knew there was a but) without the odbl=clean tag
that I added, it would have been deleted. Or, at least, OSMI says it
would have been deleted.
Why are we deleting public domain data from OSM? If it says
source=PGS, it should not be deleted no matter who did the import.
If it was subsequently edited by a decliner, well, that's different.
It's particularly galling that anonymous users who haven't accepted or
declined are having their copyright respected. If you don't post your
land *with your name and address* in New York State, you cannot
successfully pursue a claim of trespass.
--
--my blog is at http://blog.russnelson.com
Crynwr supports open source software
521 Pleasant Valley Rd. | +1 315-600-8815
Potsdam, NY 13676-3213 | Sheepdog
David Groom
----- Original Message -----
From: "Russ Nelson" <nel...@crynwr.com>
To: "Paul Norman" <peno...@mac.com>
Cc: <ta...@openstreetmap.org>
Sent: Saturday, March 24, 2012 2:42 PM
Subject: Re: [OSM-talk] ODbL-clean Coastlines
> Paul Norman writes:
> > I have been running a nightly coastline generation on my server, using
> > the
> > latest data from my jxapi server. Tonight I switched it over to filter
> > out
> > data that WTFE reports as dirty. This is somewhat more aggressive than
> > the
> > rebuild will be, but the results are worrisome.
>
> Nahhhh, not particularly. If you've looked at the PGS, you'll see that
> it is 99% crap. My problem with it is that 1) it's public domain, 2)
> imported by an anonymous user, 3) I fixed it in my region, leaving
> nothing remaining from the original import except the node and way
> existence, BUT (you knew there was a but) without the odbl=clean tag
> that I added, it would have been deleted. Or, at least, OSMI says it
> would have been deleted.
>
> Why are we deleting public domain data from OSM? If it says
> source=PGS, it should not be deleted no matter who did the import.
> If it was subsequently edited by a decliner, well, that's different.
I guess there are at least two problems.
Firstly the PGS import script had a "simplification factor" variable, which
the person running the import could change. I know that prior to doing my
imports I played "around with" a number of different values for this
variable to strike what I thought was an acceptable trade off between
number of nodes created, and the complexity of the resulting ways.
Therefore what was uploaded to OSM was not simply PGS data, but was PGS data
as amended by my decision making process. I guess you would have to know if
the user who did the imports in question made any similar changes.
Secondly you could use the PGS import script in two ways. Either (i) run
the script against the PGS data and let the script directly upload to OSM ;
or (ii) use script to create an OSM file, which could then be edited in
JOSM, and then use JOSM to upload the data. If choosing method (ii) you
were then able to look at the data in JOSM and make corrections to it before
uploading to OSM. Although when doing my imports I started using (i) I
later switched to method (ii) because that way what I uploaded to OSM was
more error free. Had I now been a CT decliner I see no legal difference
between the resulting data in this instance and data which "If it was
subsequently edited by a decliner, well, that's different".
David
Russ Nelson
> more error free. Had I now been a CT decliner I see no legal difference
> between the resulting data in this instance and data which "If it was
> subsequently edited by a decliner, well, that's different".
How would we ever know? They're anonymous. If they want to come
forward and claim their copyright, then sure. But if not, then not.
You can't be anonymous and claim a copyright. The two concepts are
incompatible.
Stephan Knauss
> David Groom writes:
> > more error free. Had I now been a CT decliner I see no legal difference
> > between the resulting data in this instance and data which "If it was
> > subsequently edited by a decliner, well, that's different".
>
> How would we ever know? They're anonymous. If they want to come
> forward and claim their copyright, then sure. But if not, then not.
They are not anonymous. You just can't see their nickname.
If they log in and agree to the contributor terms their edits will be
contained in the ODbL version of the planet.
Here is a list of changesets of agreeing users still listed as anonymous
UID 0
http://planet.openstreetmap.org/users_agreed/anon_changesets_agreed.txt
Users not agreeing or not responding have to be treated the same way,
regardless of their nick known or not.
Stephan
Paul Norman
> From: Russ Nelson [mailto:nel...@crynwr.com]
> Sent: Saturday, March 24, 2012 7:42 AM
> To: Paul Norman
> Cc: ta...@openstreetmap.org
> Subject: Re: [OSM-talk] ODbL-clean Coastlines
>
> > I have been running a nightly coastline generation on my server,
> using the > latest data from my jxapi server. Tonight I switched it
> over to filter out > data that WTFE reports as dirty. This is somewhat
> more aggressive than the > rebuild will be, but the results are
> worrisome.
>
> Nahhhh, not particularly. If you've looked at the PGS, you'll see that
> it is 99% crap. My problem with it is that 1) it's public domain, 2)
> imported by an anonymous user, 3) I fixed it in my region, leaving
> nothing remaining from the original import except the node and way
> existence, BUT (you knew there was a but) without the odbl=clean tag
> that I added, it would have been deleted. Or, at least, OSMI says it
> would have been deleted.
>
> Why are we deleting public domain data from OSM? If it says source=PGS,
> it should not be deleted no matter who did the import.
> If it was subsequently edited by a decliner, well, that's different.
>
> It's particularly galling that anonymous users who haven't accepted or
> declined are having their copyright respected. If you don't post your
> land *with your name and address* in New York State, you cannot
> successfully pursue a claim of trespass.
Copyright applies when you author something, regardless of if you state your
name on it. As OSMF is the party wanting to do something not allowed by the
(implied) license of the contributer, they have to seek permission. If the
copyright holder decides not to reply, OSMF doesn't have permission to
license it under different terms.
Russ Nelson
> Users not agreeing or not responding have to be treated the same way,
> regardless of their nick known or not.
Non-responsive.
--
--my blog is at http://blog.russnelson.com
Crynwr supports open source software
521 Pleasant Valley Rd. | +1 315-600-8815
Potsdam, NY 13676-3213 | Sheepdog
_______________________________________________
Paul Norman
Yes - You'd need to know the process the importer used to see if they had
any IP in what was uploaded. This has been done for some CORINE imports with
changeset overrides
(http://wiki.openstreetmap.org/wiki/Quick_History_Service)
Paul Norman
> -----Original Message-----
> From: Russ Nelson [mailto:nel...@crynwr.com]
> Sent: Saturday, March 24, 2012 2:21 PM
> To: ta...@openstreetmap.org
> Subject: Re: [OSM-talk] ODbL-clean Coastlines
>
> Paul Norman writes:
> > > It's particularly galling that anonymous users who haven't accepted
> or > > declined are having their copyright respected. If you don't post
> your > > land *with your name and address* in New York State, you
> cannot > > successfully pursue a claim of trespass.
> >
> > Copyright applies when you author something, regardless of if you
> state your > name on it.
>
> I can make any kind of claims of ownership of land that I want. Unless I
> go to the county clerk's office and register my claim under my name,
> they won't enforce my claim against anyone else.
>
> But all theories of law aside, as a practical matter, if someone hasn't
> bothered to decline, they're not going to bother to sue.
There are people strongly opposed to the license change who have not
declined.
OSMF has the exact same rights for redistributing contributions from someone
who has not responded as someone has declined.
> You have to
> consider what copyright law is for: it's a temporary monopoly granted to
> protect revenue. If there's no potential for restricting distribution
> under CC-By-SA, and there's no potential for restricting distribution
> under the OdBL, what loss in revenue has anybody suffered? What nutcase
> is going to bother to sue anybody over distributing under one free
> license versus a different free license where neither one has the
> potential for proprietary distribution?
> Nobody's making money here, and it costs money to sue. A LOT of money.
> No lawyer is going to take your case on unless there are punitive or
> actual damages.
Copyright holders have plenty of options short of suing. I believe in some
jurisdictions damages are automatic if the copyright is registered.
> Worst comes to worse, you claim innocent infringement because you
> thought you were distributing under fair use, you delete the offending
> data, and life goes on. In other words, the worst a lawsuit is going to
> cost the OSMF is THE HARM IT'S VOLUNTARILY DOING TO ITSELF.
You think that OSMF could claim that distributing contributions that it
knows it only has a license to under cc-by-sa under a different license
could be fair use?
Also remember that these people are not anonymous to the OSMF. They've got
registration information on them just like they do on every other user. It's
just not displayed in what they make public through the API and planet
files.
David Groom
----- Original Message -----
From: "Russ Nelson" <nel...@crynwr.com>
To: <ta...@openstreetmap.org>
Sent: Saturday, March 24, 2012 9:21 PM
Subject: Re: [OSM-talk] ODbL-clean Coastlines
> Paul Norman writes:
> > > It's particularly galling that anonymous users who haven't accepted or
> > > declined are having their copyright respected. If you don't post your
> > > land *with your name and address* in New York State, you cannot
> > > successfully pursue a claim of trespass.
> >
> > Copyright applies when you author something, regardless of if you state
> > your
> > name on it.
>
> I can make any kind of claims of ownership of land that I want. Unless
> I go to the county clerk's office and register my claim under my name,
> they won't enforce my claim against anyone else.
>
> But all theories of law aside, as a practical matter, if someone
> hasn't bothered to decline, they're not going to bother to sue. You
The argument "hey, we understand we don't know if we have any right to use
this data, buts lets leave it in and hope no one complains" doesn't sound a
particularly moral one to me. It also seems to set a rather dangerous
precedent.
> have to consider what copyright law is for: it's a temporary monopoly
> granted to protect revenue. If there's no potential for restricting
> distribution under CC-By-SA, and there's no potential for restricting
> distribution under the OdBL, what loss in revenue has anybody
> suffered? What nutcase is going to bother to sue anybody over
> distributing under one free license versus a different free license
> where neither one has the potential for proprietary distribution?
> Nobody's making money here, and it costs money to sue. A LOT of
> money. No lawyer is going to take your case on unless there are
> punitive or actual damages.
>
> Worst comes to worse, you claim innocent infringement because you
> thought you were distributing under fair use, you delete the offending
> data, and life goes on. In other words, the worst a lawsuit is going
> to cost the OSMF is THE HARM IT'S VOLUNTARILY DOING TO ITSELF.
>
Your worse case sounds so harmless. Of course it's possible to sketch an
alternative worse case scenario:
At some time in the future after being asked to delete the offending data,
the data is deleted and ... ...
a) all those contributors who had made edits to that data after 1 April
2012 get very annoyed because they see the results of their work deleted.,
and when they query this they are told "hey, its a risk we thought we'd
take, and by the way we may have to delete a load more data in the future,
so be careful what bits of OSM you edit".
b) users of OSM data get very annoyed because having seen masses of data
disappear once, they suddenly see masses of data disappear again, and when
they query this they are told "hey, its a risk we thought we'd take, and by
the way we may have to delete a load more data in the future, so be careful
which bits of OSM data you use".
David
> --
> --my blog is at http://blog.russnelson.com
> Crynwr supports open source software
> 521 Pleasant Valley Rd. | +1 315-600-8815
> Potsdam, NY 13676-3213 | Sheepdog
>
Russ Nelson
> > But all theories of law aside, as a practical matter, if someone
> > hasn't bothered to decline, they're not going to bother to sue. You
>
> The argument "hey, we understand we don't know if we have any right to use
> this data, buts lets leave it in and hope no one complains" doesn't sound a
> particularly moral one to me. It also seems to set a rather dangerous
> precedent.
Copyright isn't property. It's a bargain. We agree not to copy a work
for a period of time, unless the copying is not for profit, is
educational, is limited in quantity and/or a number of other nuances
that don't fit into "not particularly moral."
> Your worse case sounds so harmless. Of course it's possible to sketch an
> alternative worse case scenario:
>
> At some time in the future after being asked to delete the offending data,
> the data is deleted and ... ...
>
> a) all those contributors who had made edits to that data after 1 April
> 2012 get very annoyed because they see the results of their work deleted.,
You are describing the current case, more or less. "We had to kill OSM
to save it." We already are finding that swathes of data are
contaminated and will be deleted, e.g. most of the west coast of the
U.S. and most of the Great Lakes, which also means going up a fair
ways into the continent because the larger rivers are considered
coastline. That was a gotcha for me. The Racquette, the Grass, and the
Oswegatchie were all going to be deleted because they were coastline,
public domain data imported by a user we cannot ask to sign onto the
CT, but who supposedly might sue us even though they can't even be
bothered to decline. I'm FINE with preserving the anonymity, and I'm
even FINE with allowing anonymous users to decline.
I'm NOT FINE with allowing unnamed people to hold a copyright without
bothering to decline. There's a REASON why copyright notices have to
contain the name of the copyright holder.
This situation has just gotten ridiculous. The OSMF is saying that it
cannot use:
o public domain data,
o which may have been edited, but which shows no signs of having been,
o which has been modified until it is not recognizable as the original.
o imported by a user whom nobody can contact,
o who has not bothered to decline acceptance of the OdBL,
o a license nearly identical to the existing license.
And people are trying to defend this, against all logic, because they
think copyright is inviolate, and that judges are just computers,
executing the law as if their JUDGEment cannot come into play.
Nathan Edgars II
> ----- Original Message ----- From: "Paul Norman" <peno...@mac.com>
>> - Adopting changesets. Many of the dirty ways and nodes appear to be
>> imported. If the imported just imported PD data than they have no IP
>> in the
>> ways and they can be retained.
>
> to ever have IP in it, no matter what else is subsequently done to it?
No. What he's saying is that the person who imported, by simply
importing PD data without adding anything to it, has no copyright
interest in it.