GeoJSON and geospatial data extensions
37 views
Skip to first unread message
Labian Gashi
Jun 9, 2021, 7:28:25 AM6/9/21
to OpenRefine Development
Hi everyone,
I am a student in OST, Rapperswil in Switzerland, completing a master's degree in Computer Science with a combination of Data Science and Computer Science modules.
I am currently doing my master thesis and I am creating a couple of extensions for OpenRefine.
One of the extensions will allow OpenRefine to export data to GeoJSON (which I have already implemented) and the other one will be related to geospatial data. The second extension has not been fully decided yet but the main idea is to allow OpenRefine to access the Overpass API webservice as a URL Service. Overpass API is a read-only API that serves up custom selected parts of the OSM (OpenStreetMap) map data. This would allow some sort of integration between OpenRefine and OpenStreetMap (for all of you OSM enthusiasts out there!). Again... the requirements for the OSM extension have not been clearly specified yet.
You can find the code repository here: https://gitlab.com/labiangashi/geo-openrefine
and the GeoJSON extension here: https://gitlab.com/labiangashi/geo-openrefine/-/tree/master/extensions/geojson
I just want to hear your general thoughts on these extensions... AFAIK, Mr. Delpeuch is already aware of these extensions since me and my mentor had a discussion with him about this a couple of months ago.
Thanks for the attention,
Labian Gashi.
Thad Guidry
Jun 11, 2021, 6:56:40 AM6/11/21
to openref...@googlegroups.com
Hi Labian,
I have a few thoughts.
Recently, OpenRefine was given a grant for improving editing, uploading of Structured Data on Commons (Wikimedia).
Many GLAMs (Gallaries, Libraries, Archives, Museums) have data that they intend to upload as well as improve the metadata on existing media items stored in Commons.
In doing so, sometimes part of their workflows will be uploading, tagging, describing, etc. GeoJSON for items in Commons https://www.wikidata.org/wiki/Wikidata:Map_data
I can envision that some sub-workflows might be reconciling and using not only OSM but much more important to GLAMs would be OpenHistoricalMap data https://wiki.openstreetmap.org/wiki/Open_Historical_Map#Using_the_data
Something to think about,
--
You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/2a25f358-ccd6-4282-9296-cdd19529b619n%40googlegroups.com.
Włodzmierz Bartczak
Jun 11, 2021, 8:39:22 AM6/11/21
to openref...@googlegroups.com
Hi Labian,
I use openrefine for my daily GIS work and OSM related projects. If you need help with the osm plugin, feel free to contact me.
--
You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/2a25f358-ccd6-4282-9296-cdd19529b619n%40googlegroups.com.
Antonin Delpeuch (lists)
Jun 11, 2021, 10:42:13 AM6/11/21
to openref...@googlegroups.com
Hi Labian,
Thanks for sharing, this is really nice!
If I remember correctly, at some point there was the idea of providing a
reconciliation service for OpenStreetMap's data. Is this something that
is still on the cards? I assume the code for it would not live in the
extension you have developed, but rather somewhere external.
Best,
Antonin
Thanks for sharing, this is really nice!
If I remember correctly, at some point there was the idea of providing a
reconciliation service for OpenStreetMap's data. Is this something that
is still on the cards? I assume the code for it would not live in the
extension you have developed, but rather somewhere external.
Best,
Antonin
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com
> <mailto:openrefine-de...@googlegroups.com>.
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openrefine-dev/2a25f358-ccd6-4282-9296-cdd19529b619n%40googlegroups.com
> <https://groups.google.com/d/msgid/openrefine-dev/2a25f358-ccd6-4282-9296-cdd19529b619n%40googlegroups.com?utm_medium=email&utm_source=footer>.
> https://groups.google.com/d/msgid/openrefine-dev/2a25f358-ccd6-4282-9296-cdd19529b619n%40googlegroups.com
Antonin Delpeuch (lists)
Jun 11, 2021, 10:43:51 AM6/11/21
to openref...@googlegroups.com
Hi Włodzmierz,
Woaw, it's great to have you on this list! If you have ideas of things
that would be beneficial for a better OpenRefine/OSM integration, I
would be very interested to read them.
Best,
Antonin
On 11/06/2021 14:39, Włodzmierz Bartczak wrote:
> Hi Labian,
>
> I am one of the Openrefine translators (English - Polish), at the same
> time quite active as OSM editor and on the board of the local OSMF chapter.
>
> I use openrefine for my daily GIS work and OSM related projects. If you
> need help with the osm plugin, feel free to contact me.
>
> śr., 9 cze 2021 o 13:28 Labian Gashi <labia...@gmail.com
> <mailto:labia...@gmail.com>> napisał(a):
> <mailto:openrefine-de...@googlegroups.com>.
>
>
>
> --
> Pozdrawiam.
>
>
> Włodzimierz Bartczak
>
> mail: wzbar...@gmail.com <mailto:wzbar...@gmail.com>
> <https://groups.google.com/d/msgid/openrefine-dev/CABQAVfJHpEeJ%2Br3N76niOU42VsS7frmLfQhaki1A1b%3DHDzxJhA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Woaw, it's great to have you on this list! If you have ideas of things
that would be beneficial for a better OpenRefine/OSM integration, I
would be very interested to read them.
Best,
Antonin
On 11/06/2021 14:39, Włodzmierz Bartczak wrote:
> Hi Labian,
>
> I am one of the Openrefine translators (English - Polish), at the same
> time quite active as OSM editor and on the board of the local OSMF chapter.
>
> I use openrefine for my daily GIS work and OSM related projects. If you
> need help with the osm plugin, feel free to contact me.
>
> śr., 9 cze 2021 o 13:28 Labian Gashi <labia...@gmail.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openrefine-dev/2a25f358-ccd6-4282-9296-cdd19529b619n%40googlegroups.com
> <https://groups.google.com/d/msgid/openrefine-dev/2a25f358-ccd6-4282-9296-cdd19529b619n%40googlegroups.com?utm_medium=email&utm_source=footer>.
> https://groups.google.com/d/msgid/openrefine-dev/2a25f358-ccd6-4282-9296-cdd19529b619n%40googlegroups.com
>
>
>
> --
> Pozdrawiam.
>
>
> Włodzimierz Bartczak
>
> mail: wzbar...@gmail.com <mailto:wzbar...@gmail.com>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com
> <mailto:openrefine-de...@googlegroups.com>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openrefine-dev/CABQAVfJHpEeJ%2Br3N76niOU42VsS7frmLfQhaki1A1b%3DHDzxJhA%40mail.gmail.com
> <https://groups.google.com/d/msgid/openrefine-dev/CABQAVfJHpEeJ%2Br3N76niOU42VsS7frmLfQhaki1A1b%3DHDzxJhA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Włodzmierz Bartczak
Jun 13, 2021, 2:17:28 PM6/13/21
to openref...@googlegroups.com
OpenRefine/OSM integration hmmm...
Yes, but only one way. Data acquisition. Semi-automated or any "unsupervised" data entry is against community recommendations. It usually causes more problems than it does good. This is a topic for a separate discussion, it can be debated, but personally I wish such tools were not widely available.
Integration for data import is what would be really useful especially for data validation. Here I would have some ideas.
Integration with the http://overpass-turbo.eu/ service, i.e. creating queries directly in OR to retrieve data. This is where I have benefited most from OR standardising data for my needs. Currently only for points but as we have geojson support this can be greatly extended. I see room for innovation here.
One could also consider a step further, not only integration with OSM, but also with GIS software (of course QGIS), this would significantly broaden the scope of application.
If you have any OSM questions, please feel free to ask.
Yes, but only one way. Data acquisition. Semi-automated or any "unsupervised" data entry is against community recommendations. It usually causes more problems than it does good. This is a topic for a separate discussion, it can be debated, but personally I wish such tools were not widely available.
Integration for data import is what would be really useful especially for data validation. Here I would have some ideas.
Integration with the http://overpass-turbo.eu/ service, i.e. creating queries directly in OR to retrieve data. This is where I have benefited most from OR standardising data for my needs. Currently only for points but as we have geojson support this can be greatly extended. I see room for innovation here.
One could also consider a step further, not only integration with OSM, but also with GIS software (of course QGIS), this would significantly broaden the scope of application.
If you have any OSM questions, please feel free to ask.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/23ecb4fa-603f-88b2-1a18-1c7d0a16647b%40antonin.delpeuch.eu.
Thad Guidry
Jun 20, 2021, 5:47:33 PM6/20/21
to openref...@googlegroups.com
Another aspect of OSM integration might involve use of OpenLayers for example, pulling tiles or rendering vector data itself within OpenRefine extension panels or overlays for visualization and mapping controls.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/CABQAVfJnnrw-XO5SW%2Bj1DPM85C1fXv7GOtz2NVhi9DAtZ9dsNQ%40mail.gmail.com.
Włodzmierz Bartczak
Jun 21, 2021, 4:42:07 AM6/21/21
to openref...@googlegroups.com
Super idea. (Leftlet.js can also be used.) Further you can use geocoding (https://nominatim.org/release-docs/develop/api/Overview/) to make the presentation of data more functional by using map bases.
I would also add the possibility of presenting data in the form of map charts, this would greatly expand the use of the software
I would also add the possibility of presenting data in the form of map charts, this would greatly expand the use of the software
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/CAChbWaNt5tx7JW%2Bm7bmChF%3DGrzi_qiAr_3z27H63-mZXFLAfHQ%40mail.gmail.com.
Labian Gashi
Jun 23, 2021, 4:37:34 AM6/23/21
to OpenRefine Development
Thanks for all your replies and your help. I will post on this thread once I have some new updates on the OSM integration extension.
As for the GeoJSON export extension, it is already finished: https://gitlab.com/labiangashi/geo-openrefine/-/tree/master/extensions/geojson
Thanks everyone.
Reply all
Reply to author
Forward
0 new messages