GeoJSON and geospatial data extensions

14 views
Skip to first unread message

Labian Gashi

unread,
Jun 9, 2021, 7:28:25 AMJun 9
to OpenRefine Development
Hi everyone,

I am a student in OST, Rapperswil in Switzerland, completing a master's degree in Computer Science with a combination of Data Science and Computer Science modules.

I am currently doing my master thesis and I am creating a couple of extensions for OpenRefine.

One of the extensions will allow OpenRefine to export data to GeoJSON (which I have already implemented) and the other one will be related to geospatial data. The second extension has not been fully decided yet but the main idea is to allow OpenRefine to access the Overpass API webservice as a URL Service. Overpass API is a read-only API that serves up custom selected parts of the OSM (OpenStreetMap) map data. This would allow some sort of integration between OpenRefine and OpenStreetMap (for all of you OSM enthusiasts out there!). Again... the requirements for the OSM extension have not been clearly specified yet.

You can find the code repository here: https://gitlab.com/labiangashi/geo-openrefine

I just want to hear your general thoughts on these extensions... AFAIK, Mr. Delpeuch is already aware of these extensions since me and my mentor had a discussion with him about this a couple of months ago.

Thanks for the attention,
Labian Gashi.

Thad Guidry

unread,
Jun 11, 2021, 6:56:40 AMJun 11
to openref...@googlegroups.com
Hi Labian,

I have a few thoughts.
Recently, OpenRefine was given a grant for improving editing, uploading of Structured Data on Commons (Wikimedia).
Many GLAMs (Gallaries, Libraries, Archives, Museums) have data that they intend to upload as well as improve the metadata on existing media items stored in Commons.
In doing so, sometimes part of their workflows will be uploading, tagging, describing, etc. GeoJSON for items in Commons https://www.wikidata.org/wiki/Wikidata:Map_data

I can envision that some sub-workflows might be reconciling and using not only OSM but much more important to GLAMs would be OpenHistoricalMap data https://wiki.openstreetmap.org/wiki/Open_Historical_Map#Using_the_data
Some quick slides OHM Overview Lightning Talk Geo4Lib 2021, Jeff Meyer, Geo4LibCamp 2021

Something to think about,



--
You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/2a25f358-ccd6-4282-9296-cdd19529b619n%40googlegroups.com.

Włodzmierz Bartczak

unread,
Jun 11, 2021, 8:39:22 AMJun 11
to openref...@googlegroups.com
Hi Labian,

I am one of the Openrefine translators (English - Polish), at the same time quite active as OSM editor and on the board of the local OSMF chapter.

I use openrefine for my daily GIS work and OSM related projects. If you need help with the osm plugin, feel free to contact me.

--
You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/2a25f358-ccd6-4282-9296-cdd19529b619n%40googlegroups.com.


--
Pozdrawiam.


Włodzimierz Bartczak

mail: wzbar...@gmail.com


Antonin Delpeuch (lists)

unread,
Jun 11, 2021, 10:42:13 AMJun 11
to openref...@googlegroups.com
Hi Labian,

Thanks for sharing, this is really nice!
If I remember correctly, at some point there was the idea of providing a
reconciliation service for OpenStreetMap's data. Is this something that
is still on the cards? I assume the code for it would not live in the
extension you have developed, but rather somewhere external.

Best,
Antonin
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com
> <mailto:openrefine-de...@googlegroups.com>.
> <https://groups.google.com/d/msgid/openrefine-dev/2a25f358-ccd6-4282-9296-cdd19529b619n%40googlegroups.com?utm_medium=email&utm_source=footer>.

Antonin Delpeuch (lists)

unread,
Jun 11, 2021, 10:43:51 AMJun 11
to openref...@googlegroups.com
Hi Włodzmierz,

Woaw, it's great to have you on this list! If you have ideas of things
that would be beneficial for a better OpenRefine/OSM integration, I
would be very interested to read them.

Best,
Antonin

On 11/06/2021 14:39, Włodzmierz Bartczak wrote:
> Hi Labian,
>
> I am one of the Openrefine translators (English - Polish), at the same
> time quite active as OSM editor and on the board of the local OSMF chapter.
>
> I use openrefine for my daily GIS work and OSM related projects. If you
> need help with the osm plugin, feel free to contact me.
>
> śr., 9 cze 2021 o 13:28 Labian Gashi <labia...@gmail.com
> <mailto:labia...@gmail.com>> napisał(a):
> <mailto:openrefine-de...@googlegroups.com>.
> <https://groups.google.com/d/msgid/openrefine-dev/2a25f358-ccd6-4282-9296-cdd19529b619n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
>
>
> --
> Pozdrawiam.
>
>
> Włodzimierz Bartczak
>
> mail: wzbar...@gmail.com <mailto:wzbar...@gmail.com>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com
> <mailto:openrefine-de...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openrefine-dev/CABQAVfJHpEeJ%2Br3N76niOU42VsS7frmLfQhaki1A1b%3DHDzxJhA%40mail.gmail.com
> <https://groups.google.com/d/msgid/openrefine-dev/CABQAVfJHpEeJ%2Br3N76niOU42VsS7frmLfQhaki1A1b%3DHDzxJhA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Włodzmierz Bartczak

unread,
Jun 13, 2021, 2:17:28 PMJun 13
to openref...@googlegroups.com
OpenRefine/OSM integration hmmm...

Yes, but only one way. Data acquisition. Semi-automated or any "unsupervised" data entry is against community recommendations. It usually causes more problems than it does good. This is a topic for a separate discussion, it can be debated, but personally I wish such tools were not widely available.

Integration for data import is what would be really useful especially for data validation. Here I would have some ideas.

Integration with the http://overpass-turbo.eu/ service, i.e. creating queries directly in OR to retrieve data. This is where I have benefited most from OR standardising data for my needs. Currently only for points but as we have geojson support this can be greatly extended. I see room for innovation here.

One could also consider a step further, not only integration with OSM, but also with GIS software (of course QGIS), this would significantly broaden the scope of application.

If you have any OSM questions, please feel free to ask.

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/23ecb4fa-603f-88b2-1a18-1c7d0a16647b%40antonin.delpeuch.eu.


--
Pozdrawiam.


Włodzimierz Bartczak

mail: wzbar...@gmail.com


Thad Guidry

unread,
Jun 20, 2021, 5:47:33 PMJun 20
to openref...@googlegroups.com
Another aspect of OSM integration might involve use of OpenLayers for example, pulling tiles or rendering vector data itself within OpenRefine extension panels or overlays for visualization and mapping controls.

Włodzmierz Bartczak

unread,
Jun 21, 2021, 4:42:07 AMJun 21
to openref...@googlegroups.com
Super idea. (Leftlet.js can also be used.) Further you can use geocoding (https://nominatim.org/release-docs/develop/api/Overview/) to make the presentation of data more functional by using map bases.
I would also add the possibility of presenting data in the form of map charts, this would greatly expand the use of the software

Labian Gashi

unread,
Jun 23, 2021, 4:37:34 AMJun 23
to OpenRefine Development
Thanks for all your replies and your help. I will post on this thread once I have some new updates on the OSM integration extension.

As for the GeoJSON export extension, it is already finished: https://gitlab.com/labiangashi/geo-openrefine/-/tree/master/extensions/geojson

Thanks everyone.
Reply all
Reply to author
Forward
0 new messages