Reorganizing public bodies data model, harmonizing with EU standards and connecting with PublicBodies.org and Wikidata.org

61 views
Skip to first unread message

Mattias Axell

unread,
Nov 10, 2021, 2:26:49 PM11/10/21
to Alaveteli Dev
Hi,

I'm curious if there potentially could be a redesign of the public bodies data model used by Alaveteli?

Why? Many Alaveteli-based platforms run custom lists of public bodies which might not be up to date and miss out on relevant contact data and metadata. There are projects such as Wikidata.org (e.g. used by GovDirectory.org) and publicbodies.org which have relevant data and perhaps are more up-to-date than the Alaveteli sites.

Harmonizing with these could improve on the experience for managing, importing and exporting public bodies and keeping it up to date both for users, public bodies and users.
The public body database that Alaveteli sites run might also be a very valuable open data set for governments in different countries as it potentially could be the most up to date database with correct info. There are also a lack of persistent identifiers in many countries for public bodies and public sector owned companies. E.g. an organizational number is seldom persistent.  In Sweden it seems we run the most complete and open database with government info, maybe it is true to other countries as well. I know it also is a hassle for us to keep our Alaveteli public body database up to date as much as we can with a lot of manual work.

How could this be done? I suggest having a look at this
- publicbodies.org project.
- GovDirectory.org (enabled by wikidata.org)

What? Potential developments I can see happening:
- It would be awesome to be able to do a git pull from a corresponding country csv at publicbodies.org instead of managing it locally in the Alaveteli installation (even though there might be a backup at another spreadsheet).
- Connecting Alaveteli sites to Wikidata.org and thus in the future enable made-with-Alaveteli-FOI-requests as Wikidata entities?
- Handling the public body database in sync with csv from publicbodies.org via git thus always having a backup at the Alaveteli site and Git.
- Alaveteli sites offering a service with up-to-date database via publicbodies.org to national governments.

Otherwise I'm curious to hear of approaches to this from developers and other Alaveteli sites? Are you already using a git repo to keep your Alaveteli database up to date?

Best,
Mattias

Laurent Savaëte

unread,
Nov 14, 2021, 7:13:25 PM11/14/21
to alavet...@googlegroups.com

Hi Mattias,

We at madada.fr are actually about to look into this question. We've got a bit of funding to work on a few topics, including this one. We haven't really started on this point yet, but we'll definitely share what we find and come up with. If you're keen, it could be useful to arrange a call at some point to discuss a bit further.

France is probably world champion of administrative complexity, we currently have over 50,000 public bodies on the site, and we know there are more. We are not aware of any updated and comprehensive database, and we think ours is probably the most comprehensive one, at least of the ones including "PRADAs" (the person in charge of FOI).

Our current system is very manual, and we've tried setting up some watchers to get notified of changes in some of the online datasets we know of using git, but that's mostly a failure so far :(

Cheers,

Laurent

--
You received this message because you are subscribed to the Google Groups "Alaveteli Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alaveteli-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/alaveteli-dev/c04bbc13-7bb0-4388-97a0-d66f8f1e7addn%40googlegroups.com.

Gareth Rees

unread,
Nov 16, 2021, 4:43:56 AM11/16/21
to Alaveteli Dev
Just to note it is an area of interest to us [1].

We do currently have a few minor links with external databases [2] where we show extra info based on tags (e.g. a tag of `ch:1234` would link to the official company registration page for the company ID 1234).

I think for any solution we'd want to retain the existing local database of authorities, but tools to pull from external sources to make updates easier would be great!

Best,

Mattias Axell

unread,
Nov 17, 2021, 1:56:49 PM11/17/21
to Alaveteli Dev
Thanks for your replies, Laurent and Gareth. It sounds like we share challenges and similar visions. I agree about adding to Wikidata. GovDirectory has done some relevant work here indexing countries: https://www.wikidata.org/wiki/Wikidata:WikiProject_Govdirectory

Maybe something to build upon and collaborate with. There should be a long term sustainable financing model for organizing this data to help citizens and governments. I'm up for a call. Can you make it to join the GovDirectory CollabHour (https://www.wikidata.org/wiki/Wikidata:WikiProject_Govdirectory/Events) this Friday 15:00 (Central European Time) at http://meet.jit.si/GovdirectoryCollabHour ?

Best,
Mattias

WhatDoTheyKnow Support

unread,
Nov 19, 2021, 3:45:05 AM11/19/21
to Alaveteli Dev
At WhatDoTheyKnow we have been keen to improve the way we manage our public bodies database, and link it to official registers / databases / systems.

In the UK we don't think there is a more comprehensive database of public bodies than ours, so there is no master database to link to.

In the UK we have some databases of public bodies which we try to link public bodies in our database to:

The Online System for Central Accounting and Reporting (OSCAR) Tool (Previously COINS):

The national schools database:

The National Health Service Organisational Data Service:

The data protection register:

The company register:

The charity register:

We use key:value tags in the tags field to store the ID/reference numbers for these other databases.

We have used these links already to help keep our database up to date. We have done this in a low-tech manner, downloading our all authorities csv file, and using scripts and spreadsheet formulae, to compare the official databases with ours, and eg. identify bodies which have ceased to exist.

In the UK we have to be mindful of database rights, copyright, and licencing even when dealing with public, government, data.

We have sought to recruit volunteer database administrators to specifically help us with the challenge of automating and improving our integrations with these databases, but while we've had quite a few expressions of interest we've not managed to recruit any volunteers to that specialist role yet.

The links to the tag based searches given above show some sense of the scale of our inter-database links, but the search counts are only approximate.

Regards,

Richard

--
WhatDoTheyKnow Support Team

WhatDoTheyKnow is a project of mySociety run by a small team of staff and dedicated volunteers.

If you’d like to help us maintain the site you can make a donation. Your contributions, however small, really help: https://www.mysociety.org/donate

Find out more about how we handle your data at: https://www.whatdotheyknow.com/help/privacy

mySociety is a registered charity in England and Wales (1076346) and a limited company (03277032).




Magnus Sälgö

unread,
Nov 21, 2021, 8:32:11 PM11/21/21
to Alaveteli Dev
Thanks for sharing

WhatDoTheyKnow is  wikidata Property:P8167 see map

en.png

Question Do you synch with WIkidata to check diffa and do uou update Wikidata with errors you have found?

FYI there is a new Wikidata project  Wikidata:Mismatch_Finder fpor better synch with external data..

Regards
Magnus Sälgö
Wikidata salgo60
Sweden



Reply all
Reply to author
Forward
0 new messages