Help Sought: Wikibase + OpenRefine

[Lucy Hinnie]

Hi all,

First post so apologies if this doesn't land in the right place. 

I'm a resident Wikimedian and I'm working with a large set of data pertaining to index cards. I've set up a Wikibase instance and successfully created a manifest to connect it to OpenRefine. However, I am hitting a brick wall when it comes to using OpenRefine to add items to the Wikibase instance.

Does anyone have relevant experience of this, and if so, would you be willing to chat a little about it?

Best wishes,

Lucy

[Owen Stephens]

Hi Lucy,

This isn't an area I have any specific expertise in, but I wonder if you could give any more detail about exactly where you get to and what happens.

Are you able to carry out any reconciliation between OpenRefine and the wikibase?

Is there any information in the OpenRefine console when you try to add items to the wikibase?

Do you have any authentication setup on your wikibase?

What does your schema look like when you try to add new items?

These are slightly scattergun questions but aimed at the places where I could see things going wrong in the attempt to create new items in a wikibase - any other detail or information you can offer may help me or others diagnose the problem.

Best wishes

Owen

[Lucy Hinnie]

Thanks Owen! Answers in red below, hope this is helpful and not a bark up the wrong tree. I work Mon-Weds so happy to arrange a call if easier? A lot of this may be me being utterly dense as I am quite inexperienced with Wikibase so far!

Questions/answers:

Are you able to carry out any reconciliation between OpenRefine and the wikibase?

I can make them talk to each other a little but as soon as I try to reconcile, I am met with this: it doesn't seem to see Wikibase as a service, only Wikidata proper.

Is there any information in the OpenRefine console when you try to add items to the wikibase?

Not that I can see, but I could be very wrong!

Do you have any authentication setup on your wikibase?

None to my knowledge, but may be wrong.

What does your schema look like when you try to add new items?

Schema is an area I'm stuck on, and would love some advice and/or pointing to the necessary documentation on how to compose/set this up!

Manifest:

This is the manifest, for reference:

{

  "version": "1.0",

  "mediawiki": {

    "name": "Lord Chamberlain's Plays Index",

    "root": "https://lcpindex.wikibase.cloud/wiki/",

    "main_page": "https://lcpindex.wikibase.cloud/wiki/Main_Page",

    "api": "https://lcpindex.wikibase.cloud/w/api.php"

  },

  "wikibase": {

    "site_iri": "http://lcpindex.wikibase.cloud/entity/",

    "maxlag": 5,

    "properties": {

      "instance_of": "P1",

      "subclass_of": "P11"

    }

  },

  "reconciliation": {

    "endpoint": "https://wdreconcile.toolforge.org/${lang}/api"

  }

}

[Owen Stephens]

Thanks Lucy

So the first thing that I can see is that in your manifest you have:

"reconciliation": {

    "endpoint": "https://wdreconcile.toolforge.org/${lang}/api"

  }

This definitely looks wrong to me - this is pointing at the Wikidata reconciliation service and you will need a reconciliation service for your own Wikibase, which can be done by running your own version of the OpenRefine-Wikibase reconciliation interface. The documents for this are at https://openrefine-wikibase.readthedocs.io/en/latest/

I'm afraid I don't know anything about how to actually go ahead and run this (although the documentation looks thorough) but my understanding is that you would need that in place to get reconciliation working - and my understanding is that having a reconciliation service in place for your wikibase is a pre-requisite for being able to create items in the wikibase from an OpenRefine project

Hope this helps

Owen