Incomplete result from query. Intermittent. Solved when optimising database.

[Håvard Ottestad]

Hi,

I have been tracking down a very strange issue today that results in a query stopping to work, and having to optimize the database for the query to work again. Tested on 4.2.1 and 4.2.2.

Essentially I have two components. One that uploads data, and another that queries stardog to create json files to index in elasticsearch. I've created a whole system that can deal with data being loaded in an arbitrary order. An incoming file is initially split into several pieces, then uploaded one at a time to stardog. Every time a piece is uploaded, a message gets sent to the service that indexes to elasticsearch to query stardog again. Eventually the last piece is uploaded to stardog, a message is sent to the indexing service. The query in the indexing service now returns the correct answer and updates elasticsearch.

I've created a test system that does the following.

• Two threads. UPLOAD_THREAD and QUERY_THREAD. 
  
• UPLOAD_THREAD will
  
• Clear stardog
• Upload the ontology
• notify QUERY_THREAD
• start uploading data
• wait for notification from QUERY_THREAD
• QUERY_THREAD will
• Wait for notification from UPLOAD_THREAD
• Query stardog
• If the query returns the correct result, it will notify UPLOAD_THREAD
• if not it will keep querying

The data is a linked list:

A -parent-> B -parent-> C ..... -parent->E

and E has a property arkiv:arkivskaper with a resource.

The ontology has two main axioms.

1. A parentTransitive property that is super of parent

2. An arkiv:arkivskaper property that is the chain of "parentTransitive o arkivskaper".

The query asks for "A arkiv:arkivskaper ?anything" which should result in the traversal down to E and then over the declared arkiv:arkivskaper property to the resource.

Eventually though the query stops returning the correct answer. Stardog will also not return the correct answer. The data is in stardog, I've checked. And here is the strange part, optimising or taking they db offline/online will make the query start to work again.

Here is a video for demonstration: https://youtu.be/w0nArJi9HKo

We are using stardog as our master database and we are counting on it to be a strict serialisation layer for our system. We are assuming that when a transaction commits, then a query started after that transaction will return based on that transactions modifications.

I will upload the code as soon as I can, but possibly not until tomorrow.

Cheers,

Håvard M. Ottestad

[Håvard Ottestad]

Hi,

Here is the proof of concept: https://github.com/hmottestad/staleRead

Couldn't attach it as a zip :(

I have narrowed this down to a bug with reasoning. When I rewrite the query to not require reasoning, and query without reasoning enabled everything is fine. However, when I use reasoning and let stardog rewrite my query, it will fail after a minute of two.

Cheers,

Håvard

[Stephen Nowell]

Hi Håvard,

Thanks for the very detailed github/youtube bug report. It certainly
made my job easier.
We have definitely recreated the bug and are working on determining the
solution. I can update here when we've found something.

Cheers,
Stephen

PS - In your uploadAndReplace function you aren't releasing the
Connection you obtain. A few times I ran out of Connections before
encountering your issue :)

> --
> -- --
> You received this message because you are subscribed to the C&P
> "Stardog" group.
> To post to this group, send email to sta...@clarkparsia.com
> To unsubscribe from this group, send email to
> stardog+u...@clarkparsia.com
> For more options, visit this group at
> http://groups.google.com/a/clarkparsia.com/group/stardog?hl=en

[Håvard Ottestad]

Hi Stephen,

Thanks for looking into this.

Sorry about forgetting to release that connection. I copied over code from our project, and while trying to decouple it I swapped out some things, including the uploadAndReplace method, which as you saw doesn't actually upload and replace anymore :P

Cheers,

Håvard