Vocbench integration with AWS Neptune

[arun s]

Hi,

    I would like to know if there is any support for integrating vocbench with AWS Neptune DB ?

Regards

Arun S

[Manuel Fiorelli]

Dear Arun

tl;dr AWS Neptune is not currently supported and it is unlikely that AWS Neptune will be ever supported because of some limitations of this DB.

Let me expand the summary above.

First of all, the idea of choosing AWS Neptune somehow follows the general suggestion to prefer managed services to owned ones, usually to reduce total cost of ownership. I think that this argument is stronger, when also deploying VB to AWS: otherwise, I don't know how the communication delay between your premises and the cloud affects performance. In fact, in all production deployments I know about, VB and the DB (usually Ontotext GraphDB) are co-located on the same machine: however, I am quite confident that in the context of a deployment within AWS, the usage of AWS Neptune could be viable.

Unfortunately, VocBench 3 (or more precisely, the underlying Semantic Turkey server) can't connect to AWS Neptune, which seems not to implement the RDF4J Rest API used by VocBench to communicate with remote triple stores via an RDF4J's HttpRepository. This API combines the HTTP protocol for submitting SPARQL queries and interacting with a graph store, extending them with transactions and further guarantees concerning the identity of blank nodes. Looking at the documentation of AWS Neptune, I only found evidence that it supports the HTTP protocol for submitting SPARQL queries (and updates) and perhaps to interact with graphs. Indeed, the documentation contains an example using SPARQLRepository: a kind of repository which has several limitations because of the use of the sole SPARQL HTTP Protocol.

One might argue that it could be possible to implement a sort of wrapper re-implementing the RDF4J Rest API over the API of AWS Neptune, or to extend VocBench with the ability to use different APIs. However, according to my findings reported in previous paragraph, AWS Neptune seems not to cover a lot of the capabilities of the RDF4J API. Actually, AWS Neptune also supports transactional semantics, but it seems to me that there is no explicit API to manage transactions, which are handled according to a predefined strategy and isolation level.

Another limitation, which was noticed by a colleague of mine, is that it might be impossible to create multiple repositories on the same AWS Neputune instance (probably you have to create different instances of AWS Neptune).

Furthermore, validation and history could never work on AWS Neptune, because I don't think it is possible to deploy the change tracker sail into that DB.

If you are acquainted with AWS Neptune, please verify the claims above and possibly provide further information.

Best Regards

Manuel

--
You received this message because you are subscribed to the Google Groups "vocbench-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vocbench-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vocbench-user/CAEGjdLx9kftTGAfEA5VJ0QG7vqcPAzdXayeMvNg7_%3D4PR%3Dvv7A%40mail.gmail.com.

--

Manuel Fiorelli

[Lou Vasquez - NOAA Affiliate]

I noticed that I was able to use rdf4j console to connect to neptune and do a few basic operations. I wonder if things have changed in the years that have passed and this is a possibility now?

I will note that my attempt to connect has failed with a bad request exception and a bad route of /sparql/repositories. This implies that the repositories API isn't fully implemented, and even if rdf4j console doesn't have a problem, the libraries described above may still be a problem for VocBench.

Can you confirm? Any thoughts or hope here?

Thanks much,

Lou

[Manuel Fiorelli]

Hi Lou,

I've given a look at the documentation on accessing Neptune through the RDF4J console: https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-sparql-rdf4j-console.html

In my understanding, what they are doing is to create a SPARQLRepository and then issue queries against it.

A SPARQLRepository (https://rdf4j.org/javadoc/latest/org/eclipse/rdf4j/repository/sparql/SPARQLRepository.html) is a (partial) implementation of the interface Repository (https://rdf4j.org/javadoc/latest/org/eclipse/rdf4j/repository/Repository.html) based on the SPARQL 1.1 Protocol (https://www.w3.org/TR/sparql11-protocol/).

The SPARQL protocol mainly deals with issuing queries and updates against a SPARQL endpoint: most importantly, this protocol ignores transaction management. These limitations motivated the development of the RDF4J REST API (https://rdf4j.org/documentation/reference/rest-api/), which extends the SPARQL protocol with proper transaction management and other stuff (e.g. blank node identity).

VocBench uses the HTTPRepository (https://rdf4j.org/javadoc/latest/org/eclipse/rdf4j/repository/http/HTTPRepository.html) to access triple stores compliant to this API.

So the question is: can VB just use a SPARQL Repository? The answer is: out-of-box, no. Unfortunately, I can't quantify the effort required to add support for it, even for a proof-of-concept. Indeed, I expect that the eventual result would be quite unsatisfying:i) inability to define namespace prefixes, ii) corrupted data if an exception is thrown in the middle of sequence of updates within the same service, iii) without a proper RDF4J backend it would not be possible to deploy the custom sails that make it possible to handle history, validation, trivial inference SHACL validation.

Regards,

Manuel

To view this discussion on the web visit https://groups.google.com/d/msgid/vocbench-user/78f23771-82c7-4d82-944d-1463a8eae15cn%40googlegroups.com.

--

Manuel Fiorelli

[Lou Vasquez - NOAA Affiliate]

Thank you! This is very helpful into. It's disappointing that neptune didn't implement rdf4j well, but I imagine they're still getting their feet under themselves.

Y'all have done your due diligence in implementing to a standard (rdf4j). I'll keep an eye out for AWS doing better and try again if I see that.

[Armando Stellato]

Dear Lou,

 

thanks for your kind words. To be totally fair though, I must say that, while RDF4J is a very much used middleware, it is not a standard, the standard (with due limitations) is the SPARQL protocol.

Obviously, if one is a vendor of a triple store, might consider what’s around there and look for compliancy with it.

But this can go only up to a limited compliancy: implementing REST API is doable with a proper wrapper if they already implement all of those features somewhat else. Compliancy with the sail architecture (which indeed only necessary for certain features such as history, validation, trivial reasoning, undo), server side, is very much more like telling how to sort your own house from inside it :-)

 

So, it is to me perfectly understandable that a certain vendor may or may not adhere to certain specs. In VB2 we had a lightweight layer for compliancy with different middleware (e.g. Jena) and, for how light it was, it was a pain to keep it aligned with evolutions of the specs and it was in any case always missing some peculiarities of each specific middleware. Eventually, with VB3, we took the decision to take a stance, merry one technology, and that was RDF4J.

 

Seen the other way round, that is also the reason for which we cannot simply rely on standards (I’m talking about this specific case, not standards in general), as they are good for sending the same SPARQL query over different triple stores, they are not enough for backing a complex system such as VocBench. There’s more to that, under the general topic of limited interoperability (more, interchangeability) of triple store, we’ll write a section on the manual one of these days :-)

Just to give you a brief example: even though they share not only the same API, but the same architecture, it is not uncommon for us to deal with restating queries in different ways so that they behave well on both RDF4J stores and GDB stores.

As you know, you can always have a line passing through 2 points in space. With three you need luck :-)

 

Kind Regards,

 

Armando

 

 

To view this discussion on the web visit https://groups.google.com/d/msgid/vocbench-user/d7c2a5b9-cf62-46b9-80da-38bbb99a93c3n%40googlegroups.com.