VIVO Triple Store Roadmap Proposal

[Andrew Woods]

Hello All,

VIVO strives to evolve in-pace with the dependencies and technologies on which it is built. Recognizing the technical trajectory of the Jena triple store projects (SDB, TDB, TDB2), as well as a general community interest in the opportunity to use any, institution-appropriate triple store (Jena-based or otherwise), a recent “special topics” meeting was held to discuss the current state of the Jena triple stores with the goal of coming to a recommendation for VIVO’s default triple store installation.
https://wiki.lyrasis.org/display/VIVO/2019-12-06+-+Special+Topic+-+TDB+vs+SDB

This call resulted in a feature table [1] comparing Jena’s TDB and SDB, and a rich conversation with the Apache Jena project chair, Andy Seaborne, regarding Jena’s usage profiles and technical direction.

Based on the special topic and subsequent discussions, we would like to put forward the following proposal:
1. Short-term goal of VIVO continuing to support native connections to both SDB and TDB, with the default configuration changing to TDB in VIVO 1.12

2. Long-term goal of removing VIVO’s support for connecting to specific triple stores via their native APIs and instead focusing future development effort on improving VIVO's existing ability to connect to any standalone triple store that is compliant with SPARQL 1.1 Update/Query [2]. This would also include the ability to configure the SPARQL requests that VIVO makes in order to accommodate the idiosyncrasies of different triple stores

3. VIVO will continue to deliver a complete open stack with both optional and replaceable components to support the broad VIVO community

As a VIVO community, our first priority is to ensure that any technical choices for the application allow current VIVO installations to move forward with on-going VIVO software releases. That being the case, your input on the above proposal is vital.

Please respond to this thread with any questions, concerns, expressions of support or opposition.

Regards,
Andrew Woods (on behalf of the VIVO Committers)

[1] https://docs.google.com/document/d/1xUsxvcYo4N9U2nzN0OUmR44wZPU7A7DJESRA7H_Hfe0/edit#heading=h.ou4i9fgruixk
[2] https://www.w3.org/TR/sparql11-overview/

[Andrew Woods]

Hello All,

> As a VIVO community, our first priority is to ensure that any technical choices for the application allow current VIVO installations to move forward with on-going VIVO software releases. That being the case, your input on the above proposal is vital.
>
> Please respond to this thread with any questions, concerns, expressions of support or opposition.

Should we assume silence means consent/support? If you have any comments, please let them be known.

Thanks,
Andrew

________________________________________
From: Andrew Woods
Sent: Monday, February 3, 2020 10:56 AM
To: VIVO Tech
Cc: VIVO Committers; leadersh...@vivoweb.org
Subject: VIVO Triple Store Roadmap Proposal

[Leandro Tabares Martín]

Hello,

I agree with the matters proposed. I think that improving VIVO's compatibility with triple stores is a very important step to fit the evolution of semantic technologies.

Best,

Leandro

--
You received this message because you are subscribed to the Google Groups "VIVO Tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vivo-tech+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vivo-tech/MWHPR2201MB1759482D070EB8150449A81A84180%40MWHPR2201MB1759.namprd22.prod.outlook.com.

[Michel Héon]

Good Morning, everyone,

Here is my contribution to the community's call for opinion.

About point 1- the transition from SDB to TDB for version 1.12. This seems to me a wise choice... even natural.

On point 2:

• The intent is honorable. However, for bulk loading of triplet sets, the approach using Jena's Java-API is a great advantage over a SPARQL query. In my opinion, SPARQL should be kept for light weight data exchanges (a few KB) and with a medium frequency invocation. In case of heavier calls (several KB or even GB) and with high invocation frequency than the use of java-apis seems to me an essential possibility to keep in Vivo's technological architecture.

• About reusability with other triplestores I don't anticipate any particular problem since Jena is recognized as an industry standard, the few references below [1-4] show that the main triplestores in the industry offer the jena-java-API to access the data. The only exception I know is Blazegraph which is based on the openrdf Framework (the ancestor of RDF4J [5] and the competitor standard to Jena). In my opinion, in the immediate future, I would not cut the bridges with the use of the Jena Java-API. Better yet, perhaps it would be a good idea to create an VIVO-TripleStore-Facade-API that encapsulates Jena or RDF4J calls to ensure maximum interoperability with the various triplestores available on the market.

Point 3: It seems crucial to me.

1.   Maklogic -jena https://github.com/marklogic/marklogic-jena
2.  Stardog – Jena https://gist.github.com/mhgrove/1045572
3. Topbraid EDG - https://github.com/TopQuadrant/shacl
4.  Virtuoso – Jena http://vos.openlinksw.com/owiki/wiki/VOS/VirtJenaProvider
5. RDF4J Wiki https://en.wikipedia.org/wiki/RDF4J

Regards

Michel Héon

-------

[Andrew Woods]

Thanks for the input, Michel.

Regarding "point 2" and specifically:

> "perhaps it would be a good idea to create an VIVO-TripleStore-Facade-API that encapsulates Jena or RDF4J calls to ensure maximum interoperability with the various triplestores"

..if you are available, this would be a great topic to discuss during tomorrow's dev meeting:
https://wiki.lyrasis.org/display/VIVO/2020-02-18+-+VIVO+Development+IG

Thanks,
Andrew

________________________________________
From: vivo...@googlegroups.com <vivo...@googlegroups.com> on behalf of Michel Héon <cote...@gmail.com>
Sent: Thursday, February 13, 2020 3:16 AM
To: VIVO Tech
Subject: [vivo-tech] Re: VIVO Triple Store Roadmap Proposal

Good Morning, everyone,
Here is my contribution to the community's call for opinion.

About point 1- the transition from SDB to TDB for version 1.12. This seems to me a wise choice... even natural.

On point 2:

* The intent is honorable. However, for bulk loading of triplet sets, the approach using Jena's Java-API is a great advantage over a SPARQL query. In my opinion, SPARQL should be kept for light weight data exchanges (a few KB) and with a medium frequency invocation. In case of heavier calls (several KB or even GB) and with high invocation frequency than the use of java-apis seems to me an essential possibility to keep in Vivo's technological architecture.

* About reusability with other triplestores I don't anticipate any particular problem since Jena is recognized as an industry standard, the few references below [1-4] show that the main triplestores in the industry offer the jena-java-API to access the data. The only exception I know is Blazegraph which is based on the openrdf Framework (the ancestor of RDF4J [5] and the competitor standard to Jena). In my opinion, in the immediate future, I would not cut the bridges with the use of the Jena Java-API. Better yet, perhaps it would be a good idea to create an VIVO-TripleStore-Facade-API that encapsulates Jena or RDF4J calls to ensure maximum interoperability with the various triplestores available on the market.

Point 3: It seems crucial to me.

1. Maklogic -jena https://github.com/marklogic/marklogic-jena
2. Stardog – Jena https://gist.github.com/mhgrove/1045572
3. Topbraid EDG - https://github.com/TopQuadrant/shacl
4. Virtuoso – Jena http://vos.openlinksw.com/owiki/wiki/VOS/VirtJenaProvider
5. RDF4J Wiki https://en.wikipedia.org/wiki/RDF4J

Regards
Michel Héon
-------


Le lundi 3 février 2020 10:56:33 UTC-5, Andrew Woods a écrit :
Hello All,

VIVO strives to evolve in-pace with the dependencies and technologies on which it is built. Recognizing the technical trajectory of the Jena triple store projects (SDB, TDB, TDB2), as well as a general community interest in the opportunity to use any, institution-appropriate triple store (Jena-based or otherwise), a recent “special topics” meeting was held to discuss the current state of the Jena triple stores with the goal of coming to a recommendation for VIVO’s default triple store installation.
https://wiki.lyrasis.org/display/VIVO/2019-12-06+-+Special+Topic+-+TDB+vs+SDB

This call resulted in a feature table [1] comparing Jena’s TDB and SDB, and a rich conversation with the Apache Jena project chair, Andy Seaborne, regarding Jena’s usage profiles and technical direction.

Based on the special topic and subsequent discussions, we would like to put forward the following proposal:
1. Short-term goal of VIVO continuing to support native connections to both SDB and TDB, with the default configuration changing to TDB in VIVO 1.12

2. Long-term goal of removing VIVO’s support for connecting to specific triple stores via their native APIs and instead focusing future development effort on improving VIVO's existing ability to connect to any standalone triple store that is compliant with SPARQL 1.1 Update/Query [2]. This would also include the ability to configure the SPARQL requests that VIVO makes in order to accommodate the idiosyncrasies of different triple stores

3. VIVO will continue to deliver a complete open stack with both optional and replaceable components to support the broad VIVO community

As a VIVO community, our first priority is to ensure that any technical choices for the application allow current VIVO installations to move forward with on-going VIVO software releases. That being the case, your input on the above proposal is vital.

Please respond to this thread with any questions, concerns, expressions of support or opposition.

Regards,
Andrew Woods (on behalf of the VIVO Committers)

[1] https://docs.google.com/document/d/1xUsxvcYo4N9U2nzN0OUmR44wZPU7A7DJESRA7H_Hfe0/edit#heading=h.ou4i9fgruixk
[2] https://www.w3.org/TR/sparql11-overview/

--
You received this message because you are subscribed to the Google Groups "VIVO Tech" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vivo-tech+...@googlegroups.com<mailto:vivo-tech+...@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/vivo-tech/d2eeae74-cda6-4c8e-b3e7-dc23df0d87df%40googlegroups.com<https://groups.google.com/d/msgid/vivo-tech/d2eeae74-cda6-4c8e-b3e7-dc23df0d87df%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Andrew Woods]

Hello All,

Thank you for the input on the topic of VIVO's Triple Store Roadmap.
https://wiki.lyrasis.org/display/VIVO/VIVO+Triple+Store+Roadmap

Based on the largely positive response, we will be moving forward with the proposed roadmap.

Regarding the first item ("default configuration changing to TDB"), we have taken the initial step of changing the default triple store from SDB to TDB in the example configuration.
https://jira.lyrasis.org/browse/VIVO-1741

As a prerequisite for the second item (moving towards decoupled triple stores), we have started to document the performance characteristics of current VIVO/triple-store configurations so that we have a baseline against which to measure the effect of this decoupling. The performance tests and results will be documented on the wiki:
https://wiki.lyrasis.org/display/VIVO/Performance+Testing

Thanks again for your engagement and input.

Regards,

Andrew


________________________________________
From: Andrew Woods
Sent: Monday, February 3, 2020 10:56 AM
To: VIVO Tech
Cc: VIVO Committers; leadersh...@vivoweb.org

Subject: VIVO Triple Store Roadmap Proposal

[Pablo]

Hello everyone, I will begin to develop on the VIVO platform and I would like to know what IDES tools and work environment settings recommend me, jdk version and other configurations Thank you very much.

 

Sent from Mail for Windows 10

--

You received this message because you are subscribed to the Google Groups "VIVO Tech" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vivo-tech+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vivo-tech/MWHPR2201MB1759411D4ACDCBB20644166B84EB0%40MWHPR2201MB1759.namprd22.prod.outlook.com.

 

[Ralph O'Flinn]

Pablo -

While these wiki pages are 100% complete yet it should give you a good idea on how to setup a dev environment:

https://wiki.lyrasis.org/display/VIVO/Full+Development+VM+Setup

To unsubscribe from this group and stop receiving emails from it, send an email to vivo...@googlegroups.com.

[Gross, Benjamin]

I expect Ralph meant to say those wiki pages aren’t 100% complete. As for IDEs, It’s up to personal preference. There was a workshop that covered setting up Eclipse for VIVO development at the conference in Durham, so I know a handful of developers use that. It seems Ralph uses Visual Studio Code, which I have also started to use after IntelliJ changed their license for open source projects.

Benjamin

_____________________________
Benjamin Gross

VIVO Product Manager

Web of Science Group | webofsciencegroup.com

To unsubscribe from this group and stop receiving emails from it, send an email to vivo-tech+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vivo-tech/e3d45c33-e795-445f-9727-35709b5e31bc%40googlegroups.com.