How to build a REST API in EDG to validate external JSON-LD data?

[Fan Li]

For my project, I need to have web service to validate JSON-LD data using SHACL. It would be similar to the SHACL playground (https://shacl.org/playground/) but using shapes and data in EDG. I have read the EDG developer guide (http://wiki.topbraid.net/display/TBS61/Developer+Guide) but not sure where to start. Any guidance is greatly appreciated.

[Holger Knublauch]

Hi Fan Li,

if you have TBC-ME open, please go to

http://localhost:8083/tbl/swp?_viewClass=tblshacl%3AValidationPage

This page can also be reached from the Server Administration -> Available Web Services.

It allows you to upload files to validate or use local files by their base URI. So for example, you can use this service to validate an external file stored in JSON-LD against shapes stored already in the EDG workspace. The URL of the web service is tbl/shaclValidation and you can see how to call it by looking at what the user interface is sending to the server.

If you want to take more control, one alternative would be to use SWP and SPARQLMotion, but before you go there maybe the solution above is sufficient for your needs?

Holger

On 21/07/2019 15:51, Fan Li wrote:

For my project, I need to have web service to validate JSON-LD data using SHACL. It would be similar to the SHACL playground (https://shacl.org/playground/) but using shapes and data in EDG. I have read the EDG developer guide (http://wiki.topbraid.net/display/TBS61/Developer+Guide) but not sure where to start. Any guidance is greatly appreciated.

--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/f4203d1f-35e8-42c1-b9f0-7d8932f65316%40googlegroups.com.

[Fan Li]

Thanks, Holger! This is exactly what I need. I see how I can use it make a POST call with JSON-LD data payload to validate against an EDG shape graph via URI.

On Sunday, July 21, 2019 at 7:30:05 PM UTC-4, Holger Knublauch wrote:

Hi Fan Li,

if you have TBC-ME open, please go to

http://localhost:8083/tbl/swp?_viewClass=tblshacl%3AValidationPage

This page can also be reached from the Server Administration -> Available Web Services.

It allows you to upload files to validate or use local files by their base URI. So for example, you can use this service to validate an external file stored in JSON-LD against shapes stored already in the EDG workspace. The URL of the web service is tbl/shaclValidation and you can see how to call it by looking at what the user interface is sending to the server.

If you want to take more control, one alternative would be to use SWP and SPARQLMotion, but before you go there maybe the solution above is sufficient for your needs?

Holger

On 21/07/2019 15:51, Fan Li wrote:

For my project, I need to have web service to validate JSON-LD data using SHACL. It would be similar to the SHACL playground (https://shacl.org/playground/) but using shapes and data in EDG. I have read the EDG developer guide (http://wiki.topbraid.net/display/TBS61/Developer+Guide) but not sure where to start. Any guidance is greatly appreciated.

--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to topbrai...@googlegroups.com.

[Stefan Verweij]

Hi Holger,

When I try to validate my ontology using your validation page I get an "HTTP Status 400 – Bad Request" following:

---

Type Status Report

Message Current user does not have read access for graph urn:x-shacl-shapes-graph:47bd4c09-695a-4f8c-89f2-f046b16ff9ff

Description The server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing).

---

Apache Tomcat/8.5.41

How do I get read access or solve this error?

Kind regards,

Stefan

Op maandag 22 juli 2019 01:30:05 UTC+2 schreef Holger Knublauch:

[Holger Knublauch]

Hi Stefan,

this page is currently limited to Administrator accounts. If you log in with the default account of TBC-ME then it should work. Are you using a TopBraid EDG installation?

I think we'll also make it available to other users in the next release.

Holger

--

You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/f50a63fb-3628-4560-bf93-7cc92dd0821c%40googlegroups.com.

[Stefan Verweij]

Hi Holger,

I only have editor rights within the EDG project (I am not the administrator with the default account). I think it would indeed be helpful to include the availability to editors in the upcoming release, since I cannot validate the ontology I am editing (the SHACL playground is an alternative, but not ideal due to the size of the ontology).

Keep up the good work!

Stefan

Op vrijdag 2 augustus 2019 04:47:38 UTC+2 schreef Holger Knublauch:

[Stefan Verweij]

I have another question (sorry to ask this much questions about the EDG ontology, I am quite new with the tools and just starting to understand how everything works). When I export my ontology as a graph (both JSON-LD as well as Turtle) I am still finding old pieces of code in the file that I deleted earlier within the UI of the EDG ontology, how is this possible?

Op zondag 21 juli 2019 07:51:50 UTC+2 schreef Fan Li:

[Irene Polikoff]

Why can’t you simply go to Reports tab and run the Problems and Suggestions report?

Sent from my iPhone

--

You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/f93305fc-a251-4523-b5c0-a668cd18ed14%40googlegroups.com.

[Irene Polikoff]

If data was deleted, it would not be in the exported file. 

Is it possible that you deleted in the context of a workflow and have not commuted it yet and you are exporting production version that still has this code?

 The only other explanation I can think of is that you did not delete everything you thought you did.

--

You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/fd77ca33-8350-44a1-a3bf-c86b3a799279%40googlegroups.com.

[Stefan Verweij]

If I search for the deleted classes and attributes I can still find it in the UI, but they are not part of my ontology anymore since I deleted them and seem to be isolated (and invisible for the user if you are not searching for the specific label). Can this be the result of only deleting the class for example so its attributes become detached? See picture for example of attribute 'instant_property' from a class 'time' that I have deleted earlier, as you can see it doesn't have any Local Property Characteristics anymore.

And about the validation, I have some use cases in JSON-LD code (and even more data in CSV format which I need to convert first) which I would like to test to my ontology, but 'Problems and Suggestions' only looks at instances and irregularities of the ontology right?

Thanks for the fast responses btw, it's really helpful!

Stefan

Op vrijdag 2 augustus 2019 13:10:10 UTC+2 schreef Irene Polikoff:

If data was deleted, it would not be in the exported file. 

Is it possible that you deleted in the context of a workflow and have not commuted it yet and you are exporting production version that still has this code?

 The only other explanation I can think of is that you did not delete everything you thought you did.

Op vrijdag 2 augustus 2019 13:00:14 UTC+2 schreef Irene Polikoff:

[Irene Polikoff]

Yes, I believe currently if you delete a class, the associated property shape gets deleted, but not the property itself.

Since you found the property, just click on Delete button to remove it.

Problems and Suggestions will validate the collection you are in. It will validate either everything in it (including information in the included collections) or, optionally, only resources of a certain type. The validation will be against the shapes found in the collection - directly or because they are included through owl:imports. In other words, in the validation the current collection is used as both, the data graph and the shapes graph.

You could load your instance data into your ontology or, better, create a Data Graph collection and load it there. Then you can validate it.

If you decide to use Data Graph collections, note that when you create one you will need to select an Ontology it is based on. It must contain the schema (shapes) your data will be based on.

In the Ontology, you'll need to declare classes which instances you will be storing in the Data Graph ‘public classes’ of its GraphQL Schema. This can be done once on a parent class if these classes have a common parent. This set up step is explained here: http://wiki.topquadrant.com/display/master/Ontology+View+or+Edit#OntologyVieworEdit-SHACLrequirementsforworkingwithReferenceDatasetsandDataGraphs.

If your data is based on SKOS, you could also use a Taxonomy to store it.

You may not need to convert CSV data outside of EDG. EDG lets you import data from CSV, converting it to RDF. This option is available under Imports tab.

On Aug 2, 2019, at 7:36 AM, Stefan Verweij <stefan....@wur.nl> wrote:

If I search for the deleted classes and attributes I can still find it in the UI, but they are not part of my ontology anymore since I deleted them and seem to be isolated (and invisible for the user if you are not searching for the specific label). Can this be the result of only deleting the class for example so its attributes become detached? See picture for example of attribute 'instant_property' from a class 'time' that I have deleted earlier, as you can see it doesn't have any Local Property Characteristics anymore.

<isolated_property.PNG>

And about the validation, I have some use cases in JSON-LD code (and even more data in CSV format which I need to convert first) which I would like to test to my ontology, but 'Problems and Suggestions' only looks at instances and irregularities of the ontology right?

Thanks for the fast responses btw, it's really helpful!

Stefan

Op vrijdag 2 augustus 2019 13:10:10 UTC+2 schreef Irene Polikoff:

If data was deleted, it would not be in the exported file. 

Is it possible that you deleted in the context of a workflow and have not commuted it yet and you are exporting production version that still has this code?

 The only other explanation I can think of is that you did not delete everything you thought you did.

Op vrijdag 2 augustus 2019 13:00:14 UTC+2 schreef Irene Polikoff:

Why can’t you simply go to Reports tab and run the Problems and Suggestions report?

Sent from my iPhone 

--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/04ca0d17-bad5-4827-8f57-76093b61152f%40googlegroups.com.

<isolated_property.PNG>

[Irene Polikoff]

One more thing to add about “invisibility to the user”.

I expect that by this you mean that they do not show in the class hierarchy. Class hierarchy starts with a root class (by default owl:Thing) and shows all subclasses, then their subclasses, etc. Root can be configured on the Manage tab.

Under each class, properties associated with it are shown.

So, if something does not show in the hierarchy, it does not mean it does not exist and was deleted. It simply means that it is not connected in the way this particular view expects. As you have discovered, user can still find and navigate to these resources.

[Stefan Verweij]

Thanks, I indeed discovered if something is not (in)directly linked to Thing it will not be visible.So I deleted them manually by navigating and using the 'Problems and Suggestions' option to find unlinked properties resulting in errors. Since I am not the manager I do not have the right permissions to use the manage tab to configure root, so the only ways to find unlinked classes and properties is by using Problems and Suggestions (or manually scrolling through the lines of code)?

About the validation, thanks again for the problems and suggestions option, it helped debugging the ontology. I have one question about importing my instance data: Where do I create a Data Graph collection? If I understand it correctly I have to create a collection in 'Data Assets collections', but, again, since I am not the admin nor a manager I cannot create one. So the only option is by importing instances through the import tab then?

Again, thanks for all the valuable help!

Stefan

Op vrijdag 2 augustus 2019 15:26:14 UTC+2 schreef Irene Polikoff:

Yes, I believe currently if you delete a class, the associated property shape gets deleted, but not the property itself.

Since you found the property, just click on Delete button to remove it.

Problems and Suggestions will validate the collection you are in. It will validate either everything in it (including information in the included collections) or, optionally, only resources of a certain type. The validation will be against the shapes found in the collection - directly or because they are included through owl:imports. In other words, in the validation the current collection is used as both, the data graph and the shapes graph.

You could load your instance data into your ontology or, better, create a Data Graph collection and load it there. Then you can validate it.

If you decide to use Data Graph collections, note that when you create one you will need to select an Ontology it is based on. It must contain the schema (shapes) your data will be based on.

In the Ontology, you'll need to declare classes which instances you will be storing in the Data Graph ‘public classes’ of its GraphQL Schema. This can be done once on a parent class if these classes have a common parent. This set up step is explained here: http://wiki.topquadrant.com/display/master/Ontology+View+or+Edit#OntologyVieworEdit-SHACLrequirementsforworkingwithReferenceDatasetsandDataGraphs.

If your data is based on SKOS, you could also use a Taxonomy to store it.

You may not need to convert CSV data outside of EDG. EDG lets you import data from CSV, converting it to RDF. This option is available under Imports tab.

 

Op vrijdag 2 augustus 2019 18:04:51 UTC+2 schreef Irene Polikoff:

[Irene Polikoff]

When you import RDF, EDG will insert a subclass relationship to Thing for all classes that have no parent.

Of course, you could later remove such relationship from a class and then the class will disappear from the view. In general, having such unconnected resources that should have been deleted but were not, is not a problem, they will not cause any issues. If you want to find and remove them, then yes, use Problems and Suggestions. Or, alternatively, SPARQL. 

What package of EDG do you have? All collection types you have licensed appear on the left in the blue navigator. To create a new collection, click on one of the types and then click on Create.

By default, all users can create new collections. However, your organization may set up permissions such a way that only certain users can do this. You would need to discuss this with your EDG administrator.

If you can’t and/or should not be creating new collections, then you would have to import your data into one of already existing collections. 

--

You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/4067ddbf-81ff-4c77-b89f-c038c9ecce99%40googlegroups.com.

[Stefan Verweij]

Problems and suggestions raise errors for these kind of unconnected properties when the property has a constraint that says min count => 1, but also shows you which unconnected property raises the error so deleting them is indeed quite easy. And I will reach out to my administrator about the creation of Data Graphs.

Thanks for the help.

Stefan

Op maandag 5 augustus 2019 14:35:30 UTC+2 schreef Irene Polikoff:

When you import RDF, EDG will insert a subclass relationship to Thing for all classes that have no parent.

Of course, you could later remove such relationship from a class and then the class will disappear from the view. In general, having such unconnected resources that should have been deleted but were not, is not a problem, they will not cause any issues. If you want to find and remove them, then yes, use Problems and Suggestions. Or, alternatively, SPARQL. 

What package of EDG do you have? All collection types you have licensed appear on the left in the blue navigator. To create a new collection, click on one of the types and then click on Create.

By default, all users can create new collections. However, your organization may set up permissions such a way that only certain users can do this. You would need to discuss this with your EDG administrator.

If you can’t and/or should not be creating new collections, then you would have to import your data into one of already existing collections. 

On Aug 5, 2019, at 4:00 AM, Stefan Verweij <stefan...@wur.nl> wrote:

Thanks, I indeed discovered if something is not (in)directly linked to Thing it will not be visible.So I deleted them manually by navigating and using the 'Problems and Suggestions' option to find unlinked properties resulting in errors. Since I am not the manager I do not have the right permissions to use the manage tab to configure root, so the only ways to find unlinked classes and properties is by using Problems and Suggestions (or manually scrolling through the lines of code)?

About the validation, thanks again for the problems and suggestions option, it helped debugging the ontology. I have one question about importing my instance data: Where do I create a Data Graph collection? If I understand it correctly I have to create a collection in 'Data Assets collections', but, again, since I am not the admin nor a manager I cannot create one. So the only option is by importing instances through the import tab then?

Again, thanks for all the valuable help!

Stefan

Op vrijdag 2 augustus 2019 15:26:14 UTC+2 schreef Irene Polikoff:

Yes, I believe currently if you delete a class, the associated property shape gets deleted, but not the property itself.

Since you found the property, just click on Delete button to remove it.

Problems and Suggestions will validate the collection you are in. It will validate either everything in it (including information in the included collections) or, optionally, only resources of a certain type. The validation will be against the shapes found in the collection - directly or because they are included through owl:imports. In other words, in the validation the current collection is used as both, the data graph and the shapes graph.

You could load your instance data into your ontology or, better, create a Data Graph collection and load it there. Then you can validate it.

If you decide to use Data Graph collections, note that when you create one you will need to select an Ontology it is based on. It must contain the schema (shapes) your data will be based on.

In the Ontology, you'll need to declare classes which instances you will be storing in the Data Graph ‘public classes’ of its GraphQL Schema. This can be done once on a parent class if these classes have a common parent. This set up step is explained here: http://wiki.topquadrant.com/display/master/Ontology+View+or+Edit#OntologyVieworEdit-SHACLrequirementsforworkingwithReferenceDatasetsandDataGraphs.

If your data is based on SKOS, you could also use a Taxonomy to store it.

You may not need to convert CSV data outside of EDG. EDG lets you import data from CSV, converting it to RDF. This option is available under Imports tab.

 

Op vrijdag 2 augustus 2019 18:04:51 UTC+2 schreef Irene Polikoff:

One more thing to add about “invisibility to the user”.

I expect that by this you mean that they do not show in the class hierarchy. Class hierarchy starts with a root class (by default owl:Thing) and shows all subclasses, then their subclasses, etc. Root can be configured on the Manage tab.

Under each class, properties associated with it are shown.

So, if something does not show in the hierarchy, it does not mean it does not exist and was deleted. It simply means that it is not connected in the way this particular view expects. As you have discovered, user can still find and navigate to these resources.

--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to topbrai...@googlegroups.com.

[Stefan Verweij]

Hi,

I would like to know if this issue is solved in the 6.3 release? 

Cheers,

Stefan

Op vrijdag 2 augustus 2019 04:47:38 UTC+2 schreef Holger Knublauch:

I think we'll also make it available to other users in the next release.

[Holger Knublauch]

Yes, this has been addressed for 6.3, see the release notes at

https://www.topquadrant.com/wp-content/uploads/2019/12/TopBraid-Suite-6.3-Change-Log-Release.pdf

- Bug fix: SHACL valdiation service now also works for non-admin users.

Holger

--

You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/9599f2d9-dbae-4e93-849c-6224077e528f%40googlegroups.com.

[Rob Atkinson]

that very useful sounding link:

http://wiki.topquadrant.com/display/master/Ontology+View+or+Edit#OntologyVieworEdit-SHACLrequirementsforworkingwithReferenceDatasetsandDataGraphs.

 

resolves to this less useful contents page

https://doc.topquadrant.com/#OntologyVieworEdit-SHACLrequirementsforworkingwithReferenceDatasetsandDataGraphs  

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/270ab66b-0367-b5e5-1935-e6d00a405510%40topquadrant.com.

[TopQuadrant]

wiki.topquadrant.com is no longer used. I believe you are looking for this page: https://doc.topquadrant.com/6.3/ontologies/. You will find the same information for working with SHACL. 

On Monday, January 27, 2020 at 7:41:56 PM UTC-5, Rob Atkinson wrote:

that very useful sounding link:

http://wiki.topquadrant.com/display/master/Ontology+View+or+Edit#OntologyVieworEdit-SHACLrequirementsforworkingwithReferenceDatasetsandDataGraphs.

 

resolves to this less useful contents page

https://doc.topquadrant.com/#OntologyVieworEdit-SHACLrequirementsforworkingwithReferenceDatasetsandDataGraphs  

On Tue, 28 Jan 2020 at 11:00, Holger Knublauch <hol...@topquadrant.com> wrote:

Yes, this has been addressed for 6.3, see the release notes at

https://www.topquadrant.com/wp-content/uploads/2019/12/TopBraid-Suite-6.3-Change-Log-Release.pdf

- Bug fix: SHACL valdiation service now also works for non-admin users.

Holger

On 28/01/2020 00:35, Stefan Verweij wrote:

Hi,

I would like to know if this issue is solved in the 6.3 release? 

Cheers,

Stefan

Op vrijdag 2 augustus 2019 04:47:38 UTC+2 schreef Holger Knublauch:

I think we'll also make it available to other users in the next release.

--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/9599f2d9-dbae-4e93-849c-6224077e528f%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-users+unsubscribe@googlegroups.com.

[Rob Atkinson]

OK - did find that page already.  good to know there isnt something else i'm missing.

On Tue, 28 Jan 2020 at 12:59, TopQuadrant <tma...@topquadrant.com> wrote:

wiki.topquadrant.com is no longer used. I believe you are looking for this page: https://doc.topquadrant.com/6.3/ontologies/. You will find the same information for working with SHACL. 

On Monday, January 27, 2020 at 7:41:56 PM UTC-5, Rob Atkinson wrote:

that very useful sounding link:

http://wiki.topquadrant.com/display/master/Ontology+View+or+Edit#OntologyVieworEdit-SHACLrequirementsforworkingwithReferenceDatasetsandDataGraphs.

 

resolves to this less useful contents page

https://doc.topquadrant.com/#OntologyVieworEdit-SHACLrequirementsforworkingwithReferenceDatasetsandDataGraphs  

On Tue, 28 Jan 2020 at 11:00, Holger Knublauch <hol...@topquadrant.com> wrote:

Yes, this has been addressed for 6.3, see the release notes at

https://www.topquadrant.com/wp-content/uploads/2019/12/TopBraid-Suite-6.3-Change-Log-Release.pdf

- Bug fix: SHACL valdiation service now also works for non-admin users.

Holger

On 28/01/2020 00:35, Stefan Verweij wrote:

Hi,

I would like to know if this issue is solved in the 6.3 release? 

Cheers,

Stefan

Op vrijdag 2 augustus 2019 04:47:38 UTC+2 schreef Holger Knublauch:

I think we'll also make it available to other users in the next release.

--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/9599f2d9-dbae-4e93-849c-6224077e528f%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/270ab66b-0367-b5e5-1935-e6d00a405510%40topquadrant.com.

--

You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/ef144758-0321-4481-a019-368bccf9531b%40googlegroups.com.

[Stefan Verweij]

Cool, thanks! Couldn't find it before in the notes when I searched for 'validat' haha

Op dinsdag 28 januari 2020 01:00:50 UTC+1 schreef Holger Knublauch:

Yes, this has been addressed for 6.3, see the release notes at

https://www.topquadrant.com/wp-content/uploads/2019/12/TopBraid-Suite-6.3-Change-Log-Release.pdf

- Bug fix: SHACL valdiation service now also works for non-admin users.

Holger

On 28/01/2020 00:35, Stefan Verweij wrote:

Hi,

I would like to know if this issue is solved in the 6.3 release? 

Cheers,

Stefan

Op vrijdag 2 augustus 2019 04:47:38 UTC+2 schreef Holger Knublauch:

I think we'll also make it available to other users in the next release.

--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to topbrai...@googlegroups.com.