EDG Files vs Collection

[Maatary Okouya]

Hi, 

When wrapping Files, into EDG Collection. Does it become a completely separate entity e.g. ontology, or the two can be kept in sync, as in what change on files is propagate to the ontology wrapped ?

I guess my question is about understanding the intended behavior

[David Price]

“wrapping” files is not a good way to think about this. The EDG behaviour between collections and files is normal owl:imports.

EDG Files contain a named graph. So do collections, and collections can include (i.e. owl:imports) a graph that happens to be in a file or a collection.

If you change graph in a File at any point in time, then those changes are visible immediately in all including collections (accepting that caches may need to be refreshed for any currently in-use collections via a page reload for example).

FYI there are a few “special” files in EDG - api.ttl files contents are made visible via a software mechanism rather than owl:imports. These usually contain SPARQL Functions, for example, which are really “software” not "data model” or “data”. See 

https://archive.topquadrant.com/doc/latest/ext/process.html#working-with-extension-files

which is part of

https://archive.topquadrant.com/doc/latest/ext/index.html

Cheers,

David

UK +44 (0) 7788 561308
US +1 (336) 283-0808‬

--
The topics of this mailing list include TopBraid EDG and related technologies such as SHACL.
To post to this group, send email to topbrai...@googlegroups.com
---
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/80f75b5d-564c-4c2b-ae3e-923be77f3a8an%40googlegroups.com.

[Maatary Okouya]

Then what is really the rational for having files and collection ? I work with studio mostly, Files are perfect for me. I just want to make sure that i fully graps the philosophy here. Files did not exist in previous version, why was it added ?

[David Price]

On 10 Mar 2024, at 06:35, Maatary Okouya <maatar...@gmail.com> wrote:

Then what is really the rational for having files and collection ? I work with studio mostly, Files are perfect for me. I just want to make sure that i fully graps the philosophy here. Files did not exist in previous version, why was it added ?

Put simply, Studio/Files is how developers/ontologists deliver a software system, and Collections are how users create and manage data using that software system.

Studio was built for EDG server-supporting developers, ontologists, etc. who need to build and test ontologies, scripts, etc. before they get deployed to an EDG server as a “project” for use by the users of that EDG server.  

Files is a feature for those developers that understands SPARQL, SHACL, etc. and so is better than using an external source code editor on TTL files, for example. Files is not intended to be a feature available on operational EDG servers due to security and system support/uptime risks.   

The graphs deployed to an EDG server as a “project” are not “user data” so we don’t want users changing them.

Collections *are* the "user data” created and changed by users of an EDG server, assuming they have the right role/permissions.

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/188d2eea-dd43-479a-b646-eba1ac4a4452n%40googlegroups.com.

[Maatary Okouya]

Understood. thank you

[Maatary Okouya]

| Collections *are* the "user data” created and changed by users of an EDG server, assuming they have the right role/permissions.

Ok, then in that logic, why would a user be allowed to create a collection of type ontology ? This feels more like a software system by your definition. They should only be able to create instance data no? Instance of ontology or simply taxonomy. But i can also understand that as the technology itself, difference between instance data and schema data can be blur sometime. The all workflow thingy, feels to me like collaborative ontology editing. 

In any case at this point, I would like to be able to deploy the ontologies we create with EDG Studio to EDG Server, not for user to edit user data (on top of it), but simply to consult the ontologies. What the best workflow for that ? doesn't the Git integration enable that ? 

[David Price]

On 22 Mar 2024, at 16:49, Maatary Okouya <maatar...@gmail.com> wrote:

| Collections *are* the "user data” created and changed by users of an EDG server, assuming they have the right role/permissions.

Ok, then in that logic, why would a user be allowed to create a collection of type ontology ? This feels more like a software system by your definition. They should only be able to create instance data no? Instance of ontology or simply taxonomy. But i can also understand that as the technology itself, difference between instance data and schema data can be blur sometime.

If an EDG user is an ontologist, then classes and properties are user data (i.e. data created by an EDG user).  

Sometimes an ontology is the end product for some EDG users. 

Sometimes the ontologies are used by a different set of EDG users to create instances in a Data Graph, for example.

EDG is flexible … and the EDG UI is almost all "model-driven” driven by the fact that ontologies are also a kind of data.

The all workflow thingy, feels to me like collaborative ontology editing. 

Workflows are for editing anything in EDG collections, not just ontologies.

In any case at this point, I would like to be able to deploy the ontologies we create with EDG Studio to EDG Server, not for user to edit user data (on top of it), but simply to consult the ontologies. What the best workflow for that ? doesn't the Git integration enable that ? 

There is no single best practice for deployment. It depends on who is responsible for what, the skills of the team, how many are on the ontogists team, are external staff often used, how often ontologies need to change, how tightly controlled/regulated the organization is, etc.

Cheers,

David

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/0fce4b0b-295a-431f-b07a-5de98c5bcc8bn%40googlegroups.com.

[Maatary Okouya]

>>  There is no single best practice for deployment. It depends on who is responsible for what, the skills of the team, how many are on the ontogists team, are external staff often used, how often ontologies need to change, how tightly controlled/regulated the organization is, etc.

We have currently 2 experts ontologist working on models on studio and collaborating via Git.  Our goal is simply to upload the work of the ontologist as frequently as necessary to EDG Server, and have developers and product owners consult the only. It support data contract discussion between all the parties involved in the project. Ideally we wanted to go through git integration, but it seems that it does not work since the File Asset Type has been removed from EDG Server. Would have been nice to keep it as read only at least. From what i have red online, it seems we only have the manual upload possible as in extra step but git push. But we are not sure here exactly what are the exact steps to follow. 

 No external staff, the ontology currently is in active development, but very soon about to settle and expected to change not that frequently. In term of control/regulation, we are establishing the practice at the moment. 

[David Price]

On 22 Mar 2024, at 18:53, Maatary Okouya <maatar...@gmail.com> wrote:

>>  There is no single best practice for deployment. It depends on who is responsible for what, the skills of the team, how many are on the ontogists team, are external staff often used, how often ontologies need to change, how tightly controlled/regulated the organization is, etc.

We have currently 2 experts ontologist working on models on studio and collaborating via Git.  Our goal is simply to upload the work of the ontologist as frequently as necessary to EDG Server, and have developers and product owners consult the only. It support data contract discussion between all the parties involved in the project. Ideally we wanted to go through git integration, but it seems that it does not work since the File Asset Type has been removed from EDG Server. Would have been nice to keep it as read only at least. From what i have red online, it seems we only have the manual upload possible as in extra step but git push. But we are not sure here exactly what are the exact steps to follow. 

 No external staff, the ontology currently is in active development, but very soon about to settle and expected to change not that frequently. In term of control/regulation, we are establishing the practice at the moment. s, 

Seems like using Server Admin, Project Upload when your team gets to a “stable release” would work. Here’s a process we use for some customers that works for us:

1. make a folder in git called something like myontologies.topbraidlive.org (call it what you want but adopt a convention and follow it)
2. put your TTLs in the folder making sure they have rdf:type owl;Ontology statements
3. zip the folder when ready to deploy to server
4. Project Upload (from then on I usually use a the “Delete old project option and always completely replace everything, but that’s up to you, update is available)
5. Settings, Includes will now show your ontologies under the “Other” category.

That said, with two ontologist it see like just using the EDG server and workflows would work for your situation. Your two experts can share workflows and test them before committing them to the production copy allowing the “users” to see the result.  Workflows are not 100% equivalent of git branches, but not a million miles away. If you are using git as a kind of backup or as shared corporate knowledge, then after the commit the ontologist can just Export RDF sorted TTL and update the git repo.

Cheers,

David

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/35dc24c3-dcc9-4a8d-8c4d-b78d72e5fa91n%40googlegroups.com.

[Maatary Okouya]

Thank you for the instruction

I did not fully understand this 

>> Settings, Includes will now show your ontologies under the “Other” category.

Questions:

1) can we use the option 

• Send Projects to Another Server
  

I have tried it, but keep failing, it seems like i am not entering the right address ? What should be the address or any other pre-requisite for that option to work ?

I am using https://server-address/edg/. and using my admin credential. Yet it is failing. I'm also selecting the relevant subset of file i want to send. 

2) Working with EDG Server directly, unfortunately is not an option for multiple reasons unfortunately.

3) Something that seem problematic with your approach, is that we work with Files in EDG Studio.

- If i create a folder to put my TTL in the same git repo and project, then i will have multiple files with the same BaseURI which TopBraid Product typically do not like. 

- Moreover The Upload Option also requires that the folder contains a .project file otherwise it rejects the upload.

- Once the upload is done, it seems we have to use the option create ontology from Existing Files to actually see the upload. And that's a problem because that import transform things in a way we don't want to. For instance our ontology separate shapes from class, and when using that function, the collection created add shapes to the class directly i.e. making all our class nodeshape and keeping the separate shape too, and we end up with the wrong thing.

Hence does not work for us. Isn't a way to import your ontology without modyfing it, so long it follow all the standard given that it is developped with Files on Studio

[Maatary Okouya]

I see now that there is an option to not modify the ontology when "creating ontology from existing Files". 

However, is it what you had in mind, that we would upload and then use the option create ontology from Existing Files?

This approach still change the URI of the ontology, it clearly state that it is a sort of wrapper for the actually ontology in the workspace. 

I guess there is no way to actually see the ontology developed with STUDIO, in the exact same way as it looks in STUDIO. is that correct ?

[Maatary Okouya]

Just for clarification i understood that wrapper was not the right term but in this case, i just did not know what short term to use other than the all statement 

"This is the Home resource of the Asset Collection used to edit a file from the workspace. The Asset Collection only exists while you are making edits and this resource here will be discarded once you have closed the Asset Collection. It is therefore not editable. Use the 'Home' button to edit the file's owl:Ontology resource."

Btw, i am clearly able to edit the the collection, and when i click the home button nothing happens. 

[David Price]

On 23 Mar 2024, at 13:30, Maatary Okouya <maatar...@gmail.com> wrote:

Thank you for the instruction

I did not fully understand this 

>> Settings, Includes will now show your ontologies under the “Other” category.

Questions:

1) can we use the option 

• Send Projects to Another Server
  

No, you must do as I described.

Your Studio cannot be configured  in the same manner as your EDG server or the git repo. Of course, there are other possibilities..  just described one.

Cheers

David

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/a6678bb4-84c6-400c-af73-8dc9d8473a34n%40googlegroups.com.

[Maatary Okouya]

Thanks for taking the time. 

I understand the approach described. I have tried it and more. This sum up my understanding so far

2 ways to export

- From the File collection itself (i.e the Ontology as developped with studio) or via the Administration Page Export all production Graph as Zip or Trig. (With EDG)

- Zip the files of choice manually from your repo making sure that .project file is there (Without EDG)

2 ways to import

- From the Admin page  - Upload which only works if a .project is there

- From the Home Page - New -> Import multiple asset collection from Trig or Zip file

In any event files are uploaded in the workspace. 

- With the new -> import, EDG create the ontology collection automatically from the existing Files that are saved in the workspace at the same time.

- With the upload, one has to create the ontology collection from the existing files manually from the ontologies menu

In any event, this created ontology collection is a separate graph from the one saved in the workspace, and is displayed with its own uri. If their are a lot of import, navigating to the import is manual, as in, go back to ontologies and open the imported collection ontology corresponding to the import (original uri). Several Options available in studio like seeing the whole graph won't be available and so on.

I guess that does wrap it for this thread. Thanks for taking the time.

[Maatary Okouya]

My bad there is one last thing that you said that i want to confirm that i understood properly. 

>> Settings, Includes will now show your ontologies under the “Other” category.

I think i understand that now. This essentially entail, that a user will create a collection, and in the setting include section, will be able to see what has been uploaded in the workspace. In other words they will have access once they import the FILE in their working collection (whatever that collection is)

>> That said, with two ontologist it see like just using the EDG server and workflows would work for your situation. Your two experts can share workflows and test them before committing them to the production copy allowing the “users” to see the result. 

Again, "to see the results" here means via the include. 

I can see that what i tested is slightly different from what you suggested with the include. 

That's because my intent was never to serve files that will feed into other collection that users would be working on. Hence i did not understand the statement at first.

Anyway, this has been very useful. I believe I understand better EDG Server Scope. 

[David Price]

Exactly.  Until Included no graph is visible in a user collection, regardless of whether that referenced graph is a collection or deployed/uploaded RDF file.

Cheers,

David

On 23 Mar 2024, at 20:02, Maatary Okouya <maatar...@gmail.com> wrote:

My bad there is one last thing that you said that i want to confirm that i understood properly. 

To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/62a5a17e-2347-43d1-9ac7-0d32ba627f1en%40googlegroups.com.