Including content in a data graph

40 views
Skip to first unread message

Rob Atkinson

unread,
May 19, 2020, 10:38:21 PM5/19/20
to TopBraid Suite Users

Use Case: 

I have a large data graph I wish to navigate and allow users to annotate so I expose in an EDG editor.  Its not really "reference data" - but its generated by scripting so the body should only be changed by external processes - and the idea of annotation is to provide feedback on the content, manage additional information we need to attach etc.

If I include the data, the class tree works but it shows no instance - obviously only working on the current graph, not with imports.

How can we control the query scope to allow a-box import heirarchies to be handled - effectively i want to distinguish between a-box and t-box imports I think.  No problem adding a property to the graph the widget could look at to use in an import closure.

Obviously GraphQL in general needs to be able to access the a-box graph closure.

I can workaround with data load in the short term - but it means I'd have to handle annotations in working copies and never commit them - or perhaps have an export for the working copies I can save before committing them.

Irene Polikoff

unread,
May 19, 2020, 11:18:52 PM5/19/20
to topbrai...@googlegroups.com
Please see below

On May 19, 2020, at 10:38 PM, Rob Atkinson <robatki...@gmail.com> wrote:


Use Case: 

I have a large data graph I wish to navigate and allow users to annotate so I expose in an EDG editor. 

What asset collection type are you using for this? Data graph?

Its not really "reference data" - but its generated by scripting so the body should only be changed by external processes - and the idea of annotation is to provide feedback on the content, manage additional information we need to attach etc.

If I include the data, the class tree works but it shows no instance - obviously only working on the current graph, not with imports.

Include where?

If you are not working with classes and properties, why are you using the class tree? Why are you expecting a class tree to show instances? It only displays classes You could bring in Instances panel, it would then show instances for a selected class, but it is not the most convenient view for working with instances. Typically, the Search table panel works better and you can use Asset Navigator to select a class of instances you want to focus on.

A screenshot would help.


How can we control the query scope to allow a-box import heirarchies to be handled - effectively i want to distinguish between a-box and t-box imports I think. 

I am having hard time understanding what you are talking about. Editors work over the entire graph closure. All information is shown - whether it is in the currently opened asset collection or in any of the imported graphs.

No problem adding a property to the graph the widget could look at to use in an import closure.

Obviously GraphQL in general needs to be able to access the a-box graph closure.

I can workaround with data load in the short term - but it means I'd have to handle annotations in working copies and never commit them - or perhaps have an export for the working copies I can save before committing them.

What do you mean by “annotations”? You don’t have to commit a working copy. All exports work for a working copy exactly the same as production copy.

--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/b2f26203-7dd7-4f86-a803-ae8f9aa31c5e%40googlegroups.com.

Rob Atkinson

unread,
May 20, 2020, 1:20:23 AM5/20/20
to TopBraid Suite Users

inline

On Wednesday, 20 May 2020 13:18:52 UTC+10, Irene Polikoff wrote:
Please see below

On May 19, 2020, at 10:38 PM, Rob Atkinson <robatki...@gmail.com> wrote:


Use Case: 

I have a large data graph I wish to navigate and allow users to annotate so I expose in an EDG editor. 

What asset collection type are you using for this? Data graph?

yes 
Its not really "reference data" - but its generated by scripting so the body should only be changed by external processes - and the idea of annotation is to provide feedback on the content, manage additional information we need to attach etc.

If I include the data, the class tree works but it shows no instance - obviously only working on the current graph, not with imports.

Include where?

via owl:imports in the data graph
 
If you are not working with classes and properties, why are you using the class tree?

the classes and instances layout is a reasonable way to navigate the instances by selecting classes of interest
 
Why are you expecting a class tree to show instances?
i dont
 
It only displays classes You could bring in Instances panel, it would then show instances for a selected class, but it is not the most convenient view for working with instances. 

it is the default that shows up
 
Typically, the Search table panel works better and you can use Asset Navigator to select a class of instances you want to focus on.


OK - thats useful to know the direction - but Search gives me nothing useful - it obviously needs more than just the ontology and the instances to work.

It shows the 'rootClass" and a free text box - the free text box is not active and the "asset type" dialog  (is this the "asset navigator" ? and the drop down shows "no options'

this is for an ontology that can be navigated via classes and instances - so it know about everything. 

Does the search perhaps rely on generating shapes for every class and property? If so, perhaps it should detect there are classes but no shapes and provide a prompt to generate shapes (but of course I'd never want to pollute the data with generated shapes - they belong in the T-box world!)

 
A screenshot would help.

How can we control the query scope to allow a-box import heirarchies to be handled - effectively i want to distinguish between a-box and t-box imports I think. 

I am having hard time understanding what you are talking about. Editors work over the entire graph closure. All information is shown - whether it is in the currently opened asset collection or in any of the imported graphs.

nope - it may be the design but if so its a bug -if i copy the same data into the graph it shows, if i owl:import the data it doesnt show. 

No problem adding a property to the graph the widget could look at to use in an import closure.

Obviously GraphQL in general needs to be able to access the a-box graph closure.

I can workaround with data load in the short term - but it means I'd have to handle annotations in working copies and never commit them - or perhaps have an export for the working copies I can save before committing them.

What do you mean by “annotations”? You don’t have to commit a working copy. All exports work for a working copy exactly the same as production copy.

Dont sweat on this - i mean any additional statements I want to manage about the entities in the data graph without editing the underlying data graph itself. suffice to say data is not editable, but I want to attach additional statements to the data in EDG workflows.  Its not relevant to the actual issue being discussed - except to make clear the 'doing something fundamentally different than managing different aspects of instance data in separate related graphs' is not the answer I am seeking - at least until I get what seems to be basic functions working - there are good reasons not to munge everything into a single graph.

--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbrai...@googlegroups.com.

Richard Cyganiak

unread,
May 20, 2020, 1:51:29 AM5/20/20
to topbraid-users list
On 20 May 2020, at 06:20, Rob Atkinson <robatki...@gmail.com> wrote:

the classes and instances layout is a reasonable way to navigate the instances by selecting classes of interest
  
It only displays classes You could bring in Instances panel, it would then show instances for a selected class, but it is not the most convenient view for working with instances. 

it is the default that shows up

What do you mean by this? The default layout for Data Graphs has the Search panel on the left and the Form panel on the right.

Does the search perhaps rely on generating shapes for every class and property?

The search panel is indeed shapes-based. It will show subclasses of the root class. They need to be typed as *both* Class and as NodeShape. New classes created in the EDG ontology editor have these types, but if your ontology came in via a different route it may not have them.

If the Form panel works for you as you expect, then you already have the required shapes for your properties.

Typically, the shapes would be in the Ontology that the Data Graph is based on.

When working with ontologies from a third party source, it is often reasonable to have a graph with the original ontology (perhaps kept as a static file in the workspace) and an EDG ontology that includes the original and adds shape statements to make it work with EDG.

I am having hard time understanding what you are talking about. Editors work over the entire graph closure. All information is shown - whether it is in the currently opened asset collection or in any of the imported graphs.

nope - it may be the design but if so its a bug -if i copy the same data into the graph it shows, if i owl:import the data it doesnt show. 

There is a “gear menu” in the top right corner of the Instances pane. It has a checkbox for “Return local results only”. Do you have that checked perchance?

there are good reasons not to munge everything into a single graph.

Agreed.

Richard

Irene Polikoff

unread,
May 20, 2020, 2:39:02 AM5/20/20
to topbrai...@googlegroups.com
The only thing to add to what Richard has written is that I am guessing that the ontology your data is based on is not in SHACL.

A process for creating a data graphs as follows:

1. In the create dialog you pick an ontology your data is based on. This ontology must be in SHACL. It says so in the Create dialog.

It must have at least one class that is also a NodeShape. And at least one of the NodeShape classes must be declared public.

2. After creation, go to the Manage tab and select your main class. 

If you do not have any public classes, you will see a message as shown below



--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/46E4D21C-F3B4-44AB-96C6-EF161E0C5237%40topquadrant.com.

Rob Atkinson

unread,
May 20, 2020, 4:25:44 AM5/20/20
to TopBraid Suite Users
OK search is _nearly_ working using the shapes once I save them and force them to be imported.

I had to set the graphql:publicClass and force-refresh the editor to pick up changes.  

If I put in no search term it finds everything - but anything in the free text box gets zero results.  Is there some other step required to make individual fields searchable ( this is the field! in the graphql schema ?)

Also i can find any indicator showing if search is local.   At any rate I will want to set a flag to force it to allow search on imported graphs - is there such a property that sets that as default? and/or one that forces it (disables option)

Richard Cyganiak

unread,
May 20, 2020, 4:50:31 AM5/20/20
to topbraid-users list
On 20 May 2020, at 09:25, Rob Atkinson <robatki...@gmail.com> wrote:

OK search is _nearly_ working using the shapes once I save them and force them to be imported.

I had to set the graphql:publicClass and force-refresh the editor to pick up changes.  

If I put in no search term it finds everything - but anything in the free text box gets zero results.  Is there some other step required to make individual fields searchable ( this is the field! in the graphql schema ?)

I'm not sure. I know that rdfs:label is indexed, and I believe (but not sure) that *all* literals are indexed.

Sometimes, Server Administration > Text Indices > Rebuild can resolve problems with the search. In theory that shouldn't be necessary.

Also i can find any indicator showing if search is local.   At any rate I will want to set a flag to force it to allow search on imported graphs - is there such a property that sets that as default? and/or one that forces it (disables option)

Like in the Instances panel, results from included graphs are shown by default, but that can be turned off via “Return local results only” in the search panel's “gear” menu (here for a Glossary but should be the same for Data Graphs):


Irene Polikoff

unread,
May 20, 2020, 9:34:45 AM5/20/20
to topbrai...@googlegroups.com

On May 20, 2020, at 4:50 AM, Richard Cyganiak <ric...@topquadrant.com> wrote:


On 20 May 2020, at 09:25, Rob Atkinson <robatki...@gmail.com> wrote:

OK search is _nearly_ working using the shapes once I save them and force them to be imported.

I had to set the graphql:publicClass and force-refresh the editor to pick up changes.  

If I put in no search term it finds everything - but anything in the free text box gets zero results.  Is there some other step required to make individual fields searchable ( this is the field! in the graphql schema ?)

I'm not sure. I know that rdfs:label is indexed, and I believe (but not sure) that *all* literals are indexed.

Yes, by default all text literals are indexed. This can be configured to be less.


Sometimes, Server Administration > Text Indices > Rebuild can resolve problems with the search. In theory that shouldn't be necessary.

In my experience, this step is only needed if asset collections are brought in through project import into TBC workspace e.g., if you download and install samples - as described in the instructions. But do try to see if it helps.

Also i can find any indicator showing if search is local.   At any rate I will want to set a flag to force it to allow search on imported graphs - is there such a property that sets that as default? and/or one that forces it (disables option)

Like in the Instances panel, results from included graphs are shown by default, but that can be turned off via “Return local results only” in the search panel's “gear” menu (here for a Glossary but should be the same for Data Graphs):

<local-results-only.png>


--
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.

Rob Atkinson

unread,
May 20, 2020, 10:09:04 PM5/20/20
to TopBraid Suite Users
Ok - a few layers of challenges here - gradually unpicking it:

Query in both the UI and GraphiQL return data if not filter text is specified.

The graphql schema shows properties that are queryable, but if i put any sort of filter against these in Graphql or the UI Free text search returns no results

A few UI glitches made this harder to track down,  The CSS for checkboxes is broken in 6.3.2.

here are the things i'd suggest reviewing
1) The CSS for the checkbox on the "use local results" is 
element.style {
    }
    wiring_rule….editor:156
    input[type="checkbox"] {
    1. position: absolute;
    2. /* left: -9999px; */
    }

    i have switched it off as above and lo and behold the checkbox appears

    2) hiding the indicator for whether local results are used under the gear icon seems suboptimal. 

    3) Need to know how to set the value of this setting by default when generating an Asset collection as a wrapper that includes the data of interest

    4) possibly want to have a special type of include that limits queries to included data, not all included shapes

    5) New button on instances only works with graphql:publicSchema defined 

    6) Graphql UI doesnt work (finds schema but get no results if data included via an import) - after setting graphql:publicSchema

    The functionality issues I need a solution for are #3 and #6

    To unsubscribe from this group and stop receiving emails from it, send an email to topbrai...@googlegroups.com.

    Rob Atkinson

    unread,
    May 20, 2020, 10:12:05 PM5/20/20
    to TopBraid Suite Users
    oops - forgot to update item #6 - thats doesnt work if a filter is included if data is included - but does work if data is copied into graph

    Rob Atkinson

    unread,
    May 21, 2020, 2:00:43 AM5/21/20
    to TopBraid Suite Users
    An update..

    tested this on a colleagues machine using the exact same version and build of TBC, and the search works fine out of the box. I created a fresh repo and reran the same configuration script and it still fails.

    The UI is different too - i have a gear icon and a broken CSS and he has a series of icons in the tab. (both using chrome).

    Will try on some other machines and compare.

    Wondering if it some cached stuff as i have used all previous versions.

    Rob Atkinson

    unread,
    May 21, 2020, 2:58:29 AM5/21/20
    to TopBraid Suite Users

    edg_ui.PNG

    and more details.

    Cleared browser cache - no fix.
    Tried on Edge instead of chrome - same

     Ran through load on a different machine and got the same results as on mine, with the same broken CSS

    snapshot is the broken styles and empty search results.

    will try a clean install of TBC

    Richard Cyganiak

    unread,
    May 21, 2020, 3:29:20 PM5/21/20
    to topbraid-users list


    > On 21 May 2020, at 07:00, Rob Atkinson <robatki...@gmail.com> wrote:
    >
    > tested this on a colleagues machine using the exact same version and build of TBC, and the search works fine out of the box. I created a fresh repo and reran the same configuration script and it still fails.

    Sounds like the search problem is related to the configuration script then?

    > The UI is different too - i have a gear icon and a broken CSS and he has a series of icons in the tab. (both using chrome).

    In the page header there is a button with three vertical dots. It opens a menu that has a checkbox for “Display Edit Actions as Icons”. Sounds like your colleague has that checked and you have it unchecked.

    I've looked through the EDG stylesheets for 6.3.2 and didn't find anything that looked like the style rule you reported. That means it could be a rule generated dynamically by some JS code. I'll ask colleagues about that possibility.

    It could be a local customisation in your workspace. Does your workspace include any .ui.ttlx files that do anything with ui:Script, ui:Style or ui:override, ui:headIncudes? Sorry for making such allegations. It's difficult to investigate these things.

    Richard

    Rob Atkinson

    unread,
    May 21, 2020, 8:28:58 PM5/21/20
    to TopBraid Suite Users


    On Friday, 22 May 2020 05:29:20 UTC+10, Richard Cyganiak wrote:


    > On 21 May 2020, at 07:00, Rob Atkinson <robatki...@gmail.com> wrote:
    >
    > tested this on a colleagues machine using the exact same version and build of TBC, and the search works fine out of the box. I created a fresh repo and reran the same configuration script and it still fails.

    Sounds like the search problem is related to the configuration script then?

    unlikely to be directly causing it as the same script is used in all three cases, and one case finds content and the other two dont - but I expect there is some critical piece of metadata missing, and some local config - like the selection of icon choices - that affects the UI behaviour - so its narrowing down what it may be then identify the mechanisms to enforce a necessary configuratin.
     
    > The UI is different too - i have a gear icon and a broken CSS and he has a series of icons in the tab. (both using chrome).

    In the page header there is a button with three vertical dots. It opens a menu that has a checkbox for “Display Edit Actions as Icons”. Sounds like your colleague has that checked and you have it unchecked.

    OK - thats cool - I hadnt discovered that (it would make sense that this would be available as some clue on the gear icon itself - such as a hover  - or maybe a last option in the drop down to say "show settings as icons" - too many places to look

    I've looked through the EDG stylesheets for 6.3.2 and didn't find anything that looked like the style rule you reported. That means it could be a rule generated dynamically by some JS code. I'll ask colleagues about that possibility.

    It could be a local customisation in your workspace. Does your workspace include any .ui.ttlx files that do anything with ui:Script, ui:Style or ui:override, ui:headIncudes? Sorry for making such allegations. It's difficult to investigate these things.
    No worries- fair enough!    No - (I have overriden this in a custom SWP _beacuse the style was broken_ and I needed to access the checkbox. I did a brute force text search and the only style that looked like this is

    .chosen-container-single.chosen-container-single-nosearch .chosen-search {
      position: absolute;
      left: -9999px;
    }

    in TopBraid/SWA/assets.www/lib/chosen/chosen.css

    Looking at the styles in chrome dev tools - it doesnt claim this style came from any particular CSS element - so I think this means it was injected programmatically by some JS - and my guess is that is what the "chosen" CSS is doing.

    certairnly I've never touched these styles - only custom CSS is for customised logos and icons.




    Richard

    Rob Atkinson

    unread,
    May 25, 2020, 11:43:51 PM5/25/20
    to TopBraid Suite Users
    OK - solved the style issue - I wasnt looking wide enough and a colleagues code had polluted the styles in a mod to the main menu - so at leas the checkboxes are back and I've learned how to force scope for CSS - another layer of contract UI mods have to adhere to. 

    Richard Cyganiak

    unread,
    May 26, 2020, 1:32:09 AM5/26/20
    to topbraid-users list

    On 26 May 2020, at 04:43, Rob Atkinson <robatki...@gmail.com> wrote:

    OK - solved the style issue - I wasnt looking wide enough and a colleagues code had polluted the styles in a mod to the main menu - so at leas the checkboxes are back and I've learned how to force scope for CSS - another layer of contract UI mods have to adhere to. 

    Ok, thanks for letting us know.

    Richard

    Rob Atkinson

    unread,
    May 26, 2020, 3:31:28 AM5/26/20
    to TopBraid Suite Users

    So I still need to work out why in some cases we can search included content, and other we cant.

    On the same machine, same initialisation script, same version of EDG we have workspaces where it works and where it doesnt - an we cant work out what we did to the working cases or repeat it - they were ones where experimentation with Lucene was happening though.

     We have checked and they definitely include (not hold copies of)  the data, and if we import the data instead of including then all searches work as expected - so perhaps there is something influencing Lucene's indexing of content in the included graph for local search. It seems reasonable we might need to trigger something on data setup, but cant see what it is.

    Holger Knublauch

    unread,
    May 26, 2020, 3:51:33 AM5/26/20
    to topbrai...@googlegroups.com


    On 26/05/2020 17:31, Rob Atkinson wrote:

    So I still need to work out why in some cases we can search included content, and other we cant.

    On the same machine, same initialisation script, same version of EDG we have workspaces where it works and where it doesnt - an we cant work out what we did to the working cases or repeat it - they were ones where experimentation with Lucene was happening though.

     We have checked and they definitely include (not hold copies of)  the data, and if we import the data instead of including then all searches work as expected - so perhaps there is something influencing Lucene's indexing of content in the included graph for local search. It seems reasonable we might need to trigger something on data setup, but cant see what it is.

    Our Lucene index will not be informed if imported graphs change - it will only incrementally update after changes to editable teamwork graphs from asset collections. Did you try out manual rebuilds of the Lucene-text index at the server admin page?

    Holger




    On Tuesday, 26 May 2020 15:32:09 UTC+10, Richard Cyganiak wrote:

    On 26 May 2020, at 04:43, Rob Atkinson <robatki...@gmail.com> wrote:

    OK - solved the style issue - I wasnt looking wide enough and a colleagues code had polluted the styles in a mod to the main menu - so at leas the checkboxes are back and I've learned how to force scope for CSS - another layer of contract UI mods have to adhere to. 

    Ok, thanks for letting us know.

    Richard
    --
    You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.
    To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/8d94380c-2ef9-4e91-9798-95542fd07a96%40googlegroups.com.

    Rob Atkinson

    unread,
    May 26, 2020, 4:34:28 AM5/26/20
    to topbrai...@googlegroups.com
    as usual you are on the money Holger

    it wasnt the luce index - it was the text indices - I rebuilt these from the server admin and joy.

    So how can one trigger these in a script, ideally just for a specific graph?



    To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/6df3c821-6f87-c054-89e6-ec5823e1f252%40topquadrant.com.

    Holger Knublauch

    unread,
    May 26, 2020, 5:22:43 AM5/26/20
    to topbrai...@googlegroups.com

    See \server.topbraidlive.org\system-applications\tbladmin\tbladmin.ui.ttlx: tbladmin:TextIndicesPage:

    <textindex:rebuild arg:id="teamwork"/>

    There is no way to only rebuild individual graphs.

    Holger

    Rob Atkinson

    unread,
    May 26, 2020, 9:26:16 PM5/26/20
    to topbrai...@googlegroups.com
    Thanks Holger

    so does that mean that if you make a single edit to content of a graph it triggers a re-index of the whole deployment? Or you need to manually intervene regularly to update indices?

    If not, is there a roundabout way to trigger a single graph by inserting some content temporarily?




    To view this discussion on the web visit https://groups.google.com/d/msgid/topbraid-users/da4355e6-e837-a6b6-88f5-faf13303a72d%40topquadrant.com.

    Holger Knublauch

    unread,
    May 26, 2020, 11:02:43 PM5/26/20
    to topbrai...@googlegroups.com


    On 27/05/2020 11:26, Rob Atkinson wrote:
    Thanks Holger

    so does that mean that if you make a single edit to content of a graph it triggers a re-index of the whole deployment? Or you need to manually intervene regularly to update indices?
    It just updates the individual triples (of course).


    If not, is there a roundabout way to trigger a single graph by inserting some content temporarily?

    No, if you did that then it would only apply the delta of what you have inserted.

    Holger


    Reply all
    Reply to author
    Forward
    0 new messages