Model loading...taking forever

[Jack Hodges]

I added an import into a model I am constructing. This import is the FMA (anatomy ontology) and has tens of thousands of classes. It took a couple of minutes to add it. I later had to exit TBC and when I reopened the application the Progress Information dialog said the app was "Applying Superclass Inferences" for an hour (I have since changed the preferences to suppress this step). Then the Progress Information dialog told me it was "Checking Visibility" which it did for another hour. I could not find a preference to suppress this. Considering that it didn't take much time to initially import the models, why would it take so long to reload everything, and can I configure TBC to load faster? Btw, the models just finished loading and its been a couple of hours. Thank you anyone!

Jack

[Scott Henninger]

Jack; There are a number of models that fail to assert "root" classes, i.e. all classes that are subclasses of owl:Thing.  While technically correct, these are difficult to process, as the system has to find all of the class definitions that have no subClassOf definitions.  This is the "Applying Superclass Inferences" statement.

The way to speed this up is to assert these inferences, either in the model or another import.  

One way to easily assert the inferences is to choose Model > Open with SWA Default Application in Composer.  Again, this will take some time, but in the end it will make the assertions and you can save them so you do not have to do this in the future.

-- Scott

On May 28, 2014, at 3:32 PM, Jack Hodges <jhodg...@gmail.com> wrote:

I added an import into a model I am constructing. This import is the FMA (anatomy ontology) and has tens of thousands of classes. It took a couple of minutes to add it. I later had to exit TBC and when I reopened the application the Progress Information dialog said the app was "Applying Superclass Inferences" for an hour (I have since changed the preferences to suppress this step). Then the Progress Information dialog told me it was "Checking Visibility" which it did for another hour. I could not find a preference to suppress this. Considering that it didn't take much time to initially import the models, why would it take so long to reload everything, and can I configure TBC to load faster? Btw, the models just finished loading and its been a couple of hours. Thank you anyone!

Jack

--
-- You received this message because you are subscribed to the Google
Group "TopBraid Suite Users", the topics of which include Enterprise Vocabulary Network (EVN), TopBraid Composer, TopBraid Live, TopBraid Insight, SPARQLMotion, SPARQL Web Pages and SPIN.
To post to this group, send email to
topbrai...@googlegroups.com
To unsubscribe from this group, send email to
topbraid-user...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/topbraid-users?hl=en
---
You received this message because you are subscribed to the Google Groups "TopBraid Suite Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to topbraid-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jack Hodges]

Scott,

Thank you for the help!

I did as you suggested (a few times) and then closed and reopened the model. Here is the timeline (just finished):

2:15 pm: Opened the model

2:20 pm: Get the "Applying default superclass inferences to class..." message in the Progress Information dialog (in the end there were just over 90,000 classes)

2:21 pm: Get the "Checking visibility" message in the Progress Information dialog

2:23 pm: Get the "Showing model" message in the Progress Information dialog

2:38 pm: Get the "Opening editor" message in the Progress Information dialog

2:43 pm: Load complete

So there was (a lot of) improvement, but it is till taking almost 30 minutes to load. Is this the best I can expect or are there more tricks I can apply?

Thank you!

Jack

[Holger Knublauch]

I assume the remaining slowness is from the tree component, that may not be prepared to handle that many root classes. As the tree is rather useless, you may want to close it anyway. Unfortunately, TBC will always open up the class tree view when you open a new graph. So the following may work:

a) Assert an artificial root class ex:MyRoot that has all other roots as children

b) Make rdfs:Resource the root class of the tree and never open owl:Thing

Then close the Class Tree view.

(This assumes that you have set up enough memory for TBC and we are not just seeing out-of-memory slowness).

I will look next week what's causing this slowness.

Holger

[Holger Knublauch]

Hi Jack,

I have created an artificial set of 100k classes, all rdfs:subClassOf owl:Thing to confirm some of the slowness that you are seeing. I can confirm that you should switch the class tree to have rdfs:Resource as its root as this will prevent all subclasses of owl:Thing from appearing by default. You can reach this option from the drop down menu of the Classes view - uncheck owl:Thing as root class. That should allow you to open your file with reasonably speed and then close the (useless) class tree before it drags down the system.

With 100k classes, it takes about 1 minute to open owl:Thing and I can see in the Jave profiler that this is a general slowness of the Eclipse platform's tree component as only a fraction of this time is spent in TopBraid-specific code. But my question is why is your specific ontology taking 20 times as long. Did you activate human-readable labels (if yes please try switching that off)? What hardware are you using and do you have enough memory?

Thanks
Holger

[Jack Hodges]

I am using a Macbook Pro with 4 GB of memory on OS X 10.8.5. I do not see in Preferences or Help anything about human-readable labels.

Jack

[Jack Hodges]

I brought the workspace home and loaded it up onto my iMac (iCore 7, 8 GB), same version of OS X. I have dedicated 4 GB of memory to TBCME, and it still takes about 5 minutes to load. And any searches take a few minutes.

By the way, I also get those annoying 'stack unavailable' dialogs on startup and have been for a couple of years.

Jack

[Holger Knublauch]

On 6/3/2014 23:37, Jack Hodges wrote:
> I brought the workspace home and loaded it up onto my iMac (iCore 7, 8
> GB), same version of OS X. I have dedicated 4 GB of memory to TBCME,
> and it still takes about 5 minutes to load.

FWIW I have changed the TBC classes tree for version 4.5 so that it will
use the global preference on the maximum number of instances to display
(which defaults to 1000). This means that the tree should not cause any
further slowness and users can decide to close the tree at their
convenience.

> And any searches take a few minutes.

For other cases of slowness that you report, I would appreciate more
details - what do you mean with "any searches"? I can certainly imagine
some SPARQL queries to be very slow on such a large model, but which
other features are you using?

>
> By the way, I also get those annoying 'stack unavailable' dialogs on
> startup and have been for a couple of years.

I believe those will disappear with 4.5 too.

Holger

[Jack Hodges]

My apology, I meant class name searches in the navigator.

On my new Windows 7 laptop these searches take less than a minute, but are still noticeable.

I tried many configurations of memory and they have no impact on load times for my models on this new machine. They remain at 11 minutes. I am not sure why it was faster on the iMac but will time it again when I am home again (where the iMac is).

Jack

[Holger Knublauch]

On 6/6/2014 5:06, Jack Hodges wrote:
> My apology, I meant class name searches in the navigator.
>
> On my new Windows 7 laptop these searches take less than a minute, but
> are still noticeable.

Ok, auto-complete is indeed not optimized in TBC and the system may need
to walk through all 70k classes. I will record this as an enhancement
request.

> I tried many configurations of memory and they have no impact on load
> times for my models on this new machine. They remain at 11 minutes. I
> am not sure why it was faster on the iMac but will time it again when
> I am home again (where the iMac is).

Is the load time still 11 minutes if you have switched to TDB?

Also, I would like to drill down into your scenario a bit: could you
point me at the RDF files that you are playing with?

Thanks
Holger

[Jack Hodges]

Holger,

Here is the link to the major player in this exercise (the Foundational Model of Anatomy):

http://www.bioontology.org/wiki/index.php/FMAInOwl

I downloaded and load the owl file:

http://bioontology.org/projects/ontologies/fma/fmaOwlFullComponent_2_0.owl

but they say it can be referenced:

http://bioontology.org/projects/ontologies/fma

Jack

[Holger Knublauch]

Well, this (FMA) is a very unusual model - everything is an instance of some other instance (many levels deep) and the root of all that (dl:Anatomical_entity_template) is an instance of itself but is not even declared as a class! My previous speculations about the class tree were irrelevant, but I assumed you were using some kind of flat class structure used by many other bio ontologies.

Anyway, I have loaded that file (took two minutes on my 8 GB Windows 7 machine) and exported it to TDB. Loading the TDB into TBC takes about 2 seconds. A SPARQL query such as

SELECT ?type COUNT(?type)
WHERE {
    ?subject rdf:type ?type .
} GROUP BY ?type

also responded within a second or two. Auto-complete in the main search box in the tool bar is also responding very quickly for me (e.g. try dl:Liq + CTRL-Space) which takes one second on my machine.

The model has about 215k triples, which is not very large but certainly a challenge for a machine with little memory (unless a database such as TDB is used).

I am not really sure what else to do next, as I am not able to reproduce the severe slowness you report.

Thanks,
Holger

[Jack Hodges]

I must have missed something because to me they 'all' look like classes rather than all looking like instances.

Jack

[Jack Hodges]

Creating a TDB seems like a good idea but I cannot find the 'Add Files' button referenced in the help blurb. Unless it is called something else in the UI.

[Irene Polikoff]

Just open your model then do File -> Export -> Export/Merge RDF Graphs and pick TDB as the format.

[Jack Hodges]

I am a bit unsure of Holger's comment about the size of the FMA repository. When I did the export as you suggested, Irene, it says about 1.7 million triples. This is what I also heard from someone else. Anyway, the export seems to have worked.

Now the question is how to use this TDB of the FMA model/repository with my models. I can open the database, but cannot import it into my model. Do you have more documentation on how to use repositories in TBC? I dislike having to ask questions when I can answer them in documentation.

Jack

[Scott Henninger]

Jack; I haven't been following this too closely, but I assume you now have a .tdb connector file for your data.  If you want to import that data into any other model use the owl:imports, just like any other model.  The easiest is to drag/drop the .tdb file connector into the Imports view for the target model.

To do this manually, specify the base URI (the graph name) of the .tdb connector for import into the target files.

There are a few sections in Composer Help that you will find useful:
  TopBraid Composer > Importing Data Sources > Working with imported ontologies
  TopBraid Composer > User Interface Overview > Imports View
  TopBraid Composer > Importing Data Sources > Import external information > Add or Import External Triples

-- Scott

[Jack Hodges]

Hello Scott,

That is exactly what I did.  Oops, maybe I dragged the wrong file into the imports, because I just did it again and it worked as originally expected. Many apologies. Yes, this does speed things up. The access to items in the TDB is faster though the search for items in the class navigator is still taking several seconds. Thanks Scott, Irene, and Holger.

Jack