importing the ESCO turtle file

[Vito Spinelli]

Hello,

first of all, since it's the first time I'm posting here, many thanks for this wonderful software and for all the support you provide.

I've had success exporting and importing our SKOS ontology.

I'm now trying to import the ESCO turtle file (around 700Mb)  because I would like to run some sparql queries and possibly export some slices of it in RDF using CONSTRUCT queries.

However, I'm finding it really challenging to make the import work. I have been following the instructions (from both the documentation and from other questions in this forum) to increase the JAVA heap size and GC memory (directly in the karaf.bat and setenv.bat files).

In the karaf.bat file I added:

rem Check/Set up some easily accessible MIN/MAX params for GDB mem usage
if "%JAVA_MIN_MEM%" == "" (
    set JAVA_MIN_MEM=4048M
)
if "%JAVA_MAX_MEM%" == "" (
    set JAVA_MAX_MEM=24092M
)


rem Check/Set up some easily accessible MIN/MAX params for JVM mem usage
if "%JAVA_MIN_MEM%" == "" (
    set JAVA_MIN_MEM=4048M
)
if "%JAVA_MAX_MEM%" == "" (
    set JAVA_MAX_MEM=24092M
)

At the bottom of the setenv.bat file I added:

rem SET GDB_MIN_MEM
rem SET GDB_MAX_MEM

I tried both pre-loading the file and without pre-loading. My laptop has 32GB of ram so that should not be an issue.

After submitting the ESCO ttl file I just get an endless spinning wheel. No error is displayed. It's now been running for about an hour and nothing is happening... (I'm not sure if anything would happen at any point if I let it run).

Strangely, I managed to get the import work on another PC with 16GB of ram... but even on that PC the import is now failing when I tried it again (as I wanted to import a newer version of ESCO).

I assume this should not be that difficult and I must be doing something wrong.

I would appreciate any advice.

Best regards,

Vito Spinelli

[Vito Spinelli]

Just a small addendum... after an additional hour I stopped the process. Upon trying again this time I got a server error 500.

[Thomas Francart]

Are you using a triplestore as a backend ? I suggest using Graphdb as a triplestore for such large ontologies, and not use in-memory storage. 

You could try importing the *zipped* ttl directly from the triplestore management interface (not from VB). 

Thomas

--
You received this message because you are subscribed to the Google Groups "vocbench-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vocbench-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vocbench-user/5cea5f19-c748-4fae-a0d0-3427f790779dn%40googlegroups.com.

--

Thomas Francart - SPARNA
Web de données | Architecture de l'information | Accès aux connaissances
blog : blog.sparna.fr, site : sparna.fr, linkedin : fr.linkedin.com/in/thomasfrancart
tel :  +33 (0)6.71.11.25.97, skype : francartthomas

[Armando Stellato]

Dear Vito,

 

thanks for the appreciation :-)

 

there is one thing not clear from your msg. You jave added the setup for the memory to the karaf.bat file. Karaf.bat is for starting VocBench. However in one of the two rem lines you wrote “for GDB mem usage” so you don’t make any mention of GraphDB. Even more, you used the GDB specific env vars within, again, a file specific to karaf (in GDB there is a similar file called setvars, so I’m assuming you are still talking about a file in Karaf which, however, would be local to karaf and thus not affecting GraphDB)

 

A 700Mb file is totally manageable by VocBench (even larger ones) but, either you assign loot of memory or you work with an external triple store (better option, as suggested by Thomas in his other msg). As from your msg I’m not sure if you are using it or not, I suggest you revise this section:

https://vocbench.uniroma2.it/doc/sys/#separate_triple_store

which should make things clearer.

 

Pls let us know if, after a proper setup, you manage to load the file. I’d ask you for a link to the ESCO file for trying it out myself but most probably there’s just some misunderstanding in the use that will emerge by the above analysis.

 

Kind Regards,

 

Armando

 

 

 

 

 

 

 

--

[Vito Spinelli]

Dear Thomas, dear Armando,

many thanks for your quick replies.

Indeed, I think I got confused and mixed up some configuration settings. I decided to simplify things and start from scratch.

Using the last VB version, I tried the import once again without changing any settings. However, I got the same result.

Following your advice, I downloaded the suggested version of GraphDB (GDB 9.8.1) and followed the instructions in the link you sent. I copied the files for history and validation in the lib folder of GDB. Then I created a new project and configured the remote data store. I created the new project and I saw in the shell that VB connected with GDB. 

The import took some time, probably around 15-20 minutes, but was finally successful! 

Many thanks for your support!

Best regards,
Vito

[Armando Stellato]

Dear Vito,

 

Glad it worked. Happy modeling :-)

 

Armando

 

To view this discussion on the web visit https://groups.google.com/d/msgid/vocbench-user/157cf9d3-a761-432d-8e26-6593491213f2n%40googlegroups.com.