Preload data error

76 views
Skip to first unread message

Lotus Ong

unread,
Feb 25, 2021, 9:29:47 AM2/25/21
to vocben...@googlegroups.com
Hi,

When I preload a big local file into a project, it showed the following errors

The preloaded data size exceeds the profiler treshold: 1048576 bytes

May I know where I can increase the profiler threshold size ?

Thanks

Manuel Fiorelli

unread,
Feb 25, 2021, 10:07:43 AM2/25/21
to Lotus Ong, vocbench-user
Hi Lotus,

the administrator can increase the threshold in the administration page (click on the avatar icon in top-right corner), specifically in the tab "Configuration"

Regards
Manuel


--
You received this message because you are subscribed to the Google Groups "vocbench-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vocbench-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vocbench-user/CAG5Fr3JKZec8T8YcUFHFotgZUEVGqAyt79nEYT1gjc4M%2BJUK7g%40mail.gmail.com.


--
Manuel Fiorelli

Lotus Ong

unread,
Feb 25, 2021, 8:42:54 PM2/25/21
to vocbench-user
Hi Manuel,

thanks for the info.
I increased the threshold size. But I hit another error "Server Error (status: 500)"
The upload file size is about 936 MB.
Most likely the server resource couldn't cater to such big data file.

Armando Stellato

unread,
Feb 26, 2021, 5:50:31 AM2/26/21
to Lotus Ong, vocbench-user

Hi Lotus,

 

this is pretty much normal. While the data loaded within a dataset through ordinary procedures (once the project is created) is stored directly in the assigned storage space (which depends on the chosen triplestore and repository technology, but usually will be mass memory) preloaded data is always fetched in memory, just for conducting some preliminary analysis that simplifies the configuration for the user (e.g. finding the baseuri, analyzing the semantic model, lexical model etc..). The data will be then loaded into the assigned repository.

 

So, if you are trying to load 1Gb, you need to have assigned 1Gb (and more) to the java process starting VocBench. While this can be technically done (if you have memory to use, it’s not bad, I personally assign 2Gb to it) I would personally not recommend to preload 1Gb, it would do the contrary of what you are trying to achieve: saving time, as it will be loaded twice, first with the preload and then with the load, which is more than the time spared in filling those fields manually.

 

To avoid possible misunderstandings: this consideration is only related to the preload. In the more general terms of scalability, you can easily load even much larger resources. Pls give a look at this: http://vocbench.uniroma2.it/doc/sys/#data_management for more details.

 

King Regards,

 

Armando

 

 

Lotus Ong

unread,
Feb 28, 2021, 9:55:44 AM2/28/21
to vocbench-user
Hi Armando,


JAVA_MAX_MEM=2048M
JAVA_MIN_MEM=256M

If we used the RDF4J , what are the configuration?

Armando Stellato

unread,
Mar 1, 2021, 9:31:13 AM3/1/21
to Lotus Ong, vocbench-user

Dear Lotus,

 

you need to configure environment variables for Java. The procedure is described here:

 

https://www.java.com/en/download/help/path.html

 

As an alternative, you can edit the karaf file that you use to run VocBench. E.g. in karaf.bat you can find those variables in rows 41 and 44.

 

For what concerns the RDF4J vs GDB, since you were asking about the preload, it doesn’t change. The preload is done on a RDF4J in-memory repository managed by VocBench.

 

Kind regards,

--
You received this message because you are subscribed to the Google Groups "vocbench-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vocbench-use...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "vocbench-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vocbench-use...@googlegroups.com.

Reply all
Reply to author
Forward
0 new messages