Big Import fails with the command ‘Load data’

73 views
Skip to first unread message

Stephane Schneider

unread,
Feb 27, 2025, 8:55:30 AM2/27/25
to vocbench-user
We have an import problem in Vocbench v12.2 and  v13.0.0. with  ‘Load data’ command in a new project.
We do not have any difficulty importing files smaller than 1 MB. As soon as this file size is exceeded, it is impossible to import.  A large file must be divided into several files < 1 MB in order to be imported into Vocbench.

It is a Docker installation. The disk spaces have been checked.

Have you encountered this kind of problem?

Kind Regards,

Stéphane Schneider
INIST/CNRS

stel...@uniroma2.it

unread,
Feb 27, 2025, 10:23:00 AM2/27/25
to Stephane Schneider, vocbench-user

Dear Stephane,

 

There are two places where limits can be set.

 

The general one for any upload is here:

https://bitbucket.org/art-uniroma2/semantic-turkey/src/master/distribution/src/distribution/config/application.yml

this is by default set to us to 1Gb,. So there should be no issue with that and unless you need very large files, there is generally no need to change it.

 

Through the interface, what can be changed is the “preload”. Are you using that one? Since you write “in a new project” I suspect you are using this feature (for which the file is by first analyzed and the loaded into the system upon confirmation).

 

For that, you need an administrator going on “Administration” à “Configuration” and then set the value for the second widget called “Preload file threshold”, which is set by default to 1Mb (other evidence leading me to suspect this is your case)

 

Kind Regards,

 

Armando

 

--
You received this message because you are subscribed to the Google Groups "vocbench-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vocbench-use...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/vocbench-user/0e54ea48-7509-46f7-b382-d78c55938b7dn%40googlegroups.com.

Stephane Schneider

unread,
Mar 3, 2025, 2:18:09 AM3/3/25
to vocbench-user
Dear Armando

Thank you for your advice. We have tried to increase the threshold but the file is still not imported.
The file is 1.5MB and the limit is 100MB.
I don't know if there is another explanation.
I can send it to you for a test.

Thanks again

Stéphane

Thomas Francart

unread,
Mar 3, 2025, 2:46:00 AM3/3/25
to Stephane Schneider, vocbench-user
Hello

An hypothesis is that is limit is set by the webserver in which VocBench is deployed, or by a proxy server in front of VocBench. Are you going through a proxy ?

Thomas



--

Thomas Francart - SPARNA
linked data | domain ontologies | knowledge graphs
blog :
blog.sparna.fr, site : sparna.fr, linkedin : fr.linkedin.com/in/thomasfrancart
tel : 
 +33 (0)6.71.11.25.97

stel...@uniroma2.it

unread,
Mar 3, 2025, 3:39:05 AM3/3/25
to Thomas Francart, Stephane Schneider, vocbench-user

Thomas is right and it could be something from outside (e.g. a reverse proxy).

 

One way to (almost certainly) rule out the effect of external factors is to use some other feature of VB.

 

For instance: Stephane, have you tried using “load data”?

 

Let’s proceed with order, to avoid any misunderstanding: from your answer I didn’t get for sure a couple of facts.

Given the premises in my email, I can only guess that you didn’t touch the one on the Spring Boot server (because it should be already properly set) and you were talking about the “Preload profiler threshold” in the admin configuration.

Also, you did not confirm that you were using the “preload” function, it’s that one we are talking about right?

My apologies for asking something possibly trivial, but misunderstanding is always ‘round the corner :-)

 

So, in the hypothesis that you already set everything and that you were talking about “preload”, let’s go with load data. So:

 

  1. Just create a project, set its baseuri, namespace, etc.. manually
  2. After the project has been created, go on the “Global Data Management” menu, and select “load data” (it’s pretty simple but, to clarify all aspects of it: https://vocbench.uniroma2.it/doc/user/global_data_management.jsf#load_data )

 

If it works with files larger than 1Mb, then it *shouldn’t* be something external, rather a problem with the pre-load (then we need to confirm which threshold was adjusted). If it still has problem with 1Mb, then it’s pretty sure that the problem is not in VB.

Stephane Schneider

unread,
Mar 19, 2025, 4:02:12 AM3/19/25
to vocbench-user
Dear Armando and Thomas

We've found the culprit. It's an nginx !!
So you were right.
Thanks for your help

Kind regards,

Stephane


Reply all
Reply to author
Forward
0 new messages