file size limit for tagged upload?

[Torsten Rohlfing]

Hi -

We are trying to upload some large tgz files (2.2GB) to our XNAT 1.6.1 server, but experience consistent failure.

Is there a maximum supported file size, and if there is, can it be increased? Unfortunately, the tgz file contains only one really large file, so uploading the archive in parts is not an option unfortunately.

Thanks!
  Torsten

[Torsten Rohlfing]

Also inspected the log files now and found this in turbine.log:

2013-01-07 14:59:10,270 [http-bio-8080-exec-569] ERROR org.apache.turbine.util.parser.DefaultParameterParser - File upload failed
org.apache.turbine.util.TurbineException: the request was rejected because it's size is unknown
        at org.apache.turbine.services.upload.TurbineUploadService.parseRequest(TurbineUploadService.java:210)
        at org.apache.turbine.services.upload.TurbineUpload.parseRequest(TurbineUpload.java:144)
        at org.apache.turbine.util.parser.DefaultParameterParser.setRequest(DefaultParameterParser.java:187)
        at org.apache.turbine.services.rundata.DefaultTurbineRunData.getParameters(DefaultTurbineRunData.java:398)

Might this explain?

And to clarify - this is related to tagged file uploads into an existing imaging session.

Thanks!

[Gabriele]

If it's the case that the request body is too large there used to be a maximum file and/or post size that you can set in WEB-INF\web.xml, something like:

<multipart-config>

  <!-- 52MB max -->

  <max-file-size>52428800</max-file-size>

  <max-request-size>52428800</max-request-size>

  <file-size-threshold>0</file-size-threshold>

</multipart-config>

Not sure if that's still the case. In TC 6.0, there you may want to look at maxPostSize as well.


-G

[Torsten Rohlfing]

Thanks -

So I found nothing in any of the web.xml files, including Tomcat's own. But I did add "maxPostSize=0" to all Connector entries in /etc/tomcat/server.xml.

This changed the turbine error message to:

2013-01-08 10:57:17,051 [http-bio-8080-exec-7] ERROR org.apache.turbine.util.parser.DefaultParameterParser - File upload failed
org.apache.turbine.util.TurbineException: the request was rejected because it's size exceeds allowed range
        at org.apache.turbine.services.upload.TurbineUploadService.parseRequest(TurbineUploadService.java:215)
        at org.apache.turbine.services.upload.TurbineUpload.parseRequest(TurbineUpload.java:144)

I did some digging and it seems I should change

services.UploadService.size.max=1048576000

in the Turbine properties file - alas, all that I can find are:

xnat/plugin-resources/cache/maven-turbine-plugin-1.2/plugin-resources/flavor/common/conf/TurbineResources.properties
xnat/deployments/xnat/conf/TurbineResources.properties
xnat/xnat/deployments/xnat/target/xnat/WEB-INF/conf/TurbineResources.properties

and these all seem to be generated from some template.

Now where would I change the size.max setting so it propagates into the generated files?

Thanks!
TR

[Torsten Rohlfing]

So... thanks to Rick's suggestion in a different thread, I was able to modify the Turbine maximum upload size via adding an "ant:replace" to maven.xml. I did not find anything of the sort Gabriele suggested in any of the web.xml files, though. I also put maxPostSize="0" in all the Connector items in the tomcat/server.xml, and put a 4GB max upload size in the TurbineResources.properties (confirmed that this is in the version deployed with the webapp),

Unfortunately, making the two aforementioned changes didn't fix the upload problem yet. Actually, it did exactly one time, on one of my two servers, but I never got it to work on the second one, and without me doing anything it stopped working again on the first also. Very strange.

For every upload attempt larger than about 1GB, I have the two following in my turbine.log:

2013-01-11 16:00:27,490 [http-bio-8080-exec-14] ERROR org.apache.turbine.util.parser.DefaultParameterParser - File upload failed

org.apache.turbine.util.TurbineException: the request was rejected because it's size is unknown
        at org.apache.turbine.services.upload.TurbineUploadService.parseRequest(TurbineUploadService.java:210)
        at org.apache.turbine.services.upload.TurbineUpload.parseRequest(TurbineUpload.java:144)

.... [shortened here]
2013-01-11 16:00:27,616 [http-bio-8080-exec-14] ERROR org.apache.turbine.Turbine - Turbine.handleException:
java.lang.Exception: Invalid submit value (POST on URL: http://localhost:8080/xnat/app/action/ExptFileUpload from /128.18.20.155 (40328) user: 0:0:0:0:0:0:0:1
Headers:
host: localhost:8080
user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20100101 Firefox/17.0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
accept-language: en-US,en;q=0.5
accept-encoding: gzip, deflate
referer: https://...
cookie: JSESSIONID=...; SERVER_RESPONSE_TIME=256.5; server_name=...; SESSION_TIMEOUT_TIME=1357941855463; SESSION_EXPIRATION_TIMEOUT=false; SESSION_EXPIRATION_TIME_DIALOG_DISPLAYING=false; WARNING_BAR=OPEN; SESSION_EXPIRATION_TIME="0,900000"; PHPSESSID=...; authchallenge=...
content-type: multipart/form-data; boundary=---------------------------14265898961818864561123961800
x-forwarded-for: 128.x...
x-forwarded-host: ...
x-forwarded-server: ...
connection: Keep-Alive
content-length: 3097243346

 Cookies:
JSESSIONID ... -1 null
SERVER_RESPONSE_TIME 256.5 -1 null
server_name ... -1 null
SESSION_TIMEOUT_TIME 1357941855463 -1 null
SESSION_EXPIRATION_TIMEOUT false -1 null
SESSION_EXPIRATION_TIME_DIALOG_DISPLAYING false -1 null
WARNING_BAR OPEN -1 null
SESSION_EXPIRATION_TIME 0,900000 -1 null
PHPSESSID ... -1 null
authchallenge ... -1 null
)

(I removed potentially confidential-looking stuff and replaced it with "...").

It seems to me that the key here is Turbine's claim that the request size is unknown, yet in the following error message, the content-length is properly set (and essentially the size of my test file).

If this makes any sense to anybody, I'd be very grateful for further suggestions what to try and where to look.

[Torsten Rohlfing]

Huh... I though I already posted this yesterday, but apparently I didn't.

Well, here we go - I figured out the underlying problem with large file uploads: apparently Turbine represents the size of a POST request as a signed 32bit int, which limits the size to 2GB, regardless of whatever one may set in any configuration files.

Bummer.

Thanks everyone for your help! (At least I was now able to increase the upload size from the default 1GB to 2GB; that might be just enough for our current application.)

Torsten

[Torsten Rohlfing]

By the way, Rick and all on the developing end:

Might you guys consider making the 2GB upper limit the default (via putting the ant:replace that you suggested into maven.xml)? The current 1GB limit seems a bit arbitrary anyway (in fact, I noticed it's not even a real GB - just the original default 1MB with three zeroes added ;))

Best,
  Torsten