File upload issue

[Andrew Dixon]

Hi All,

Has anyone else had an intermittent issue with Lucee and file uploads? Basically what is happening is every now and again I'm getting an error from the upload code saying the form field the file is contained within doesn't exist within the form structure. The field is 100% in the form from which it is being submitted and all other environment variables suggest the form field should exist, for example:

Content-Type = multipart/form-data;

Content-Length = 500826

The form only has this one file, it is an image upload form, but for some reason, from time to time, Lucee seems to think it is not there and throws an exception. I've tried various things to isolate the issue without success, there is no correlation to browser, etc... either.

Anyone else seen this behaviour or have any idea what might be causing it? Could it be some sort of timeout on the browser side or something? It's all very odd.

Thanks.

Kind regards,

Andrew

about.me - mso - Lucee Association Member

[Michael van Leest]

I've seen that happening behind ngnix.

Can't remember exactly what fixed it, but it had to do something by setting some kind of buffer size.

Let me know if you're behind a ngnix server too and I'll check my notes on it later this weekend.

Mike

--
See Lucee at CFCamp Oct 22 & 23 2015 @ Munich Airport, Germany - Get your ticket NOW - http://www.cfcamp.org/
---
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/CAG1WijXPMhxf4OHACry4%3DdATYBxM7D7ABXEbSK2ibkujCP6ocw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
Sent from Gmail Mobile

[Brad Wood]

Can you try/catch it and put in some debugging that sends you the form scope and output of getHttpRequestData()?

Thanks!

~Brad

[Andrew Dixon]

Hi Michael,

No I'm using Apache. Thanks anyway, I will check to see if Apache has anything similar.

Kind regards,

Andrew

about.me - mso - Lucee Association Member

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/CAMaJE6t8htmnSOuAwZxM49CLjNX_s-vu5vvdDCOtnEKAWww0Tg%40mail.gmail.com.

[Andrew Dixon]

Hi Brad,

Good plan, going to add more data to the error capturing and see what else there is to see. Will report back.

Kind regards,

Andrew

about.me - mso - Lucee Association Member

--

See Lucee at CFCamp Oct 22 & 23 2015 @ Munich Airport, Germany - Get your ticket NOW - http://www.cfcamp.org/
---
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/0913d43c-83c2-4516-a678-66ce684361ab%40googlegroups.com.

[Andrew Dixon]

Hi,

Think I have found the issue. It is a Tomcat setting. In Tomcat there are 3 settings for the timeout for receiving a request from the client, here is a snippet of the docs.

connectionTimeout	The number of milliseconds this Connector will wait, after accepting a connection, for the request URI line to be presented. Use a value of -1 to indicate no (i.e. infinite) timeout. The default value is 60000 (i.e. 60 seconds) but note that the standard server.xml that ships with Tomcat sets this to 20000 (i.e. 20 seconds). Unless disableUploadTimeout is set to false, this timeout will also be used when reading the request body (if any).
connectionUploadTimeout	Specifies the timeout, in milliseconds, to use while a data upload is in progress. This only takes effect if disableUploadTimeout is set to false.
disableUploadTimeout	This flag allows the servlet container to use a different, usually longer connection timeout during data upload. If not specified, this attribute is set to true which disables this longer timeout.

With the installer that Railo came with back when this server was install (not check the Lucee installer yet), the AJP connector port in the configuration file (server.xml) doesn't specify any of these values meaning it uses the default, which is 60 seconds. So if the upload takes longer than 60 seconds it times out, however it appears the script in Lucee keeps running, meaning when it gets to the part of the script that is looking for the form field data, it is not there, hence the error. This also explains why the issue was so intermittent.

Not sure if this should be considered a bug in Lucee as surely it should not process the request if it doesn't receive all the data from the client, but then surely Tomcat should not have passed it to Lucee in the first place, so maybe a bug in Tomcat instead?!?!?!?!?!?

I'm going to try adding:

connectionUploadTimeout="300000" disableUploadTimeout="false"

To the AJP connector configuration line to see if that helps with my issue, which I'm thinking it will, but it requires a Tomcat restart to change them so will need to wait until later to do that.

Ref: https://tomcat.apache.org/tomcat-7.0-doc/config/http.html

Kind regards,

Andrew

about.me - mso - Lucee Association Member

[Nando Breiter]

Andrew,

Once you have worked this out conclusively, I think it deserves a blog post. Seems to be a very important detail to be aware of!

Aria Media Sagl
Via Rompada 40
6987 Caslano
Switzerland

+41 (0)91 600 9601
+41 (0)76 303 4477 cell
skype: ariamedia

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/CAG1WijV2R-9mNcR26wFZzNDJ3%2BUvuNDNdjy%2BOHoMyp%3DAS_GNCg%40mail.gmail.com.

[Andrew Dixon]

Yep, good idea... will sort something once I have a final answer...

Kind regards,

Andrew

about.me - mso - Lucee Association Member

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/CAGHrs%3D_uhm6W65MVfsky6uKtfW3OfWLsCS5j%2BDERC%3DUtZhV_eQ%40mail.gmail.com.

[Andrew Dixon]

So it turns out it wasn't actually a Lucee or Tomcat issue at all, it was an AWS Elastic Load Balancer (ELB) issue. The ELB has a default "Idle Timeout" of 60 seconds which was, after 60 seconds terminating the connection between the client and the server, so if the image had not been uploaded from the client in the 60 seconds it would terminate the connection but the script on the server side appeared to continue anyway. Increasing the "Idle Timeout" timeout setting on the ELB has fixed the issue completely, no more occurrences since increasing it. The Tomcat settings appear to have been a red herring after all.

Kind regards,

Andrew

about.me - mso - Lucee Association Member