File Upload Questions (Related to Larger Uploads Crashing Lucee)

[Jamie Jackson]

Hi Folks,

I'm having some problems where larger files are crashing my Lucee (4.5.1.023) server.

First question:

If I give fileUpload(All) a file in the 0-50MB range, it flushes all at once to the temp directory (full-sized file appears); however, if I give it a file greater than about 55MB, it flushes to the temp directory incrementally (file grows until it's full-sized).

Can anyone explain this threshold/behavior?

I looked at the guts of fileUploadAll(), but I didn't spot anything.

Thanks,

Jamie

[Jamie Jackson]

After doing some step-debugging in the Lucee core, I see that (on Windows, at least), there is no difference in behavior between Lucee (via, FileOutputStream) writing a large vs. small file. Both seem to write incrementally on each iteration of out.write().

So, I'm not sure why I saw the behavior I'd previously seen/described (on Linux), but I wasn't as scientific with the previous experiment (since I can't easily step debug on the server), and have to watch files/directories change full-speed.

Back to the drawing board, I guess.

In the meantime, please let me know if you've had any experience with large(r) files crashing your Railo/Lucee server.

Thanks,

Jamie

[Robert Munn]

I have had serious issues with large file uploads crashing servers in the past, so much so that in a new project I wrote an AJAX-based file uploader that splits uploads into 1MB increments, uploads them separately and then re-assembles the files on the other end. If you are interested in my solution, I am going to put it into an open source project soon. I need to add broken upload resume and some other features before I open source it.

There is at least one other solution like mine, PLUpload, though I think you need to buy a commercial license for production. Ben Nadel has written about it in the past.

Robert

-- 
See Lucee at CFCamp Oct 22 & 23 2015 @ Munich Airport, Germany - Get your ticket NOW - http://www.cfcamp.org/
--- 
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/3f2cae40-d105-4a68-a2c3-9bfb00a0e340%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[mail...@gmail.com]

I have also noticed this in the passed (on railo), firstly you need to max your jvm memory I have mine set at 2GB but still never managed files larger than ~50mb, it causes outofmemoryexeption's so it seems the limitation is between java and lucee since it tries to loas the entire file in memory.

This should be done using a filestream instead but I have never dug into it.

Regards

GX

[Andrew Dixon]

Can you raise a bug for this on JIRA please:

https://luceeserver.atlassian.net/secure/Dashboard.jspa

Thanks.

Kind regards,

Andrew

about.me - mso - Lucee Association Member

--

See Lucee at CFCamp Oct 22 & 23 2015 @ Munich Airport, Germany - Get your ticket NOW - http://www.cfcamp.org/
---
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.

[Jamie Jackson]

Here's the ticket: https://luceeserver.atlassian.net/browse/LDEV-548

[Jamie Jackson]

Thanks, Robert. Were you ever able to track down what the ultimate problem was? I started a ticket, but it isn't a very clean bug report: https://luceeserver.atlassian.net/browse/LDEV-548

I'll keep an eye on your uploader if nothing comes of the ticket.

[Jamie Jackson]

I'm back to looking at this flushing behavior on Linux. I seem to have been right, on Linux, files smaller than the magic threshold (~55MB) write to the temp directory all at once, while larger files flush incrementally, from the very beginning.

So, to sum up my questions. (I've got a new one, too.)

Flushing

1. Why/how do files get written differently to the temp directory, depending on their size, on Linux JVMs? (Incrementally for files greater than ~55MB, and all at once for smaller files.)

Transfer Rate

2. Why is a cURL upload 2.5x faster than a real form upload from a browser? https://gist.github.com/jamiejackson/3963d2bf293a46742b18 (Using Chrome, FWIW.)

Thanks,

Jamie

[Alex Skinner]

Hi Guys,

in the mean while check this out

https://gist.github.com/denuno/5864210#file-gistfile1-cfm

I will see if we can use this as the basis of the fix

Thanks

Alex

--

See Lucee at CFCamp Oct 22 & 23 2015 @ Munich Airport, Germany - Get your ticket NOW - http://www.cfcamp.org/
---
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/ffa8600b-a023-4236-abb6-1f84b9e309e0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Alex Skinner
Managing Director

Pixl8 Interactive, 3 Tun Yard, Peardon Street, London SW8 3HT, United Kingdom T: +44 [0] 845 260 0726• W: www.pixl8.co.uk• E: in...@pixl8.co.uk

Follow us on: Facebook Twitter LinkedIn

CONFIDENTIAL AND PRIVILEGED - This e-mail and any attachment is intended solely for the addressee, is strictly confidential and may also be subject to legal, professional or other privilege or may be protected by work product immunity or other legal rules. If you are not the addressee please do not read, print, re-transmit, store or act in reliance on it or any attachments. Instead, please email it back to the sender and then immediately permanently delete it. Pixl8 Interactive Ltd Registered in England. Registered number: 04336501. Registered office: 8 Spur Road, Cosham, Portsmouth, Hampshire, PO6 3EB

[Terry Whitney]

What is the Web server your running...

I would bet its not a CF issue

On Wednesday, September 23, 2015 at 10:20:02 AM UTC-4, Jamie Jackson wrote:

[Jamie Jackson]

I'm running Apache HTTPD on the (Linux) servers.

When I was step-debugging locally, I was on Windows, using the debugging server which runs on Jetty.

Why, what are you thinking?

--

See Lucee at CFCamp Oct 22 & 23 2015 @ Munich Airport, Germany - Get your ticket NOW - http://www.cfcamp.org/
---
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/779f3186-2c76-4c03-abe4-b6e6db38b760%40googlegroups.com.

[Pete Freitag]

On Tue, Sep 29, 2015 at 9:14 AM, Jamie Jackson <jamie...@gmail.com> wrote:

I'm back to looking at this flushing behavior on Linux. I seem to have been right, on Linux, files smaller than the magic threshold (~55MB) write to the temp directory all at once, while larger files flush incrementally, from the very beginning.

So, to sum up my questions. (I've got a new one, too.)

Flushing

1. Why/how do files get written differently to the temp directory, depending on their size, on Linux JVMs? (Incrementally for files greater than ~55MB, and all at once for smaller files.)

It sounds to me like you might be hitting something in the servlet container that might be causing this, perhaps the <file-size-theshold> in web.xml is set, or needs to be set: 

https://tomcat.apache.org/tomcat-7.0-doc/servletapi/javax/servlet/annotation/MultipartConfig.html#fileSizeThreshold()

--

Pete Freitag
https://foundeo.com/ - ColdFusion Consulting & Products
http://hackmycf.com - CFML Server Security Scanner

[Terry Whitney]

Tomcat has a max upload set of 50 meg by default.

On Wednesday, September 23, 2015 at 10:20:02 AM UTC-4, Jamie Jackson wrote:

[Jamie Jackson]

Hi Terry. I don't see anything specified in web.xml (is that where I should be looking), and from what I can tell, the default is "-1L", which is crappily documented, but I assume translates to "unlimited."

Can you tell me how you know about this default?

[Jamie Jackson]

Hi Pete, according to (my interpretation of) the docs you sent, the default is unlimited (-1), and since I don't see any overrides in the stock Lucee web.xml, I don't think this is in play.

[Terry Whitney]

I have not seen where -1 ever is equal to unlimited in tomcat, but Meh

Make sure you restart the service after you make the change

/etc/init.d/tomcat(versionNumber) restart or on RHel 7 / CentOs 7

systemctl restart tomcat

webapps/manager/WEB-INF/web.xml

I would edit the the web.xml file to look like this below and then restart the java server (tomcat)

<multipart-config>

  

      <max-file-size>252428800</max-file-size>

      <max-request-size>252428800</max-request-size>

      <file-size-threshold>0<</file-size-threshold>

 

  </multipart-config> 

[Terry Whitney]

Just a random thought, 

Are you using the express version with jetty?

On Wednesday, September 23, 2015 at 10:20:02 AM UTC-4, Jamie Jackson wrote:

[Jochem van Dieten]

On Sep 30, 2015 22:05, "Terry Whitney" wrote:
> webapps/manager/WEB-INF/web.xml

That is unlikely to be the correct one: it will control the manager, but not Lucee. Depending on how you installed Tomcat the correct web.xml is either in /<catalina-root>/conf/, /etc/tomcat?/ or /<catalina-root>/webapps/ROOT/.

Jochem

[Robert Munn]

Based on the rest of the discussion, it looks like maybe the Tomcat config issue will fix it, but I never got that far because i decided that i would rather chunk large files into parts. A benefit of this strategy is broken file upload resume so if, for instance, your 700MB ISO upload fails midway through, you can resume without re-uploading the entire file.

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/b6bc302e-4e44-4080-867a-24593c5389e7%40googlegroups.com.

[Jamie Jackson]

Contrary to some folks' misconceptions, Tomcat doesn't have a default max file size--it's unlimited by default. (I confirmed on the Tomcat mailing list.)

Also, my questions still stand, but I'm still investigating--I'm not quite ready to give up on stock Lucee uploads (I really want to find out what's going on).

I'll keep you guys' uploaders in mind if I can't get to the bottom of this, though, so thanks for mentioning them.

Thanks,

Jamie

--

See Lucee at CFCamp Oct 22 & 23 2015 @ Munich Airport, Germany - Get your ticket NOW - http://www.cfcamp.org/
---
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/ffa8600b-a023-4236-abb6-1f84b9e309e0%40googlegroups.com.

[Jamie Jackson]

On Wed, Sep 30, 2015 at 3:06 PM, Terry Whitney <twhitn...@gmail.com> wrote:

Just a random thought, 

Are you using the express version with jetty?

When I was step debugging, I was using whichever solution comes with the Lucee  "RunAsJavaApp.java" (or whatever that's called), from the Lucee source code. I assume it was Express, but I'm not sure.

FWIW, normally I use the full installer version, which bundles Tomcat and configures Apache httpd, and that's what's running on the server that crashed.

Thanks,

Jamie

On Wednesday, September 23, 2015 at 10:20:02 AM UTC-4, Jamie Jackson wrote:

Hi Folks,

I'm having some problems where larger files are crashing my Lucee (4.5.1.023) server.

First question:

If I give fileUpload(All) a file in the 0-50MB range, it flushes all at once to the temp directory (full-sized file appears); however, if I give it a file greater than about 55MB, it flushes to the temp directory incrementally (file grows until it's full-sized).

Can anyone explain this threshold/behavior?

I looked at the guts of fileUploadAll(), but I didn't spot anything.

Thanks,

Jamie

--

See Lucee at CFCamp Oct 22 & 23 2015 @ Munich Airport, Germany - Get your ticket NOW - http://www.cfcamp.org/
---
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/920bed17-a9d1-47ce-9169-d59f0829c650%40googlegroups.com.

[JP]

Tomcat doesn't have a max upload file size, but it does have a file upload timeout. Go to server.xml and then your HTTP connector. You should add the following attributes:

connectionUploadTimeout="36000000" disableUploadTimeout="false"

This sets the upload timeout at 10 hrs. (set your timeout value based on what's reasonable for your app)

See: http://blog.somepixels.net/en/502-proxy-error-uploading-from-apache-mod_proxy-to-tomcat-7/

[JP]

Also, if you are running in IIS, there are a couple IIS settings you need to tweak:

1. configuration editor

1. system.web/httpRuntime
   https://msdn.microsoft.com/en-us/library/e1f13641(v=vs.85).aspx

1. executionTimeout: 10:00:00

2. maxRequestLength: 512000 (500 MB)

2. Request Filtering -> edit feature setting

1. Maximum allowed content length: 512000000

On Wednesday, September 23, 2015 at 8:20:02 AM UTC-6, Jamie Jackson wrote: