URLDecoder timeout for very large POST requests

[mar...@cubicstate.com]

Hi,

I am experiencing a problem with inbound emails from Mandrill that contain large attachments. 

Mandrill is posting a large URLEncoded JSON string - the last one that failed was around 15MB.

The error that I receive (see stack trace below) seems to suggest that Lucee is timing out after 10 seconds during the decoding of this string - before it even gets to run any of our code where we have set a request timeout of two minutes.

I am guessing that this is where Lucee is parsing the 'form' struct?

Can anybody think of a way around this problem or how to improve performance here??

Your hopefully,

Martin

request ((fail to retrieve

path:java.lang.NullPointerException:null):49) has run into a timeout (10

seconds) and has been stopped. 

        StackTrace 

            string

request ((fail to retrieve path:java.lang.NullPointerException:null):49) has

run into a timeout (10 seconds) and has been stopped.

at

lucee.commons.net.URLDecoder.decode(URLDecoder.java:65):65

at

lucee.runtime.type.scope.ScopeSupport.fillDecoded(ScopeSupport.java:179):179

at

lucee.runtime.type.scope.FormImpl.initializeUrlEncodedOrTextPlain(FormImpl.java:261):261

at

lucee.runtime.type.scope.FormImpl.initialize(FormImpl.java:134):134

at

lucee.runtime.PageContextImpl.formScope(PageContextImpl.java:1172):1172

at

lucee.runtime.type.scope.UndefinedImpl.reinitialize(UndefinedImpl.java:595):595

at

lucee.runtime.type.scope.UndefinedImpl.initialize(UndefinedImpl.java:574):574

at

lucee.runtime.PageContextImpl.initialize(PageContextImpl.java:546):546

at

lucee.runtime.CFMLFactoryImpl.getPageContextImpl(CFMLFactoryImpl.java:161):161

at

lucee.runtime.CFMLFactoryImpl.getLuceePageContext(CFMLFactoryImpl.java:140):140

at

lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:435):435

at

lucee.loader.servlet.CFMLServlet.service(Unknown Source):-1

at

javax.servlet.http.HttpServlet.service(HttpServlet.java:727):727

at

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303):303

at

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208):208

at

org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52):52

at

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241):241

at

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208):208

at

com.seefusion.Filter.doFilter(Filter.java:49):49

at

com.seefusion.SeeFusion.doFilter(SeeFusion.java:1500):1500

at

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241):241

at

org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208):208

at

org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220):220

at

org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122):122

at

org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504):504

at

org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170):170

at

org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103):103

at

org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116):116

at

org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421):421

at

org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190):190

at

org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611):611

at

org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314):314

at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145):1145

at

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615):615

at

org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61):61

at

java.lang.Thread.run(Thread.java:745):745

 

        TagContext Array

[Michael van Leest]

Isn't there also a setting in tomcat and in the lucee config for max upload/form size? Not sure if that is related to your issue, but might be worth looking at.

Mike

--
Love Lucee? Become a supporter and be part of the Lucee project today! - http://lucee.org/supporters/become-a-supporter.html
---
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/a09ac603-c09d-4b41-aa9e-1f1f2fac5a26%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Michael van Leest

[mar...@cubicstate.com]

Hi Mike,

Thanks for the suggestion, however, I do not think that it is related to this. We are running in Windows using IIS8 so I think that IIS takes care of the file size restrictions?? I may be wrong there, but we have already setup the server to accept file uploads of up to 50MB.

It looks like this is simply Lucee timing out when trying to decode the large JSON string, but the timeout occurs at a point before we can control the request in our code - although I am guessing that the 10 seconds is taken from the default request timeout set in the Lucee admin.

I have done a quick test in Lucee to read the string in question, decode it and write it back to disk, that shows each of these tasks takes around a second for a 15MB file leaving me a bit confused by the timeout. Could it be that it is taking more than 10 seconds to actually download/upload the data from Mandrill to our server once the Lucee request has been started?

I am guessing that the only solution may be to increase the default request timeout, but I really wanted to avoid that if possible. I will try it at 20 seconds.

I have just noticed that you can turn on the RequestTimout URL parameter... I could maybe add that to the webhook URL, but I guess there is then a risk of something abusing that somehow?

Martin