File upload streaming and progress monitoring ...

[Marius]

HI there,

I know, there have been several other threads around the topic but I
just wanted to yield somethings here. In the past few days I've been
talking with Tim as he was having problems with his FileUpload widget.
Even with short polling the progress was not reported even though the
LiftSession.progressListener was properly called.

Today I started to look into it and here is what I've done:

1. Here is the markup:

<iframe id="upload_target" name="upload_target" src="#" style="width:
0;height:0;border:0px solid #fff;"></iframe>

<lift:Upload.render form="POST" multipart="true" id="upload_form"
target="upload_target">
<f:inputFile/>
<input type="submit" value="Upload"/>
</lift:Upload.render>

A simple snippet that uses the invisible iframe trick to upload
stuff in the background. The so called "Ajax" upload. (I had to add
the target attribute into the supported form snippet attributed but
that's ok)

2. In boot had something like:

LiftSession.onSetupSession = ((session: LiftSession) => {
session.progessListener = Full((chunk, total, index) => {
Thread.sleep(100)
println(index + " read " + chunk + " out of " + total + " " +
Thread.currentThread)
})
}) :: Nil

Just monitor the progress ... and to make it last a little longer
I'm sleeping for 100 milliseconds

3. I set the GC polling to 10 seconds. The think is that GC polling
would be just like the short polling for getting the progress status
back in the browser.

The behavior that I observed was that while I was uploading a 30 Mb +
file the polling requests timed out after 5 secconds per
LiftRules.ajaxPostTimeout. This means that our progress monitoring
ajax request would have timed out as well. On the server side the
execution got stuck until the upload was complete.
LiftServlet.handleAjax was not called until the upload terminates.

Initially I thought that this has to do to limited request channels
that browser have but the thing is that we have several places in Lift
code where we synchronize things. Such as LiftSession.getSession,
LiftSession.fixSessionTime etc. Acquiring these locks made the
progress polling to hang until the upload terminated. Of course
removing th synchronized block made the polling to work right during
the file upload. Now this was just for test ... I'm not suggesting to
remove the synchronized blocks as we'd loose consistency but this
looks to stay in the way of properly dealing with multiple parallel
request per same session like fileupload and progress monitoring.

It appears that there is either a thread starvation situation or
simply a lock that is held while upload is is progress.


Br's,
Marius

[marius d.]

A little more on this ...

Living the above synchronized blocks where they are but removing the
one from LiftSession.runParams made it work properly. Looks like this
one was holding the lock while the file upload progress was happening.
Dave I'm not really sure why in runParams the the toRun is obtained
from a synchronized block ... is it because we'rreading from
messageCallback there ? ..cause otherwise the entire expression is
about functions composition that does not change any state.


Br's,
Marius

[marius d.]

Just committed a fix ...

Br's,
Marius

[Timothy Perrett]

Marius,

This is awesome. Things are now working as expected - I knew this
wasn't a browser connection issue causing the problems.

The code in the widget package is pretty rubbish right now, but i just
had time to hack together a hardcoded sample to make sure that your
commit actually fix the problem... Im happy to report that it does
indeed now work exactly as required.

The next step is to work backwards and clean all the code up and fix
the cross-browser issues as right now it works in firefox and IE but
not in safari or chrome et al.

Cheers, Tim

[marius d.]

Great. Let me know if I can help with anything.

Br's,
Marius

[Viktor Klang]

Awesome work guys!

--
Viktor Klang

Java Specialist
Scala Loudmouth
Lift Committer