Chunked transfer encoding in Play! framework

[unkx80]

Hi,

Many thanks for creating and maintaining the Play! framework. It
simplifies the development of web applications quite a bit.

However, it seems that trying to stream data is unusually difficult,
particularly binary data. In some cases, I am interested to stream
large amounts of data. In other cases, I am interested in streaming
live data. Either way, it is not practical to write the data to a file
and then render it. Therefore, I am looking into chunked transfer
encoding. However, I ran into some problems trying to stream data, and
documentation for this area appears to be almost non-existent.

public class Application extends Controller {
public static void test1() {
response.setContentTypeIfNotSet("text/plain");
response.writeChunk("a");
response.writeChunk("b");
response.writeChunk("c");
}
public static void test2() {
response.setContentTypeIfNotSet("text/plain");
byte[] bytes = "abc".getBytes();
response.writeChunk(bytes);
}
}

For both test1 and test2, I was expecting the following output:

abc

However, I got this instead for test1:

a
b
c

And for test2, I got this:

[B@129552

For both cases, it appears that there is no support for sending out
binary data in chunks. In particular, the first case seems to
introduce extra CRLF bytes, which breaks certain kinds of string data
as well.

I would like advice on whether I am streaming the data correctly, or
whether there will be better support for the chunked transfer encoding
in future versions of Play! framework. In addition, I would like to
suggest that code examples for streaming be included in the Play!
documentation. For your information, I am running Play! framework
1.2.2 as a standalone web server.

Thanks in advance.

Regards,
Yee Fan TAN

[unkx80]

Ticket 920 (http://play.lighthouseapp.com/projects/57987/tickets/920-
response-streaming-chunked-encoding-bug) pointed me to the location to
fix these two issues. Hence, I have created a ticket (http://
play.lighthouseapp.com/projects/57987/tickets/1043-chunked-transfer-
encoding-issues) and a pull request (https://github.com/playframework/
play/pull/312). My pull request is intended to supersede ticket 920.

I am quite new to this process, so I hope I have followed the
contributor guide correctly. Please consider my pull request. Many
thanks!

Regards,
Yee Fan

[Nicolas Leroux]

Thank you very much. We will review the pull request as soon as possible.

Nicolas

> --
> You received this message because you are subscribed to the Google Groups "play-framework" group.
> To post to this group, send email to play-fr...@googlegroups.com.
> To unsubscribe from this group, send email to play-framewor...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/play-framework?hl=en.
>

[unkx80]

Hi Nicholas,

On a related note, I am wondering whether there is a way to know that
the client has closed the HTTP connection in the middle of sending a
chunked response stream, so that the server can stop sending as well.
Controller code would be similar to the following:

public class Application extends Controller {

public static void index() {
response.setContentTypeIfNotSet("text/plain");
while (true) {
response.writeChunk("somedata");
boolean closed = // ???
if (closed)
break;
}
}
}

This kind of use case is for streaming of live data, where the server
keeps sending the data indefinitely until the client closes the HTTP
connection. An example is streaming of live video from a camera.

Thanks.

Regards,
Yee Fan

[Nicolas]

Hi,

Hmm, I think that writeChunck will then throw some sort of exception because it would not be able to write to the socket, no?

Nicolas

[unkx80]

No, at least not in Play 1.2.2. The index counter in the code below
keeps printing even though the browser is closed.

public static void index() {
response.setContentTypeIfNotSet("text/plain");

int index = 0;
while (true) {
response.writeChunk("data");
System.out.println(index);
try {
Thread.sleep(1000L);
}
catch (InterruptedException e) {
}
index++;
}
}

Let me see whether I can figure this out.

Regards,
Yee Fan

[unkx80]

Okay I figured it out. I pushed a commit into the branch with the pull
request.
https://github.com/playframework/play/pull/312

It turns out that in PlayHandler.java, the writeChunk() method in
LazyChunkedInput class did not check for the closed state, so I added
an exception for that case.

Regards,
Yee Fan

[unkx80]

Dear Play! developers,

As of version 1.2.4, I still see no fix to the chunked transfer encoding issues.
Any chance I see these issues fixed in 1.2.5?

Thanks,
Yee Fan