Gzip compression/decompression in Netty

[Guillaume Bort]

Hi all,

I just noticed that we still have this gzip compression enabled in Netty:

https://github.com/playframework/Play20/blame/master/framework/src/play/src/main/scala/play/core/server/NettyServer.scala#L56

It was introduced by a pull-request, but we shouldn't have it enabled at this level.

If you want it in the JVM, it should be possible to enable at the application level (via a Gzip Filter). And if you want it at the server level, it is better to handle it at the reverse proxy, or network infrastructure level.

I propose to remove this. 

(Actually we found strange bugs with this compression on one of our application. Not sure what is the cause, but having if hard-wired by default without any option to disable it, or to enable it for sone Actions only, makes it painful).

--
Guillaume Bort, http://guillaume.bort.fr

[Guillaume Bort]

Erratum:

We should handle 'decompression' to be able to receive gzip encoded requests. But not compress automatically the response. So I propose to remove only the 'compressor' handler.

[Sadek Drobi]

I agree.

--
 
 

[James Roper]

I agree too.  Over the weekend I wrote gzip/gunzip enumeratees.  They can easily be used on a per action basis, and it wouldn't be hard to write a filter that used it automatically if people wanted.

https://github.com/jroper/play-iteratees-extras/blob/master/src/main/scala/play/extras/iteratees/Gzip.scala

--
 
 

[Guillaume Bort]

Cool! Would it make sense to create a Filter from this and to have it built-in in the play-filters project?

[Ivan Meredith]

+1 to that.

Compression is an especially huge deal when you have high latencies between server and client, and filters seem like the right place to use this. Of course if the iteratee was exposed it could be reused per action too.

[Sadek Drobi]

Great. What about having Enumeratee[A,Array[Byte]] having and implicit of Writeable[A]

def gzip[A:Writeable](...):Enumeratee[A,Array[Byte]]

On Tue, Dec 11, 2012 at 4:56 AM, James Roper <james...@typesafe.com> wrote:

[Sadek Drobi]

On Tue, Dec 11, 2012 at 9:18 PM, Ivan Meredith <iv...@ivan.net.nz> wrote:

+1 to that.

Compression is an especially huge deal when you have high latencies between server and client, and filters seem like the right place to use this. Of course if the iteratee was exposed it could be reused per action too.

It is exposed, you can send an Enumerator to play.

[Ivan Meredith]

Err sorry I meant that the impl of the enumator is not private for the
filter it could be reused.

[James Roper]

On Wed, Dec 12, 2012 at 8:09 AM, Sadek Drobi <s...@zenexity.com> wrote:

Great. What about having Enumeratee[A,Array[Byte]] having and implicit of Writeable[A]

I feel like this should be a separate enumeratee, ie

def transformWriteable[A](writeable: Writeable[A]) = Enumeratee.map(writeable.transform _)

and then composed with the gzip enumeratee.  We could also provide the following helper function:

def writeable[A](content: A*)(implicit writeable: Writeable[A]) = Enumerator(content) &> transformWritable(writeable)

then, using it in an action, you can:

return Ok.stream(writeable(myContent) &> gzip())

 

Or we could even provide a gzip method on Status:

return Ok.gzip(Enumerator(myContent))

--
 
 

[James Roper]

I guess it depends on whether we want to support it or not yet.  I can tell you this enumeratee works fine with trivial tests in memory, but I've got no idea if there are any performance issues or other bugs.  It feels like it's a bit late to include this in 2.1 for me.  I've created this iteratees extras project because I've written a few iteratees that people might find useful, with the outlook that once an iteratee/enumeratee has proved itself stable and useful enough, it could be moved into Play.  The really nice thing about iteratees is that they are so easy to compose, so even if we don't include it in Play, it will still be really easy for people to use it.