Questions about how the GWT obfuscation process works

[David Lindsay]

I've been vaguely aware of GWT and what it does for a few years, but learning Java is still on my todo list so I haven't yet explored GWT in too much depth.

While recently poking around in my browser devtools as I tried to figure out how a particular webapp worked I noticed some obfuscated cache files, and after a bit of digging and research I learned this was because the code was GWT-generated.

My curiosity to figure out what I was looking at was motivated by the obvious obfuscation I could see, particularly the fact that the obfuscation process appears to be compressing the JavaScript in question - if I'm not mistaken, it looks like it's identifying similar code snippets and placing them in common functions. Is this what's happening? It'd be a very interesting technique from a bandwidth-saving perspective.

Even if the obfuscation engine doesn't compress as a side effect, the result certainly seems much harder to read and follow to me than the output of the various other JS minifiers out there, and as such, the obfuscation component is very appealing to me on its own.

Is it possible to point the GWT obfuscator at my own JavaScript and have it obfuscate that, or does it only work on GWT-generated code?

- David Lindsay

PS. As an aside, I'm very curious why so many of the functions I see are empty. Is there a reason for this? Why can't eg 527 empty functions (with no arguments) be turned into 1? Are they being used as objects?

[Jens]

GWT first optimizes the Java AST, then converts it into a JavaScript AST and optimizes it as well. Once that is done the actual obfuscation is just a simple renaming as far as I know. Its also kind of worthless doing such a compression as you have mentioned because that is exactly what GZIP compression does when activated on your web server. 

The obfuscation is not a standalone tool so you can not run it against any JS library.

In the future it is likely that GWT use a different compiler called J2CL (in development inside Google) which takes the Java code and directly converts it to ES6 JavaScript classes in a way that they are really good understandable by Google Closure Compiler. Then JS optimization + minification will be done by Google Closure Compiler which you can of course use for any JS library as well. So the minification/obfuscation you are seeing today in GWT will likely change in the future.

-- J.

[Thomas Broyer]

On Thursday, June 30, 2016 at 11:36:19 AM UTC+2, Jens wrote:

GWT first optimizes the Java AST, then converts it into a JavaScript AST and optimizes it as well. Once that is done the actual obfuscation is just a simple renaming as far as I know.

Once the optimization (aka tree-shaking; includes inlining, deadcode pruning, etc.) is done, code is also reorganized so it'll compress better with gzip ("sorts functions by size and clusters them by edit distance"); and obfuscation allocates obfuscated identifiers "in the original order of their declaration, which produces a lot more common substrings in the obfuscated output which significantly improves gzip/deflate compression."

(quotes come from commit messages)

See https://timepedia.blogspot.fr/2009/08/on-reducing-size-of-compressed.html about reorganization.

Also, up to GWT 2.7, there was an experimental flag to use Closure Compiler for the JS AST optimisation and obfuscation (it has been removed for GWT 2.8), which would use different algorithms and produce different results.

[David Lindsay]

GWT first optimizes the Java AST, then converts it into a JavaScript AST and optimizes it as well.

I had a sneaking suspicion the obfuscation might not be working off human-readable JS. Neat.

 

Once that is done the actual obfuscation is just a simple renaming as far as I know.

Ah, right.

 

Its also kind of worthless doing such a compression as you have mentioned because that is exactly what GZIP compression does when activated on your web server.

I was wondering whether it made that much of a difference with gzip configured. As Thomas Broyer noted, GWT reorganizes the code so it fits better into gzip's sliding window and gets compressed more readily.

As an aside, I recently learned just how much gzip makes a difference: http://jvns.ca/blog/2013/10/24/day-16-gzip-plus-poetry-equals-awesome/

 

The obfuscation is not a standalone tool so you can not run it against any JS library.

Duly noted.

 

In the future it is likely that GWT use a different compiler called J2CL (in development inside Google) which takes the Java code and directly converts it to ES6 JavaScript classes in a way that they are really good understandable by Google Closure Compiler.

Interesting! Thanks for the headsup!

 

Then JS optimization + minification will be done by Google Closure Compiler which you can of course use for any JS library as well.

I just had a look at the Closure Compiler web service and fed it some JS I was recently golfing. To quote, it "Saved -68.96% off the gzipped size (-107.55% without gzip)", although in all fairness it did give me some possible ideas for saving a few bytes.

 

So the minification/obfuscation you are seeing today in GWT will likely change in the future.

 I see. I'll keep an eye out for the changes! :)

[David Lindsay]

On Thursday, June 30, 2016 at 11:36:19 AM UTC+2, Jens wrote:

GWT first optimizes the Java AST, then converts it into a JavaScript AST and optimizes it as well. Once that is done the actual obfuscation is just a simple renaming as far as I know.

Once the optimization (aka tree-shaking; includes inlining, deadcode pruning, etc.) is done, code is also reorganized so it'll compress better with gzip ("sorts functions by size and clusters them by edit distance"); and obfuscation allocates obfuscated identifiers "in the original order of their declaration, which produces a lot more common substrings in the obfuscated output which significantly improves gzip/deflate compression."

(quotes come from commit messages)

See https://timepedia.blogspot.fr/2009/08/on-reducing-size-of-compressed.html about reorganization.

Thanks for the link - this is really interesting reading.

 

Also, up to GWT 2.7, there was an experimental flag to use Closure Compiler for the JS AST optimisation and obfuscation (it has been removed for GWT 2.8), which would use different algorithms and produce different results.

 Was the flag removed in preparation for J2CL?