Guice, JarJar, and Google Collections

[Stuart McCulloch]

Hi,

I think I've tracked down why jarjar isn't shrinking the collections library by much,
it's related to the static methods that simplify the creation of generic collections.

For example, from the injector implementation:

  multimap = Multimaps.newArrayListMultimap();

Unfortunately, Multimaps (and similar classes like Maps, Sets, etc.) have a lot
of static methods that touch classes throughout the collection - so once you've
pulled in Multimaps you end up pulling in almost the complete library.

In fact, even a simple class like:

  public class main {
    com.google.common.collect.Multimaps m;
  }

will end up as a 431k jarfile when jarjar'd with Google Collections!

Not sure how this could be resolved - technically you do need these classes as
they're referenced from the same classfile (although not all the static methods
will be called).

The only solution would be to change back to creating generic collections using
"new XYZ<...>" instead of the static approach "Foo.newXYZ()" as then jarjar can
safely remove the unused classes, but this would affect readability...

Any thoughts?

--
Cheers, Stuart

[Gili Tzabari]

Why doesn't jarjar strip out unused static methods? I believe
Proguard does.

Gili

[Stuart McCulloch]

2008/10/27 Gili Tzabari <gili.t...@gmail.com>

   Why doesn't jarjar strip out unused static methods? I believe
Proguard does.

no idea, certainly that would help although you want to make
sure static methods that form part of the public API are kept
(like the ones in the Guice class)

but that's just a configuration detail, should still be possible...

--
Cheers, Stuart

[Stuart McCulloch]

2008/10/27 Stuart McCulloch <mcc...@gmail.com>

2008/10/27 Gili Tzabari <gili.t...@gmail.com>

   Why doesn't jarjar strip out unused static methods? I believe
Proguard does.

no idea, certainly that would help although you want to make
sure static methods that form part of the public API are kept
(like the ones in the Guice class)

but that's just a configuration detail, should still be possible...

based on a quick hack of the Guice codebase to switch to reduced
versions of Maps, Lists, etc. (as the number of statics used is small)
we could shrink the jar to at least 600k, ie. a saving of 30%

hmm, I might take a peek at hacking JarJar to do this :)
 

--
Cheers, Stuart

[Stuart McCulloch]

2008/10/27 Stuart McCulloch <mcc...@gmail.com>

based on a quick hack of the Guice codebase to switch to reduced

versions of Maps, Lists, etc. (as the number of statics used is small)
we could shrink the jar to at least 600k, ie. a saving of 30%

hmm, I might take a peek at hacking JarJar to do this :)

FYI, I tried switching to use proguard but I couldn't find a way to make
proguard rename the packages but keep the original class names :(

but you can use it to strip unused static methods and then use jarjar
to rename the packages (see patch below) - though it would be cool
if we could get jarjar to do all of this...

the final guice jar after proguard'ing and jarjar'ing is ~600k

--
Cheers, Stuart


Index: build.xml
===================================================================
--- build.xml   (revision 649)
+++ build.xml   (working copy)
@@ -11,15 +11,30 @@

   <target name="jar" depends="compile, manifest"
        description="Build jar.">
+    <taskdef resource="proguard/ant/task.properties"
+      classpath="lib/build/proguard.jar"/>
     <taskdef name="jarjar" classname="com.tonicsystems.jarjar.JarJarTask"
       classpath="lib/build/jarjar-1.0rc8.jar"/>
     <mkdir dir="${build.dir}/dist"/>
+    <proguard>
+      -libraryjars ${java.home}/lib/rt.jar
+      -libraryjars ${ant.home}/lib/ant.jar
+      -libraryjars lib/aopalliance.jar
+      -injars ${build.dir}/classes
+      -injars lib/build/cglib-2.2.jar(!META-INF/**)
+      -injars lib/build/asm-3.1.jar(!META-INF/**)
+      -injars lib/build/google-collect-snapshot-20080530.jar(!META-INF/**)
+      -outjars ${build.dir}/stripped.jar
+      -keep class com.google.inject.** { *; }
+      -keepclassmembers class !com.google.common.** { *; }
+      -keepattributes
+      -dontobfuscate
+      -dontoptimize
+      -dontnote
+    </proguard>
     <jarjar jarfile="${build.dir}/dist/guice-${version}.jar"
         manifest="${build.dir}/META-INF/MANIFEST.MF">
-      <fileset dir="${build.dir}/classes"/>
-      <zipfileset src="lib/build/cglib-2.2.jar"/>
-      <zipfileset src="lib/build/asm-3.1.jar"/>
-      <zipfileset src="lib/build/google-collect-snapshot-20080530.jar"/>
+      <zipfileset src="${build.dir}/stripped.jar"/>
       <rule pattern="com.google.common.**" result="com.google.inject.internal.@1"/>
       <rule pattern="net.sf.cglib.**" result="com.google.inject.internal.cglib.@1"/>
       <rule pattern="org.objectweb.asm.**" result="com.google.inject.internal.asm.@1"/>

[Stuart McCulloch]

2008/10/27 Stuart McCulloch <mcc...@gmail.com>

2008/10/27 Stuart McCulloch <mcc...@gmail.com>

based on a quick hack of the Guice codebase to switch to reduced

versions of Maps, Lists, etc. (as the number of statics used is small)
we could shrink the jar to at least 600k, ie. a saving of 30%

hmm, I might take a peek at hacking JarJar to do this :)

FYI, I tried switching to use proguard but I couldn't find a way to make
proguard rename the packages but keep the original class names :(

I've opened an issue to track this:

   http://code.google.com/p/google-guice/issues/detail?id=264

ProGuard obfuscation doesn't play well with CGLIB, and adapting
JarJar to trim static methods is non-trivial in the current codebase
so IMHO combining the two in "build.xml" is the best option...
 

--
Cheers, Stuart

[Gili Tzabari]

For what it's worth, here is the Proguard configuration I've used to
prevent removal of required CGLIB components:

# Referenced by ReflectUtils.newInstance
-keepclassmembers class com.google.inject.cglib.reflect.FastClass
{
<init>(java.lang.Class);
}

# Referenced by com.google.inject.cglib.proxy.Enhancer.getCallbacksSetter
-keep class com.google.inject.cglib.proxy.MethodProxy
{
*;
}

# Referenced by cglib-enhanced proxies
-keep class com.google.inject.cglib.core.ReflectUtils
{
public <methods>;
}

# Referenced by cglib proxies
-keep class com.google.inject.cglib.proxy.MethodInterceptor
{
public java.lang.Object intercept(java.lang.Object,
java.lang.reflect.Method, java.lang.Object[],
com.google.inject.cglib.proxy.MethodProxy);
}

Gili

Stuart McCulloch wrote:
> 2008/10/27 Stuart McCulloch <mcc...@gmail.com <mailto:mcc...@gmail.com>>
>
> 2008/10/27 Stuart McCulloch <mcc...@gmail.com
> <mailto:mcc...@gmail.com>>

[Gili]

I'm thinking that maybe all you need is a Proguard configuration
(above and more below) and some third-party (such as JarJar) to move
some packages around and you're done. You'd run Proguard with
shrinking enabled, obfuscation disabled, and look into whether
optimization should be enabled or not.

Here is the second part of the Proguard configuration you will need to
prevent Guice-specific classes from being removed:

# Referenced by com.google.inject.AbstractModule
-keep class com.google.inject.Binder
{
*;
}

# Keep any constructors annotated with @Inject
-keepclassmembers class *
{
@com.google.inject.Inject <init>(...);
}

At least, this is what I used for Guice 1.0

[Stuart McCulloch]

2008/10/28 Gili <gili.t...@gmail.com>

I'm thinking that maybe all you need is a Proguard configuration
(above and more below) and some third-party (such as JarJar) to move
some packages around and you're done. You'd run Proguard with
shrinking enabled, obfuscation disabled, and look into whether
optimization should be enabled or not.

exactly - that's basically the patch I attached to issue 264 (ProGuard then JarJar)
 

Here is the second part of the Proguard configuration you will need to
prevent Guice-specific classes from being removed:

# Referenced by com.google.inject.AbstractModule
-keep class com.google.inject.Binder
{
       *;
}

# Keep any constructors annotated with @Inject
-keepclassmembers class *
{
       @com.google.inject.Inject <init>(...);
}

At least, this is what I used for Guice 1.0

--
Cheers, Stuart

[Gili]

I am hoping that one of these days someone ends up forking Proguard
altogether (this is something the JarJar authors might want to
consider). I like Eric well enough but he seems unwilling to give
anyone else commit rights. I am uncomfortable relying on a single
developer projects and of course he is severely overloaded with work
so development goes very slowly.

There are plenty of features I'd like to see added on the obfuscation
and ease-of-use front, much less so on the optimization front which he
seems to be focused on for the past couple of years. Just my 2
cents ;)

Gili

On Oct 27, 1:13 pm, "Stuart McCulloch" <mccu...@gmail.com> wrote:
> 2008/10/28 Gili <gili.tzab...@gmail.com>

[Chris Nokleberg]

FYI the JarJar authors == me, so it may not solve your single
developer issue :-)

The main reason that JarJar only does class-based elimination is that
it is 100x simpler. Basically it was trivial to add as a feature to
the existing jarjar codebase so I did it, but I'm less enthused about
making it a proguard alternative. I think using a combination of
proguard + jarjar is totally reasonable and in fact I have done that
myself on past projects.

[Stuart McCulloch]

2008/11/6 Chris Nokleberg <chris.n...@gmail.com>

FYI, I finally got round to writing patch that removes "unused" static methods:

   http://code.google.com/p/jarjar/issues/detail?id=22

I've tested it with Guice, but not checked impact on performance/memory use

--
Cheers, Stuart