Quick test of ICBM

[Evan Jones]

I figured I'd spend 10 minutes trying ICBM on our relatively simple project:

1. Running icbm in our project's root doesn't work because the path isn't src (java/server/src/co/mitro/core/server/Main.java)

2. Running icbm from the java/server directory "worked", but one of our dependencies in a .jar depends on log4j in another jar. ICBM didn't find the dependency on log4j, so the binary doesn't start. This is sort of a "bug", but I understand why this happens. However, you *could* actually detect this by trying to load the dependent types from the .jars using Class.fromName(), catching any NoClassDefFoundError, and then doing ICBM's normal lookup for those types to add any dependent jars.

3. Added an icbm.cfg which let me build from my project root, but I couldn't remember enough of the build.spec stuff to figure out how to add the right dependencies in the right places.

Anyway, just thought I'd let you know that it mostly worked, but you'll need some additional documentation about icbm.cfg and build.spec if you want others to use this.

Cheers,

Evan Jones

[Ilia Mirkin]

Hi Evan,

Thanks for trying ICBM! As you noticed, there are a few rough edges,
hopefully they won't deter you from future attempts.

On Thu, Sep 26, 2013 at 2:38 PM, Evan Jones <e...@evanjones.ca> wrote:
> I figured I'd spend 10 minutes trying ICBM on our relatively simple project:
>
> 1. Running icbm in our project's root doesn't work because the path isn't
> src (java/server/src/co/mitro/core/server/Main.java)

Right, as you discovered, you need to declare the java/server/src
module. If the above isn't a typo and you indeed use a tld of "co",
some regexes inside genautodep.py will need to be changed as well. I
don't think they have been moved out to the cfg yet, but they will
need to be. (Just search for "com", should be obvious.) Otherwise it
won't pick up on fully-qualified class name references.

> 2. Running icbm from the java/server directory "worked", but one of our
> dependencies in a .jar depends on log4j in another jar. ICBM didn't find the
> dependency on log4j, so the binary doesn't start. This is sort of a "bug",
> but I understand why this happens. However, you *could* actually detect this
> by trying to load the dependent types from the .jars using Class.fromName(),
> catching any NoClassDefFoundError, and then doing ICBM's normal lookup for
> those types to add any dependent jars.

It would also be probably reasonable to use one of the java bytecode
parsers to detect explicit references. And you could perform similar
logic on all strings as genautodep.py does for the ones detected in
the jar (to pick up Class.forName & co). Quite frankly, I was just too
lazy to implement it because (a) jars tend not to have a lot of
dependencies on one another, so it's not a big deal to add manually,
and (b) jars and their deps don't change often, so it's not a huge
pain to maintain manually either. Don't know if Austin would be up for
a jar parser that added the deeper parsing (the current one just
extracts the list of classes contained). It would increase the time of
"first" autodep run significantly, since jars tend to have lots of
classes and the classes are "harder" to parse, but should have little
impact on the later runs. Perhaps it'd be sufficient to use javap -c
(although I'm sure that faster solutions could exist).

>
> 3. Added an icbm.cfg which let me build from my project root, but I couldn't
> remember enough of the build.spec stuff to figure out how to add the right
> dependencies in the right places.
>
> Anyway, just thought I'd let you know that it mostly worked, but you'll need
> some additional documentation about icbm.cfg and build.spec if you want
> others to use this.

Yeah, detailed docs are sorely missing. For your specific situation,
if you have jars a.jar, b.jar in some directory and you want b to
depend on a, you add a build.spec in that dir that reads

java_library(name="a", deps=['":b"])

If it's in another directory in the same module you could do

java_library(name="a", deps=["path/to/dir:b"])

and if it's in another module, you would do

java_library(name="a", deps=["other/module=path/to/dir:b"])

What happens behind the scenes is that autodep creates
java_library(name="a") for a.jar and java_library(name="b") for b.jar,
and the build.spec just extends existing objects (and can obviously
define new ones). For a Foo.java file, the lib name is libFoo.

-ilia

[Evan Jones]

On Sep 26, 2013, at 15:59 , Ilia Mirkin <imi...@alum.mit.edu> wrote:
> Right, as you discovered, you need to declare the java/server/src
> module. If the above isn't a typo and you indeed use a tld of "co",
> some regexes inside genautodep.py will need to be changed as well. I
> don't think they have been moved out to the cfg yet, but they will
> need to be. (Just search for "com", should be obvious.) Otherwise it
> won't pick up on fully-qualified class name references.

Ah, interesting! Yes .co is not a typo, see https://www.mitro.co/ , thanks to the unavailability of good .com names these days. I didn't notice any problems with my cursory inspection of the output, since we probably don't have any fully-qualified references.

> Quite frankly, I was just too
> lazy to implement it because (a) jars tend not to have a lot of
> dependencies on one another, so it's not a big deal to add manually,
> and (b) jars and their deps don't change often, so it's not a huge
> pain to maintain manually either.

I agree with (b), but my experience is that (a) is not true, particularly as more projects rely on Maven or Ivy for dependency resolution. In *most* cases a code base will directly depend on all jars involved in a dependency, but we probably have a few. log4j is just the first one that I encountered. I wouldn't be surprised to see some Jetty-related exceptions.

Still your point is valid: one time setup, not a huge deal. Particularly since all that data is already available in something like Maven/Ivy, so theoretically you could generate the right dependencies from that.

Evan

--
Work: https://www.mitro.co/ Personal: http://evanjones.ca/