Mark Reinhold on jigsaw - live notes from devoxx.

[Reinier Zwitserloot]

Notes from his presentation at devoxx, 17:40-18:40 Central European
Time, November 19th, 2009.

Section 1: Problems and Solutions (why jigsaw, what is it trying to
solve? Subtext: What is it trying to solve that OSGi can't?)

jigsaw exists because they want to have modules-all-the-way-down. OSGi
depends on java classes, but how can you then modularize the core java
classes like java.util.List or even java.lang.String with OSGi? So,
the module system needs to be done in the JVM core. OSGi comes with
baggage such as lifecycle management that isn't needed for slicing
RT.jar into modules.

Secondly, jar files are a fine distro tool, but a crappy runtime tool,
so a new package processing system is required. If everyone versions,
then we can go back to the model of installing 'jars' (whatever they
will be) into the system, without running into version conflicts
anymore. By changing the format between running and distro, java can
verify at install time, do some ahead-of-time compilation, and store
JIT profiles straight into the local module base for future executions
of the JVM. This is not something OSGi does (or even should do). Also,
class files are great for distribution, but they aren't that great for
efficient loading.

The java core libraries are what they are. They can't be changed now.
And yet, e.g. java.io.FileInputStream has a method to get a
java.nio.FileChannel. If java.io and java.nio are to be different
packages, and you don't want io to be dependent on nio, then what?
Even worse, some non-util packages contribute a class or two to
java.util. OSGi is very much anti split packages. If java were to be
designed from scratch with modularization in mind, that might be okay,
but java wasn't. Thus, split packages, in-package optional
dependencies are all fairly complicated requirements that are needed
for slicing rt.jar into modules.

Big point he keeps making: He looked at OSGi to slice rt.jar, decided
it just wasn't going to work, so came up with a new module system that
is mostly not intersecting with OSGi's bailiwick. And then he decided
that, as java code itself will have to have the capability of stating
that it doesn't, for example, need 'swing', or 'nio', or
'corba' (heh), it makes sense to allow other people to make modules in
this as well. Hence, jigsaw.

Section 2: Jigsaw - technical details.

Part A: Grouping.

A module system needs to group classes and other resources in a single
module.

jigsaw solution: module-info.java file, at the top of the relevant
source tree (so at ~/module.info if your class is at ~/com/foo/
Whatever.class)

Flipside of coin: Need ability to state dependency, better than
'import foo;'.

example:

module com.foo {
requires org.bar.lib;
requires edu.baz.util;
}

and with versions:

module com.foo @ 1.0.0 {
requires org.bar.lib @ 2.1-alpha;
requires edu.baz.util @ 5.2_11;
}

Version syntax is now chosen to be non-descriptive at all. Anything
goes. Just like apt-get. Awareness of the need to then look up
versioning scheme per module, and there WILL be syntaxes like 2.(1+).
Preferable to OSGi forced versioning system. Tokenizes on a limited
number of characters (dots and dashes, pretty much), and each
subsequence can be matched against with wildcards. It seems to work
well for debian, and for rpm (their their algo is slightly different),
hence confidence this will be fine, and should make for much easier
adoption amongst existing projects, such as Oracle
v1.9.12.0.0.0.0.0.0.0.0.0.2 (red: I'm channeling Joe here, not Mark).
As there is to be interop between e.g. rpm and debian, anything more
rigid will also not translate very well (e.g. a complex wildcard just
couldn't be translated to a deb file!)

'requires' by default flows through to dependents. So if A requires B,
and C requires B, then C has access to A's stuff. Get around via
'requires private ...'.

You can also get around this FROM A:

module com.foo.secret {
permits com.foo.lib;
}

means that only com.foo.lib can 'requires' it, and can't export it
(forced to say 'private', or compiler error).



optional dependencies: requires optional...

Of course, your code will need to deal with the ClassNotFoundErrors
and such that could result.

virtual modules: Like apt-get's virtual 'java' package. It's a bit
like a module interface concept, though it's just a name, there's no
structure in a virtual module.

So:

module com.foo.lib {
}

module com.foo.lib.impl.phone {
provides com.foo.lib;
}

If another module then wants 'com.foo.lib', it'll get
com.foo.lib.impl.phone.


Part B: Interop with other packaging systems, and module distribution.

You can for example create a debian package (For apt-get, so you can
install on ubuntu, debian, etc):

jpkg - m classes deb com.foo.app com.foo.lib

creates a .deb file for apt-get from the moudles com.foo.app and
com.foo.lib. All dependencies are reflected into the debian package,
so if com.foo.app requires foo.bar.baz, this is reflected in the apt
file's Requires block.

For windows, with no such package manager, or in any java-centric
environment, or even on phones with a stripped-down linux without rpm
or apt-get, you just stick with java's own.

There will be a platform independent transfer format (basically v2.0
of jar files. Won't be zip on the inside, will be opaque so the format
can be revved by sun over time, but it's mostly the same concept). It
will have a new file extension.

Part C: Slicing the JDK.

Here's a chart of dependencies per package in java core library:
http://moby.to/bbwmcv

Here's what base is dependent on in JDK6 (so, almost everything):
http://moby.to/v7y35j

Here's what it looks like now in an experimental JDK7. with base
dependent on very little, and nearly acyclic: http://moby.to/zlosc8

And even simpler with optional deps removed (this photo you should be
able to read): http://moby.to/8xpyaw

*JIGSAW BINARIES* will go live in a few minutes after this talk!

http://openjdk.java.net/projects/jigsaw
http://dl.openjdk.java.net/jigsaw

only binaries for unbuntu jaunty, but you can always build from source
(that's just because Mark is an ubuntu user). It's all very raw, but
it includes the packaging tools, as well as support for module-info
and the like.

Part D: OSGi interop.

(Not mentioned in main presentation, but someone asked).

Status unknown, but it's something they'll look at. Realization that
compatibility is rather difficult due to different requirements, such
as split-packages. Realization that having 2 competing module formats,
forcing lib devvers to make 2 versions. Something Mark is looking at
is extending jigsaw libraries to just wrap around an OSGi library.
That way, you can choose to just make an OSGi library and ship that.


NB: There's a BOF coming up on this at 20:00 Central European about
the Modular Java Platform and Jigsaw with Mark, so this isn't the last
word.

NB2: Slicing rt.jar is well on its way, but not quite done yet. Still
dependencies exist that will be eliminated before JDK7 RC. Also, so
far, no measurable performance benefits (yet). However, new installed-
module stuff (ahead-of-time implementation, second version of class
file format optimized for reading, jit profiles) aren't live yet, and
should bring significant performance benefits.

[MassH]

Wow. Can't wait to finally try this out. Thanks for posting all of
that.

[Brian Ehmann]

GREAT post. Thank you...

> http://openjdk.java.net/projects/jigsawhttp://dl.openjdk.java.net/jigsaw