clojure.lang.Reflector in java9

259 views
Skip to first unread message

Herwig Hochleitner

unread,
Oct 3, 2016, 8:02:07 PM10/3/16
to cloju...@googlegroups.com
While looking through data.xml's tickets, an odd one stood out: http://dev.clojure.org/jira/browse/DXML-32

The stacktrace is thrown by the new jigsaw infrastructure, preventing clojure.lang.Reflector from accessing an implementation class.
Is anybody running clojure code on java9 yet and can confirm that behavior?

This has serious implications, because accessing implementation classes, is what clojure.lang.Reflector does and if java9 is released with such restrictions, we should start thinking about how to keep clojure code working.


If this is confirmed (which seems likely to me, from what I know about jigsaw), what to do about it? Run clojure with permission to access all classes (if even possible, would still be alienating)? Deal with restricted classes and try to reflect on public interfaces?

kind regards

Alex Miller

unread,
Oct 3, 2016, 9:34:24 PM10/3/16
to cloju...@googlegroups.com
I have not spent enough time with Java 9 modules yet to know the exact path but I assume there is some reasonable solution, as existing code is designed to work without change (and this code is very similar to the equivalent Java code to do the same thing). 


--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev+unsubscribe@googlegroups.com.
To post to this group, send email to cloju...@googlegroups.com.
Visit this group at https://groups.google.com/group/clojure-dev.
For more options, visit https://groups.google.com/d/optout.

Toby Crawley

unread,
Nov 21, 2016, 2:17:17 PM11/21/16
to cloju...@googlegroups.com
On Mon, Oct 3, 2016 at 9:34 PM, Alex Miller <al...@puredanger.com> wrote:
> I have not spent enough time with Java 9 modules yet to know the exact path
> but I assume there is some reasonable solution, as existing code is designed
> to work without change (and this code is very similar to the equivalent Java
> code to do the same thing).
>
> Seems to be in the ballpark:
>
> http://openjdk.java.net/projects/jigsaw/spec/sotms/#reflective-readability

(Note: apologies if the following is disjoint - I've been looking at various
Java 9 issues the last few days, and wanted to capture and share what I have
so far before I have to move on to other things)

That section of the spec is related, but only applies when the class
being reflected on is exported publicly:

> We therefore, instead, revise the reflection API simply to assume
> that any code that reflects upon some type is in a module that can
> read the module that defines that type. This enables the above
> example, and other code like it, to work without change. This
> approach does not weaken strong encapsulation: A public type must
> still be in an exported package in order to be accessed from outside
> its defining module, whether from compiled code or via reflection.

In this case, the class in question isn't exported, but can be
exported via an option to java/javac (discussed in [1] as
`-XaddExports`, but since renamed to `--add-exports`).

Using that option, the data.xml build gets past all the reflection
issues with:

mvn install -Dclojure.vmargs="--add-exports=java.xml/com.sun.xml.internal.stream=ALL-UNNAMED
--add-exports=java.xml/com.sun.xml.internal.stream.writers=ALL-UNNAMED
--add-exports=java.xml/com.sun.org.apache.xerces.internal.impl=ALL-UNNAMED"

This was with the latest java 9 early access release:

java version "9-ea"
Java(TM) SE Runtime Environment (build 9-ea+144)
Java HotSpot(TM) 64-Bit Server VM (build 9-ea+144, mixed mode)

This would mean that any Clojure + Java 9 user would need to provide
the same `--add-exports` to use data.xml, and export any other
internal classes that run into this same issue.

Changes to reflection on non-exported types is currently being
discussed on the jigsaw-dev list ([2] [3]), but it's not yet clear if
the built-in modules will be exposed, it may just be intended for
external module authors (from [3]):

> This proposal primarily addresses "friendly" uses of reflection,
> such as dependency injection and persistence, in which the author of
> a module knows in advance that one or more, or possibly all,
> packages must be opened at run time for deep reflective access by
> frameworks. Intrusive access to arbitrary packages of arbitrary
> modules by, e.g., serialization frameworks or debugging tools, will
> still require the use of sharp knives such as `--add-exports` or
> `--add-opens` command-line options, the legacy unsupported
> `sun.misc.Unsafe` API and related APIs, or JVM TI.

Though they may provide an escape via allowing specification of
exports in MANIFEST.MF (in this case, any library that knows it
requires reflection against internal classes would be able expose the
required classes via the manifest. Developers touching other internal
classes on their own would still be responsible for their own
--add-exports) (also from [3]):

> Using `--add-exports` or `--add-opens` command-line options, or
> their equivalents, remains awkward, but sometimes they're the only
> way out. To ease migration it's worth considering some way for an
> application packaged as a JAR file to include such options in its
> `MANIFEST.MF` file, as suggested by Simon Nash [b]. This is
> addressed in the proposal for #AddExportsInManifest [c], which is
> hereby amended to rename the `Add-Exports-Private` attribute to
> `Add-Opens`.

However, I haven't read the full threads discussing these changes -
it's possible our concerns are addressed in the comments, and much of
this is moot.

If this ends up not being addressed in a satisfactory way by jigsaw,
it would be possible to detect and handle this case automatically in
Reflector.java for 1.9: if reflection fails against the object's
class, we can walk the ancestor tree until we find a class or
interface that provides an instance of the method we're looking for
that we can actually call. I have a very rough patch locally that does
that successfully, with just the added cost of an additional try/catch
around every reflective method call in the happy path case. If such a
solution proves necessary, I can file a JIRA for this and provide a
cleaned up patch.

The target release date for Java 9 is currently July 27, 2017[4], but I
suspect that will slip again, as that schedule uses May 26, 2016 as
the feature-complete date, and jigsaw is still in flux.

- Toby

[1]: http://mreinhold.org/blog/jigsaw-module-system
[2]: http://mail.openjdk.java.net/pipermail/jigsaw-dev/2016-October/009818.html
[3]: http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2016-October/000430.html
[4]: http://www.java9countdown.xyz

Alex Miller

unread,
Nov 21, 2016, 2:24:14 PM11/21/16
to Clojure Dev
Thanks Toby, very interesting info. I'd be ok with a jira now, if even just to serve as a location to consolidate discussion and tracking.

Toby Crawley

unread,
Nov 22, 2016, 11:04:23 AM11/22/16
to cloju...@googlegroups.com
On Mon, Nov 21, 2016 at 2:24 PM, Alex Miller <al...@puredanger.com> wrote:
> Thanks Toby, very interesting info. I'd be ok with a jira now, if even just
> to serve as a location to consolidate discussion and tracking.

My pleasure. I filed http://dev.clojure.org/jira/browse/CLJ-2066

- Toby
Reply all
Reply to author
Forward
0 new messages