[ANN] Clojure 1.8.0-alpha3

[Alex Miller]

Clojure 1.8.0-alpha3 is now available.

Try it via

- Download: https://repo1.maven.org/maven2/org/clojure/clojure/1.8.0-alpha3

- Leiningen: [org.clojure/clojure "1.8.0-alpha3"]

Tuples have been disabled after further analysis of performance impacts.

This alpha has initial support for direct linking which you can enable with -Dclojure.compiler.direct-linking=true

Direct linking allows functions compiled with direct linking on to make direct static method calls to most other functions, instead of going through the var and the Fn object. This can enable further optimization by the jit, at a cost in dynamism. In particular, directly-linked calls will not see redefinitions.

In alpha3, clojure.core is compiled with direct linking by default and therefore other namespaces cannot redefine core fns and have those redefinitions seen by core code.

Functions declared as dynamic will never be linked to directly.

As with all alphas, this represents ongoing work-in-progress that is subject to change.

[Rangel Spasov]

Hey guys,

I'm getting this compiler error after upgrading to alpha3, I assume it has something to do with the direct linking changes?

I think it's related to clj-ssh library, and this method specifically:

(defn- ^{:tag (Class/forName "[B")} as-bytes

  "Return arg as a byte array.  arg must be a string or a byte array."

  [arg]

  (if (string? arg)

    (.getBytes ^String arg ascii)

    arg))

The exception is below:

#error {

 :cause Unable to resolve classname: (Class/forName "[B")

 :via

 [{:type clojure.lang.Compiler$CompilerException

   :message java.lang.IllegalArgumentException: Unable to resolve classname: (Class/forName "[B"), compiling:(clj_ssh/ssh.clj:94:1)

   :at [clojure.lang.Compiler analyzeSeq Compiler.java 6891]}

  {:type java.lang.IllegalArgumentException

   :message Unable to resolve classname: (Class/forName "[B")

   :at [clojure.lang.Compiler$HostExpr tagToClass Compiler.java 1132]}]

 :trace

 [[clojure.lang.Compiler$HostExpr tagToClass Compiler.java 1132]

  [clojure.lang.Compiler tagClass Compiler.java 8384]

  [clojure.lang.Compiler$FnMethod parse Compiler.java 5309]

  [clojure.lang.Compiler$FnExpr parse Compiler.java 3977]

  [clojure.lang.Compiler analyzeSeq Compiler.java 6882]

  [clojure.lang.Compiler analyze Compiler.java 6685]

  [clojure.lang.Compiler analyzeSeq Compiler.java 6872]

  [clojure.lang.Compiler analyze Compiler.java 6685]

  [clojure.lang.Compiler access$300 Compiler.java 38]

  [clojure.lang.Compiler$DefExpr$Parser parse Compiler.java 593]

  [clojure.lang.Compiler analyzeSeq Compiler.java 6884]

  [clojure.lang.Compiler analyze Compiler.java 6685]

  [clojure.lang.Compiler analyze Compiler.java 6646]

  [clojure.lang.Compiler eval Compiler.java 6947]

  [clojure.lang.Compiler load Compiler.java 7390]

  [clojure.lang.RT loadResourceScript RT.java 372]

  [clojure.lang.RT loadResourceScript RT.java 363]

  [clojure.lang.RT load RT.java 453]

  [clojure.lang.RT load RT.java 419]

  [clojure.core$load$fn__5436 invoke core.clj 5869]

  [clojure.core$load invokeStatic core.clj 5868]

  [clojure.core$load_one invokeStatic core.clj 5669]

  [clojure.core$load_one invoke core.clj -1]

  [clojure.core$load_lib$fn__5385 invoke core.clj 5714]

  [clojure.core$load_lib invokeStatic core.clj 5713]

  [clojure.core$load_lib doInvoke core.clj -1]

  [clojure.lang.RestFn applyTo RestFn.java 142]

  [clojure.core$apply invokeStatic core.clj 635]

  [clojure.core$load_libs invokeStatic core.clj 5751]

  [clojure.core$load_libs doInvoke core.clj -1]

  [clojure.lang.RestFn applyTo RestFn.java 137]

  [clojure.core$apply invokeStatic core.clj 635]

  [clojure.core$require invokeStatic core.clj 5773]

  [clojure.core$require doInvoke core.clj -1]

  [clojure.lang.RestFn invoke RestFn.java 930]

  [cloud_monkey.ssh$eval21511$loading__5328__auto____21512 invoke ssh.clj 1]

  [cloud_monkey.ssh$eval21511 invokeStatic ssh.clj 1]

  [cloud_monkey.ssh$eval21511 invoke ssh.clj -1]

  [clojure.lang.Compiler eval Compiler.java 6943]

  [clojure.lang.Compiler eval Compiler.java 6932]

  [clojure.lang.Compiler load Compiler.java 7390]

  [clojure.lang.RT loadResourceScript RT.java 372]

  [clojure.lang.RT loadResourceScript RT.java 363]

  [clojure.lang.RT load RT.java 453]

  [clojure.lang.RT load RT.java 419]

  [clojure.core$load$fn__5436 invoke core.clj 5869]

  [clojure.core$load invokeStatic core.clj 5868]

  [clojure.core$load_one invokeStatic core.clj 5669]

  [clojure.core$load_one invoke core.clj -1]

  [clojure.core$load_lib$fn__5385 invoke core.clj 5714]

  [clojure.core$load_lib invokeStatic core.clj 5713]

  [clojure.core$load_lib doInvoke core.clj -1]

  [clojure.lang.RestFn applyTo RestFn.java 142]

  [clojure.core$apply invokeStatic core.clj 635]

  [clojure.core$load_libs invokeStatic core.clj 5751]

  [clojure.core$load_libs doInvoke core.clj -1]

  [clojure.lang.RestFn applyTo RestFn.java 137]

  [clojure.core$apply invokeStatic core.clj 635]

  [clojure.core$require invokeStatic core.clj 5773]

  [clojure.core$require doInvoke core.clj -1]

  [clojure.lang.RestFn invoke RestFn.java 1289]

  [cloud_monkey.execution_pipeline$eval21505$loading__5328__auto____21506 invoke execution_pipeline.clj 1]

  [cloud_monkey.execution_pipeline$eval21505 invokeStatic execution_pipeline.clj 1]

  [cloud_monkey.execution_pipeline$eval21505 invoke execution_pipeline.clj -1]

  [clojure.lang.Compiler eval Compiler.java 6943]

  [clojure.lang.Compiler eval Compiler.java 6932]

  [clojure.lang.Compiler load Compiler.java 7390]

  [clojure.lang.RT loadResourceScript RT.java 372]

  [clojure.lang.RT loadResourceScript RT.java 363]

  [clojure.lang.RT load RT.java 453]

  [clojure.lang.RT load RT.java 419]

  [clojure.core$load$fn__5436 invoke core.clj 5869]

  [clojure.core$load invokeStatic core.clj 5868]

  [clojure.core$load_one invokeStatic core.clj 5669]

  [clojure.core$load_one invoke core.clj -1]

  [clojure.core$load_lib$fn__5385 invoke core.clj 5714]

  [clojure.core$load_lib invokeStatic core.clj 5713]

  [clojure.core$load_lib doInvoke core.clj -1]

  [clojure.lang.RestFn applyTo RestFn.java 142]

  [clojure.core$apply invokeStatic core.clj 635]

  [clojure.core$load_libs invokeStatic core.clj 5751]

  [clojure.core$load_libs doInvoke core.clj -1]

  [clojure.lang.RestFn applyTo RestFn.java 137]

  [clojure.core$apply invokeStatic core.clj 635]

  [clojure.core$require invokeStatic core.clj 5773]

  [clojure.core$require doInvoke core.clj -1]

  [clojure.lang.RestFn invoke RestFn.java 1523]

  [cloud_monkey.events$eval21499$loading__5328__auto____21500 invoke events.clj 1]

  [cloud_monkey.events$eval21499 invokeStatic events.clj 1]

  [cloud_monkey.events$eval21499 invoke events.clj -1]

  [clojure.lang.Compiler eval Compiler.java 6943]

  [clojure.lang.Compiler eval Compiler.java 6932]

  [clojure.lang.Compiler load Compiler.java 7390]

  [clojure.lang.RT loadResourceScript RT.java 372]

  [clojure.lang.RT loadResourceScript RT.java 363]

  [clojure.lang.RT load RT.java 453]

  [clojure.lang.RT load RT.java 419]

  [clojure.core$load$fn__5436 invoke core.clj 5869]

  [clojure.core$load invokeStatic core.clj 5868]

  [clojure.core$load_one invokeStatic core.clj 5669]

  [clojure.core$load_one invoke core.clj -1]

  [clojure.core$load_lib$fn__5385 invoke core.clj 5714]

  [clojure.core$load_lib invokeStatic core.clj 5713]

  [clojure.core$load_lib doInvoke core.clj -1]

  [clojure.lang.RestFn applyTo RestFn.java 142]

  [clojure.core$apply invokeStatic core.clj 635]

  [clojure.core$load_libs invokeStatic core.clj 5751]

  [clojure.core$load_libs doInvoke core.clj -1]

  [clojure.lang.RestFn applyTo RestFn.java 137]

  [clojure.core$apply invokeStatic core.clj 635]

  [clojure.core$require invokeStatic core.clj 5773]

  [clojure.core$require doInvoke core.clj -1]

  [clojure.lang.RestFn invoke RestFn.java 1789]

  [cloud_monkey.aleph_netty$eval1588$loading__5328__auto____1589 invoke aleph_netty.clj 1]

  [cloud_monkey.aleph_netty$eval1588 invokeStatic aleph_netty.clj 1]

  [cloud_monkey.aleph_netty$eval1588 invoke aleph_netty.clj -1]

  [clojure.lang.Compiler eval Compiler.java 6943]

  [clojure.lang.Compiler eval Compiler.java 6932]

  [clojure.lang.Compiler load Compiler.java 7390]

  [clojure.lang.RT loadResourceScript RT.java 372]

  [clojure.lang.RT loadResourceScript RT.java 363]

  [clojure.lang.RT load RT.java 453]

  [clojure.lang.RT load RT.java 419]

  [clojure.core$load$fn__5436 invoke core.clj 5869]

  [clojure.core$load invokeStatic core.clj 5868]

  [clojure.core$load_one invokeStatic core.clj 5669]

  [clojure.core$load_one invoke core.clj -1]

  [clojure.core$load_lib$fn__5385 invoke core.clj 5714]

  [clojure.core$load_lib invokeStatic core.clj 5713]

  [clojure.core$load_lib doInvoke core.clj -1]

  [clojure.lang.RestFn applyTo RestFn.java 142]

  [clojure.core$apply invokeStatic core.clj 635]

  [clojure.core$load_libs invokeStatic core.clj 5751]

  [clojure.core$load_libs doInvoke core.clj -1]

  [clojure.lang.RestFn applyTo RestFn.java 137]

  [clojure.core$apply invokeStatic core.clj 635]

  [clojure.core$require invokeStatic core.clj 5773]

  [clojure.core$require doInvoke core.clj -1]

  [clojure.lang.RestFn invoke RestFn.java 512]

  [cloud_monkey.core$eval14$loading__5328__auto____15 invoke core.clj 1]

  [cloud_monkey.core$eval14 invokeStatic core.clj 1]

  [cloud_monkey.core$eval14 invoke core.clj -1]

  [clojure.lang.Compiler eval Compiler.java 6943]

  [clojure.lang.Compiler eval Compiler.java 6932]

  [clojure.lang.Compiler load Compiler.java 7390]

  [clojure.lang.RT loadResourceScript RT.java 372]

  [clojure.lang.RT loadResourceScript RT.java 363]

  [clojure.lang.RT load RT.java 453]

  [clojure.lang.RT load RT.java 419]

  [clojure.core$load$fn__5436 invoke core.clj 5869]

  [clojure.core$load invokeStatic core.clj 5868]

  [clojure.core$load_one invokeStatic core.clj 5669]

  [clojure.core$load_one invoke core.clj -1]

  [clojure.core$load_lib$fn__5385 invoke core.clj 5714]

  [clojure.core$load_lib invokeStatic core.clj 5713]

  [clojure.core$load_lib doInvoke core.clj -1]

  [clojure.lang.RestFn applyTo RestFn.java 142]

  [clojure.core$apply invokeStatic core.clj 635]

  [clojure.core$load_libs invokeStatic core.clj 5751]

  [clojure.core$load_libs doInvoke core.clj -1]

  [clojure.lang.RestFn applyTo RestFn.java 137]

  [clojure.core$apply invokeStatic core.clj 635]

  [clojure.core$require invokeStatic core.clj 5773]

  [clojure.core$require doInvoke core.clj -1]

  [clojure.lang.RestFn invoke RestFn.java 408]

  [user$eval5 invokeStatic form-init3436449082767268851.clj 1]

  [user$eval5 invoke form-init3436449082767268851.clj -1]

  [clojure.lang.Compiler eval Compiler.java 6943]

  [clojure.lang.Compiler eval Compiler.java 6932]

  [clojure.lang.Compiler eval Compiler.java 6932]

  [clojure.lang.Compiler load Compiler.java 7390]

  [clojure.lang.Compiler loadFile Compiler.java 7328]

  [clojure.main$load_script invokeStatic main.clj 275]

  [clojure.main$init_opt invokeStatic main.clj 277]

  [clojure.main$init_opt invoke main.clj -1]

  [clojure.main$initialize invokeStatic main.clj 308]

  [clojure.main$null_opt invokeStatic main.clj 342]

  [clojure.main$null_opt invoke main.clj -1]

  [clojure.main$main invokeStatic main.clj 421]

  [clojure.main$main doInvoke main.clj -1]

  [clojure.lang.RestFn invoke RestFn.java 421]

  [clojure.lang.Var invoke Var.java 383]

  [clojure.lang.AFn applyToHelper AFn.java 156]

  [clojure.lang.Var applyTo Var.java 700]

  [clojure.main main main.java 37]]}

[Sean Corfield]

Somewhat related, I had this code which "worked" on Alpha 2:

(defn ^java.nio.ByteBuffer to-byte-array [^com.eaio.uuid.UUID u]
(let [lo (.getClockSeqAndNode u)
hi (.getTime u)]
(-> (java.nio.ByteBuffer/allocate 16)
(.putLong hi)
(.putLong lo)
(.array))))

(can anyone spot the error?)

…which gave this exception on Alpha 3:

[exec] Caused by: java.lang.ClassCastException: [B cannot be cast to java.nio.ByteBuffer
[exec] at worldsingles.data.uuid$to_byte_array.invokeStatic(uuid.clj:33)
[exec] at worldsingles.data.uuid$to_byte_array.invoke(uuid.clj)
[exec] at worldsingles.data.uuid$uuid_as_byte_array.invokeStatic(uuid.clj:36)
[exec] at worldsingles.data.uuid$uuid_as_byte_array.invoke(uuid.clj)
[exec] at worldsingles.news$add_basic_event_data.invokeStatic(news.clj:24)
[exec] at worldsingles.news$add_basic_event_data.invoke(news.clj)

Changing it to the following solved the problem:

(defn ^bytes to-byte-array [^com.eaio.uuid.UUID u]
(let [lo (.getClockSeqAndNode u)
hi (.getTime u)]
(-> (java.nio.ByteBuffer/allocate 16)
(.putLong hi)
(.putLong lo)
(.array))))

It returns a byte[] not a java.nio.ByteBuffer. I actually had three functions with this incorrect type hint (that, yes, all failed on Alpha 3).

I’m assuming that direct linking is causing return type hints to be a) static class names — which is breaking Rangel’s code — and b) are more strictly type-checked — which is breaking my code?

(we’re continuing to run tests on Alpha 3 but this is what we tripped over immediately)

Sean

[Mikera]

Hi Alex, 

1.8.0-alpha3 is breaking core.matrix at the moment.

Root cause appears to be related to how protocols are being handled when used with Java arrays:

e.g. for the protocol implementation:

(extend-protocol mp/PImplementation

  (Class/forName "[Ljava.lang.Object;")

    (implementation-key [m] :object-array)

    (meta-info [m]

      {:doc "Clojure.core.matrix implementation for Java Object arrays"})

    (new-vector [m length] (construct-object-vector (long length)))

    (new-matrix [m rows columns]

      (let [columns (long columns)

            m (object-array rows)]

        (dotimes [i rows]

          (aset m i (construct-object-vector columns)))

        m))

    (new-matrix-nd [m shape]

      (construct-nd shape))

    (construct-matrix [m data]

      (construct-object-array data))

    (supports-dimensionality? [m dims]

      (>= dims 1)))

When called as:

(clojure.core.matrix.protocols/construct-matrix (object-array 1) [1])

Gives exception:

VerifyError (class: clojure/core/matrix$eval10586, method: invokeStatic signature: ()Ljava/lang/Object;) Incompatible object argument for function call  java.lang.Class.getDeclaredConstructors0 (:-2)

[Daniel Compton]

For those wondering like me what "Direct Linking" is, here are a few links I could find. This seems to be a very old possible improvement to Clojure which is being looked at again.

http://dev.clojure.org/display/design/Build+Profiles

https://groups.google.com/forum/#!msg/clojure-dev/guT3QBEg5w0/9deJ5E0AW9sJ

https://groups.google.com/forum/#!msg/clojure-dev/guT3QBEg5w0/e6m0RZ0yk4gJ

[Alex Miller]

Thanks, I forwarded this stuff over to Rich. We appreciate people trying it and reporting stuff!

[Eduardo Aquiles Affonso Radanovitsck]

Not sure if this is expected or not:

(map (fn [v] {:k v}) #{1 2 3})

=> ({:k 1} {:k 3} {:k 2})

(map #({:k %}) #{1 2 3})

ArityException Wrong number of args (0) passed to: PersistentArrayMap  clojure.lang.AFn.throwArity (AFn.java:429)

--

Eduardo Aquiles Radanovitsck

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ben Wolfson]

On Fri, Jul 31, 2015 at 1:57 PM, Eduardo Aquiles Affonso Radanovitsck <eduardoa...@gmail.com> wrote:

Not sure if this is expected or not:

(map (fn [v] {:k v}) #{1 2 3})

=> ({:k 1} {:k 3} {:k 2})

(map #({:k %}) #{1 2 3})

ArityException Wrong number of args (0) passed to: PersistentArrayMap  clojure.lang.AFn.throwArity (AFn.java:429)

The same behavior is visible in earlier versions of clojure and has to do with how the #(...) syntax is parsed. The workaround is #(do {:k %}).

Basically, whatever occurs between the #( and the ) is spliced into a list, so if you have just one single expression, it turns into a function call:

user=> (defmacro p [k] (println k))

#'user/p

user=> (p #({:k %}))

(fn* [p1__7143#] ({:k p1__7143#}))    ;; <---- attempting to invoke a map with zero args

nil

user=> (p #(do {:k %}))

(fn* [p1__7144#] (do {:k p1__7144#}))

nil

--

Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks, which may be sweet, aromatic, fermented or spirit-based. ... Family and social life also offer numerous other occasions to consume drinks for pleasure." [Larousse, "Drink" entry]

[Eduardo Aquiles Affonso Radanovitsck]

Thanks for the clarification!

--

Eduardo Aquiles Radanovitsck

ThoughtWorks Brasil

[Dragan Djuric]

Is there any introduction/example/short intro on direct linking?

[Alex Miller]

I think 1.8.0-alpha4 addresses this issue? Can you verify?

On Wednesday, July 29, 2015 at 8:23:16 PM UTC-5, Mikera wrote:

[Sean Corfield]

Clojure 1.8.0-alpha3 is now available.

…

Tuples have been disabled after further analysis of performance impacts.

Out of curiosity, could this have surfaced in alpha2 as a memory leak?

We had alpha2 in production for about a week and it looked like we were seeing a very slow memory leak. Our next production build was based on alpha4 and we have not seen that same memory curve (in a slightly longer period).

I looked over the alpha3 / alpha4 change logs and didn’t see anything specific about memory leaks (that would be new in alpha2 compared to 1.7.0 final).

Sean Corfield -- (904) 302-SEAN

World Singles -- http://worldsingles.com/

[Alex Miller]

Not to my knowledge but could be I suppose.

[Sean Corfield]

Clojure 1.8.0-alpha3 is now available.

…

Tuples have been disabled after further analysis of performance impacts.

Out of curiosity, could this have surfaced in alpha2 as a memory leak?

We had alpha2 in production for about a week and it looked like we were seeing a very slow memory leak. Our next production build was based on alpha4 and we have not seen that same memory curve (in a slightly longer period).

Now that we’ve had longer in production, we are still seeing a slow memory leak. We are prepping to move back to 1.7.0 for the time being. If we don’t see the leak on 1.7.0 (so we can isolate the leak as coming from our code vs "your" code), we’ll put one server back on 1.8.0 Alpha 4 and see if we can identify what is actually leaking.

[Sean Corfield]

Just to keep everyone informed, we’ve now had enough time back on Clojure 1.7.0 to rule out 1.8.0 as the source of our very slow memory leak.

Now we have the fun task of figuring out exactly what has introduced it :)

Sean Corfield -- (904) 302-SEAN

An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood