[ANN] 1.10.0-beta5

988 views
Skip to first unread message

Alex Miller

unread,
Nov 6, 2018, 8:42:45 AM11/6/18
to Clojure
1.10.0-beta5 is now available.

You can try it with clj using:

      clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-beta5"}}}'

Changes in 1.10.0-beta5:

  • In addition to prior methods of extension, values can now extend protocols by adding metadata where keys are fully-qualified symbols naming protocol functions and values are function implementations. Protocol implementations are checked first for direct definitions (defrecord, deftype, reify), then metadata definitions, then external extensions (extend, extend-type, extend-protocol). datafy has been updated to use this mechanism.
  • `symbol` can now be passed vars or keywords to obtain the corresponding symbol
  • CLJ-2420 error reporting enhancements - more refined phase reporting, new clojure.main/ex-triage split out of clojure.main/ex-str, execution errors now report the top *user* line in the stack trace omitting frames from core, enhancements to providing file and line via meta on a form
  • CLJ-2425 add java 11 javadoc url
  • CLJ-2424 fix test bug from CLJ-2417

You can read the full 1.10 changelog here: https://github.com/clojure/clojure/blob/master/changes.md

John Schmidt

unread,
Nov 6, 2018, 10:25:31 AM11/6/18
to Clojure
Nice to see continued progress on Clojure 1.10!

It is not clear to me what metadata extension provides that is not already possible with direct definitions or external extensions. Some additional background or a small motivating example would be much appreciated in clearing up the confusion!

Alex Miller

unread,
Nov 6, 2018, 10:50:55 AM11/6/18
to Clojure

On Tuesday, November 6, 2018 at 9:25:31 AM UTC-6, John Schmidt wrote:
Nice to see continued progress on Clojure 1.10!

It is not clear to me what metadata extension provides that is not already possible with direct definitions or external extensions. Some additional background or a small motivating example would be much appreciated in clearing up the confusion!

Metadata extension allows you to implement protocols on a per-value basis (the others are type/class-oriented). This opens up a whole new range of potential uses for protocols. Rich was already using this approach for the new datafy/nav functionality - this made it generic for all protocols.

Any collection instance can carry metadata, so you can take any collection instance and dynamically extend a protocol to it by adding metadata. So if you think about something like Component, which has a Lifecycle protocol for start/stop, you can now make lightweight components without needing to make a type or reify:

$ clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-beta5"}, com.stuartsierra/component {:mvn/version "0.3.2"}}}'
Clojure 1.10.0-beta5
user
=> (require '[com.stuartsierra.component :as component])
nil
user=> (def c (with-meta {:state :init}
                {`component/start (fn [c] (assoc c :state :started))
                 `component/stop (fn [c] (assoc c :state :stopped))}))
#'
user/c
user
=> (component/start c)
{:state :started}
user
=> (component/stop c)
{:state :stopped}


Jason Felice

unread,
Nov 6, 2018, 10:58:05 AM11/6/18
to clo...@googlegroups.com
This pattern appears in clojure.zip, also.  (Any point in updating that?)

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Alex Miller

unread,
Nov 6, 2018, 11:18:20 AM11/6/18
to clo...@googlegroups.com
On Tue, Nov 6, 2018 at 9:58 AM Jason Felice <jason.m...@gmail.com> wrote:
This pattern appears in clojure.zip, also.  (Any point in updating that?)

clojure.zip is not currently protocol-oriented, but could be, in which case it's impl would be pretty similar to what it is now (but using the instance protocol support).

John Schmidt

unread,
Nov 7, 2018, 2:06:50 AM11/7/18
to Clojure
Wow, that's a nice example, makes it clear that it brings something new to the table!

Khalid Jebbari

unread,
Nov 7, 2018, 3:12:44 AM11/7/18
to Clojure
Alex, it's funny that the example you showed about using the new extension mechanism looks very much like implementing lifecycle hooks in React.js (and the various CLJ/CLJS wrappers).

Indeed having value-based extension makes it much more flexible than having to use deftype/defrecord. My understanding is that since it's based on metadata, one could extend after the fact with `reset-meta!`/`alter-meta!` right?

Didier

unread,
Nov 7, 2018, 3:40:28 AM11/7/18
to Clojure
Hum, just noticed tap. Seems really interesting. I'm not thinking of any concrete usage for now, but I like it!

Also, is the object in the datafy description referring to a Java object? If so, is it a way to transform nested Java objects into Clojure data, and possibly back?

Khalid Jebbari

unread,
Nov 7, 2018, 4:19:27 AM11/7/18
to Clojure
Example usage of tap: https://quanttype.net/posts/2018-10-18-how-i-use-tap.html

I've also briefly read the source code of the datafy namespace (short and easy to understand), and my understanding is that it's currently for one-way transformation of Java Objects into Clojure data through the Datafiable protocol but it will attach the original object as a metadata to the Clojure data produced. My feeling tells me that datafy will be used to transform a clojure.spec object into data... (to avoid breaking the current spec API). I haven't understood the nav function and associated Navigable protocol yet though.

Alex Miller

unread,
Nov 7, 2018, 9:02:44 AM11/7/18
to Clojure

On Wednesday, November 7, 2018 at 2:12:44 AM UTC-6, Khalid Jebbari wrote:
Alex, it's funny that the example you showed about using the new extension mechanism looks very much like implementing lifecycle hooks in React.js (and the various CLJ/CLJS wrappers).

Indeed having value-based extension makes it much more flexible than having to use deftype/defrecord. My understanding is that since it's based on metadata, one could extend after the fact with `reset-meta!`/`alter-meta!` right?

Yep.

Alex Miller

unread,
Nov 7, 2018, 9:03:48 AM11/7/18
to Clojure


On Wednesday, November 7, 2018 at 3:19:27 AM UTC-6, Khalid Jebbari wrote:
Example usage of tap: https://quanttype.net/posts/2018-10-18-how-i-use-tap.html

I've also briefly read the source code of the datafy namespace (short and easy to understand), and my understanding is that it's currently for one-way transformation of Java Objects into Clojure data through the Datafiable protocol but it will attach the original object as a metadata to the Clojure data produced. My feeling tells me that datafy will be used to transform a clojure.spec object into data... (to avoid breaking the current spec API). I haven't understood the nav function and associated Navigable protocol yet though.

That's a good summary. Wasn't really created for spec, but that is one possible use of it, still TBD. 

Rick Moynihan

unread,
Nov 8, 2018, 5:33:10 AM11/8/18
to clo...@googlegroups.com
Cool, so I guess it's the clojure of equivalent of Ruby's eigenclasses:

f = "some object"
class << f
  def foo
     "new foo method on f"
  end
end

f.foo # => "new foo method on f"

It's a shame this mechanism only works for values implementing IMeta, but I understand the JVM is a little prohibitive in this regard.

Even so I suppose the main use of this is that it lets a caller give you a value, and you can return a plussed up version with new capabilities, without having to return a wrapped value.  Wrapped values are sometimes problematic because they introduce new representations of existing types, and this allows an API to return values to the caller that behave the same as what went in.  Neat!

R.

alex

unread,
Nov 8, 2018, 9:36:35 AM11/8/18
to Clojure
No, that will not work exactly like Ruby eigenclass, cause eigenclass has a priority when method lookup is performed. In Clojure case if instance's class has implementation of method, it will be preferred instead of meta version. If I understand everything correctly.

четверг, 8 ноября 2018 г., 12:33:10 UTC+2 пользователь Rick Moynihan написал:

Rick Moynihan

unread,
Nov 8, 2018, 11:24:45 AM11/8/18
to clo...@googlegroups.com
On Thu, 8 Nov 2018 at 14:36, alex <fmn...@gmail.com> wrote:
No, that will not work exactly like Ruby eigenclass, cause eigenclass has a priority when method lookup is performed. In Clojure case if instance's class has implementation of method, it will be preferred instead of meta version. If I understand everything correctly.

Yes, clearly it'll not be exactly like Ruby eigenclasses; because this is clojure! :-)  My point was really that it's spiritually similar if you consider them as a means of extending individual instances you're given with new functionality.  Though yes, thankfully you can't use this for monkey patching over existing functionality; but who would want to do that?! ;-)

R.

Timothy Baldridge

unread,
Nov 8, 2018, 11:44:34 AM11/8/18
to clo...@googlegroups.com
The instance based polymorphism is a bit wonky in some cases. Can we get some sort of spec that tells us what the rules are for resolution? See these cases where it breaks in some rather strange ways.

Clojure 1.10.0-beta5
user=> (defprotocol ILevel (level [this]))
ILevel
user=> (extend-protocol ILevel
         clojure.lang.IPersistentVector
         (level [this] "interface"))
nil
user=> (extend-protocol ILevel
         Object
         (level [this] "object"))
nil
user=> (level [])
"interface"
user=> (level :key)
"object"
user=> (level (with-meta 'foo {`level (fn [this] "instance")}))
"instance"
user=> (level (with-meta [] {`level (fn [this] "instance")}))
"instance"
user=> (defrecord MyType [] ILevel (level [this] "type"))
user.MyType
;; Fails to override
user=> (level (with-meta (->MyType) {`level (fn [this] "instance")}))
"type"
user=> (defrecord MyType3 [])
user.MyType3
user=> (extend-protocol ILevel
         MyType3
         (level [this] "type"))
nil
user=> (level (->MyType3))
"type"
;; Overrides
user=> (level (with-meta (->MyType3) {`level (fn [this] "instance")}))
"instance"
user=>

So it looks like if someone uses inline protocol extension that class cannot participate in instance level polymorphism. If however they use extend on the type after it's defined then the behavior changes and instance polymorphism is supported.
 
“One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.”
(Robert Firth)

Alex Miller

unread,
Nov 8, 2018, 12:12:17 PM11/8/18
to Clojure

On Thursday, November 8, 2018 at 10:44:34 AM UTC-6, tbc++ wrote:
The instance based polymorphism is a bit wonky in some cases. Can we get some sort of spec that tells us what the rules are for resolution?

From the top of the thread: "Protocol implementations are checked first for direct definitions (defrecord, deftype, reify), then metadata definitions, then external extensions (extend, extend-type, extend-protocol)."  Was that unclear in some way? I think it explains everything you're seeing.

We do plan to update the protocols reference page for final release, but waiting on that. 

We are still doing performance eval on this too - still might change a bit more. 

Timothy Baldridge

unread,
Nov 8, 2018, 12:28:09 PM11/8/18
to clo...@googlegroups.com
Nope, you're right, I missed the "extend, extend-type, extend-protocol" part of the original post. 

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Andy Fingerhut

unread,
Nov 15, 2018, 1:32:20 AM11/15/18
to clo...@googlegroups.com
I have run versions 0.2.6 and 0.3.3 of the Eastwood Clojure lint tool on 84 projects in its "crucible" set of projects that we use for testing Eastwood.  You can find the list at [2].

I tried comparing the output for these pairs of version combinations of (Eastwood version, Clojure version, JDK version):

(0.2.6, 1.9.0, Oracle 1.8.0_192) compared with (0.2.6, 1.10.0-beta5, Oracle 1.8.0_192)
(0.2.6, 1.9.0, OpenJDK 11.0.1) compared with (0.2.6, 1.10.0-beta5, OpenJDK 11.0.1)
(0.3.3, 1.9.0, Oracle 1.8.0_192) compared with (0.3.3, 1.10.0-beta5, Oracle 1.8.0_192)
(0.3.3, 1.9.0, OpenJDK 11.0.1) compared with (0.3.3, 1.10.0-beta5, OpenJDK 11.0.1)

All test runs were run on an Ubuntu Linux 18.04.1 system in a VM on my Intel x86_64 based Mac laptop.

The only differences I found were:

+ differing line numbers of Clojure source code that appeared in stack traces, which is expected in a new Clojure release
+ fewer deprecation warnings when linting Clojure namespace clojure.java.browse-ui, because of intentional changes made for Clojure 1.10.0 to eliminate the use of deprecated APIs [3]
+ a couple of reflection warnings present in namespace clojure.main are gone.  There was no change in the Java interop calls, so perhaps this is because of changes made in how the Clojure compiler resolves Java method calls.

I didn't see anything more than 3 to 4 percent difference in run times across all of the version combinations above.

Andy



--

Alex Miller

unread,
Nov 15, 2018, 9:07:34 AM11/15/18
to Clojure
Thanks Andy!

I'd be curious to see what reflection errors you're talking about in clojure.main just to make sure I understand the difference.

Andy Fingerhut

unread,
Nov 15, 2018, 2:04:41 PM11/15/18
to clo...@googlegroups.com
Understood.  I am curious, too, and will reply to this email thread if I get more details in a coherent form.

The main reason for taking more time here is that it is possible that Eastwood-generated reflection warnings are different than the reflection warnings without Eastwood -- Eastwood reads, analyzes with tools.analyzer.jvm, emits new forms, and then evals those.  tools.analyzer.jvm _might_ be causing the emitted code to generate different reflection warnings than Clojure would on the original code.

Probably the quickest way for me to double-check is just to stick a (set! *warn-on-reflection* true) at the beginning of clojure/main.clj source file for Clojure 1.9.0 and 1.10.0-beta5 and compare the output there.  If those are identical/similar-enough, then it is tools.analyzer.jvm or something else in Eastwood causing the different reflection warnings.

Andy


--

Alex Miller

unread,
Nov 15, 2018, 4:12:51 PM11/15/18
to clo...@googlegroups.com
I looked at current version of main and found some unintentional reflection in new code and expect that to be fixed for next beta. There is also some intended reflection in there, based on the docstrings so I left that there.

Andy Fingerhut

unread,
Nov 16, 2018, 1:59:56 AM11/16/18
to clo...@googlegroups.com
Great, sounds like you are on top of it.  I tried compiling Clojure 1.9.0 and 1.10.0-beta5 with a line (set! *warn-on-reflection* true) in the src/clj/clojure/main.clj source file of both, and the reflection warnings appear to be different from each other when building Clojure, in a similar way to how they are different when running Eastwood on that namespace of those two different versions, and the differences in the reflection warnings don't surprise me at all given the code changes there.

Andy

Reply all
Reply to author
Forward
0 new messages