[ANN] Clojure 1.9.0-alpha11

[Alex Miller]

Clojure 1.9.0-alpha11 is now available.

Try it via

- Download: https://repo1.maven.org/maven2/org/clojure/clojure/1.9.0-alpha11

- Leiningen: [org.clojure/clojure "1.9.0-alpha11"]

1.9.0-alpha11 includes the following changes since 1.9.0-alpha10:

Clojure now has specs for the following clojure.core macros: let, if-let, when-let, defn, defn-, fn, and ns. Because macro specs are checked during macroexpansion invalid syntax in these macros will now fail at compile time whereas some errors were caught at runtime and some were not caught at all.

- CLJ-1914 - Fixed race condition in concurrent range realization

- CLJ-1870 - Fixed reloading a defmulti removes metadata on the var

- CLJ-1744 - Clear unused locals, which can prevent memory leaks in some cases

- CLJ-1423 - Allow vars to be invoked with infinite arglists (also, faster)

- CLJ-1993 - Added *print-namespace-maps* dynamic var that controls whether to use namespace map syntax for maps with keys from the same namespace. The default is false, but standard REPL bindings set this to true.

- CLJ-1985 - Fixed with-gen of conformer losing unform fn

- Fixed clojure.spec.test/check to skip spec'ed macros

- Fixed regression from 1.9.0-alpha8 where type hints within destructuring were lost

- Fixed clojure.spec/merge docstring to note merge doesn't flow conformed values

- Fixed regex ops to use gen overrides if they are used

[Alex Miller]

One thing to note is that the new specs for clojure.core means that there is a reasonable amount of (broken) code in the wild that now does not compile.

I have filed PRs in a number of commonly used projects and while those were all merged, most have not yet been released into new versions yet. Any project that uses any of those projects via dependencies will also have issues. So, expect to see some breakage if using the latest alpha. In particular, ClojureScript itself had a bad ns declaration through version 1.9.93 so anything not on a fresh CLJS will likely see that.

The three most common things I saw were:

1. (ns ... (require ...) (import ...) )  instead of (ns ... (:require ...) (:import ...) )

2. (fn a.b [] ...) - fully-qualified symbol names in fn (these are pretty much always the result of a sloppy macro that expands to fn)

3. (let [{ ... :or {:a 1}} {}] ... ) - :or keys that are not simple symbols. There were some accidental cases where this did something for fully-qualified keyword keys in the past, but those were not intended and are no longer accepted as of Clojure 1.9.0-alpha8. All of the cases I found were actually typos though where the keyword was used instead of the symbol.

[Howard M. Lewis Ship]

I saw your PR for io.aviso/pretty ... thanks!

I can't imagine how much work it must be to track all of those things down ...

[Rangel Spasov]

Ran into this problem with ClojureScript/ReactNative/Re-natal project:

Compiling ClojureScript...

Exception in thread "main" java.lang.IllegalArgumentException: Call to clojure.core/ns did not conform to spec:

In: [3 0] val: import fails spec: :clojure.core.specs/ns-refer-clojure at: [:args :clauses :refer-clojure :clause] predicate: #{:refer-clojure}

In: [3 0] val: import fails spec: :clojure.core.specs/ns-require at: [:args :clauses :require :clause] predicate: #{:require}

In: [3 0] val: import fails spec: :clojure.core.specs/ns-import at: [:args :clauses :import :clause] predicate: #{:import}

In: [3 0] val: import fails spec: :clojure.core.specs/ns-use at: [:args :clauses :use :clause] predicate: #{:use}

In: [3 0] val: import fails spec: :clojure.core.specs/ns-refer at: [:args :clauses :refer :clause] predicate: #{:refer}

In: [3 0] val: import fails spec: :clojure.core.specs/ns-load at: [:args :clauses :load :clause] predicate: #{:load}

In: [3 0] val: import fails spec: :clojure.core.specs/ns-gen-class at: [:args :clauses :gen-class :clause] predicate: #{:gen-class}

:clojure.spec/args  (cljsbuild.crossover (:use [clojure.java.io :only [as-url resource]]) (:require [cljsbuild.util :as util] [clojure.string :as string] [fs.core :as fs]) (import java.io.File java.net.URLDecoder))

, compiling:(cljsbuild/crossover.clj:1:1)

at clojure.lang.Compiler.macroexpand1(Compiler.java:6795)

at clojure.lang.Compiler.macroexpand(Compiler.java:6861)

at clojure.lang.Compiler.eval(Compiler.java:6935)

at clojure.lang.Compiler.load(Compiler.java:7403)

at clojure.lang.RT.loadResourceScript(RT.java:374)

at clojure.lang.RT.loadResourceScript(RT.java:365)

at clojure.lang.RT.load(RT.java:455)

at clojure.lang.RT.load(RT.java:421)

at clojure.core$load$fn__7645.invoke(core.clj:6008)

at clojure.core$load.invokeStatic(core.clj:6007)

at clojure.core$load.doInvoke(core.clj:5991)

at clojure.lang.RestFn.invoke(RestFn.java:408)

at clojure.core$load_one.invokeStatic(core.clj:5812)

at clojure.core$load_one.invoke(core.clj:5807)

at clojure.core$load_lib$fn__7590.invoke(core.clj:5852)

at clojure.core$load_lib.invokeStatic(core.clj:5851)

at clojure.core$load_lib.doInvoke(core.clj:5832)

at clojure.lang.RestFn.applyTo(RestFn.java:142)

at clojure.core$apply.invokeStatic(core.clj:659)

at clojure.core$load_libs.invokeStatic(core.clj:5889)

at clojure.core$load_libs.doInvoke(core.clj:5873)

at clojure.lang.RestFn.applyTo(RestFn.java:137)

at clojure.core$apply.invokeStatic(core.clj:659)

at clojure.core$require.invokeStatic(core.clj:5911)

at clojure.core$require.doInvoke(core.clj:5911)

at clojure.lang.RestFn.invoke(RestFn.java:436)

at user$eval15.invokeStatic(form-init1217511081856523183.clj:1)

at user$eval15.invoke(form-init1217511081856523183.clj:1)

at clojure.lang.Compiler.eval(Compiler.java:6951)

at clojure.lang.Compiler.eval(Compiler.java:6940)

at clojure.lang.Compiler.load(Compiler.java:7403)

at clojure.lang.Compiler.loadFile(Compiler.java:7341)

at clojure.main$load_script.invokeStatic(main.clj:276)

at clojure.main$init_opt.invokeStatic(main.clj:278)

at clojure.main$init_opt.invoke(main.clj:278)

at clojure.main$initialize.invokeStatic(main.clj:309)

at clojure.main$null_opt.invokeStatic(main.clj:343)

at clojure.main$null_opt.invoke(main.clj:340)

at clojure.main$main.invokeStatic(main.clj:422)

at clojure.main$main.doInvoke(main.clj:385)

at clojure.lang.RestFn.applyTo(RestFn.java:137)

at clojure.lang.Var.applyTo(Var.java:700)

at clojure.main.main(main.java:37)

Caused by: java.lang.IllegalArgumentException: Call to clojure.core/ns did not conform to spec:

In: [3 0] val: import fails spec: :clojure.core.specs/ns-refer-clojure at: [:args :clauses :refer-clojure :clause] predicate: #{:refer-clojure}

In: [3 0] val: import fails spec: :clojure.core.specs/ns-require at: [:args :clauses :require :clause] predicate: #{:require}

In: [3 0] val: import fails spec: :clojure.core.specs/ns-import at: [:args :clauses :import :clause] predicate: #{:import}

In: [3 0] val: import fails spec: :clojure.core.specs/ns-use at: [:args :clauses :use :clause] predicate: #{:use}

In: [3 0] val: import fails spec: :clojure.core.specs/ns-refer at: [:args :clauses :refer :clause] predicate: #{:refer}

In: [3 0] val: import fails spec: :clojure.core.specs/ns-load at: [:args :clauses :load :clause] predicate: #{:load}

In: [3 0] val: import fails spec: :clojure.core.specs/ns-gen-class at: [:args :clauses :gen-class :clause] predicate: #{:gen-class}

:clojure.spec/args  (cljsbuild.crossover (:use [clojure.java.io :only [as-url resource]]) (:require [cljsbuild.util :as util] [clojure.string :as string] [fs.core :as fs]) (import java.io.File java.net.URLDecoder))

at clojure.spec$macroexpand_check.invokeStatic(spec.clj:627)

at clojure.spec$macroexpand_check.invoke(spec.clj:616)

at clojure.lang.AFn.applyToHelper(AFn.java:156)

at clojure.lang.AFn.applyTo(AFn.java:144)

at clojure.lang.Var.applyTo(Var.java:700)

at clojure.lang.Compiler.macroexpand1(Compiler.java:6789)

... 42 more

Error encountered performing task 'cljsbuild' with profile(s): 'prod'

[Alex Miller]

That looks like a bug in lein-cljsbuild. cljsbuild/crossover.clj is failing due to (import java.io.File java.net.URLDecoder) - should be (:import java.io.File java.net.URLDecoder).

PR filed here: https://github.com/emezeske/lein-cljsbuild/pull/447

[Rangel Spasov]

Great - thanks Alex! All the efforts around clojure.spec will definitely bring consistency in the world of Clojure codebases : )

[se...@corfield.org]

The cljsbuild.crossover namespace has (import …) instead of (:import …) – file a bug report against that project so the maintainer can update it to be compatible with Clojure 1.9.0.

 

The error message says In: [3 0] so you can see it is the 0^th element of the item with index 3 in the ns form that failed the spec.

 

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

 

[Alan Moore]

All,

Is anyone else experiencing problems using danielz/system and/or stuartsierra/component? I'm getting the following:

Caused by: java.lang.IllegalArgumentException: Call to clojure.core/ns did not conform to spec:

In: [1] val: ((require [com.stuartsierra.component :as component] [clojure.tools.namespace.track :as track] [system.reload :as reload] [clojure.stacktrace :as st] [io.aviso.ansi :refer [bold-red bold-yellow]])) fails at: [:args] predicate: (cat :docstring (? string?) :attr-map (? map?) :clauses :clojure.core.specs/ns-clauses),  Extra input

:clojure.spec/args  (system.repl (require [com.stuartsierra.component :as component] [clojure.tools.namespace.track :as track] [system.reload :as reload] [clojure.stacktrace :as st] [io.aviso.ansi :refer [bold-red bold-yellow]]))

I noticed Alex's PR for aviso but I'm not using it explicitly - it must be coming in transiently but I don't think that is the problem:

I can see the use of (require ...) in the implicated ns-form, here:

(system.repl (require [com.stuartsierra.component :as component] ...

but I searched my project for any "(require" and didn't find any.

There must be some macro or namespace magic happening in either of those two libraries. I've been searching them for any use of (require ...) vs (:require ...) and only found a couple in danielz/system, here:

src/system/reload.clj

src/system/repl.clj

but I'm not exactly sure how those would cause my error. The second one looks like it is related.

Alex - does a previously compiled library, like system, pulled from clojars uses "(require" be subject to the spec conformance? How much of the ns-form is resolved at compile time and how much happens at runtime?

Clearly I've not done my deep-dive homework on that form (yet) and how it works... I know those two libraries do a bit of namespace manipulation to deal with reloading repls and whatnot so it very well could be something they are doing that is beyond my current understanding and pay grade.

Alan

On Friday, August 19, 2016 at 11:35:31 AM UTC-7, Alex Miller wrote:

[Alex Miller]

Those are bugs in system. When you see "(system.repl (require [com.stuartsierra.component :as component] ..." those are the arguments to the ns form, so the "system.repl" is the namespace name. You cut out the stack trace, but I'd expect the top frame to be in that ns.

PRs filed at https://github.com/danielsz/system/pull/93 and https://github.com/danielsz/system/pull/94 (found two problems)

All Clojure code is compiled. Some is compiled early (during AOT) but most is compiled late (during load). At the point where you loaded these namespaces, they were compiled to be invoked and you triggered the errors.

Alex

[Alex Miller]

system also depends on the latest released version of monger, which has an invalid defn in it (:or [ ] instead of :or { }) in monger.core. afaict, this problem has been fixed in master of monger, but no new version is available yet.

[Alan Moore]

That is what I figured/suspected but wasn't sure. Thanks for filing those pull requests so quickly!

Alan

--

You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

clojure+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/_slHTn-Ej1Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

-- 
"Whatever you can do, or dream you can do, begin it. Boldness has genius, power, and magic in it. Begin it now." - Goethe

[Colin Fleming]

(ns ... (require ...) (import ...) )  instead of (ns ... (:require ...) (:import ...) )

Is this not intended to be allowed? The docstring implies no but it has always worked fine - I wouldn't call the code broken.

Does the spec also limit the refs to refer-clojure/require/use/import/load/gen-class rather than allowing, say, println? One not on that list that I have seen people using in the wild is refer.

--

You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---

You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.

[Nicola Mometto]

I've used `:alias` to alias clojure.core in the past, I've seen it being used in more than one project

[Alex Miller]

On Saturday, August 20, 2016 at 2:32:29 AM UTC-5, Colin Fleming wrote:

(ns ... (require ...) (import ...) )  instead of (ns ... (:require ...) (:import ...) )

Is this not intended to be allowed? The docstring implies no but it has always worked fine - I wouldn't call the code broken.

It has never been intended to work and afaik there is no documentation that would lead you to believe it should.

 

Does the spec also limit the refs to refer-clojure/require/use/import/load/gen-class rather than allowing, say, println? One not on that list that I have seen people using in the wild is refer.

Yes - it is limited (you can read the ns spec at https://github.com/clojure/clojure/blob/master/src/clj/clojure/core/specs.clj#L187). 

refer is included. While it's not explicitly listed in the docstring its use is implied (by :refer-clojure) so I ended up including it.

 

[Alex Miller]

alias is not included and I have yet to find code using it in the wild. That said, it exists in the gray area as well. If it becomes an issue and/or Rich thinks it should be added, that could be done. I suspect that if we have any changes related to aliasing non-existing namespaces (for qualified keyword ease of use) that we might consider it more in that context.

[Roc King]

I have two questions about spec and ns. In clojure/core/specs.clj:

  * both ::ns-require and ::prefix-list use s/cat.
  * ::refer uses s/coll-of.

So both list and vector could be used, right?

i.e. the following ns declarations work in 1.9.0-alpha11:

  * (ns n (:require [clojure.set :refer [map-invert]]))
  * (ns n [:require [clojure.set :refer [map-invert]]])
  * (ns n [:require [clojure.set :refer (map-invert)]])

Q1: Are all of these intended behaviors?

There are cases that some of the expanded expressions will fail at runtime while the original (ns ...) will pass the spec:

  * (require '(clojure.set :refer [map-invert])) one of the expression produced by (ns n [:require (clojure.set :refer [map-invert])]).
  * (require) produced by (ns n (:require)).

In the (ns n (:require)) case, we could simply:

  * change ns so that it will discard (:require) completely.
  * change s/* to s/+ in ::ns-require.

But there exists many other mismatches between specs and implementations. Some of them may be subtler or more complicated than (ns n (:require)) and couldn't be fixed easily.

Q2: When mismatches occurs, which one(the spec, the current implementation or none of them) should be taken as the formal definition of Clojure language?

Thanks!

[Ambrose Bonnaire-Sergeant]

Hi,

There seems to be some issues instrumenting functions that

compile to .invokePrim calls.

user=> (s/fdef blah :args (s/cat :foo int?) :ret any?)

user/blah

user=> (defn blah [^long a])

#'user/blah

user=> (st/instrument)

[clojure.core/symbol user/blah]

user=> (blah 1)

ClassCastException clojure.spec.test$spec_checking_fn$fn__13057 cannot be cast to clojure.lang.IFn$LO  user/eval70134 (form-init2184258571072833082.clj:1)

user=> 

I'm about to tackle this problem in core.typed, so I should have some useful

utilities to fix this soon (if interested).

Thanks,

Ambrose

[Kevin Downey]

instrument is going to have issues with any calling convention outisde
of the normal deref var + invoking path.
http://dev.clojure.org/jira/browse/CLJ-1941 is a similar case with some
discussion in the comments.

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to

> clojure+u...@googlegroups.com

> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to clojure+u...@googlegroups.com
> <mailto:clojure+u...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.


--

And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?

[Ambrose Bonnaire-Sergeant]

invokePrim seems to be the most common alternative calling convention.

I've worked around this myself for core.typed, basically generating a

deftype with the correct primitive invoke interfaces based on the :arglists

of the current var we're instrumenting.

I can prepare a patch if this approach isn't too hacky for core.

https://gist.github.com/frenchy64/7eb65d244a4edbb3a10dd4d9f889a3bd

Thanks,

Ambrose