[ANN] Clojure 1.7.0-alpha6 released

2,285 views
Skip to first unread message

Alex Miller

unread,
Mar 31, 2015, 12:51:13 PM3/31/15
to clo...@googlegroups.com, cloju...@googlegroups.com
Clojure 1.7.0-alpha6 is now available.

Try it via
- Leiningen: [org.clojure/clojure "1.7.0-alpha6"]

Regression fixes from previous alphas (and one from 1.6):

1) CLJ-1544 was rolled back and will be investigated for a future release.
2) CLJ-1637 fixed regression with vec on MapEntry 
3) CLJ-1663 fixed regression in classloader (affected Cursive)
4) CLJ-1638 fixed regression with removed PersistentVector.create(List) method
5) CLJ-1681 fixed regression in 1.6 with NPE on reflection warning for literal nil arg
6) CLJ-1604 fixed problem with AOT and shadowing clojure.core symbols that prevented update

Some highlights new in alpha6:

## Transducer-related changes:

The LazyTransformer introduced to create lazy transforming sequences has been 
replaced with a TransformingIterator. This was done to simplify the code
around transformations and to make certain use cases around eduction more efficient.

## Faster reduce, iterator, and sequence paths

A lot of work has been done across a set of tickets to improve the ability of
collections to provide more efficient reduce or iterator performance, and also to
make common sequence generators create faster sequence and reduce paths. You
should see significant performance in many reduce-related paths (this includes 
reduce, transduce, into, and anything else built on reduce). 

Many of those changes also have beneficial sequence performance, so you may see
some benefits even in code that does not use transducers.

* Most uses of SeqIterator have now been replaced with iterators that directly walk
the underlying source for improved efficiency. This includes maps, sets, records, etc.
* repeat - now returns a faster sequence with a fast reduce path
* cycle - now returns a faster sequence with a fast reduce path
* iterate - now returns a faster sequence with a fast reduce path
* range - (did not quite make it in, but coming soon...)
* keys - iterates directly over the keys of a map, without seq or MapEntry allocation
* vals - iterates directly over the vals of a map, without seq or MapEntry allocation
* iterator-seq - now creates a chunked sequence when previously it was unchunked
* vec and set - were not changed in this release but were set up in a previous alpha
  to take advantage of the reduce and iterator changes above

## Reader conditionals

Reader Conditionals is a new capability to support portable code that
can run on multiple Clojure platforms with only small changes. In
particular, this feature aims to support the increasingly common case
of libraries targeting both Clojure and ClojureScript.

Code intended to be common across multiple platforms should use a new
supported file extension: ".cljc". When requested to load a namespace,
the platform-specific file extension (.clj, .cljs) will be checked
prior to .cljc.

A new reader form can be used to specify "reader conditional" code in
cljc files (and *only* cljc files). Each platform defines a feature
identifying the platform (:clj, :cljs, :cljr). The reader conditional
specifies code that is read conditionally based on the feature/

Form #? takes a list of alternating feature and expression. These are
checked like cond and the selected expression is read and returned. Other
branches are unread. If no branch is selected, the reader reads nothing
(not nil, but literally as if reading ""). An optional ":default" branch
can be used as a fallthrough.

Reader conditional with 2 features and a default:

#?(:clj     Double/NaN
  :cljs    js/NaN
  :default nil)

There is also a reader conditional splicing form. The evaluated expression
should be sequential and will be spliced into the surrounded code, similar
to unqoute-splicing.

For example:

   [1 2 #?@(:clj [3 4] :cljs [5 6])]

This form would read as [1 2 3 4] on Clojure, [1 2 5 6] on ClojureScript,
and [1 2] on any other platform.

Additionally, the reader can now be invoked with options for the features
to use and how to interpret reader conditionals. By default, reader conditionals
are not allowed, but that can be turned on, or a "preserve" mode can be used to
preserve all branches (most likely useful for tooling or source transforms).

In the preserve mode, the reader conditional itself and any tagged literals
within the unselected branches are returned as tagged literal data.

For more information, see:

Two important side notes:
- Clojure dependencies have been updated and you must re-run antsetup.sh if you
build Clojure locally with ant.
- The equivalent ClojureScript support for reader conditionals is not yet available
but is a top priority to release as soon as possible... stay tuned.

## Printing as data

There have been enhancements in how the REPL prints values without a
print-method, specifically Throwable and the fallthrough Object case.
Both cases now print in a tagged literal data form that can be read
by the reader.

Unhandled objects print with the class, hash code, and toString:

user=> *ns*
#object[clojure.lang.Namespace 0x55aa628 "user"]

Thrown exceptions will still be printed in the normal way by the default
REPL but printing them to a stream will show a different form:

user=> (/ 1 0)
ArithmeticException Divide by zero  clojure.lang.Numbers.divide (Numbers.java:158)
user=> (println *e)
#error{:cause Divide by zero,
  :via [{:type java.lang.ArithmeticException,
     :message Divide by zero,
 :at [clojure.lang.Numbers divide Numbers.java 158]}],
  :trace
    [[clojure.lang.Numbers divide Numbers.java 158]
 [clojure.lang.Numbers divide Numbers.java 3808]
 [user$eval5 invoke NO_SOURCE_FILE 3]
 ;; elided ...
 ]]}
 
For all changes new in alpha6, see the issues marked "(alpha6)" in the
full changes below.

------------------------------------------------------------
Clojure 1.7.0-alpha6 has the following updates since 1.6.0:

## 1 New and Improved Features

### 1.1 Transducers

Transducers is a new way to decouple algorithmic transformations from their
application in different contexts. Transducers are functions that transform
reducing functions to build up a "recipe" for transformation.


Many existing sequence functions now have a new arity (one fewer argument
than before). This arity will return a transducer that represents the same
logic but is independent of lazy sequence processing. Functions included are:

* conj (conjs to [])
* map
* mapcat
* filter
* remove
* take
* take-while
* drop
* drop-while
* take-nth
* replace
* partition-by
* partition-all
* keep
* keep-indexed
* map-indexed
* distinct
* interpose

Additionally some new transducer functions have been added:

* cat - concatenates the contents of each input
* dedupe - removes consecutive duplicated values
* random-sample - returns items from coll with random probability

And this function can be used to make completing transforms:

* completing

There are also several new or modified functions that can be used to apply
transducers in different ways:

* sequence - takes a transformation and a coll and produces a lazy seq
* transduce - reduce with a transformation (eager)
* eduction - returns a reducible/iterable of applications of the transducer to items in coll. Applications are re-performed with every reduce/iterator.
* run! - run the transformation for side effects on the collection

There have been a number of internal changes to support transducers:

* volatiles - there are a new set of functions (volatile!, vswap!, vreset!, volatile?) to create and use volatile "boxes" to hold state in stateful transducers. Volatiles are faster than atoms but give up atomicity guarantees so should only be used with thread isolation.
* array iterators - added support for iterators over arrays

Some related issues addressed during development:

### 1.2 Reader Conditionals (alpha6)

Reader Conditionals is a new capability to support portable code that
can run on multiple Clojure platforms with only small changes. In
particular, this feature aims to support the increasingly common case
of libraries targeting both Clojure and ClojureScript.

Code intended to be common across multiple platforms should use a new
supported file extension: ".cljc". When requested to load a namespace,
the platform-specific file extension (.clj, .cljs) will be checked
prior to .cljc.

A new reader form can be used to specify "reader conditional" code in
cljc files (and *only* cljc files). Each platform defines a feature
identifying the platform (:clj, :cljs, :cljr). The reader conditional
specifies code that is read conditionally based on the feature/

Form #? takes a list of alternating feature and expression. These are
checked like cond and the selected expression is read and returned. Other
branches are unread. If no branch is selected, the reader reads nothing
(not nil, but literally as if reading ""). An optional ":default" branch
can be used as a fallthrough.

Reader conditional with 2 features and a default:

#?(:clj     Double/NaN
  :cljs    js/NaN
  :default nil)

There is also a reader conditional splicing form. The evaluated expression
should be sequential and will be spliced into the surrounded code, similar
to unqoute-splicing.

For example:

   [1 2 #?@(:clj [3 4] :cljs [5 6])]

This form would read as [1 2 3 4] on Clojure, [1 2 5 6] on ClojureScript,
and [1 2] on any other platform.

Additionally, the reader can now be invoked with options for the features
to use and how to interpret reader conditionals. By default, reader conditionals
are not allowed, but that can be turned on, or a "preserve" mode can be used to
preserve all branches (most likely useful for tooling or source transforms).

In the preserve mode, the reader conditional itself and any tagged literals
within the unselected branches are returned as tagged literal data.

For more information, see:


### 1.3 Keyword and Symbol Construction

In response to issues raised in [CLJ-1439](http://dev.clojure.org/jira/browse/CLJ-1439),
several changes have been made in symbol and keyword construction:

1) The main bottleneck in construction of symbols (which also occurs inside keywords) was
interning of the name and namespace strings. This interning has been removed, resulting
in a performance increase.

2) Keywords are cached and keyword construction includes a cache check. A change was made
to only clear the cache reference queue when there is a cache miss.

### 1.4 Warn on Boxed Math

One source of performance issues is the (unintended) use of arithmetic operations on
boxed numbers. To make detecting the presence of boxed math easier, a warning will now
be emitted about boxed math if \*unchecked-math* is set to :warn-on-boxed (any truthy
value will enable unchecked-math, only this specific value enables the warning).

Example use:

    user> (defn plus-2 [x] (+ x 2))  ;; no warning, but boxed
#'user/plus-2
    user> (set! *unchecked-math* :warn-on-boxed)
true
    user> (defn plus-2 [x] (+ x 2)) ;; now we see a warning
    Boxed math warning, NO_SOURCE_PATH:10:18 - call: public static java.lang.Number
clojure.lang.Numbers.unchecked_add(java.lang.Object,long).
    #'user/plus-2
user> (defn plus-2 [^long x] (+ x 2)) ;; use a hint to avoid boxing
#'user/plus-2


### 1.5 update - like update-in for first level

`update` is a new function that is like update-in specifically for first-level keys:

    (update m k f args...)

Example use:

    user> (update {:a 1} :a inc)
{:a 2}
user> (update {:a 1} :a + 2)
{:a 3}
user> (update {} :a identity)  ;; missing returns nil
{:a nil}


### 1.6 Faster reduce and iterator paths (alpha6)

Several important Clojure functions now return sequences that also
contain fast reduce() (or in some cases iterator()) paths. In many
cases, the new implementations are also faster for lazy sequences

* repeat - now implements IReduce
* cycle - implements IReduceInit
* iterate - implements IReduceInit
* range - implements IReduce, specialized case handles common case of all longs
* keys - iterates directly over the keys of a map, without seq or MapEntry allocation
* vals - iterates directly over the vals of a map, without seq or MapEntry allocation
* iterator-seq - creates a chunked sequence when previously it was unchunked

Additionally, hash-maps and hash-sets now provide iterators that walk
the data structure directly rather than via a sequence.

A new interface (IMapIterable) for direct key and val iterators on maps
was added. External data structures can use this interface to provide
direct key and val iterators via keys and vals.

These enhancements are particularly effective when used
in tandem with transducers via transduce, sequence, into, and
eduction.


### 1.7 Printing as data (alpha6)

There have been enhancements in how the REPL prints values without a
print-method, specifically Throwable and the fallthrough Object case.
Both cases now print in a tagged literal data form that can be read
by the reader.

Unhandled objects print with the class, hash code, and toString:

user=> *ns*
#object[clojure.lang.Namespace 0x55aa628 "user"]

Thrown exceptions will still be printed in the normal way by the default
REPL but printing them to a stream will show a different form:

user=> (/ 1 0)
ArithmeticException Divide by zero  clojure.lang.Numbers.divide (Numbers.java:158)
user=> (println *e)
#error{:cause Divide by zero,
  :via [{:type java.lang.ArithmeticException,
     :message Divide by zero,
 :at [clojure.lang.Numbers divide Numbers.java 158]}],
  :trace
    [[clojure.lang.Numbers divide Numbers.java 158]
 [clojure.lang.Numbers divide Numbers.java 3808]
 [user$eval5 invoke NO_SOURCE_FILE 3]
 ;; elided ...
 ]]}

## 2 Enhancements

### 2.1 Error messages

  Invalid defrecord results in exception attributed to consuming ns instead of defrecord ns
  Report line,column, and source in defmacro errors
  Give more specific hint if namespace with "-" not found to check file uses "_"

### 2.2 Documentation strings

  clojure.java.io/input-stream has incorrect docstring
  Fix typo in gen-class doc-string
  Fix typo in filterv example
  Fix typo in defmulti docstring
  Fix typo in deftype docstring
  Fix typo in clojure.main usage

### 2.3 Performance

  Improve performance of partial with more unrolling
  clojure.core/set should use transients for better performance
  Cache unknown multimethod value default dispatch
  Reduce compile times by avoiding unnecessary calls to Class.forName()
  vec is now faster on almost all inputs
  set is now faster on almost all inputs

### 2.4 Other enhancements

  Improve apropos to show some indication of namespace of symbols found
  Hints don't work with #() form of function
  Removes owner-thread check from transients - this check was preventing some valid usage of transients in core.async where a transient is created on one thread and then used again in another pooled thread (while still maintaining thread isolation).
  Extracted IAtom interface implemented by Atom.
  Don't initialize classes when importing them
  Class name clash between top-level functions and defn'ed ones
  Update to latest test.generative and add dependency on test.check
  vec now works with things that only implement Iterable or IReduceInit
  set now works with things that only implement Iterable or IReduceInit
  PersistentList/creator doesn't handle ArraySeqs correctly
  Clean up unused paths in InternalReduce
  Add setLineNumber() to LineNumberingPushbackReader
  Change test to avoid using hard-coded socket port
  Change reduce tests to better catch reduce without init bugs

## 3 Bug Fixes

  Reduce broken on some primitive vectors
  Equality bug on records created with nested calls to map->record
  Unable to set compiler options via system properties except for AOT compilation
  NPE when AOTing overrided clojure.core functions
  reductions does not check for reduced value
  Using def with metadata {:type :anything} throws ClassCastException during printing
  Error when calling primitive functions with destructuring in the arg vector
  Piping seque into seque can deadlock
  <= is incorrect when args include Double/NaN
  Make cached string value of Keyword and Symbol transient
  clojure.core/bean should implement Iterable
  Make refer of Clojure core function not throw exception on reload
  LazySeq equals() should not use equiv() logic
  into (and other fns that rely on reduce) require only IReduceInit
  PersistentVector now directly implements reduce without init
  Transient collections should guarantee thread visibility
  Some IReduce/IReduceInit implementors don't respect reduced
  Clojure resolves to wrong deftype classes when AOT compiling or reloading
  Fix intermittent SeqIterator problem by removing use of this as a sentinel
  Fix regression from CLJ-1546 that broke vec on MapEntry
  Fix regression from CLJ-979 for DynamicClassLoader classloader delegation
  Fix error from AOT'ed code defining a var with a clojure.core symbol name
  Fix incorrect line number reporting for error locations
  Fix incorrect line number reporting for error locations
  Fix regression from CLJ-1546 removed PersistentVector.create(List) method
  Fix regression from CLJ-1248 (1.6) in reflection warning with literal nil argument



Alex Miller

unread,
Mar 31, 2015, 1:29:06 PM3/31/15
to clo...@googlegroups.com, cloju...@googlegroups.com
In case anyone is curious about the path from here to a final release, the remaining items on the 1.7 list can always be found here:


The main "feature" things to be done before a beta are optimized range and the socket repl (which is undergoing a bit of a refactor in its conception). However, I am prioritizing the completion of reader conditionals for tools.reader and ClojureScript first.

After that, it's just a matter of working any issues or regressions down to release. The more people that try alpha6 and give feedback (positive or negative), the faster we can know what if anything needs to be done before release, so your assistance is welcomed!! Please give it a shot on your own project or library.

Because a lot of performance-related changes went into alpha6, I'm also interested in feedback related to that (good or bad).

Thanks,
Alex

Geraldo Lopes de Souza

unread,
Mar 31, 2015, 1:40:31 PM3/31/15
to clo...@googlegroups.com, cloju...@googlegroups.com
Hi,

The fancy printing of exceptions is not working for me. 

Ubuntu 14.04.2 LTS
java version "1.8.0_40"
plain clojure jar or lein 2.5.1
gnome terminal

Regards,

Geraldo

Karsten Schmidt

unread,
Mar 31, 2015, 1:42:19 PM3/31/15
to cloju...@googlegroups.com, clo...@googlegroups.com
This is exciting! Thanks Alex et al! Been using alpha5 w/o problems in
several projects (and w/ noticeable improvements when I started using
transducers) and will switch to alpha6 asap.
> --
> You received this message because you are subscribed to the Google Groups
> "Clojure Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure-dev...@googlegroups.com.
> To post to this group, send email to cloju...@googlegroups.com.
> Visit this group at http://groups.google.com/group/clojure-dev.
> For more options, visit https://groups.google.com/d/optout.



--
Karsten Schmidt
http://postspectacular.com | http://toxiclibs.org | http://toxi.co.uk

Alex Miller

unread,
Mar 31, 2015, 2:36:40 PM3/31/15
to clo...@googlegroups.com, cloju...@googlegroups.com
As I mentioned in the change log, the current repl chooses to continue printing exceptions with no change. But if you print an exception yourself with, for example, (println *e), you should see it. Currently you'll see that printed without any pretty-printing - there will likely still be some changes around the exception printing before release.
...

Sean Corfield

unread,
Mar 31, 2015, 6:21:42 PM3/31/15
to cloju...@googlegroups.com, clo...@googlegroups.com
Looks like a great set of updates!

Unfortunately, as several of us found out today, the change to the StringReader invoke() signature breaks Instaparse so I’m blocked from testing the World Singles code base with alpha6 (or master) at the moment. Is that just a hazard of relying on the internals of reader classes or would Clojure/core consider that breakage avoidable (by adding an overloaded signature instead of just changing the current signature)? Since those readers extend AFn, it seems that there is an expectation that they would be used as functions in Clojure code out in the wild...

Sean

Michael Blume

unread,
Mar 31, 2015, 6:36:49 PM3/31/15
to cloju...@googlegroups.com, clo...@googlegroups.com
I've proposed a patch to instaparse to fix this, I realize it's not the most elegant version check ever, but it should fix the problem https://github.com/Engelberg/instaparse/pull/94

Alex Miller

unread,
Mar 31, 2015, 6:57:08 PM3/31/15
to clo...@googlegroups.com
From my perspective, Instaparse is reaching pretty far into the guts there. I'll talk to Rich about it though.

Mark Engelberg

unread,
Mar 31, 2015, 7:01:02 PM3/31/15
to clojure
I can push a new version of instaparse incorporating Michael Blume's pull request by the end of the day.

On Tue, Mar 31, 2015 at 3:57 PM, Alex Miller <al...@puredanger.com> wrote:
From my perspective, Instaparse is reaching pretty far into the guts there. I'll talk to Rich about it though.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

Sean Corfield

unread,
Mar 31, 2015, 7:03:53 PM3/31/15
to clo...@googlegroups.com
On Mar 31, 2015, at 3:57 PM, Alex Miller <al...@puredanger.com> wrote:
> From my perspective, Instaparse is reaching pretty far into the guts there. I'll talk to Rich about it though.

Thanks. I know we've had several discussions about what should be considered "API" around some of the Java code that implements Clojure and it struck me as odd that these pieces were clearly designed to be called from Clojure yet otherwise seemed to be "internal" so I'd definitely be curious as to the thinking there.

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

dev.Objective() - Developing Apps, Developing Skills, Developing Community
-- May 12-15, 2015 - Bloomington, MN - http://devobjective.com



Sean Corfield

unread,
Mar 31, 2015, 7:04:10 PM3/31/15
to clo...@googlegroups.com
On Mar 31, 2015, at 4:00 PM, Mark Engelberg <mark.en...@gmail.com> wrote:
> I can push a new version of instaparse incorporating Michael Blume's pull request by the end of the day.

That would be great, Mark! Thank you!

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)



Mark Engelberg

unread,
Mar 31, 2015, 7:12:09 PM3/31/15
to clojure
The reason instaparse uses Clojure's string reader is to make sure that strings inside the context-free grammar DSL are handled consistently with Clojure's logic for interpreting strings.

Mark Engelberg

unread,
Mar 31, 2015, 7:57:36 PM3/31/15
to clojure
Included Michael Blume's pull request in version 1.3.6.

Alexander Gunnarson

unread,
Mar 31, 2015, 8:25:08 PM3/31/15
to clo...@googlegroups.com, cloju...@googlegroups.com
This patch is great! It's much needed for my development workflow. One question: how do I enable conditional reading by default in the REPL? Do I set certain system properties in the command line like "cond_read=true" ?

Sean Corfield

unread,
Mar 31, 2015, 9:01:23 PM3/31/15
to cloju...@googlegroups.com, clo...@googlegroups.com
With instaparse 1.3.6 available, we’ve been able to upgrade our dev stack to Clojure 1.7.0-alpha6 and all our automated tests pass (and our full build/test script seems to be running a bit faster although we haven’t actually benchmarked our application yet). We’ll push alpha6 into QA either later today or tomorrow in preparation for our next production build (we’re running alpha5 in production right now).

Sean

Colin Fleming

unread,
Mar 31, 2015, 10:57:34 PM3/31/15
to cloju...@googlegroups.com, clo...@googlegroups.com
One question - it appears that the new #object and #error forms aren't readable at the moment. Is this something that's coming, or is the intention just that people could provide reader functions for these if they wanted to?

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev...@googlegroups.com.
To post to this group, send email to cloju...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojure-dev.

Alex Miller

unread,
Apr 1, 2015, 12:03:58 AM4/1/15
to cloju...@googlegroups.com, clo...@googlegroups.com
We do not expect to provide readers for the #object and #error forms (afaik). It's not possible to actual create a new Object or Throwable form that would be useful as an instance.

The new tagged-literal function could be used as a fallback reader function for either though, which would then allow you to retrieve either the tag or form as data. 

Colin Fleming

unread,
Apr 1, 2015, 12:27:45 AM4/1/15
to cloju...@googlegroups.com, clo...@googlegroups.com
Ok, thanks - I was looking for a way to access the data, right, I'll try tagged-literal for this.

Alex Miller

unread,
Apr 1, 2015, 11:33:10 AM4/1/15
to clo...@googlegroups.com
If this is a use case that could be lifted out into an API level function, that would be an ok enhancement jira request to consider (would need to think about it more, but that seems like one option).

Alexander Gunnarson

unread,
Apr 1, 2015, 12:56:12 PM4/1/15
to clo...@googlegroups.com, cloju...@googlegroups.com
This patch is incredibly useful! Great job to everyone that contributed. One question: how do I enable conditional reading by default in the REPL as opposed to passing a properties map to /read-string/, etc.? Do I set certain system properties in the command line like "cond_read=true"?

Alex Miller

unread,
Apr 1, 2015, 1:09:41 PM4/1/15
to clo...@googlegroups.com, cloju...@googlegroups.com
If you want to invoke the reader programmatically with reader conditionals allowed, you can do that through the (new) options map available with both read and read-string:

user=> (read-string {:read-cond :allow} "#?(:clj foo :cljs bar)")
foo

There is no way to "turn on" read conditionals by default at the REPL - it is only on by default when loading a .cljc file.
...

Sean Corfield

unread,
Apr 1, 2015, 2:52:15 PM4/1/15
to cloju...@googlegroups.com, clo...@googlegroups.com
On Apr 1, 2015, at 10:09 AM, Alex Miller <al...@puredanger.com> wrote:
There is no way to "turn on" read conditionals by default at the REPL - it is only on by default when loading a .cljc file.

This sounds like a useful feature to add to the REPL tho’ so that you can copy’n’paste code as-is and have it behave "appropriately" (i.e., you can verify that a given section of code does the right thing for a given environment)?

I guess we’ll have to see how various IDEs deal with this to know what’s really needed…

Sean

Alex Miller

unread,
Apr 1, 2015, 3:20:08 PM4/1/15
to cloju...@googlegroups.com, clo...@googlegroups.com
REPLs are of course free to choose how they read and some of the other changes coming out of the socket repl work may make those choices easier to select. The clojure.main/repl function is already (perhaps excessively) parameterized and can be given a custom read function for example. The read and read-string option maps are intentionally the first arg for easy use with partial.

Alexander Gunnarson

unread,
Apr 1, 2015, 3:26:22 PM4/1/15
to clo...@googlegroups.com, cloju...@googlegroups.com
@Alex Miller: Thanks for letting me know. I'll unfortunately have to change my workflow accordingly. 


On Tuesday, March 31, 2015 at 10:51:13 AM UTC-6, Alex Miller wrote:

Alexander Gunnarson

unread,
Apr 1, 2015, 3:32:58 PM4/1/15
to clo...@googlegroups.com, cloju...@googlegroups.com
@Sean Corfield — That's exactly my point. I use Sublime Text and I usually just copy-paste code from various buffers / open files into a REPL buffer on my workspace. Maybe that's not the most efficient way, and I want to move to some sort of auto-reload plugin for leiningen a la figwheel for Clojure, but I'm not sure if the new reader conditionals would with such a plugin anyway. I'd have to experiment. Either way, it would be very convenient to be able to enable/disable reader conditionals via something like (set! *cond-eval* true) or some such thing.


On Tuesday, March 31, 2015 at 10:51:13 AM UTC-6, Alex Miller wrote:

Alex Miller

unread,
Apr 1, 2015, 3:39:28 PM4/1/15
to clo...@googlegroups.com, cloju...@googlegroups.com
We'll consider it, thanks for the questions.

Alexander Gunnarson

unread,
Apr 1, 2015, 3:45:08 PM4/1/15
to clo...@googlegroups.com, cloju...@googlegroups.com
@Alex Miller — Thanks! I appreciate it.

Ambrose Bonnaire-Sergeant

unread,
Apr 1, 2015, 7:20:59 PM4/1/15
to clojure
Hi,

Iterate calls its function after it is finished iterating.

;; clojure 1.6
user=> (take 2 (iterate zero? 0))
(0 true)

;; clojure 1.7-alpha6
user=> (take 2 (iterate zero? 0))
ClassCastException java.lang.Boolean cannot be cast to java.lang.Number  clojure.lang.Numbers.isZero (Numbers.java:92)

Thanks,
Ambrose

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

Matching Socks

unread,
Apr 1, 2015, 8:50:04 PM4/1/15
to clo...@googlegroups.com, cloju...@googlegroups.com
Ambrose, does that "iterate" result arise from chunking?  "iterate" is advertised as producing an infinite lazy sequence.  While a suddenly chunking "iterate" will no doubt smoke out some cases like this, doesn't it seem that they are abuses?  It's not quite spot-on to employ "take" or "take-while" outside the "iterate" as a loop-ending nanny for sequences that could not genuinely be infinite.  The "take" should be concerned with what the consumer wants to consume, not preoccupied with what the producer will be able to produce.

Ambrose Bonnaire-Sergeant

unread,
Apr 1, 2015, 8:57:18 PM4/1/15
to clojure
AFAICT there is consistently one extra call, which seems to suggest an off-by-one error in the IReduce implementation of Iterate.

;; 1.6
user=> (take 11 (iterate (fn [a] (prn (str "PR" a)) (inc a)) 1))
"PR1"
"PR2"
"PR3"
"PR4"
"PR5"
"PR6"
"PR7"
"PR8"
"PR9"
"PR10"
(1 2 3 4 5 6 7 8 9 10 ...)

;; 1.7.0-alpha6
user=> (take 11 (iterate (fn [a] (prn (str "PR" a)) (inc a)) 1))
"PR1"
"PR2"
"PR3"
"PR4"
"PR5"
"PR6"
"PR7"
"PR8"
"PR9"
"PR10"
"PR11"
(1 2 3 4 5 6 7 8 9 10 ...)

Thanks,
Ambrose

On Wed, Apr 1, 2015 at 8:50 PM, Matching Socks <phill...@gmail.com> wrote:
Ambrose, does that "iterate" result arise from chunking?  "iterate" is advertised as producing an infinite lazy sequence.  While a suddenly chunking "iterate" will no doubt smoke out some cases like this, doesn't it seem that they are abuses?  It's not quite spot-on to employ "take" or "take-while" outside the "iterate" as a loop-ending nanny for sequences that could not genuinely be infinite.  The "take" should be concerned with what the consumer wants to consume, not preoccupied with what the producer will be able to produce.

--
You received this message because you are subscribed to the Google

Ambrose Bonnaire-Sergeant

unread,
Apr 1, 2015, 9:07:13 PM4/1/15
to clojure
Actually it seems the oddity is that "next" now does the computation instead of "first" in Iterate.java.

Alex Miller

unread,
Apr 1, 2015, 9:10:47 PM4/1/15
to clo...@googlegroups.com
I would love a jira for the iterate thIng.

Ambrose Bonnaire-Sergeant

unread,
Apr 1, 2015, 9:13:04 PM4/1/15
to clojure
Ok.

On Wed, Apr 1, 2015 at 9:10 PM, Alex Miller <al...@puredanger.com> wrote:
I would love a jira for the iterate thIng.

Ambrose Bonnaire-Sergeant

unread,
Apr 1, 2015, 9:14:52 PM4/1/15
to clojure

Alex Miller

unread,
Apr 1, 2015, 10:28:02 PM4/1/15
to clo...@googlegroups.com, abonnair...@gmail.com
Thanks! If anyone wants to throw a patch, would love to have one. Must include test ...

Ambrose Bonnaire-Sergeant

unread,
Apr 2, 2015, 12:14:25 AM4/2/15
to clojure
Here's some weird behaviour I found from 1.6.

user=> (take 1 (iterate zero? true))
(true)
user=> (reduce (fn [l _] (reduced l)) (iterate zero? true))
ClassCastException java.lang.Boolean cannot be cast to java.lang.Number  clojure.lang.Numbers.isZero (Numbers.java:90)

Is this a bug, and should this be fixed?

Thanks,
Ambrose

tcrayford

unread,
Apr 2, 2015, 4:22:43 AM4/2/15
to clo...@googlegroups.com, cloju...@googlegroups.com
Yeller (yellerapp.com) (which I maintain) has a comprehensive benchmark suite
(using criterium, with a heck of a lot of work put into getting real and good
results). I've run Yeller's benchmark suite against clojure 1.7 alpha6 - it's
pretty comprehensive, and covers a wide range of things. Results are below.

Mostly some things got very minorly slower (but not enough to make a real
impact for the application), but quite a few benchmarks saw **dramatic**
speedups.

Given these promising results, I ran Yeller's `test.check` suite for 8 hours in
CI, and didn't see any failures, so I've deployed 1.7 alpha6 to production, and
haven't seen any issues as of yet.

Yeller's build also got ~24 seconds faster, which is super awesome (and it no
longer breaks under CLJ-1604 because that patch is in).

Much love to all the folk who put time and work into this release.

Benchmark names retracted because they're proprietary. Each line is a seperate benchmark, recorded on production level hardware, with the same jvm options/version used in production. Each benchmark is run in its own JVM (because otherwise JIT path dependence will kill you).

Benchmarks that were slower under 1.7 alpha6 vs 1.6.0 (with percentage changed amounts):
3.12%
2.62%
2.27%
2.19%
2.17%
2.06%
1.97%
1.43%
1.29%
0.89%
0.74%
0.71%
0.58%
0.45%
0.42%
0.34%
0.27%
0.25%
0.21%
0.2%
0.08%
0.03%

Benchmarks that were faster under 1.7 alpha6 vs 1.6.0 (with percentage changed amounts):
+0.0%
+0.0%
+0.01%
+0.02%
+0.03%
+0.03%
+0.03%
+0.04%
+0.04%
+0.09%
+0.12%
+0.16%
+0.31%
+0.33%
+0.52%
+0.53%
+0.85%
+1.74%
+1.84%
+1.99%
+2.14%
+2.17%
+2.42%
+2.96%
+3.0%
+5.18%
+5.36%
+9.86%
+11.93%
+12.41%
+14.36%
+16.95%
+24.71%
+53.5%
+53.62%
+72.07%

Thanks again! Super excited about this release.

Ambrose Bonnaire-Sergeant

unread,
Apr 2, 2015, 8:35:55 AM4/2/15
to clojure
core.typed 0.2.86 should be 1.7.0-alpha6 compatible now.

Thanks,
Ambrose

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev...@googlegroups.com.
To post to this group, send email to cloju...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojure-dev.

Alex Miller

unread,
Apr 2, 2015, 8:39:54 AM4/2/15
to cloju...@googlegroups.com, clo...@googlegroups.com
Great to hear feedback like this! I'd be particularly interested if you had any suspicions about the characteristics of the ones that are slower.


Nicola Mometto

unread,
Apr 2, 2015, 8:53:03 AM4/2/15
to clo...@googlegroups.com

Not a bug imho, you're invoking reduce with no init arg so you're
forcing the realization of at least two elements in the coll, one for
init and one for step -- the step one causes the exception.
--

Leon Grapenthin

unread,
Apr 2, 2015, 2:24:46 PM4/2/15
to clo...@googlegroups.com, cloju...@googlegroups.com
http://dev.clojure.org/display/design/Reader+Conditionals

First use-case "Platform-specific require/import":

Shouldn't the reader conditional example be:
(ns cemerick.pprng
  #?(:cljs (:require math.seedrandom
                     [cljs.core :as lang]))
  #?@(:clj [
            (:require [clojure.core :as lang])
            (:import java.util.Random)])
  (:refer-clojure :exclude (double float int long boolean)))


Alex Miller

unread,
Apr 2, 2015, 2:35:39 PM4/2/15
to clo...@googlegroups.com, cloju...@googlegroups.com
Yup. Fixed.

Nicola Mometto

unread,
Apr 2, 2015, 7:12:02 PM4/2/15
to cloju...@googlegroups.com, clo...@googlegroups.com

The recent changes to iterate come with an interesting consequence:
reducing an iterate multiple times will cause the entire chain of x, (f
x), (f (f x)) .. to be recalculated every time.

I'd argue that this is not desiderable and a regression (even though
probably one considered by design), and that this change in behaviour is
going to produce a degradation in performance when the function to
iterate actually does some computation rather than being a trivial
function like the one used in the benchmarks in the CLJ-1603 ticket
(inc) since its result won't be cached.

If changing the new behaviour is out of the question I'd suggest to at
least document this new behaviour. The docstring of iterate talks about
returning a lazy-seq while when used with reduce it is actually a
generator.

To show what I mean here's a silly example using Threa/sleep to simulate
computation:

Clojure 1.7.0-alpha5
user=> (def a (iterate #(do (Thread/sleep 2) (inc %)) 0))
#'user/a
user=> (time (reduce (fn [_ x] (if (= 50 x) (reduced nil))) nil a))
"Elapsed time: 106.385891 msecs"
nil
user=> (time (reduce (fn [_ x] (if (= 50 x) (reduced nil))) nil a))
"Elapsed time: 0.560275 msecs"
nil

Clojure 1.7.0-master-SNAPSHOT
user=> (def a (iterate #(do (Thread/sleep 2) (inc %)) 0))
#'user/a
user=> (time (reduce (fn [_ x] (if (= 50 x) (reduced nil))) nil a))
"Elapsed time: 109.088477 msecs"
nil
user=> (time (reduce (fn [_ x] (if (= 50 x) (reduced nil))) nil a))
"Elapsed time: 109.51494 msecs"
nil

Nicola Mometto

unread,
Apr 2, 2015, 7:22:36 PM4/2/15
to cloju...@googlegroups.com, clo...@googlegroups.com

Actaully caching makes a difference even with inc and a sequence of just
1000 elements:

Clojure 1.7.0-alpha5
user=> (def a (iterate inc 0))
#'user/a
user=> (time (reduce (fn [_ x]) nil (take 1000 a)))
"Elapsed time: 4.170778 msecs"
nil
user=> (time (reduce (fn [_ x]) nil (take 1000 a)))
"Elapsed time: 1.589569 msecs"
nil

Clojure 1.7.0-master-SNAPSHOT
user=> (def a (iterate inc 0))
#'user/a
user=> (time (reduce (fn [_ x]) nil (take 1000 a)))
"Elapsed time: 4.831726 msecs"
nil
user=> (time (reduce (fn [_ x]) nil (take 1000 a)))
"Elapsed time: 4.058695 msecs"
nil
--

tcrayford

unread,
Apr 3, 2015, 8:10:31 AM4/3/15
to clo...@googlegroups.com, cloju...@googlegroups.com
Alex:

I'd bet heavily on the "slower" ones being measurement noise (they only differ by a few percent) - I typically see that much variation between runs. I probably shouldn't have reported them as "slower" - 1-5% is (in my experience) typically just noise, so don't think it's worth digging into.
...

Alex Miller

unread,
Apr 3, 2015, 8:39:11 AM4/3/15
to cloju...@googlegroups.com, clo...@googlegroups.com
If you want caching behavior, then use it as a sequence. If you want faster iteration, use it via reduce. The whole point of this change is to open the faster reduce path, which you won't get if you also cache.

I did some research a few months back via crossclj looking at how people typically use iterate. I can't actually remember finding any use cases other than inc or something similar (but it's been a while).

Ian Rumford

unread,
Apr 4, 2015, 10:57:06 AM4/4/15
to clo...@googlegroups.com, cloju...@googlegroups.com
Hi Alex, All,

An observation really.

I've just noticed that a literal map e.g. (def m {:a 1 :b 2 :c 3}) in alpha6 is an array map not hash map.  

In 1.6.0 and 1.7.0-alpha5  its a hash map.

Couldn't find any obvious statement in the release notes but interested in the background to the change.

Andy Fingerhut

unread,
Apr 4, 2015, 7:45:07 PM4/4/15
to clo...@googlegroups.com
I believe that change is due to this commit from early March 2015: https://github.com/clojure/clojure/commit/692645c73c86d12c93a97c858dc6e8b0f4280a0b

Nicola Mometto had opened a ticket CLJ-1593 for this issue: http://dev.clojure.org/jira/browse/CLJ-1593
The commit did not reference the ticket, but Nicola noticed the commit and closed the ticket due to that commit.

Alex, would it be appropriate to include a reference to CLJ-1593 in the Clojure 1.7 release notes?

Andy

--
You received this message because you are subscribed to the Google

Alex Miller

unread,
Apr 5, 2015, 12:19:43 AM4/5/15
to clo...@googlegroups.com
Yes, I'll get that in the next release, I forgot to put it in there...

Alexander Yakushev

unread,
Apr 5, 2015, 2:44:16 PM4/5/15
to cloju...@googlegroups.com, clo...@googlegroups.com
Hello Alex,

As I've understood from the dev.clojure.org page, additional features and feature combinations will become available later. Can we please get :clj/android (or :clja) still in 1.7? If so, what has to be done by me or Daniel to make it happen?

Thanks!

Alex Miller

unread,
Apr 5, 2015, 5:34:19 PM4/5/15
to cloju...@googlegroups.com, clo...@googlegroups.com
The reader can be invoked programmatically with any feature set you like now. We did not plan to allow custom features for 1.7.

I'm not sure what it would mean to "add" an android feature? I'll plead ignorance in not knowing how the Clojure 
Android stuff works or where a feature indicating Android could be set without support for custom features. Do you currently fork to support Android?
--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev...@googlegroups.com.
To post to this group, send email to cloju...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojure-dev.

Alexander Yakushev

unread,
Apr 5, 2015, 6:44:16 PM4/5/15
to cloju...@googlegroups.com, clo...@googlegroups.com
Well, I must say I'm even more ignorant in how the feature expansions are implemented. Is the feature a thing that each implementation defines separately? If so, then it will be easy to set our fork's feature as :clja, or whatever else. My request to you would then be: can you please include :clja into the "spec", in the list of available features? What I ask for is sort of semi-official confirmation that Clojure-Android has its own feature, so that we would have easier time to include Android-specific code into the libraries that will become cross-platform after 1.7 goes live.

Best regards,
Alex
...

Alexander Yakushev

unread,
Apr 5, 2015, 6:46:45 PM4/5/15
to cloju...@googlegroups.com, clo...@googlegroups.com
On the other hand, Clojure-Android runtime is largely similar to regular Clojure except for some small differences; so in order to use feature expressions there effectively it might make sense to wait until broader capabilities (feature combinations, feature inheritance etc.) is introduced.

Alex Miller

unread,
Apr 5, 2015, 6:59:31 PM4/5/15
to cloju...@googlegroups.com, clo...@googlegroups.com
Yes, each platform defines their own platform feature so it wouldn't be too hard to change the specified platform in the fork. However the key here is that the conditional occurs at read time, so you need to ensure that compilation happens with this platform if aot is involved. Maybe that's obvious.

The existing code in Clojure will not fail in unknown features so you would be able to read the same code in Clojure or Android.

I'll ask Rich about a preferred feature name.


--

Alexander Yakushev

unread,
Apr 5, 2015, 7:25:51 PM4/5/15
to cloju...@googlegroups.com, clo...@googlegroups.com
Awesome, thanks for the help and the clarification!
Reply all
Reply to author
Forward
0 new messages