Clojure 1.3 Alpha 4

126 views
Skip to first unread message

Stuart Halloway

unread,
Dec 12, 2010, 10:09:32 PM12/12/10
to clo...@googlegroups.com

Clojure 1.3 Alpha 2 is now available at

http://clojure.org/downloads

 0 Changes from 1.3 Alpha 3 to 1.3 Alpha 4
 1 Changes from 1.3 Alpha 2 to 1.3 Alpha 3
 2 Changes from 1.3 Alpha 1 to 1.3 Alpha 2
 3 Changes from 1.2 to 1.3 Alpha 1
 4 About Alpha Releases

= 0 Changes from 1.3 Alpha 3 to 1.3 Alpha 4
 
  * normalized unchecked-* fn names
  * added *unchecked-math* support
  * fixes to binding conveyance (and *agent*)
  
= 1 Changes from 1.3 Alpha 2 to 1.3 Alpha 3
  
  * fixed filter performance issue introduced in 1.3A2 
  * with-redefs macro (useful for stubbing)
  * print-table

= 2 Changes from 1.3 Alpha 1 to 1.3 Alpha 2

  * code path for using vars is now *much* faster for the common case,
    and you must explicitly ask for :dynamic bindability
  * new: clojure.reflect/reflect
  * new: clojure.data/diff

= 3 Changes from 1.2 to 1.3 Alpha 1

  * enhanced primitive support 
  * better exception reporting
  * ancillary namespaces no longer auto-load on startup:
    clojure.set, clojure.xml, clojure.zip

= 4 About Alpha Releases

1.3 is the first release of Clojure that will include a series of
alpha builds. We are adding these builds to support maven and
leiningen users, who want a specific artifact that they can target (as
opposed to building from master or "moving-target" snapshots).

If you are the kind of person who used to track master by building
from source, but no longer do so because you are using maven or
leiningen, alpha releases are for you.


Vagif Verdi

unread,
Dec 13, 2010, 4:27:39 PM12/13/10
to Clojure
Maybe clojure should adopt linux versioning scheme, where even numbers
are stable production clojure and odd numbers are development branch ?

Sean Corfield

unread,
Dec 13, 2010, 5:27:59 PM12/13/10
to clo...@googlegroups.com
On Sun, Dec 12, 2010 at 7:09 PM, Stuart Halloway
<stuart....@gmail.com> wrote:
> Clojure 1.3 Alpha 2 is now available at
>
> http://clojure.org/downloads

...and via Leiningen as:

[org.clojure/clojure "1.3.0-alpha4"]

which leads me to ask:

Has there been any thought to making simultaneous clojure/contrib
releases to make it easier for folks using lein / mvn to just update
their deps for all org.clojure entries at the same time?
--
Sean A Corfield -- (904) 302-SEAN
Railo Technologies, Inc. -- http://getrailo.com/
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

Mike Meyer

unread,
Dec 13, 2010, 5:39:53 PM12/13/10
to Clojure
"Vagif Verdi" <vagif...@gmail.com> wrote:

>Maybe clojure should adopt linux versioning scheme, where even numbers
>are stable production clojure and odd numbers are development branch ?

Gods please no.
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

Shantanu Kumar

unread,
Dec 13, 2010, 11:27:18 PM12/13/10
to Clojure

On Dec 14, 2:27 am, Vagif Verdi <vagif.ve...@gmail.com> wrote:
> Maybe clojure should adopt linux versioning scheme, where even numbers
> are stable production clojure and odd numbers are development branch ?

AFAICT this will seriously affect Maven/Lein/Cake transitive
dependency management when dealing with version-ranges.

Regards,
Shantanu

Stuart Sierra

unread,
Dec 14, 2010, 11:02:37 AM12/14/10
to clo...@googlegroups.com
There has been much thought lately around build and release processes. See http://dev.clojure.org/display/design/Common+Contrib+Build

The goal is to to have more libraries under the umbrella of "contrib" without requiring them to keep to the same release schedule.  Each library can have its own release schedule and version numbers.  The current "clojure-contrib", the monolithic repository with 60+ modules, will be phased out.

All 1.3.0-alpha* releases of the "clojure-contrib" module JARs are source-code-only (except for the 4 modules that require AOT) so they are compatible with any Clojure release.

For example, you can use Clojure "1.3.0-alpha4" and org.clojure.contrib/logging "1.3.0-alpha3" in the same project.

-Stuart Sierra

Sean Corfield

unread,
Dec 14, 2010, 4:27:21 PM12/14/10
to clo...@googlegroups.com
On Tue, Dec 14, 2010 at 8:02 AM, Stuart Sierra
<the.stua...@gmail.com> wrote:
> There has been much thought lately around build and release processes.
> See http://dev.clojure.org/display/design/Common+Contrib+Build

Thanx.

> The goal is to to have more libraries under the umbrella of "contrib"
> without requiring them to keep to the same release schedule.  Each library
> can have its own release schedule and version numbers.

Makes sense (although it sounds like a recipe for confusion until the
transition is completed - at 1.3.0 final, yes?).

I suspect it will get easier - or at least look a little less
confusing - when the current contrib libs actually do start having
separate version numbers :)

> For example, you can use Clojure "1.3.0-alpha4" and
> org.clojure.contrib/logging "1.3.0-alpha3" in the same project.

Is there an easy way to lookup whether a given lib has an alpha4
version (or, more generally, what the latest reasonably stable build
is)?

(should I take this to the -dev list?)

Stuart Sierra

unread,
Dec 14, 2010, 4:50:05 PM12/14/10
to clo...@googlegroups.com
For now, you can just look at the repository at http://build.clojure.org/releases/org/clojure/

Once we get automated releases from Hudson/Maven (real soon now) the JARs will be deployed to the Maven Central repository, where they will be automatically indexed by search engines like jarvana.com.

-S

Miki

unread,
Dec 14, 2010, 7:30:19 PM12/14/10
to clo...@googlegroups.com
(defn fact [n] (reduce * (range 1 (inc n))))
(fact 100)

This produces the right result on 1.2 but "ArithmeticException integer overflow" on 1.3-alpha4.
Is this intentional?

Brian Goslinga

unread,
Dec 14, 2010, 7:36:40 PM12/14/10
to Clojure
Primitive math is the default in 1.3

Benny Tsai

unread,
Dec 14, 2010, 8:23:12 PM12/14/10
to Clojure
As Brian said, primitive math is now the default in 1.3. If auto-
promotion on overflow is desired, you can use the +', -', *', inc',
dec' functions (note the single quote suffix).

http://dev.clojure.org/display/doc/Enhanced+Primitive+Support

Benny Tsai

unread,
Dec 14, 2010, 8:25:44 PM12/14/10
to Clojure
Of course, TANSTAAFL: the auto-promoting version of the functions will
be slower than their primitive counterparts.

Ken Wesson

unread,
Dec 14, 2010, 9:04:11 PM12/14/10
to clo...@googlegroups.com
On Tue, Dec 14, 2010 at 8:23 PM, Benny Tsai <benny...@gmail.com> wrote:
> As Brian said, primitive math is now the default in 1.3.  If auto-
> promotion on overflow is desired, you can use the +', -', *', inc',
> dec' functions (note the single quote suffix).

Why was this done? I preferred having +, -, etc. DTRT in general and
unchecked-+, etc. for when you really needed efficient primitive math.
My code is littered with + but has few unchecked-+s. Which means I'll
have to go through it all adding little tick-marks everywhere and
making the math look funny to keep its behavior the same whenever 1.3
is released.

David Nolen

unread,
Dec 14, 2010, 9:56:27 PM12/14/10
to clo...@googlegroups.com
For quite a few good reasons, the most important being that it makes Rich Hickey's life a lot easier, and ours as well as a result of that.

Best to read over the very, very long thread on the subject.

David 

Brian Goslinga

unread,
Dec 14, 2010, 10:02:28 PM12/14/10
to Clojure
On Dec 14, 8:04 pm, Ken Wesson <kwess...@gmail.com> wrote:
This topic has been discussed to death before on this group.

Short version:
Doing the right thing is actually harder than you might first think:
methods in Java must choose between returning a primitive and
returning in reference type. This means that you have to choose
between fast primitive math or slow reference type math. To advance
the goal of writing Clojure in Clojure, it is necessary for it be easy
to write high-performance Clojure code (although you might not care
about how hard photosynthesis is, the plants do). Since it is
generally the case that longs are sufficient for most purposes and it
generally known ahead of time when they are not (Project Euler
problems for example), primitive math is being made default in 1.3

Another change in 1.3 is that BigInteger math will not auto-reduce and
will be contagious. This means that the prime versions of operations
(+', -', etc.) are almost never necessary as one can use a BigInteger
literal to ensure the operation will not overflow. Your example will
work correctly if you change 1 to 1N. (Since java.lang.BigInteger is
slow on small numbers, clojure.lang.BigInt is also being introduced
(which is the type of 1N) that should be as fast as math was in 1.2
when the numbers fit into a long)

For better or worse (hopefully for better), Clojure is increasingly
adopting C++'s philosophy of pay for what you need.

Ken Wesson

unread,
Dec 14, 2010, 10:24:09 PM12/14/10
to clo...@googlegroups.com

Breaking source compatibility with just about every single preexisting
line of Clojure code out there is supposed to make our lives *easier*?
I'd dearly love to know how -- my cousin is a stage magician and he's
always on the lookout for new tricks, so this would make a nearly
perfect Christmas present for him. :)

Ken Wesson

unread,
Dec 14, 2010, 10:26:15 PM12/14/10
to clo...@googlegroups.com
On Tue, Dec 14, 2010 at 10:02 PM, Brian Goslinga
<quickba...@gmail.com> wrote:
> This topic has been discussed to death before on this group.

If so, it was before I joined.

> Doing the right thing is actually harder than you might first think

But it's also already being done by Clojure 1.2 so I don't see how
that's relevant. If someone proposed a novel behavior, it being
"harder than you might first think" would be a point against that
proposal. However it cannot logically ever be a point against keeping
current behavior.

Michael Gardner

unread,
Dec 14, 2010, 10:37:13 PM12/14/10
to clo...@googlegroups.com
On Dec 14, 2010, at 9:26 PM, Ken Wesson wrote:

> On Tue, Dec 14, 2010 at 10:02 PM, Brian Goslinga
> <quickba...@gmail.com> wrote:
>> This topic has been discussed to death before on this group.
>
> If so, it was before I joined.

That's what archives are for:

http://groups.google.com/group/clojure/search?group=clojure&q=primitive+math

See in particular the threads "Enhanced primitive support" and "Enhanced primitive support - redux".

>> Doing the right thing is actually harder than you might first think
>
> But it's also already being done by Clojure 1.2 so I don't see how
> that's relevant. If someone proposed a novel behavior, it being
> "harder than you might first think" would be a point against that
> proposal. However it cannot logically ever be a point against keeping
> current behavior.

Are you saying that simplifying existing code provides no benefit?

Brian Goslinga

unread,
Dec 14, 2010, 10:57:52 PM12/14/10
to Clojure
On Dec 14, 9:24 pm, Ken Wesson <kwess...@gmail.com> wrote:
> Breaking source compatibility with just about every single preexisting
> line of Clojure code out there is supposed to make our lives *easier*?
Actually, it appears that the majority of the lines of code out there
use integers that fit inside a long, so the change doesn't affect
them.

On Dec 14, 9:26 pm, Ken Wesson <kwess...@gmail.com> wrote:
> But it's also already being done by Clojure 1.2 so I don't see how
> that's relevant. If someone proposed a novel behavior, it being
> "harder than you might first think" would be a point against that
> proposal. However it cannot logically ever be a point against keeping
> current behavior.
There exists a difference in the behavior of primitive and reference
type math in 1.2; Clojure 1.3 will unify the behavior of the two. The
auto promoting operators need to return an Object currently even if
they take in primitives because the operation might overflow. This
means that the auto promoting operators are incompatible with fast
math (blame the JVM). It would be nice if there wasn't a trade-off to
make, but there is one.

Mark Engelberg

unread,
Dec 14, 2010, 11:32:58 PM12/14/10
to clo...@googlegroups.com


Search the archives for "enhanced primitive support".  There was a big battle back in June.  People like you and me, who don't want to have to think about whether their code might crash with certain inputs from arithmetic overflow, lost the battle to those who want to get more speed out of Clojure without having to specifically annotate tight loops with special primitive math ops.

The theory is that most programs work perfectly fine with longs.  If yours doesn't, you'll hopefully find out when it throws an error, and go annotate your code accordingly.  In exchange, Clojure will supposedly get faster for the common case.

In practice, I haven't seen a significant speed improvement in the new branch of Clojure (except on specific benchmarks that intentionally test Clojure's new default primitive math).  In my day-to-day code, all my numbers, despite being perfectly small enough to fit in a long, end up getting stored and retrieved from Clojure's various data structures - vectors, maps, sets, etc. and thus lose their primitiveness.  So I presumably haven't seen any speed improvement because all the numbers are boxed by the time I do math on them.  Fortunately, I also don't seem to run into arithmetic overflows because my production code isn't particularly math intensive, but I still end up feeling stressed out trying to convince myself that it can't ever overflow for any input.

It will be interesting to see how this all shakes out -- whether there are more ways to exploit the default of primitive math for better speed, and whether people still like the new direction after trying it more.

Mark Engelberg

unread,
Dec 14, 2010, 11:42:56 PM12/14/10
to clo...@googlegroups.com
On Tue, Dec 14, 2010 at 7:02 PM, Brian Goslinga <quickba...@gmail.com> wrote:
 (Since java.lang.BigInteger is
slow on small numbers, clojure.lang.BigInt is also being introduced
(which is the type of 1N) that should be as fast as math was in 1.2
when the numbers fit into a long)

AFAIK, this hasn't been done yet, and clojure.lang.BigInt is just a stub that is as slow as java.lang.BigInteger.  Is this correct?

Ken Wesson

unread,
Dec 15, 2010, 12:22:28 AM12/15/10
to clo...@googlegroups.com
On Tue, Dec 14, 2010 at 10:37 PM, Michael Gardner <gard...@gmail.com> wrote:
> On Dec 14, 2010, at 9:26 PM, Ken Wesson wrote:
>
>> On Tue, Dec 14, 2010 at 10:02 PM, Brian Goslinga
>> <quickba...@gmail.com> wrote:
>>> This topic has been discussed to death before on this group.
>>
>> If so, it was before I joined.
>
> That's what archives are for

Are you honestly suggesting I search the archives for every word of
every thought that it ever occurs to me to post here?

I don't have that kind of time and I doubt anyone else does. If anyone
started to actually enforce such a rule, participation here would drop
to practically zero overnight.

>>> Doing the right thing is actually harder than you might first think
>>
>> But it's also already being done by Clojure 1.2 so I don't see how
>> that's relevant. If someone proposed a novel behavior, it being
>> "harder than you might first think" would be a point against that
>> proposal. However it cannot logically ever be a point against keeping
>> current behavior.
>
> Are you saying that simplifying existing code provides no benefit?

If it breaks existing client code then yes. Simplifying internals
without altering API semantics is generally a good thing; when the API
semantics start changing, though, an unequivocal improvement becomes a
tradeoff that might tip either way.

Changing the behavior of arithmetic operators that are found in
virtually every extant source file is going to have a very big
downside in broken backward compatibility. If there's an upside, it
would have to be staggeringly enormous to make such a change
worthwhile.

Ken Wesson

unread,
Dec 15, 2010, 12:27:24 AM12/15/10
to clo...@googlegroups.com
On Tue, Dec 14, 2010 at 10:57 PM, Brian Goslinga
<quickba...@gmail.com> wrote:
> On Dec 14, 9:24 pm, Ken Wesson <kwess...@gmail.com> wrote:
>> Breaking source compatibility with just about every single preexisting
>> line of Clojure code out there is supposed to make our lives *easier*?
> Actually, it appears that the majority of the lines of code out there
> use integers that fit inside a long, so the change doesn't affect
> them.

You could as well argue that dosync be changed to simply lock all the
things that are altered or assured instead of doing the whole STM
thing. Most things wouldn't break ... but a few things would, with the
odd deadlock here and there. And that too would simplify the
implementation. A lot.

> There exists a difference in the behavior of primitive and reference
> type math in 1.2; Clojure 1.3 will unify the behavior of the two. The
> auto promoting operators need to return an Object currently even if
> they take in primitives because the operation might overflow.  This
> means that the auto promoting operators are incompatible with fast
> math (blame the JVM).

But we have the unchecked-foo operators for when we need fast math. We
have the longer, more awkward name in the rare case and plain old +,
-, etc. in the common case. Now you're proposing to reverse that.

Worse, from the sounds of it the new + isn't exactly the old
unchecked-+; it still checks for overflow rather than allowing
wrapping. That's going to add a compare-and-branch to every add
instruction and halve the speed of those operators on typical
hardware. Compare-and-throw-exception is hardly superior to
compare-and-box-in-BigInteger, since it's still slow AND now some
arithmetic code that used to work but be slow will now explode in your
face.

David Nolen

unread,
Dec 15, 2010, 12:38:56 AM12/15/10
to clo...@googlegroups.com
On Wed, Dec 15, 2010 at 12:22 AM, Ken Wesson <kwes...@gmail.com> wrote:

Are you honestly suggesting I search the archives for every word of
every thought that it ever occurs to me to post here?

With all due respect, the topic has already been bike shedded into the ground by many members in the community. Which isn't to say you don't have a valid point, but that ground has already been covered at great, great length. If you want to know why the decision was made read the thread. If you don't want to read the thread, well there's not much too talk about that hasn't been talked about before.

Such decisions aren't not made lightly and it would be fair to at least go read what Rich Hickey had to say on the matter.

David 

Ken Wesson

unread,
Dec 15, 2010, 12:50:12 AM12/15/10
to clo...@googlegroups.com
On Wed, Dec 15, 2010 at 12:38 AM, David Nolen <dnolen...@gmail.com> wrote:
> On Wed, Dec 15, 2010 at 12:22 AM, Ken Wesson <kwes...@gmail.com> wrote:
>>
>> Are you honestly suggesting I search the archives for every word of
>> every thought that it ever occurs to me to post here?
>
> With all due respect, the topic has already been bike shedded into the
> ground by many members in the community.

Fascinating. Your point being? That statement of yours does not, after
all, magically make searching the archive for every word in every post
you ever think of posting suddenly become practical.

David Nolen

unread,
Dec 15, 2010, 1:07:44 AM12/15/10
to clo...@googlegroups.com
I wasn't suggesting you do that. People earlier in the post gave links to the thread.

David 

Michael Gardner

unread,
Dec 15, 2010, 1:11:11 AM12/15/10
to clo...@googlegroups.com
On Dec 14, 2010, at 11:22 PM, Ken Wesson wrote:

> On Tue, Dec 14, 2010 at 10:37 PM, Michael Gardner <gard...@gmail.com> wrote:
>> That's what archives are for
>
> Are you honestly suggesting I search the archives for every word of
> every thought that it ever occurs to me to post here?
>
> I don't have that kind of time and I doubt anyone else does. If anyone
> started to actually enforce such a rule, participation here would drop
> to practically zero overnight.

That's a mighty fine straw man you have there. And how deftly you knock it down!

>> Are you saying that simplifying existing code provides no benefit?
>
> If it breaks existing client code then yes. Simplifying internals
> without altering API semantics is generally a good thing; when the API
> semantics start changing, though, an unequivocal improvement becomes a
> tradeoff that might tip either way.
>
> Changing the behavior of arithmetic operators that are found in
> virtually every extant source file is going to have a very big
> downside in broken backward compatibility. If there's an upside, it
> would have to be staggeringly enormous to make such a change
> worthwhile.

The claim I responded to was: "it cannot logically ever be a point against keeping current behavior". You are now arguing a much weaker claim, that the upside of simplifying existing code is unlikely to outweigh the drawbacks of breaking existing code.

Ken Wesson

unread,
Dec 15, 2010, 10:05:20 AM12/15/10
to clo...@googlegroups.com

The first link to the thread was posted AFTER I was castigated with
the phrase "this topic has been discussed to death before on this
group." So, no, people "earlier in the post" did NOT give links to the
thread. People "earlier in the post" DID expect me to have read it
before any of them mentioned it.

And that it what I took issue with.

Ken Wesson

unread,
Dec 15, 2010, 10:19:16 AM12/15/10
to clo...@googlegroups.com
On Wed, Dec 15, 2010 at 1:11 AM, Michael Gardner <gard...@gmail.com> wrote:
> On Dec 14, 2010, at 11:22 PM, Ken Wesson wrote:
>
>> On Tue, Dec 14, 2010 at 10:37 PM, Michael Gardner <gard...@gmail.com> wrote:
>>> That's what archives are for
>>
>> Are you honestly suggesting I search the archives for every word of
>> every thought that it ever occurs to me to post here?
>>
>> I don't have that kind of time and I doubt anyone else does. If anyone
>> started to actually enforce such a rule, participation here would drop
>> to practically zero overnight.
>
> That's a mighty fine straw man you have there. And how deftly you knock it down!

How rude. It is not a straw man. People criticized me for not having
read a thread that was posted before I came here but that happened to
be relevant. There's simply no way I could have known about that
thread unless I routinely searched the archive for everything that
might be remotely tangentially related to anything I was considering
posting, and if I did THAT I'd spend far more time searching the
archive than doing anything else.

Moreover, there's simply no way I could have known about your
EXPECTATION that I'd have read that thread short of telepathy!

Now please drop this. I have done nothing wrong nor have I been
exceptionally lazy or indolent. I may have raised a topic you don't
like and consider to have been discussed to death -- tough. Nobody's
forcing you to respond, or even to read, topics you don't like.

For example, I ignore the reams and reams of how-to-configure-emacs
posts this list gets, since I don't use emacs and thus don't find that
material interesting. I don't instead pop up every two or three posts
in those threads complaining that this is a Clojure forum, not an
Emacs forum, or remarking "gee, Emacs sure is hard to configure, why
don't you just use normal software with a normal Mac/Windows UI like
everybody else?".

Surely you can do likewise with primitive-arithmetic threads if you
dislike those.

>>> Are you saying that simplifying existing code provides no benefit?
>>
>> If it breaks existing client code then yes. Simplifying internals
>> without altering API semantics is generally a good thing; when the API
>> semantics start changing, though, an unequivocal improvement becomes a
>> tradeoff that might tip either way.
>>
>> Changing the behavior of arithmetic operators that are found in
>> virtually every extant source file is going to have a very big
>> downside in broken backward compatibility. If there's an upside, it
>> would have to be staggeringly enormous to make such a change
>> worthwhile.
>
> The claim I responded to was: "it cannot logically ever be a point against keeping current behavior". You are now arguing a much weaker claim, that the upside of simplifying existing code is unlikely to outweigh the drawbacks of breaking existing code.

No, I'm arguing a completely unrelated claim. I didn't consider my
earlier claim to be in dispute, since it is pretty much self-evidently
correct and you didn't subsequently address it.

To be exact, I had said that "it's hard to implement" can be a reason
against adding new behavior, but that it is never a reason not to keep
the current behavior -- hard to implement or not, the current behavior
is ALREADY implemented.

You didn't remark upon that, but instead said that it could simplify
the code internals. That's a separate matter from what's hard to
implement. You're not talking implementing arithmetic from scratch but
re-implementing it. Re-implementing it is, obviously, harder than not
changing anything at all. So you're doing something
harder-than-nothing and that breaks backwards compatibility. That
requires a justification and I'm not sure "it makes the
compiler/library source code simpler" is sufficient when the backwards
compatibility breakage isn't just for some relatively infrequently
used feature (such as, say, the changes to agent error handling a
version or so ago) but is going to affect almost every application and
library out there.

Again, there'd have to be a staggering further benefit from the change
than just "the clojure.core code looks cleaner in github" or even "the
code is a bit easier to maintain in the future". I'm not sure that
even massive increases in code maintainability alone suffice for
something like this. A major, massively end-user-useful new feature
that's nigh-impossible to implement otherwise that can then be
implemented *might* suffice.

Laurent PETIT

unread,
Dec 15, 2010, 10:56:03 AM12/15/10
to clo...@googlegroups.com
2010/12/15 Ken Wesson <kwes...@gmail.com>

On Wed, Dec 15, 2010 at 1:11 AM, Michael Gardner <gard...@gmail.com> wrote:
> On Dec 14, 2010, at 11:22 PM, Ken Wesson wrote:
>
>> On Tue, Dec 14, 2010 at 10:37 PM, Michael Gardner <gard...@gmail.com> wrote:
>>> That's what archives are for
>>
>> Are you honestly suggesting I search the archives for every word of
>> every thought that it ever occurs to me to post here?
>>
>> I don't have that kind of time and I doubt anyone else does. If anyone
>> started to actually enforce such a rule, participation here would drop
>> to practically zero overnight.
>
> That's a mighty fine straw man you have there. And how deftly you knock it down!

How rude. It is not a straw man. People criticized me

Hello Ken,

please, don't take it bad, but just halt for a minute, and take a deep breath.
Nobody criticized you. To convince myself with this, I've reread the first posts following your first ones:

David Nolen:

"For quite a few good reasons, the most important being that it makes Rich Hickey's life a lot easier, and ours as well as a result of that.

Best to read over the very, very long thread on the subject."

Brian:

"This topic has been discussed to death before on this group.

Short version:
Doing the right thing is actually harder than you might first think:
methods in Java must choose between returning a primitive and...."

Michael:

"On Dec 14, 2010, at 9:26 PM, Ken Wesson wrote:
> <quickba...@gmail.com> wrote:
>> This topic has been discussed to death before on this group.
> If so, it was before I joined.


Peace,

--
Laurent

nicola...@gmail.com

unread,
Dec 15, 2010, 11:07:27 AM12/15/10
to clo...@googlegroups.com
> Again, there'd have to be a staggering further benefit from the change
> than just "the clojure.core code looks cleaner in github" or even "the
> code is a bit easier to maintain in the future". I'm not sure that
> even massive increases in code maintainability alone suffice for
> something like this. A major, massively end-user-useful new feature
> that's nigh-impossible to implement otherwise that can then be
> implemented *might* suffice.
>

I think the point was not to criticize you at all (nobody have to read
every post in the archive)
but just to point the fact that this debate was somehow already behind us.

It has been already debated a lot with pretty good points on each
side. (The concerns you are expressing
are good points, as are the answers in this thread and the preceeding
on the subject.)

For the specific problem you are facing, I would advise to go through
your code looking for number literal and
replacing each number nnnn by nnnnN. (ex 15 ---> 15N).

As boxedness is contagious, you probably won't have to change much
more to solve compatibility issues with 1.3.

If you find other specific problems, it would be great to post them:
we will need a good compilation of tips to convert numerical code from
1.2 to 1.3.

Best,

Nicolas,

Ken Wesson

unread,
Dec 15, 2010, 11:09:13 AM12/15/10
to clo...@googlegroups.com
On Wed, Dec 15, 2010 at 10:56 AM, Laurent PETIT <lauren...@gmail.com> wrote:
> People criticized me
>
> Hello Ken,
>
> please, don't take it bad, but just halt for a minute, and take a deep
> breath.
> Nobody criticized you. To convince myself with this, I've reread the first
> posts following your first ones:
>
> David Nolen:
> "For quite a few good reasons, the most important being that it makes Rich
> Hickey's life a lot easier, and ours as well as a result of that.
> Best to read over the very, very long thread on the subject."

No objection to this one.

> Brian:
> "This topic has been discussed to death before on this group.

This reads as a (mild) criticism.

> Short version:
> Doing the right thing is actually harder than you might first think:
> methods in Java must choose between returning a primitive and...."

Suggests a criticism, but furthermore seems to miss the point. If we
were debating whether to make arithmetic work the 1.2 way or the 1.3
alpha 4 way while still busily creating Clojure from scratch, which
way would be easier to implement would be a key factor in the decision
and whether the choice we made might break existing code wouldn't be.

However, Clojure 1.2 exists and is already implemented, with
arithmetic working the 1.2 way, and there is a ton of existing code
that may be relying on arithmetic working the 1.2 way.

So breaking existing code IS material to the debate now, and whether
it's hard to implement the 1.2 way is not. There's no need to
implement the 1.2 way because it's already been done.

> Michael:
> "On Dec 14, 2010, at 9:26 PM, Ken Wesson wrote:
>> <quickba...@gmail.com> wrote:
>>> This topic has been discussed to death before on this group.
>> If so, it was before I joined.
> That's what archives are for:

This is the one I objected the most to.

First of all, I argued back (mildly) against the (mild) criticism from
quickbasicguru (and what, I might ask, is one of those doing on a lisp
mailing list? :)) and Michael argued instead of acceding.

So it's sort of as if an elbow came close to my face, possibly
unintentionally, I took a mild action (raising my own arm in a
blocking posture) to prevent it knocking my glasses off, and someone
grabbed my arm and tried to yank it down again to ensure the elbow hit
me in the nose. That's a deliberate attack and it makes it more likely
the elbow was also.

Furthermore, Michael's statement carries with it the implication that
I should search the archives for every word I'm about to use before
every post to this list, which would take hours of my time every day
if implemented. That simply cannot be reasonable and I feel it
necessary to make it very firmly clear that no such expectation can be
considered reasonable.

In particular, I pointed out that if Michael's
"search-the-archives-first-before-posting-anything" policy was adhered
to perfectly by everyone, contributions to this list would slow to a
crawl, if not stop entirely.

There are 2 ways to reduce that: apply the policy only to some users,
or only to some topics.

The first of these is obviously grossly unfair. That leaves the second.

The problem with the second is: which topics "require" an archive
search and which do not is not itself obvious. Such a policy turns
this list into a minefield; unless you search the archives on every
post (check for a mine at every footfall) you might oneday fail to
search on a post on one of the "forbidden" topics (fail to check for a
mine at a spot where there's actually a mine).

The only way to avoid the minefield effect, other than not requiring
(via punishing with public castigation) people to check the archives
at all, is to post the "touchy subject list" somewhere. Of course,
then there's the problem of making sure that everyone knows about the
touchy subject list! And about every update to it. It would have to be
reposted every few days like the usenet faqs of yore, with a clear
indication in the subject header of whether anything had been added.

Ken Wesson

unread,
Dec 15, 2010, 11:14:14 AM12/15/10
to clo...@googlegroups.com
On Wed, Dec 15, 2010 at 11:07 AM, nicola...@gmail.com
<nicola...@gmail.com> wrote:
>> Again, there'd have to be a staggering further benefit from the change
>> than just "the clojure.core code looks cleaner in github" or even "the
>> code is a bit easier to maintain in the future". I'm not sure that
>> even massive increases in code maintainability alone suffice for
>> something like this. A major, massively end-user-useful new feature
>> that's nigh-impossible to implement otherwise that can then be
>> implemented *might* suffice.
>
> I think the point was not to criticize you at all (nobody have to read
> every post in the archive)
> but just to point the fact that this debate was somehow already behind us.

Perhaps so, but if so, this pointing-out was done in a manner that was
open to interpretation.

> It has been already debated a lot with pretty good points on each
> side. (The concerns you are expressing
> are good points, as are the answers in this thread and the preceeding
> on the subject.)

What answers in this thread? The points I'm aware of so far are:

Pros (heard from others so far and taken on faith):
* Clojure's code base can be made internally simpler.
* It's easier to implement what was already implemented. (?)
* Code that isn't performance-critical may get slightly faster.

Cons: (easily inferred from what's been stated)

* Code that is performance-critical won't get any faster but will
need to be rewritten.
* It may even get slower if there's no way to get the old
unchecked-foo behavior where overflows wrap rather than throw OR
get promoted.
* Everything else may break randomly from time to time if not
rewritten.
* Ugly tick-marks will appear after every arithmetic operator in
ordinary code.
* Or else ugly Ns after every numeric literal in ordinary code.
* Or else it breaks randomly from time to time.
* Non-backward-compatible semantic changes affecting nearly every
existing sourcefile!

I can only assume, in the spirit of interpreting these events
charitably, that there may be a pro or two not on my list. :) But that
means that those pros were not raised in this thread. You said the
answers in this thread are good points, but the second pro on my list
is damned dubious, the third is clearly not worth wrecking backward
compatibility, and I'm highly dubious of the notion that the first is,
or even all three of those combined.

> For the specific problem you are facing, I would advise to go through
> your code looking for number literal and
> replacing each number nnnn by nnnnN. (ex 15 ---> 15N).
>
> As boxedness is contagious, you probably won't have to change much
> more to solve compatibility issues with 1.3.
>
> If you find other specific problems, it would be great to post them:
> we will need a good compilation of tips to convert numerical code from
> 1.2 to 1.3.

Thanks. I'll keep this in mind if I'm ever forced to upgrade to 1.3.

David Nolen

unread,
Dec 15, 2010, 11:24:56 AM12/15/10
to clo...@googlegroups.com
On Wed, Dec 15, 2010 at 11:14 AM, Ken Wesson <kwes...@gmail.com> wrote:
Pros (heard from others so far and taken on faith):
* Clojure's code base can be made internally simpler.
* It's easier to implement what was already implemented. (?)
* Code that isn't performance-critical may get slightly faster.

Not many people have mentioned concrete benefits from this change because they have already been enumerated in the original thread about the 1.3 primitive math change. So I don't think anyone is going to defend or critique your arguments.

From *my* experience unless you're working on Project Euler, you won't have to change your code. I've have yet to hear anyone else state otherwise, which, admittedly, is not conclusive, but perhaps a good sign.

David

Eric Schulte

unread,
Dec 15, 2010, 12:51:26 PM12/15/10
to clo...@googlegroups.com
Ken Wesson <kwes...@gmail.com> writes:

> Are you honestly suggesting I search the archives

It is common courtesy on open-source lists such as this one to check if
a question you are about to ask has already been answered. Not only
does it save a lot of noise on the list, but it often means that you
find your answer immediately rather than having to wait for a reply.

> for every word of every thought that it ever occurs to me to post
> here?

hyperbole helps no-one, generally a single short query suffices

Mike Meyer

unread,
Dec 15, 2010, 12:52:52 PM12/15/10
to clo...@googlegroups.com

One of the things those of us on your side *begged* for (and
apparently also didn't get) was that the versions with correct
behavior not have second-class names. That we didn't get them means
we'll have to provide them ourselves, but it's easy. Since we're
having to fix code anyway, just use a file containing something like:

(def add +')
(def sub -')
(def mul *')
(def add1 inc')
(def sub1 dec')

when you need it.

<mike
--
Mike Meyer <m...@mired.org> http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org

Ken Wesson

unread,
Dec 15, 2010, 1:02:13 PM12/15/10
to clo...@googlegroups.com
On Wed, Dec 15, 2010 at 12:51 PM, Eric Schulte <schult...@gmail.com> wrote:
> Ken Wesson <kwes...@gmail.com> writes:
>
>> Are you honestly suggesting I search the archives
>
> It is common courtesy on open-source lists such as this one to check if
> a question you are about to ask has already been answered.

As I believe I already mentioned, if everyone spends a while searching
some archives every time they are going to post, this list's traffic
will drop to nearly nil. Do we really want that?

Ken Wesson

unread,
Dec 15, 2010, 1:05:19 PM12/15/10
to clo...@googlegroups.com
On Wed, Dec 15, 2010 at 12:52 PM, Mike Meyer
<mwm-keyword-goo...@mired.org> wrote:
> One of the things those of us on your side *begged* for (and
> apparently also didn't get) was that the versions with correct
> behavior not have second-class names. That we didn't get them means
> we'll have to provide them ourselves, but it's easy. Since we're
> having to fix code anyway, just use a file containing something like:
>
> (def add +')
> (def sub -')
> (def mul *')
> (def add1 inc')
> (def sub1 dec')

I'd much rather exclude +, -, etc. when importing core and then (def +
+'), etc. and (def unchecked-+ clojure.core/+) etc.

Of course that adds enough boilerplate to the code to make me want to
write a my-ns macro that passes most of its arguments through to ns
but does all of the above as well.

And still leaves me no way to get the old unchecked-+ speed and
behavior (wrap on overflow rather than compare-and-branch then throw).
I can only hope the branch prediction in modern CPUs will mostly
prevent the speed loss from this change.

Eric Schulte

unread,
Dec 15, 2010, 1:12:31 PM12/15/10
to clo...@googlegroups.com
Ken Wesson <kwes...@gmail.com> writes:

1. I disagree with your assertion that traffic would drop to zero, and
2. I would not mind if posts which repeat previous posts were not sent

but maybe I'm wrong, and creating a vibrant open-source community is as
simple as a thesaurus-equipped script which re-sends old mailing list
posts with some synonym replacement. :)

Ken Wesson

unread,
Dec 15, 2010, 1:53:16 PM12/15/10
to clo...@googlegroups.com
On Wed, Dec 15, 2010 at 1:12 PM, Eric Schulte <schult...@gmail.com> wrote:
> Ken Wesson <kwes...@gmail.com> writes:
>
>> On Wed, Dec 15, 2010 at 12:51 PM, Eric Schulte <schult...@gmail.com> wrote:
>>> Ken Wesson <kwes...@gmail.com> writes:
>>>
>>>> Are you honestly suggesting I search the archives
>>>
>>> It is common courtesy on open-source lists such as this one to check if
>>> a question you are about to ask has already been answered.
>>
>> As I believe I already mentioned, if everyone spends a while searching
>> some archives every time they are going to post, this list's traffic
>> will drop to nearly nil. Do we really want that?
>
> 1. I disagree with your assertion that traffic would drop to zero, and

I said "nearly nil", not exactly zero. It would just slow down
tremendously as everyone spent ages poking around in Google instead of
coding or writing insightful mailing list posts just to be sure of not
maybe possibly duplicating something someone else said six years ago
in this list. :)

Mike Meyer

unread,
Dec 15, 2010, 3:19:22 PM12/15/10
to clo...@googlegroups.com, kwes...@gmail.com

Actually, what would drop to nearly nil would be questions that have
already been answered and points that have already been debated to
death. And yes, we *do* want that. In fact, it's part of the FAQ on
asking smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html.

Further, not only is it easy (the list is hosted by google groups, so
just googling for keywords will search it) but may well turn up the
answer faster than posting to the list.

Admittedly, that (or searching the archives proper) might not have
found what you're looking for in this case, because neither enhanced
nor primitive leap to mind as search terms when you get an unexpected
overflow error. But it works often enough that checking google before
asking a question is a productive habit to get into.

Jason Wolfe

unread,
Dec 15, 2010, 6:11:36 PM12/15/10
to Clojure


On Dec 15, 10:05 am, Ken Wesson <kwess...@gmail.com> wrote:
> On Wed, Dec 15, 2010 at 12:52 PM, Mike Meyer
>
This was one of the stated features of the new release:

user=> *clojure-version*
{:major 1, :minor 3, :incremental 0, :qualifier "alpha4"}
user=> (doc *unchecked-math*)
-------------------------
clojure.core/*unchecked-math*
While bound to true, compilations of +, -, *, inc, dec and the
coercions will be done without overflow checks. Default: false.

-Jason

Stuart Halloway

unread,
Dec 16, 2010, 8:13:11 AM12/16/10
to clo...@googlegroups.com
> Worse, from the sounds of it the new + isn't exactly the old
> unchecked-+; it still checks for overflow rather than allowing
> wrapping. That's going to add a compare-and-branch to every add
> instruction and halve the speed of those operators on typical
> hardware. Compare-and-throw-exception is hardly superior to
> compare-and-box-in-BigInteger, since it's still slow AND now some
> arithmetic code that used to work but be slow will now explode in your
> face.

This argument is based on the (provably wrong) presumption that "still slow" means "equally slow". The difference is percentage points vs. order of magnitude. Test it for yourself.

The Clojure/core team has an action item to pull the docs on the numeric stuff into one place so that it is easier to point people to a single location and prevent rehashing old issues.

Until that is done, if you want to continue this thread, please make arguments that back up notions like "slow" or "fast" or "broken" with evidence from real code.

Stu

Stuart Halloway

unread,
Dec 16, 2010, 8:17:10 AM12/16/10
to clo...@googlegroups.com
> On Tue, Dec 14, 2010 at 9:56 PM, David Nolen <dnolen...@gmail.com> wrote:

>> On Tue, Dec 14, 2010 at 9:04 PM, Ken Wesson <kwes...@gmail.com> wrote:
>>>
>>> On Tue, Dec 14, 2010 at 8:23 PM, Benny Tsai <benny...@gmail.com> wrote:
>>>> As Brian said, primitive math is now the default in 1.3. If auto-
>>>> promotion on overflow is desired, you can use the +', -', *', inc',
>>>> dec' functions (note the single quote suffix).
>>>
>>> Why was this done? I preferred having +, -, etc. DTRT in general and
>>> unchecked-+, etc. for when you really needed efficient primitive math.
>>> My code is littered with + but has few unchecked-+s. Which means I'll
>>> have to go through it all adding little tick-marks everywhere and
>>> making the math look funny to keep its behavior the same whenever 1.3
>>> is released.
>>
>> For quite a few good reasons, the most important being that it makes Rich
>> Hickey's life a lot easier, and ours as well as a result of that.
>
> Breaking source compatibility with just about every single preexisting
> line of Clojure code out there is supposed to make our lives *easier*?
> I'd dearly love to know how -- my cousin is a stage magician and he's
> always on the lookout for new tricks, so this would make a nearly
> perfect Christmas present for him. :)

We are aware that this is a breaking change. :-)

In addition to talking on IRC and the mailing list, we checked dozens of Clojure libraries (code review and test suite) and found *minimal* breakage. If anyone has different empirical evidence to offer, please do so.

Stu

Stuart Halloway

unread,
Dec 16, 2010, 8:23:00 AM12/16/10
to clo...@googlegroups.com
> In practice, I haven't seen a significant speed improvement in the new branch of Clojure (except on specific benchmarks that intentionally test Clojure's new default primitive math). In my day-to-day code, all my numbers, despite being perfectly small enough to fit in a long, end up getting stored and retrieved from Clojure's various data structures - vectors, maps, sets, etc. and thus lose their primitiveness. So I presumably haven't seen any speed improvement because all the numbers are boxed by the time I do math on them. Fortunately, I also don't seem to run into arithmetic overflows because my production code isn't particularly math intensive, but I still end up feeling stressed out trying to convince myself that it can't ever overflow for any input.

Mark nails it. The interesting thing about the numeric change in 1.3 is that a great deal of application code is unaffected in *either* direction! That is,

(1) It isn't a lot faster.

(2) Very little breaks.


Ken Wesson

unread,
Dec 16, 2010, 11:18:51 AM12/16/10
to clo...@googlegroups.com
On Thu, Dec 16, 2010 at 8:13 AM, Stuart Halloway
<stuart....@gmail.com> wrote:
>> Worse, from the sounds of it the new + isn't exactly the old
>> unchecked-+; it still checks for overflow rather than allowing
>> wrapping. That's going to add a compare-and-branch to every add
>> instruction and halve the speed of those operators on typical
>> hardware. Compare-and-throw-exception is hardly superior to
>> compare-and-box-in-BigInteger, since it's still slow AND now some
>> arithmetic code that used to work but be slow will now explode in your
>> face.
>
> This argument is based on the (provably wrong) presumption that "still slow" means "equally slow". The difference is percentage points vs. order of magnitude. Test it for yourself.

That does not make sense, since the implementations in both cases have
to test for overflow and branch. In the "overflowed" branch further
expensive actions are taken -- in both cases the creation of a Java
object, for instance (an exception or a boxed numeric). These branches
might differ in other ways in speed, but they're the rare case. The
common case is test and accept the result, returning it, in both
cases; so the common case should have comparable execution speed given
both implementations. If not, something is wrong someplace else with
at least one of the implementations (or, much less likely, with the
JVM/JIT).

> Until that is done, if you want to continue this thread, please make arguments that back up notions like "slow" or "fast" or "broken" with evidence from real code.

As for "broken", you can't honestly think changing the semantics of
the + operator, after years of code written in Clojure has
accumulated, won't break *something*. Surely. Most likely many
somethings, scattered all over the place.

Ken Wesson

unread,
Dec 16, 2010, 11:19:53 AM12/16/10
to clo...@googlegroups.com
On Thu, Dec 16, 2010 at 8:17 AM, Stuart Halloway
<stuart....@gmail.com> wrote:

> I wrote:
>> Breaking source compatibility with just about every single preexisting
>> line of Clojure code out there is supposed to make our lives *easier*?
>> I'd dearly love to know how -- my cousin is a stage magician and he's
>> always on the lookout for new tricks, so this would make a nearly
>> perfect Christmas present for him. :)
>
> We are aware that this is a breaking change. :-)
>
> In addition to talking on IRC and the mailing list, we checked dozens of Clojure libraries (code review and test suite) and found *minimal* breakage. If anyone has different empirical evidence to offer, please do so.

Define "minimal".

nicola...@gmail.com

unread,
Dec 16, 2010, 11:35:26 AM12/16/10
to clo...@googlegroups.com
> The
> common case is test and accept the result, returning it, in both
> cases; so the common case should have comparable execution speed given
> both implementations. If not, something is wrong someplace else with
> at least one of the implementations (or, much less likely, with the
> JVM/JIT).
>

I am not a specialist but, if I understood well, the new
implementation does not box the result, which is a
*significant* speed up.

David Nolen

unread,
Dec 16, 2010, 11:36:52 AM12/16/10
to clo...@googlegroups.com
On Thu, Dec 16, 2010 at 11:18 AM, Ken Wesson <kwes...@gmail.com> wrote:
On Thu, Dec 16, 2010 at 8:13 AM, Stuart Halloway
>> Worse, from the sounds of it the new + isn't exactly the old
>> unchecked-+; it still checks for overflow rather than allowing
>> wrapping. That's going to add a compare-and-branch to every add
>> instruction and halve the speed of those operators on typical
>> hardware. Compare-and-throw-exception is hardly superior to
>> compare-and-box-in-BigInteger, since it's still slow AND now some
>> arithmetic code that used to work but be slow will now explode in your
>> face.

Hacker's Delight shows how the overflow check can be done w/ around 6-8% hit on performance. Clojure implements that strategy.

Arguments without any knowledge of the details seems fruitless. Why not try the primitive branch yourself and report back w/ actual experience? I've been using 1.3 exclusively for some time and experienced no trouble.

David 

Stuart Halloway

unread,
Dec 16, 2010, 12:04:10 PM12/16/10
to clo...@googlegroups.com


What folllows is more of a characterization than a definition:

Number of projects checked: 20+ open source projects and a similar number of commercial projects.

Number of unit tests broken by changes in 1.3, across all projects: 1. (Solution: Replace "1" with "1N" in test input.)

Number of higher-level tests broken: 0.

Number of production breakages observed: 0.

It takes a lot of effort to do this checking. I have done it. It takes almost zero time to offer opinions without bothering to check.

Stu


Ken Wesson

unread,
Dec 16, 2010, 12:06:24 PM12/16/10
to clo...@googlegroups.com
On Thu, Dec 16, 2010 at 11:36 AM, David Nolen <dnolen...@gmail.com> wrote:
> On Thu, Dec 16, 2010 at 11:18 AM, Ken Wesson <kwes...@gmail.com> wrote:
>>
>> On Thu, Dec 16, 2010 at 8:13 AM, Stuart Halloway
>> <stuart....@gmail.com> wrote:
>> >> Worse, from the sounds of it the new + isn't exactly the old
>> >> unchecked-+; it still checks for overflow rather than allowing
>> >> wrapping. That's going to add a compare-and-branch to every add
>> >> instruction and halve the speed of those operators on typical
>> >> hardware. Compare-and-throw-exception is hardly superior to
>> >> compare-and-box-in-BigInteger, since it's still slow AND now some
>> >> arithmetic code that used to work but be slow will now explode in your
>> >> face.
>
> Hacker's Delight shows how the overflow check can be done w/ around 6-8% hit
> on performance. Clojure implements that strategy.

The overflow check is the same whether you react to an overflow by
boxing the result or react to an overflow by throwing an exception!

> Arguments without any knowledge of the details seems fruitless.

This reads like a personal criticism to me. And meanwhile you managed
to entirely miss my point.

Ken Wesson

unread,
Dec 16, 2010, 12:09:46 PM12/16/10
to clo...@googlegroups.com
On Thu, Dec 16, 2010 at 12:04 PM, Stuart Halloway

<stuart....@gmail.com> wrote:
> I wrote:
>> On Thu, Dec 16, 2010 at 8:17 AM, Stuart Halloway
>> <stuart....@gmail.com> wrote:
>>> I wrote:
>>>> Breaking source compatibility with just about every single preexisting
>>>> line of Clojure code out there is supposed to make our lives *easier*?
>>>> I'd dearly love to know how -- my cousin is a stage magician and he's
>>>> always on the lookout for new tricks, so this would make a nearly
>>>> perfect Christmas present for him. :)
>>>
>>> We are aware that this is a breaking change. :-)
>>>
>>> In addition to talking on IRC and the mailing list, we checked dozens of Clojure libraries (code review and test suite) and found *minimal* breakage. If anyone has different empirical evidence to offer, please do so.
>>
>> Define "minimal".
>
> What folllows is more of a characterization than a definition:
>
> Number of projects checked: 20+ open source projects and a similar number of commercial projects.
>
> Number of unit tests broken by changes in 1.3, across all projects: 1. (Solution: Replace "1" with "1N" in test input.)

So we're looking at breakage in 2% of cases, IF the testing was
thorough and included (simulated or actual) industrial-scale use of
the systems (overflow problems may well not show up with small test
cases and then blow up in your face in a production environment with
much bigger inputs) and IF that's a statistically significant sample
size.

2% may not sound like much but imagine the uproar if Oracle made a
non-compatible change that broke 2% of all Java codebases!

> It takes almost zero time to offer opinions without bothering to check.

That looks like yet another unproductive, non-constructive personal criticism.

Stuart Halloway

unread,
Dec 16, 2010, 12:13:08 PM12/16/10
to clo...@googlegroups.com
>>> Worse, from the sounds of it the new + isn't exactly the old
>>> unchecked-+; it still checks for overflow rather than allowing
>>> wrapping. That's going to add a compare-and-branch to every add
>>> instruction and halve the speed of those operators on typical
>>> hardware. Compare-and-throw-exception is hardly superior to
>>> compare-and-box-in-BigInteger, since it's still slow AND now some
>>> arithmetic code that used to work but be slow will now explode in your
>>> face.
>>
>> This argument is based on the (provably wrong) presumption that "still slow" means "equally slow". The difference is percentage points vs. order of magnitude. Test it for yourself.
>
> That does not make sense, since the implementations in both cases have
> to test for overflow and branch. In the "overflowed" branch further
> expensive actions are taken -- in both cases the creation of a Java
> object, for instance (an exception or a boxed numeric). These branches
> might differ in other ways in speed, but they're the rare case. The
> common case is test and accept the result, returning it, in both
> cases; so the common case should have comparable execution speed given
> both implementations. If not, something is wrong someplace else with
> at least one of the implementations (or, much less likely, with the
> JVM/JIT).

It will make sense once you understand the implications of primitives and objects not being unified, which come into play before the test for overflow, and hurt you even in applications where overflow never happens.

If this thread is not sufficiently clear on that point, and you don't have time to run tests for yourself, can I ask that you please hold off on beating this dead horse until we can

>> Until that is done, if you want to continue this thread, please make arguments that back up notions like "slow" or "fast" or "broken" with evidence from real code.
>
> As for "broken", you can't honestly think changing the semantics of
> the + operator, after years of code written in Clojure has
> accumulated, won't break *something*. Surely. Most likely many
> somethings, scattered all over the place.

That's why it is called a breaking change. :-) We have (and will continue to have) a large number of prereleases for people to shake out issues.

Stu

nicola...@gmail.com

unread,
Dec 16, 2010, 12:14:09 PM12/16/10
to clo...@googlegroups.com
>
> The overflow check is the same whether you react to an overflow by
> boxing the result or react to an overflow by throwing an exception!


But then all the rest of the code has to check whether things are boxed or not.
Moreover, the JVM makes it very hard (impossible) to manipulate
something that is
either a boxed or a primitive value.

Terrance Davis

unread,
Dec 16, 2010, 12:18:47 PM12/16/10
to clo...@googlegroups.com
*begin rant*

I have yet to see anyone who posts the classic "rtfm" (even politely)
response search previous posts and realize that "rtfm" responses have
already been sent and refrain from sending the same explanation of how
to use a mailing list over and over and over. Simple customer service
experience teaches that if customers are asking the same questions
multiple times, then the documentation is either, hard to find,
incomplete, or not clear enough. Improving the docs is a healthier and
more productive use of time than starting yet another thread on how to
use a mailing list.

*end rant*

Sorry. Couldn't contain myself ;-)

Baishampayan Ghose

unread,
Dec 16, 2010, 12:22:42 PM12/16/10
to clo...@googlegroups.com
>> It takes almost zero time to offer opinions without bothering to check.
>
> That looks like yet another unproductive, non-constructive personal criticism.

Why do you think so? These people are just requesting you to check
things for yourself instead engaging in this meaningless argument. The
design decision of implementing "enhanced primitives" support in
Clojure was taken months ago after a *lot* of intense debate, thinking
& research.

You are unwilling to dig the archives, read the implementation or even
test an existing codebase for issues, yet you are accusing people of
criticizing you just because they feel you should do a bit more
research about this.

Don't you think it's unfair of you?

Talk is cheap, show us the code.

Regards,
BG

--
Baishampayan Ghose
b.ghose at gmail.com

David Nolen

unread,
Dec 16, 2010, 12:24:24 PM12/16/10
to clo...@googlegroups.com
On Thu, Dec 16, 2010 at 12:06 PM, Ken Wesson <kwes...@gmail.com> wrote:
The overflow check is the same whether you react to an overflow by
boxing the result or react to an overflow by throwing an exception!

It's not the same at all.

If you box the result all further arithmetic computations slows down. You cannot preserve the primitive path.

Throwing an exception means "Hey it looks like you want big math, clearly you don't want or need primitive perf, fix your code". So use auto-promoting variants, or add a big integer literal in the right place. BigInts are contagious. In this world very little work needs to be done to satisfy the real desires of people who want the fast path and those people who want big math. You're right that this is a breaking change. Some code will get broken.

However there are other breaking changes in 1.3 that have far greater implications for real Clojure apps than this change - like dynamic binding. This affects a much large range of Clojure apps, libraries and tools.

But Clojure is not Oracle. Rich Hickey has stated that he will continue to forge ahead. Things are not going to "slow down". That is something that the Clojure community embraces. Otherwise why would any of us be writing in a crazy functional Lisp on the JVM :)

P.S. That was not a personal criticism as much as fact that can be applied to any discussion between respectful parties.

David

Ken Wesson

unread,
Dec 16, 2010, 12:55:04 PM12/16/10
to clo...@googlegroups.com
On Thu, Dec 16, 2010 at 12:13 PM, Stuart Halloway
<stuart....@gmail.com> wrote:
>>>> Worse, from the sounds of it the new + isn't exactly the old
>>>> unchecked-+; it still checks for overflow rather than allowing
>>>> wrapping. That's going to add a compare-and-branch to every add
>>>> instruction and halve the speed of those operators on typical
>>>> hardware. Compare-and-throw-exception is hardly superior to
>>>> compare-and-box-in-BigInteger, since it's still slow AND now some
>>>> arithmetic code that used to work but be slow will now explode in your
>>>> face.
>>>
>>> This argument is based on the (provably wrong) presumption that "still slow" means "equally slow". The difference is percentage points vs. order of magnitude. Test it for yourself.
>>
>> That does not make sense, since the implementations in both cases have
>> to test for overflow and branch. In the "overflowed" branch further
>> expensive actions are taken -- in both cases the creation of a Java
>> object, for instance (an exception or a boxed numeric). These branches
>> might differ in other ways in speed, but they're the rare case. The
>> common case is test and accept the result, returning it, in both
>> cases; so the common case should have comparable execution speed given
>> both implementations. If not, something is wrong someplace else with
>> at least one of the implementations (or, much less likely, with the
>> JVM/JIT).
>
> It will make sense once you understand

I don't care for your condescending tone.

If you have a personal problem with me, please take it up in personal
email rather than posting it to the list. Thank you.

Now, as I understand it, + and the like, being normal Clojure
functions (rather than, say, special forms or interop calls), take and
return boxed values. So boxing overhead is unaffected by all this. We
have a function that, say, takes two Integers, sees if the result
overflows, and if not adds them and returns the boxed result, and if
it does overflow, either throws an exception or returns a BigInteger
rather than an Integer.

The check should be about the same speed in either case.

Now, BigInteger contagion could slow things down, but it would only do
so in the case where the proposed changes result in exceptions being
thrown (= breakage, in the case of old code that assumes BigInteger
promotion will happen instead).

So, some code won't slow down or speed up appreciably; other code
would speed up if the behavior was to wrap the value but will instead
simply not work now.

I still confess I don't see a big advantage here.

Ken Wesson

unread,
Dec 16, 2010, 12:56:17 PM12/16/10
to clo...@googlegroups.com

I thought it had method overload resolution for that.

And that everything is boxed, except in let and loop forms sometimes,
and then whether it's boxed or not is generally known at compile time.

Rich Hickey

unread,
Dec 16, 2010, 12:57:46 PM12/16/10
to clo...@googlegroups.com

It's a breaking change. It will be clearly documented as such. Whether you think it is the right thing, minimal or whatever else is not going to change it. There have been very few breaking changes made in Clojure, given its age, and this is going to be one of them.

You'll either have to get over it, or move on. Your dissatisfaction is noted. The discussion period for this has passed.

Thanks,

Rich

Ken Wesson

unread,
Dec 16, 2010, 1:04:16 PM12/16/10
to clo...@googlegroups.com
On Thu, Dec 16, 2010 at 12:22 PM, Baishampayan Ghose <b.g...@gmail.com> wrote:
>>> It takes almost zero time to offer opinions without bothering to check.
>>
>> That looks like yet another unproductive, non-constructive personal criticism.
>
> Why do you think so?

Because of the implication that my opinions are uninformed ones not
worth their time. (It's interesting, though, that they apparently
consider those opinions worth their time to criticize, but not worth
their time to actually consider carefully!)

> These people are just requesting you to check things for yourself

I CAN'T check things for myself -- I only have 1.2 here and I'm not
about to break all of my OWN code by "up"grading it to an alpha
version that has at least one KNOWN massive compatibility-breaking
change as well as being likely to contain significant unfixed bugs.

Really, I was wondering if anyone would manage to top the ridiculous
suggestion that every post be preceded by two hours of thorough
archive-diving and reading of older posts.

Now you have, by making the truly *ludicrous* suggestion that one not
post unless one is using the bleedingest-edge alpha version, complete
with whatever headaches that will certainly induce (not least among
them, updating everything every few days instead of every few
months/years).

> instead engaging in this meaningless argument. The
> design decision of implementing "enhanced primitives" support in
> Clojure

We had fine primitives support in let and loop with the unchecked-foo
operations; and it didn't affect the rest of one's code, which was
generally not arithmetic-performance-critical.

I still do not see what advantage this change brings. Can the fastest
primitive operations in let and loop contexts be made any faster? Not
that I've heard. Can primitives now be passed and returned in function
calls? Not that I've heard, just "we're working on it".

> You are unwilling to dig the archives

I am unwilling to accede to a request if it's stated rudely enough. I
DID read the summary link someone posted, but it did not relieve my
concerns on this topic. This does not mean I did not read it
thoroughly and understand it. It means that I did and I STILL DISAGREE
WITH YOU. Perhaps this notion is literally inconceivable to you, but
those're the facts, Jack.

> read the implementation

Don't have it. (1.3 alpha 4, that is.)

> or even test an existing codebase for issues

Don't have it. (1.3 alpha 4, that is.)

> yet you are accusing people of criticizing you just because they feel you should do a bit more
> research about this.

I'm asking them to explain themselves better, and their responses are
not any kind of explanation.

Ken Wesson

unread,
Dec 16, 2010, 1:06:43 PM12/16/10
to clo...@googlegroups.com
On Thu, Dec 16, 2010 at 12:24 PM, David Nolen <dnolen...@gmail.com> wrote:
> On Thu, Dec 16, 2010 at 12:06 PM, Ken Wesson <kwes...@gmail.com> wrote:
>>
>> The overflow check is the same whether you react to an overflow by
>> boxing the result or react to an overflow by throwing an exception!
>
> It's not the same at all.
> If you box the result all further arithmetic computations slows down. You
> cannot preserve the primitive path.

All further arithmetic computations slow down on the one hand; halt
with an exception if the opposite choice is made. Slow-but-works is
usually preferred to broken.

> However there are other breaking changes in 1.3 that have far greater
> implications for real Clojure apps than this change - like dynamic binding.
> This affects a much large range of Clojure apps, libraries and tools.

I don't know how common dynamic binding is in application code. It
tends to be in library code more often, which is a smaller number of
changes to make. Plus, the dynamic binding changes have a rationale
behind them that actually seems to me to make the tradeoff potentially
worth it.

Baishampayan Ghose

unread,
Dec 16, 2010, 1:15:56 PM12/16/10
to clo...@googlegroups.com
>> yet you are accusing people of criticizing you just because they feel you should do a bit more
>> research about this.
>
> I'm asking them to explain themselves better, and their responses are
> not any kind of explanation.

Please try putting yourself in their shoes. They have already
explained themselves the best they could, and it's all documented (in
the archives, the wiki, etc).

Why do you think they should do it again? Anyway, as Rich said in his
response, the die has been cast and it can't be changed at the moment.

You will have to either accept it or stick to Clojure 1.2.x.

I am requesting you to end the discussion here.

David Nolen

unread,
Dec 16, 2010, 1:25:15 PM12/16/10
to clo...@googlegroups.com
On Thu, Dec 16, 2010 at 1:06 PM, Ken Wesson <kwes...@gmail.com> wrote:
I don't know how common dynamic binding is in application code. It
tends to be in library code more often, which is a smaller number of
changes to make. Plus, the dynamic binding changes have a rationale
behind them that actually seems to me to make the tradeoff potentially
worth it. 

I find this situation quite humorous:

1.3 Dynamic binding breaking change

Much faster. People don't seem to have strong opinions. But a lot of things break. People fix it and move on.

1.3 Primitive math

Much faster. Everyone seems to have a strong opinion. Hardly anything breaks. People move on.

David 

Stuart Halloway

unread,
Dec 16, 2010, 1:27:06 PM12/16/10
to clo...@googlegroups.com
> <stuart....@gmail.com> wrote:
>>>>> Worse, from the sounds of it the new + isn't exactly the old
>>>>> unchecked-+; it still checks for overflow rather than allowing
>>>>> wrapping. That's going to add a compare-and-branch to every add
>>>>> instruction and halve the speed of those operators on typical
>>>>> hardware. Compare-and-throw-exception is hardly superior to
>>>>> compare-and-box-in-BigInteger, since it's still slow AND now some
>>>>> arithmetic code that used to work but be slow will now explode in your
>>>>> face.
>>>>
>>>> This argument is based on the (provably wrong) presumption that "still slow" means "equally slow". The difference is percentage points vs. order of magnitude. Test it for yourself.
>>>
>>> That does not make sense, since the implementations in both cases have
>>> to test for overflow and branch. In the "overflowed" branch further
>>> expensive actions are taken -- in both cases the creation of a Java
>>> object, for instance (an exception or a boxed numeric). These branches
>>> might differ in other ways in speed, but they're the rare case. The
>>> common case is test and accept the result, returning it, in both
>>> cases; so the common case should have comparable execution speed given
>>> both implementations. If not, something is wrong someplace else with
>>> at least one of the implementations (or, much less likely, with the
>>> JVM/JIT).
>>
>> It will make sense once you understand
>
> I don't care for your condescending tone.

I apologize. This forum is renowned for civil discourse, and I hope it will continue to be.

> Now, as I understand it, + and the like, being normal Clojure
> functions (rather than, say, special forms or interop calls), take and
> return boxed values.

I think I can help here:

1. Clojure's numeric operators are more than normal Clojure functions. You can see hints of this in the source code (even in 1.2!) in the :inline and :inline-arities metadata.

2. As such, Clojure's numeric operators are capable of working directly with primitives.

3. In 1.3 functions can take and return primitives.

It is the combination of these factors that provides the speed boost in 1.3, so if you were unaware of some of them, perhaps we have been talking past each other.

Stu

Ken Wesson

unread,
Dec 16, 2010, 1:28:07 PM12/16/10
to clo...@googlegroups.com
On Thu, Dec 16, 2010 at 1:15 PM, Baishampayan Ghose <b.g...@gmail.com> wrote:
>>> yet you are accusing people of criticizing you just because they feel you should do a bit more
>>> research about this.
>>
>> I'm asking them to explain themselves better, and their responses are
>> not any kind of explanation.
>
> Please try putting yourself in their shoes. They have already
> explained themselves the best they could, and it's all documented

Then why didn't the link I followed to the summary assuage my doubts?

> Why do you think they should do it again? Anyway, as Rich said in his
> response, the die has been cast and it can't be changed at the moment.

That same summary implied there were two proposals, not just one; in
one of them, + and the like retain their current behavior and it is +'
and the like that you use if you want potentially faster math.

> I am requesting you to end the discussion here.

I reserve the right to respond in my own defense if personally
attacked, and to respond if asked a question.

Mike Meyer

unread,
Dec 16, 2010, 3:52:28 PM12/16/10
to clo...@googlegroups.com
On Thu, 16 Dec 2010 10:18:47 -0700
Terrance Davis <terranc...@gmail.com> wrote:

> *begin rant*
>
> I have yet to see anyone who posts the classic "rtfm" (even politely)
> response search previous posts and realize that "rtfm" responses have
> already been sent and refrain from sending the same explanation of how
> to use a mailing list over and over and over. Simple customer service
> experience teaches that if customers are asking the same questions
> multiple times, then the documentation is either, hard to find,
> incomplete, or not clear enough. Improving the docs is a healthier and
> more productive use of time than starting yet another thread on how to
> use a mailing list.
>
> *end rant*
>
> Sorry. Couldn't contain myself ;-)

No need to be sorry - it's a very good point.

In the past, I've contributed to open source projects by watching for
the same question to be raised multiple times, combining the data in
the best answers into one "best of breed", and submitting it as a
patch for the project handbook. The clojure community doesn't have
anything as spiffy as the FreeBSD handbook - instead we have a wiki
FAQ page (from clojure.org, click "wiki" then "2 FAQ" to get to
http://en.wikibooks.org/wiki/Clojure_Programming). While much more
painful than editing docbook, it's a good place to post things.

Unfortunately, we can't post excerpts from clojure source there
because the wiki license is incompatible with the source license - or
anything else using that same license. In particular, not being able
to use doc strings, etc.

Given that the FAQ itself suggests that such be posted to the clojure
group, this makes doing what I did rather problematical. Minimally, I
need to figure out whether or not a post contains such an excerpt in
order to be able to use it. Worst case, the license for content posted
to the group is *also* incompatible with the source license, so you
can't legally add any Frequent Answers from there to the FAQ.

Ok, I found a problem. Anyone got solutions?

<mike

PS: behavior-modification posts (i.e. - the "rtfm" posts) needs to be
done repeatedly because (we hope) the group keeps growing. If everyone
simply ignores improper behavior, the newcomers will assume it's
proper behavior (even if there's a covenant for the group saying
otherwise). Hence periodic reminders are called for.

--
Mike Meyer <m...@mired.org> http://www.mired.org/consulting.html
Independent Network/Unix/Perforce consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org

Stuart Halloway

unread,
Dec 16, 2010, 5:50:58 PM12/16/10
to clo...@googlegroups.com
The new FAQ (under construction at http://dev.clojure.org/display/doc/FAQ) has edit capabilities tied to signing the CA.

If you have signed a CA you can post to the FAQ, quoting from other CA-governed sources (e.g. Clojure) as makes sense.

Does that help?

Stu

Mike Meyer

unread,
Dec 16, 2010, 6:46:53 PM12/16/10
to clo...@googlegroups.com

First, that looks more like it's part of a wiki aimed at clojure
developers than at clojure users. While that's an important thing to
have, it's not what I'm looking for, and I'm pretty sure that
combining the two is bad idea.

Second, a wiki that requires you be a contributor to edit it seems to
defeat the point of having a wiki in the first place. If you're going
to require someone to be a contribute to edit the docs, why not use a
real document processing system instead, so that you can handle
documentation using the same tools (source control system, etc.) as
you're using for code, and get the added benefit of being able to
generate media-specific output (assuming you chose a good dps).

Finally, Jira needs a category for "wiki bugs" so that people who
haven't signed a CA can contribute fixes (patches? to a wiki? Another
reason to use a real document processing system....).

<mike

Reply all
Reply to author
Forward
0 new messages