Clojure 1.6.0-RC1 - LAST CHANCE, PLEASE TEST

[Alex Miller]

Hello all, 

We would love to release Clojure 1.6.0 final soon.

We need your help in checking out the current release candidate - this is your opportunity to let us know about problems *before* we release, rather than after. 

Try it via 

- Download: http://central.maven.org/maven2/org/clojure/clojure/1.6.0-RC1

- Leiningen: [org.clojure/clojure "1.6.0-RC1"]

See the full change log here:

https://github.com/clojure/clojure/blob/master/changes.md

If you have questions, feel free to drop them here. For issues, please log them in JIRA. 

Alex

[Andrey Antukh]

All test passes on my projects! It works fine for me!

Thanks!

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Andrey Antukh - Андрей Антух - <andrei....@kaleidos.net> / <ni...@niwi.be>

http://www.niwi.be

https://github.com/niwibe

[Alex Miller]

Thanks Andrey! You are the wind beneath my wings.

[Michael Klishin]

2014-03-18 18:21 GMT+04:00 Alex Miller <al...@puredanger.com>:

We need your help in checking out the current release candidate - this is your opportunity to let us know about problems *before* we release, rather than after. 

No issues to report from testing 20+ ClojureWerkz projects on 1.6.0-RC1.
--
MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin

[Alex Miller]

Thanks Michael! You're the hops in my ale.

[David Nolen]

Just tried 1.6.0-RC1 with ClojureScript master. All ClojureScript tests pass except for one which is due to assumptions about print order (different now due to hash code changes). Easy enough to fix once 1.6.0 actually ships.

David

--

[Alex Miller]

Thanks David!  You're the English horn in my symphony.

[Ken Barber]

We had some bugs related to assumed ordering in our code which
1.6.0-RC1 surfaced (https://github.com/puppetlabs/puppetdb/pull/887),
otherwise everything looks good for our project - thanks for all the
hard work on this release.

ken.

[Alex Miller]

Thanks Ken! You're the cheese on my nachos.

[Ambrose Bonnaire-Sergeant]

I think I found a difference between 1.5.1->1.6.0-RC1.

(compile 'cljs.core) fails with a NPE in 1.6.0-RC1. I realise this isn't particularly supported

by CLJS, but it still worked in 1.5.1.

https://github.com/frenchy64/clojure-16-fail

Help narrowing this is appreciated.

Thanks,

Ambrose

[Ambrose Bonnaire-Sergeant]

I believe this is because cljs.core defines unsigned-bit-shift-right, which now conflicts

with clojure.core/unsigned-bit-shift-right (added with 1.6.0).

1.6.0 doesn't seem to break anything here, aside from adding to clojure.core.

Thanks,

Ambrose

[Alex Miller]

Yeah, that's a thing. I think the name was chosen intentionally to be the same across those as cljs had it first.

[Ambrose Bonnaire-Sergeant]

FYI `mvn test` will fail on core.typed because of this issue.

I believe it otherwise passes.

Ambrose

[Alex Miller]

I assume that fixing this is a matter of updating src/clj/cljs/core.clj in clojurescript to exclude unsigned-bit-shift-right?

[David Nolen]

Yep, we'll get rid of it once 1.6.0 actually ships.

David

[Alex Baranosky]

Tests look good in some of Staples Innovation Labs' projects.  Only failures are ones that assume ordering, which is just an easily correct failure in the test cases.

[Sean Corfield]

We have 1.6.0-RC1 running in production as of this afternoon. No
problems so far...

Sean

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--

Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)

[Peter Taoussanis]

I've got RC1 running in a number of places, all working well.

[Stefan Kamphausen]

hi,

RC1 works well with our largest project.  See a 5-8% slow-down but no other problems.

Regards,
Stefan

[Alex Miller]

<record scratch> what's that about performance now?

Is that something definitive and reproducible? And if so, is there any way to track down a cause?

[Nicola Mometto]

I'm guessing it's because of the minor overhead on hashing added with
the move to Murmur3?

[Walter van der Laan]

I moved to 1.6 two weeks ago for development and have had no problem. I had to change some tests because they depended on the order of elements if you seq a hashmap.

[Alex Miller]

That is the only set of changes that I know of that negatively affect performance in 1.6.0. However, the absolute differences are quite small and codes are cached. It's hard for me to conjure a scenario where the resulting effect is an overall 5-8% slowdown. If this is one, I'd like to understand it better. 

[Michał Marczyk]

Hashes are cached for Clojure collections, keywords and symbols, but
not for strings.

I wonder if people who report these perf regressions use long string
keys in their maps... For this to explain the regression they'd have
to be looking up the same String objects a lot, of course (else
hashCode caching wouldn't have been helpful before the move to
Murmur3).

[Stefan Kamphausen]

Hi Alex,


the runtime of the program changed from 19 to 20 seconds for one set of data and from approx 6minutes to 6:30 for another set.  To me that is acceptable.

We are reading a few MB of files into memory, create lots of intermediate vectors, maps and sets and output up to a few hundred MB of data later.  All this with pure Clojure, almost no Interop, no protocols, no records or types, just one multimethod which is only used during the first second.

Comparison was done by changing the project.clj to point to the desired Clojure version, followed by lein clean && lein uberjar and the running the uberjar with a fresh JVM.  All this on a QuadCore with 16GB RAM and OpenJDK 1.7 on Linux.  We use tools.logging, tools.cli, core.async and log4j and AOT compile :all in the uberjar.

If you still consider this a possible breaker (I humbly don't) please let me know, what I should test for you.

Kind regards,
stefan

[Stefan Kamphausen]

Hi,

On Wednesday, March 19, 2014 3:55:22 PM UTC+1, Michał Marczyk wrote:

Hashes are cached for Clojure collections, keywords and symbols, but
not for strings.

I wonder if people who report these perf regressions use long string
keys in their maps...

As far as I can see, we have /lots/ of strings in sets and lots of strings as keys, but they are not particularly /long/ (usually less than 10-15 Characters).  We have one hot spot in the code, where we create lots (hundreds of thousands up to millions) of sets, each containing one to five strings.  I will run a few tests and see whether that makes a difference.

Best,
stefan

[Alex Miller]

Thanks, this is all really useful. I would appreciate any more detailed info. 

[Steve Miner]

I just tried to build master locally with the new JDK 1.8 on a Mac.  I'm getting a build test failure.  Sorry, I don't have time to look into it at the moment.  JDK 1.7 worked fine for me.

     [java] clojure.test-clojure.reflect

     [java] 

     [java] java.lang.IllegalArgumentException

     [java] at clojure.asm.ClassReader.<init>(ClassReader.java:170)

     [java] at clojure.asm.ClassReader.<init>(ClassReader.java:153)

     [java] at clojure.asm.ClassReader.<init>(ClassReader.java:424)

     [java] at clojure.reflect.AsmReflector.do_reflect(java.clj:200)

     [java] at clojure.reflect$fn__9042$G__9038__9045.invoke(reflect.clj:44)

     [java] at clojure.reflect$fn__9042$G__9037__9049.invoke(reflect.clj:44)

     [java] at clojure.lang.AFn.applyToHelper(AFn.java:156)

     [java] at clojure.lang.AFn.applyTo(AFn.java:144)

     [java] at clojure.core$apply.invoke(core.clj:626)

     [java] at clojure.core$partial$fn__4228.doInvoke(core.clj:2468)

     [java] at clojure.lang.RestFn.invoke(RestFn.java:408)

     [java] at clojure.reflect$type_reflect.doInvoke(reflect.clj:100)

     [java] at clojure.lang.RestFn.invoke(RestFn.java:439)

     [java] at clojure.test_clojure.reflect$fn__19909.invoke(reflect.clj:23)

     [java] at clojure.test$test_var$fn__7187.invoke(test.clj:704)

     [java] at clojure.test$test_var.invoke(test.clj:704)

     [java] at clojure.test$test_vars$fn__7209$fn__7214.invoke(test.clj:722)

     [java] at clojure.test$default_fixture.invoke(test.clj:674)

     [java] at clojure.test$test_vars$fn__7209.invoke(test.clj:722)

     [java] at clojure.test$default_fixture.invoke(test.clj:674)

     [java] at clojure.test$test_vars.invoke(test.clj:718)

     [java] at clojure.test$test_all_vars.invoke(test.clj:728)

     [java] at clojure.test$test_ns.invoke(test.clj:747)

     [java] at clojure.core$map$fn__4245.invoke(core.clj:2559)

     [java] at clojure.lang.LazySeq.sval(LazySeq.java:40)

     [java] at clojure.lang.LazySeq.seq(LazySeq.java:49)

     [java] at clojure.lang.Cons.next(Cons.java:39)

     [java] at clojure.lang.RT.next(RT.java:599)

     [java] at clojure.core$next.invoke(core.clj:64)

     [java] at clojure.core$reduce1.invoke(core.clj:903)

     [java] at clojure.core$reduce1.invoke(core.clj:894)

     [java] at clojure.core$merge_with.doInvoke(core.clj:2777)

     [java] at clojure.lang.RestFn.applyTo(RestFn.java:139)

     [java] at clojure.core$apply.invoke(core.clj:626)

     [java] at clojure.test$run_tests.doInvoke(test.clj:762)

     [java] at clojure.lang.RestFn.applyTo(RestFn.java:137)

     [java] at clojure.core$apply.invoke(core.clj:624)

     [java] at clojure.test.generative.runner$run_all_tests$fn__529.invoke(runner.clj:255)

     [java] at clojure.test.generative.runner$run_all_tests$run_with_counts__521$fn__525.invoke(runner.clj:251)

     [java] at clojure.test.generative.runner$run_all_tests$run_with_counts__521.invoke(runner.clj:251)

     [java] at clojure.test.generative.runner$run_all_tests.invoke(runner.clj:253)

     [java] at clojure.test.generative.runner$test_dirs.doInvoke(runner.clj:304)

     [java] at clojure.lang.RestFn.applyTo(RestFn.java:137)

     [java] at clojure.core$apply.invoke(core.clj:624)

     [java] at clojure.test.generative.runner$_main.doInvoke(runner.clj:312)

     [java] at clojure.lang.RestFn.invoke(RestFn.java:408)

     [java] at user$eval566.invoke(run_tests.clj:4)

     [java] at clojure.lang.Compiler.eval(Compiler.java:6703)

     [java] at clojure.lang.Compiler.load(Compiler.java:7130)

     [java] at clojure.lang.Compiler.loadFile(Compiler.java:7086)

     [java] at clojure.main$load_script.invoke(main.clj:274)

     [java] at clojure.main$script_opt.invoke(main.clj:336)

     [java] at clojure.main$main.doInvoke(main.clj:420)

     [java] at clojure.lang.RestFn.invoke(RestFn.java:408)

     [java] at clojure.lang.Var.invoke(Var.java:379)

     [java] at clojure.lang.AFn.applyToHelper(AFn.java:154)

     [java] at clojure.lang.Var.applyTo(Var.java:700)

     [java] at clojure.main.main(main.java:37)

     [java] {:file "ClassReader.java",

     [java]  :tstamp 1395244274721,

     [java]  :type :error,

     [java]  :clojure.test/vars (compare-reflect-and-asm),

     [java]  :level :error,

     [java]  :pid 97553,

     [java]  :line 170,

     [java]  :thread 1,

     [java]  :exception

     [java]  #<IllegalArgumentException java.lang.IllegalArgumentException>,

     [java]  :thread/name "main",

     [java]  :message "Uncaught exception, not in assertion."}

     [java] 

[Nicola Mometto]

ASM 4.1 does not work with JDK 1.8 see
http://dev.clojure.org/jira/browse/CLJ-1323

[Stefan Kamphausen]

On Wednesday, March 19, 2014 4:34:45 PM UTC+1, Alex Miller wrote:

Thanks, this is all really useful. I would appreciate any more detailed info. 

* No atoms, agents, refs
* Almost purely functional plus logging and I/O
* Multi-threaded using latest core.async with thread, no go
* JVisualVM reports top methods in CPU sampling c.l.LazySeq.sval, c.c$promise$reify_6310.deref, c.l.LazySeq.seq no notable difference for me to see.

A few experiments on two data-sets (dat1 and dat2) using either a call to set or to doall:

Clojure	Op	Time (dat1)	Time (dat1)	Average dat1	Time (dat2)
1.5.1	set	19,1	19	19,05	323
1.5.1	doall	19	19,4	19,2	330
1.6.0-RC1	set	19,5	19,8	19,65	350
1.6.0-RC1	doall	20,2	19,8	20	351

[Softaddicts]

Moved to this release 3 days ago. Nothing to report, works as expected :)

Thank to everyone for this new release :)

Luc P.

> Hello all,
>
> We would love to release Clojure 1.6.0 final soon.
>
> We need your help in checking out the current release candidate - this is
> your opportunity to let us know about problems *before* we release, rather
> than after.
>
> Try it via
> - Download: http://central.maven.org/maven2/org/clojure/clojure/1.6.0-RC1
> - Leiningen: [org.clojure/clojure "1.6.0-RC1"]
>
> See the full change log here:
> https://github.com/clojure/clojure/blob/master/changes.md
>
> If you have questions, feel free to drop them here. For issues, please log

> them in JIRA <http://dev.clojure.org/jira/secure/Dashboard.jspa>.
>
> Alex

>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

--
Softaddicts<lprefo...@softaddicts.ca> sent by ibisMail from my ipad!

[Alex Miller]

Rich just pushed a change to the String hashing to address this. We're going to grab the string hashcode (which is cached after first call) and murmur the result of that. This gives us constant time hashcode after first call with better distribution for combinations in nested collections.  Will be in presumed RC2.

[Alex Miller]

And thanks everyone for giving RC1 a look. You are (collectively) the fortune in my fortune cookie.

[Michał Marczyk]

On 19 March 2014 21:14, Alex Miller <al...@puredanger.com> wrote:

Rich just pushed a change to the String hashing to address this. We're going to grab the string hashcode (which is cached after first call) and murmur the result of that. This gives us constant time hashcode after first call with better distribution for combinations in nested collections.  Will be in presumed RC2.

If I understand correctly, this means that keywords and symbols use String.hashCode-based hashing in this commit. They are, however, capable of caching their Murmur3 hashes, so I was wondering if it might be better to have them use Murmur3 to hash name & ns?

Rationale: murmuring hashCode results in collisions whenever hashCode does, plus possibly some additonal collisions if ints collide under murmur; murmur is better all around.

Potential problems: not sure if things like case depend on symbol/keyword hashes being directly based on string hashes; hopefully not?

Cheers,

Michał

 

[Alex Miller]

Yes, agreed.

[Alex Miller]

case depends on hashCode, not hasheq, so should be unaffected by these changes.

[Tim McCormack]

On Wednesday, March 19, 2014 4:14:38 PM UTC-4, Alex Miller wrote:
> Rich just pushed a change to the String hashing to address this. We're going to grab the string hashcode (which is cached after first call) and murmur the result of that. This gives us constant time hashcode after first call with better distribution for combinations in nested collections. Will be in presumed RC2.

(Discussion continued from IRC and Github.)

This does make PHM vulnerable to "hashDoS" attacks again -- ["AaAa" "BBBB" "AaBB" "BBAa"] will all hash to the same value, so an attacker can pass a ton of these colliding strings to a webapp as a querystring or POST body and really bog down the machine. Best article I could find on this attack: http://cryptanalysis.eu/blog/2011/12/28/effective-dos-attacks-against-web-application-plattforms-hashdos/

Is there some compromise we can make between caching and good hashing?

My naïve thought would be to combine the native String hashCode with a Murmur hash of a fixed chunk of the string, then possibly run that combination through Murmur. This avoids hashing unbounded data more than once, and would be effective against basic hashDoS. (Intelligently picking the fixed chunk of the string would be essential for protecting against an adaptive hashDoS attack.)

 - Tim McCormack

[Michał Marczyk]

We're vulnerable to this problem anyway as long as hashing is
deterministic, which is why I think it would be cool to use some
universal-hashing-like scheme...

I think Murmur3 actually uses a seed that could be randomized? Not
really "safe" in the cryptographic sense of the word, but would make
this sort of attack more challenging.

The way to avoid the problem completely is to use BST-based
dictionaries -- slower, but free from pathological edge cases.

Murmuring a short initial fragment could still be cool, just because
we'd probably get a better hash.

Cheers,
Michał

[Andy Fingerhut]

Regarding Michal's comment of using BST (binary search tree)-based dictionaries, Clojure does already have sorted-maps and sorted-sets that do this, for comparable keys/elements.

A nice hybrid of the nearly-O(1) typical case of hash maps/sets, and simultaneously protecting against the cases where there are many hash collisions, is to have the 'leaves' of the current hash maps/sets be binary search trees, rather than linked lists.

That requires the keys/element to be comparable, but today's clojure.core/compare cannot compare all pairs of things, and perhaps there is no practical way to try to include "everything" in a more general compare function so that everything hashable (i.e. every Java object) is also comparable.  Would it be easier to create a 'universal compare' function for all Clojure values, assuming they only contained Clojure values inside of them?

Andy

[Andy Fingerhut]

In case it isn't clear, I don't think such drastic changes could ever hope to be done in a reasonable time frame for Clojure 1.6.0.  Probably better to have a different thread for this if there is interest in discussing it.

Andy

[Alex Miller]

On Thursday, March 20, 2014 11:56:46 AM UTC-5, Michał Marczyk wrote:

We're vulnerable to this problem anyway as long as hashing is
deterministic, which is why I think it would be cool to use some
universal-hashing-like scheme...

Agreed - whether it's the old algorithm or new, it's still a deterministic function from input to hash and an attacker could create inputs that generate the same hash regardless.

 

I think Murmur3 actually uses a seed that could be randomized? Not
really "safe" in the cryptographic sense of the word, but would make
this sort of attack more challenging.

It does and we currently set that seed to 0. It would be completely feasible to randomize that seed on a per-VM basis thus hashing differently on different JVM instances. Java 7 includes an implementation of this (after u7) for Strings. 

In Java 8, interestingly they removed all of that and just fixed the hash collection implementation to use balanced trees for collisions instead of lists. Seems like something similar could potentially be done in the HashCollisionNode impl in PHM.

 

The way to avoid the problem completely is to use BST-based
dictionaries -- slower, but free from pathological edge cases.

Murmuring a short initial fragment could still be cool, just because
we'd probably get a better hash. 

That's an interesting conjecture, but I wouldn't judge it until I saw data on the hash quality and performance impact.

I do not expect that any of these changes would be in 1.6.0 - a workaround for any of these problems would be to create a deftype that wrapped a String, implemented IHashEq, and hashed with a VM-specific seed.

[Andy Fingerhut]

On Thu, Mar 20, 2014 at 10:16 AM, Alex Miller <al...@puredanger.com> wrote:

In Java 8, interestingly they removed all of that and just fixed the hash collection implementation to use balanced trees for collisions instead of lists. Seems like something similar could potentially be done in the HashCollisionNode impl in PHM.

 

Interesting.  I found the following link describing this with a bit of Google searching:

    http://openjdk.java.net/jeps/180

I also found the following sentence on the JDK 8 page for HashMap: http://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html

    "To ameliorate impact, when keys are Comparable, this class may use comparison order among keys to help break ties."

I guess the implementation checks whether two keys are the same type, and if that class implements Comparable?

If anyone has additional details on the implementation (e.g. a direct link to the source code implementing this), that would be much appreciated.

Thanks,
Andy

[Alex Miller]

Sorry, should have pre-emptively provided links in hand...

Useful note related to this here:

http://docs.oracle.com/javase/8/docs/technotes/guides/collections/changes8.html

java.util.HashMap source for JDK8 is here:

http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/util/HashMap.java

Tie-breaking here in the tree impl:

http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/util/HashMap.java#l1883

[Alex Miller]

The internal implementation notes are (as usual) phenomenal in the Java collection classes and discusses everything:

http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/util/HashMap.java#l143

Also see the comparableClassFor() method.

[Stefan Kamphausen]

Hi,


after two days of git bisecting and running my tests over and over again, I give up.  While I can repeatedly identify the commit which causes the biggest slowdown for me between the 1.5.1 tag and 1.6.0 RC1 I simply refuse to believe that the result of my analysis is correct.

On 5045ac124e24ed1fe02be86cccca10f7f212a4dc everything is fine, on 96f5b5bdcae44ccc69e33d701e6188dc2570dadf things are slow.  And no, I am not running into the exception.

I can only guess that there is some error on my side.  I do not think, the slowdown I reported based upon this program should be no reason to delay 1.6.0. (Even though I'd still like to understand what's going on.)


Best,
stefan

[Andy Fingerhut]

That is odd.  This is a shot in the dark, and probably unhelpful because I do not know a good way to verify whether my guess is true, but perhaps the seqFrom method went from being small enough to be inlined by your JIT before that change, to being too large to consider for inlining after the change?  It isn't a big change in the code, so it would have to have been close to the threshold if this is true.

Andy

[Stefan Kamphausen]

On Saturday, March 22, 2014 12:41:55 AM UTC+1, Andy Fingerhut wrote:

That is odd.  This is a shot in the dark, and probably unhelpful because I do not know a good way to verify whether my guess is true, but perhaps the seqFrom method went from being small enough to be inlined by your JIT before that change, to being too large to consider for inlining after the change?  It isn't a big change in the code, so it would have to have been close to the threshold if this is true.

if there is anything I can do to test this, just let me know.

Best,
stefan

[Alex Miller]

That's pretty weird. 

1.6.0-RC2 is out now - I would really appreciate it if you could give it a shot.

Alex

[Stefan Kamphausen]

Hi,

On Saturday, March 22, 2014 3:52:00 AM UTC+1, Alex Miller wrote:

That's pretty weird. 

that's pretty true.
 

1.6.0-RC2 is out now - I would really appreciate it if you could give it a shot.

Sure.  Tried with Oracle JDK 7 and 8, each run two times and took the average.  Each run was done using

lein clean && lein uberjar && run.sh

where run.sh starts the uberjar and measures the times with /usr/bin/time -v.

These are the results:

Version	Java Version	User time(s)	Sys time(s)	Wall clock(s)	% CPU
1.6.0-RC2	Oracle 1.7.0_11	2087,5	104,5	326,5	671
1.5.1	Oracle 1.7.0_11	1957	104,5	311,5	661
1.6.0-RC2	Oracle 1.8.0	2022,5	110	318,5	669,5
1.5.1	Oracle 1.8.0	2087	105	323	675,5

The results for Java7 are in-line with the results of my previous experiments, I tried Java8 just out of curiosity.

Cheers,
stefan

[Michał Marczyk]

You could experiment with different values of -XX:MaxInlineSize and
-XX:FreqInlineSize, see

http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html

You can use -XX:+PrintFlagsFinal to discover what values it's currently using.

Additionally differences in -XX:+PrintCompilation ouput might be interesting.

Cheers,
Michał

[Alex Miller]

Stefan, how do these numbers compare to RC1? Is RC2 better than RC1?

[Stefan Kamphausen]

On Sunday, March 23, 2014 6:02:26 PM UTC+1, Alex Miller wrote:

Stefan, how do these numbers compare to RC1? Is RC2 better than RC1?

If my tests can be trusted, there is an improvement between RC1 and RC2, but it is still worse than 1.5.1.

Version

Java Version

User time(s)

Sys time(s)

Wall clock(s)

% CPU

1.6.0-RC1

Oracle 1.7.0_11

2145

107,5

335

672

1.6.0-RC2

Oracle 1.7.0_11

2087,5

104,5

326,5

671

 
Stefan

[Stefan Kamphausen]

Hi,

Additionally differences in -XX:+PrintCompilation ouput might be interesting.

first, I tested with -XX:+PrintInlining and there actually is a difference for seqFrom: in 1.5.1 there are more occurrences of this function in the output (see data at the end of this post).  Since the function seems to get inlined, I did not experiment with FreqInlineSize.  Some background reading indicated that MaxInlineSize is not relevant for this function which clearly is a hot spot.

There is a diff with >3000 lines in the output of PrintCompilation.  Right now, I don't know what to look for in that output.  I'll have to read about how to read those files when time permits.


Cheers,
Stefan

Data:

grep seqFrom print-inline-1.5.1
                                   @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                              @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                    @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                            @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                              @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                      @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                      @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                      @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                      @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                      @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                      @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                              @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                              @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                              @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                            @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)
                                @ 28   clojure.lang.RT::seqFrom (133 bytes)   inline (hot)


grep seqFrom print-inline-1.6.0master
                                  @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                              @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                                    @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                                @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                                    @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                                @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                            @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                                @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                              @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                                  @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                                @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                            @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)
                                @ 28   clojure.lang.RT::seqFrom (152 bytes)   inline (hot)