[ANN] Clojure 1.10.0-alpha5
Alex Miller
Sean Corfield
Anyone using taoensso/nippy will encounter a compilation exception on Clojure 1.10.0 Alpha 5. See https://dev.clojure.org/jira/browse/CLJ-2367 for details (an unexpected – and probably unintended – change in the new ASM GeneratorAdapter breaks casts from short or byte to int).
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
Sent: Wednesday, June 27, 2018 12:52:20 PM
To: Clojure
Subject: [ANN] Clojure 1.10.0-alpha5
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Sean Corfield
I happened to notice this morning that the patch for CLJ-2367 was merged and Clojure 1.10.0-alpha6 has quietly gone up to Maven Central so we’re running a full pass of our tests at work on the new ASM/Java 8+ stuff.
The 1.10.0-master-SNAPSHOT is currently lagging behind (June 27) and does not yet include this fix.
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
Sent: Thursday, June 28, 2018 4:26:36 PM
To: clo...@googlegroups.com
Subject: RE: [ANN] Clojure 1.10.0-alpha5
Alex Miller
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/ESNrjIr0JUE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
Alex Miller
Sean Corfield
Just FYI, we ran into all sorts of strange errors (class not found exceptions from Boot pods etc) trying Alpha 6 at work but I was about to head out of the country for two weeks vacation so I didn’t have time to dig into that. I’m back now and will try again shortly and see if I can repro outside of our build pipeline.
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
Sent: Wednesday, July 4, 2018 10:23:00 AM
Alex Miller
Sean Corfield
Yes, which allowed us to actually _try_ to run our build pipeline – so the problems we’re seeing are fallout from the big changes in Alpha 5… I just haven’t nailed them down yet 😊 Everything works fine on Alpha 4.
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
To: Clojure
Subject: RE: [ANN] Clojure 1.10.0-alpha5
Alex Miller
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/ESNrjIr0JUE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
Sean Corfield
Progress so far… with Alpha 6.
I’ve encountered a number of (random) JVM SEGV fatal errors running our Boot build pipeline and if I don’t hit any of those, I hit an exception like this fairly reliably:
https://gist.github.com/seancorfield/f29bdb948a2a533c14a07ff6ffd6548a
(the missing class seems to vary from run to run but the exception is always in the same place)
It _seems_ to be an interaction between something new in Alpha 5 and Boot’s pod machinery since all of the failures I’m encountering seem to have when Boot is attempting to refresh pods in its pool of pods. We rely heavily on pods to isolate various parts of our build pipeline (since we load in different sets of dependencies for different sets of tests).
Ghadi suggested I update my local JDK to a more recent version and try again so that’s next on my list.
Narrowing this down is going to be hard: if I run the build pipeline in separate “chunks” – which means less interaction between pods – each chunk always passes.
So, overall, no failures from our test suite itself for any of our application components (good). Just random failures within the build tool itself ☹
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
Sent: Thursday, July 19, 2018 1:08:33 PM
To: clo...@googlegroups.com
Sean Corfield
Things tried so far:
- Update JDK to 1.8.0_144 (to match one of our other environments). No effect on the problem (well, I haven’t seem any JVM crashes since!).
- Update Boot to 2.8.1. No effect. Ghadi wondered if Boot’s class loading might have changed since 2.7.2 and that might be a root cause.
- Change our multi-version testing so we’re only loading Clojure 1.10.0-alpha6 (we were trying both alpha 4 and alpha 6 before). No effect. I wondered if loading Clojure based on two different versions of ASM (even isolated via pods) might cause issues.
We can reliably run the Boot task on its own – clj-webdriver loads and all our tests pass as expected. All of our tasks combined up until this point run just fine. All tasks individually run just fine. The only time we see a failure is when we combine the WebDriver-based test task with all the other tasks – and it reliably fails to load some class (usually a clj-webdriver class but on one run it failed to load clojure.tools.logging.impl.Logger) originating from Boot’s pod refresh, preparing to load clj-webdriver, as far as I can tell.
I haven’t been able to produce a smaller combination of tasks that exhibits the problem. I will observe that I have _occasionally_ seen CNFEs during pod refresh in the past, when Components are being stopped (and unloaded?) asynchronously, so it may be that this is an intermittent/non-deterministic bug in Boot pods that is exacerbated with Clojure 1.10.0-alpha5 and later?
For now we’re staying on Clojure 1.9.0 (and running just our unit tests against both that and master-SNAPSHOT). As far as we can tell, all our apps run fine on Clojure 1.10.0-alpha6 so this seems to just impact our Boot-based build toolchain and we can probably work around that, so we’ll probably switch to 1.10 at some point (before release) so we can test it in production and provide feedback.
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
Sent: Thursday, July 19, 2018 4:55:47 PM
Sean Corfield
Having posted about our success with testing Alpha 7, I need to ‘fess up about the end result of all this palaver from July:
With a bit of instrumentation in our build pipeline it turned out that we were thrashing on heap space in a few places and, for whatever reason, that was causing all the ClassNotFoundException problems. Sigh. I bumped the heap size up in the JVM options for Boot and everything ran flawlessly…
…the hint that pushed me in this direction was that, even on Clojure 1.9.0, we _occasionally_ saw problems loading instant18 (which I’d assumed was a weird edge case in Boot’s async pod setup/refresh)… and that started cropping up more often, and then we started seeing the same CFNE problems with 1.9.0 and that WebDriver test when run in a long Boot pipeline…
…so clearly the issue WASN’T the new Alpha build: that just happened to push us nearer the heap/GC issue.
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
From: Sean Corfield <se...@corfield.org>
Sent: Saturday, July 21, 2018 8:22:21 PM
To: clo...@googlegroups.com
Subject: RE: [ANN] Clojure 1.10.0-alpha5
Things tried so far:
- Update JDK to 1.8.0_144 (to match one of our other environments). No effect on the problem (well, I haven’t seem any JVM crashes since!).
- Update Boot to 2.8.1. No effect. Ghadi wondered if Boot’s class loading might have changed since 2.7.2 and that might be a root cause.
- Change our multi-version testing so we’re only loading Clojure 1.10.0-alpha6 (we were trying both alpha 4 and alpha 6 before). No effect. I wondered if loading Clojure based on two different versions of ASM (even isolated via pods) might cause issues.
We can reliably run the Boot task on its own – clj-webdriver loads and all our tests pass as expected. All of our tasks combined up until this point run just fine. All tasks individually run just fine. The only time we see a failure is when we combine the WebDriver-based test task with all the other tasks – and it reliably fails to load some class (usually a clj-webdriver class but on one run it failed to load clojure.tools.logging.impl.Logger) originating from Boot’s pod refresh, preparing to load clj-webdriver, as far as I can tell.
I haven’t been able to produce a smaller combination of tasks that exhibits the problem. I will observe that I have _occasionally_ seen CNFEs during pod refresh in the past, when Components are being stopped (and unloaded?) asynchronously, so it may be that this is an intermittent/non-deterministic bug in Boot pods that is exacerbated with Clojure 1.10.0-alpha5 and later?
For now we’re staying on Clojure 1.9.0 (and running just our unit tests against both that and master-SNAPSHOT). As far as we can tell, all our apps run fine on Clojure 1.10.0-alpha6 so this seems to just impact our Boot-based build toolchain and we can probably work around that, so we’ll probably switch to 1.10 at some point (before release) so we can test it in production and provide feedback.
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
From: Sean Corfield <se...@corfield.org>
Sent: Thursday, July 19, 2018 4:55:47 PM
To: clo...@googlegroups.com
Subject: RE: [ANN] Clojure 1.10.0-alpha5
Progress so far… with Alpha 6.
I’ve encountered a number of (random) JVM SEGV fatal errors running our Boot build pipeline and if I don’t hit any of those, I hit an exception like this fairly reliably:
https://gist.github.com/seancorfield/f29bdb948a2a533c14a07ff6ffd6548a
(the missing class seems to vary from run to run but the exception is always in the same place)
It _seems_ to be an interaction between something new in Alpha 5 and Boot’s pod machinery since all of the failures I’m encountering seem to have when Boot is attempting to refresh pods in its pool of pods. We rely heavily on pods to isolate various parts of our build pipeline (since we load in different sets of dependencies for different sets of tests).
Ghadi suggested I update my local JDK to a more recent version and try again so that’s next on my list.
Narrowing this down is going to be hard: if I run the build pipeline in separate “chunks” – which means less interaction between pods – each chunk always passes.
So, overall, no failures from our test suite itself for any of our application components (good). Just random failures within the build tool itself ☹
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
From: Sean Corfield <se...@corfield.org>
Sent: Thursday, July 19, 2018 1:08:33 PM
To: clo...@googlegroups.com
Subject: RE: [ANN] Clojure 1.10.0-alpha5
Yes, which allowed us to actually _try_ to run our build pipeline – so the problems we’re seeing are fallout from the big changes in Alpha 5… I just haven’t nailed them down yet 😊 Everything works fine on Alpha 4.
Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood
From: clo...@googlegroups.com <clo...@googlegroups.com> on behalf of Alex Miller <al...@puredanger.com>
Sent: Wednesday, July 18, 2018 10:50:59 AM
To: Clojure
Subject: RE: [ANN] Clojure 1.10.0-alpha5