why couldn't clojure 1.2 get to beta 1 tomorrow?

[Stuart Halloway]

Hi all,

We are preparing to release the first beta for Clojure 1.2. What are we forgetting that needs to be addressed before we do?

Stu

[Laurent PETIT]

Maybe I missed something, but it is not clear to me if all or some of
the num/equiv ... branches will be part of 1.2 ?

2010/6/30 Stuart Halloway <stuart....@gmail.com>:

> Hi all,
>
> We are preparing to release the first beta for Clojure 1.2. What are we forgetting that needs to be addressed before we do?
>
> Stu
>

> --
> You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
> To post to this group, send email to cloju...@googlegroups.com.
> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.
>
>

[Rich Hickey]

On Jun 29, 2010, at 7:02 PM, Laurent PETIT wrote:

> Maybe I missed something, but it is not clear to me if all or some of
> the num/equiv ... branches will be part of 1.2 ?
>

No they will not. They will move into master shortly after we go to
beta on 1.2, and will be part of the next release, after some more
shake-out.

Rich

[Dimitry Gashinsky]

On Jun 29, 2010, at 19:39 , Rich Hickey wrote:

>
> On Jun 29, 2010, at 7:02 PM, Laurent PETIT wrote:
>
>> Maybe I missed something, but it is not clear to me if all or some of
>> the num/equiv ... branches will be part of 1.2 ?
>>
>
> No they will not. They will move into master shortly after we go to beta on 1.2, and will be part of the next release, after some more shake-out.
>

What about prim branch? Is it hidden in the ...

_ __
| \o/__
|_/|\_|

[Kevin Downey]

it might be late to mention this, but last var wins, in any
incarnation is horrible and I'd rather it didn't get released in any
form.

most tests (including the default stubs lein generates) load the
namespace they are testing with :reload-all, this breaks if you load
the namespace form for the test namespace again (say by hitting C-c
C-k for slime). I have tests where at the top of the test file there
are things like (ns-unmap (create-ns 'clojure.core) 'spit) to get the
tests to run.

> --
> You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
> To post to this group, send email to cloju...@googlegroups.com.
> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.
>
>

--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?

[Stuart Halloway]

I believe that the bigger problem here is misuse of :reload-all. Reloading is an interactive feature, to be driven from the repl. If I want to reload a test (and the code under test) then I can pass :reload-all at the repl. The test itself should *not* use :reload-all. If this is done across a test suite, it can cause the tests to run an order of magnitude or more slower as the entire app is recompiled unnecessarily between every test namespace.

I have already fixed a variant of this problem in leiningen itself [1], and I think the generated test stub should be similarly fixed.

Would this solve your problem, or are there cases for :reload-all in source files that I have missed?

[1] http://github.com/technomancy/leiningen/commit/12b4bf10a32810b4478ceefe3af2a49f4e0a03ee

[Kevin Downey]

I am running tests interactively, with slime and when I rewrite
something, or re-def something and want to reload the whole file to
make sure I pick up any changes *boom* it breaks

[Stuart Halloway]

Ok, I am seeing this issue now. will investigate further and get back to you.

Thanks,
Stu

[Chas Emerick]

I (and others) would like to get this shoehorned into 1.2 if at all
possible:

http://www.assembla.com/spaces/clojure/tickets/322

This issue has been a constant build and deployment problem, and has
gotten worse for me of late due to some external tooling; getting it
into 1.2 would be fantastic.

The fix is on my fork (and an early version of it has been tested by
Phil [and perhaps Mark]). I'll produce an applyable patch tomorrow
morning and attach it to the ticket when I'm back in a proper
environment.

- Chas

[Phil Hagelberg]

On Tue, Jun 29, 2010 at 7:38 PM, Chas Emerick <ceme...@snowtide.com> wrote:
> I (and others) would like to get this shoehorned into 1.2 if at all
> possible:
>
> http://www.assembla.com/spaces/clojure/tickets/322

inc

I would love to see this applied.

#315 would also be a nice very-low-impact patch that would be helpful
to have. But if it would delay the 1.2 release then I can wait.

http://www.assembla.com/spaces/clojure/tickets/315
(Add support for running -main namespace from clojure.main without AOT)

-Phil

[David Powell]

On Wed 30/06/10 00:52 , Stuart Halloway stuart....@gmail.com sent:

> Hi all,
>
> We are preparing to release the first beta for Clojure 1.2. What are we
> forgetting that needs to be addressed before we do?
> Stu

A while ago, I posted to the group a list of clojure.java.shell issues with 6
patches attached. Creating tickets for separate issues wouldn't be
contraversial, but because they all affect the same few lines of code, it would
make patching a mess; so I think it makes sense to make a single patch, but I
didn't want to do that without some positive feedback because the changes are a
bit sweeping.

I didn't get much feedback, but if nobody disagrees I can combine them and make
a ticket with the combined patch. I'd like to see these go in because the
clojure.java.shell namespace is new, and the proposals include an API change
which would be best to get in before the release.

http://groups.google.com/group/clojure-
dev/browse_thread/thread/b4a4d6eca3fd9b9d/1ebd2916345b88d9

--
Dave

[Chas Emerick]

On Jun 29, 2010, at 11:17 PM, Phil Hagelberg wrote:

> On Tue, Jun 29, 2010 at 7:38 PM, Chas Emerick
> <ceme...@snowtide.com> wrote:
>> I (and others) would like to get this shoehorned into 1.2 if at all
>> possible:
>>
>> http://www.assembla.com/spaces/clojure/tickets/322
>
> inc
>
> I would love to see this applied.

Patches now attached (one that maintains current behaviour, another
that changes it [much for the better, IMO]), with an assessment of
user impacts if the default is (hopefully) changed.

- Chas

[Stuart Halloway]

Hi Dave,

Ticket with combined patch welcome.

Stu

[Chouser]

On Wed, Jun 30, 2010 at 9:17 AM, Stuart Halloway
<stuart....@gmail.com> wrote:
> Hi Dave,
>
> Ticket with combined patch welcome.

clojure.java.shell definitely needs some love. I just now looked
through your separate patches on the google group thread, and
they look excellent -- thanks for doing all that! It looks like
this will solve ticket 387 as well?

http://www.assembla.com/spaces/clojure/tickets/387

Perhaps it's already too late, but should the 'sh' function be
renamed? 'sh' sorta implies it's launching /bin/sh which it
specifically does not do (I know, the current name is all my
fault -- sorry). Perhaps 'run' or 'exec'?

--Chouser

[Stuart Halloway]

Chas,

I changed the subject line to make sure this gets a separate discussion. I personally prefer the proposed new behavior, i.e. *transitive-compile* defaults to false.

Stu

[Laurent PETIT]

I like this.

I have had trouble in the past where in different Eclipse plugins for
ccw, sometimes I ended up, after compilation, with classes from one
plugin into classes of one of the plugins it depends on ! I had to
take care of compiling plugins in the right order. But for the more
general case where some provided plugins may not be totally AOT
compiled, having the possibility to not transitively compile
everything is great, IMHO.

2010/6/30 Stuart Halloway <stuart....@gmail.com>:

[Howard Lewis Ship]

On Wed, Jun 30, 2010 at 6:55 AM, Stuart Halloway
<stuart....@gmail.com> wrote:
> Chas,
> I changed the subject line to make sure this gets a separate discussion. I
> personally prefer the proposed new behavior, i.e. *transitive-compile*
> defaults to false.

+1. I feel the current behavior, that transitively compiles past the
current project boundaries, is broken.

There's a third option, however. You can have transitive compilation,
but only if the Clojure source file being compiled as a file: URL
(i.e., its a local file on the file system, not a file stored in a
JAR). That would make it easier to use compilation on the local
project without transitively compiling imported libraries, such as
clojure-contrib.

> Stu
>
> From: Chas Emerick <ceme...@snowtide.com>
> Date: June 30, 2010 9:14:46 AM EDT
> To: cloju...@googlegroups.com
> Subject: Re: why couldn't clojure 1.2 get to beta 1 tomorrow?
> Reply-To: cloju...@googlegroups.com
>
>
> On Jun 29, 2010, at 11:17 PM, Phil Hagelberg wrote:
>
> On Tue, Jun 29, 2010 at 7:38 PM, Chas Emerick <ceme...@snowtide.com> wrote:
>
> I (and others) would like to get this shoehorned into 1.2 if at all
>
> possible:
>
> http://www.assembla.com/spaces/clojure/tickets/322
>
> inc
>
> I would love to see this applied.
>
> Patches now attached (one that maintains current behaviour, another that
> changes it [much for the better, IMO]), with an assessment of user impacts
> if the default is (hopefully) changed.
>
> - Chas
>

> --
> You received this message because you are subscribed to the Google Groups
> "Clojure Dev" group.
> To post to this group, send email to cloju...@googlegroups.com.
> To unsubscribe from this group, send email to
> clojure-dev...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/clojure-dev?hl=en.
>

--
Howard M. Lewis Ship

Creator of Apache Tapestry

The source for Tapestry training, mentoring and support. Contact me to
learn how I can get you up and productive in Tapestry fast!

(971) 678-5210
http://howardlewisship.com

[Brian Hurt]

On Wed, Jun 30, 2010 at 9:55 AM, Stuart Halloway <stuart....@gmail.com> wrote:

Chas,

I changed the subject line to make sure this gets a separate discussion. I personally prefer the proposed new behavior, i.e. *transitive-compile* defaults to false.

Stu

Is there a mechanism to actually determine the dependencies, so I don't have to maintain dependency information by hand?

Brian
 

[Laurent PETIT]

Oh, if the "can of worms" (just joking) is open again, then may I add
my own wish to the wishlist ? :-)

More seriously: in CCW (and I guess in other IDEs as well ?), we are
always AOT compiling all namespaces to be certain that the pieces for
java interop are in place: artefacts from genclass, geninterface,
definterface, defprotocol, deftype, defrecord. The problem is that the
directories are quickly (as quick as the user can type a function :-)
) full of different classes for different versions of any fn. I'm not
sure if there's a performance impact of having all this I/O made all
the time (anytime a user saves a file, the whole project is recompiled
in memory and on disk). But there sure is a whole mess of unnecessary
files !

Would it make sense to have the possibility of requiring that only
stuff needed for host interop (the above mentioned artifacts from
genclass, geninterface, definterface, defprotocol, deftype, defrecord)
be AOT compiled. Of course, without having the user wrap its
namespaces, calls to deftype et al. with special bindings ?

2010/6/30 Brian Hurt <bhu...@gmail.com>:

[Chas Emerick]

On Jun 30, 2010, at 12:48 PM, Howard Lewis Ship wrote:

> On Wed, Jun 30, 2010 at 6:55 AM, Stuart Halloway
> <stuart....@gmail.com> wrote:
>> Chas,
>> I changed the subject line to make sure this gets a separate
>> discussion. I
>> personally prefer the proposed new behavior, i.e. *transitive-
>> compile*
>> defaults to false.
>
> +1. I feel the current behavior, that transitively compiles past the
> current project boundaries, is broken.
>
> There's a third option, however. You can have transitive compilation,
> but only if the Clojure source file being compiled as a file: URL
> (i.e., its a local file on the file system, not a file stored in a
> JAR). That would make it easier to use compilation on the local
> project without transitively compiling imported libraries, such as
> clojure-contrib.

I thought about this some, and I don't think that's a good idea, at
least for now. I'm uncomfortable with semantics changing depending
upon where code is being loaded from -- which, depending upon a tool's
implementation, might be undefined. E.g. if the com.foo.bar ns is
available in source form in one directory, but as classes from a jar,
and classpaths aren't being constructed in a stable fashion, then the
results of compilation will change.

- Chas

[Chas Emerick]

That is definitely out of scope for this change, which is
fundamentally fairly simple.

That said, I have to think that such decisions should be entirely made
and implemented by tools, at least for now. I don't think anyone
knows what is best in this area yet.

- Chas

[Allen Rohner]

I found a bug today, clojure.set/rename-keys doesn't work with
defrecords. There's an attached (trivial) fix on the issue:
https://www.assembla.com/spaces/clojure/tickets/393-fix-rename-keys-to-work-with-defrecords

Allen

[Stuart Halloway]

Moving this onto a separate subject line so the discussion doesn't get lost.

Kevin's concerns below unpack into three separate questions/issues:

(1) What is the fate of last-var-wins? Disabled, except for one case: The clojure.core namespace is special, because it is referred in on your behalf. To enable promotion of names into clojure core without breaking existing code, your namespaces can replace clojure.core vars with a warning. (Nothing new here: This has been stable for some time on master.)

(2) Why is test reloading broken? That's a bug in the implementation of #1. I am surprised this hadn't caused more pain. Anyway, should be fixed now. If this bug bit you please check the fix http://github.com/clojure/clojure/commit/daed9dfb86a34c362a614f29c259a35d781d6b1e

(3) Where is :reload-all appropriate? At the REPL, *not* in your test files. Leiningen has this wrong. Nobody has responded to any of my comments on this subject and I want to make sure people agree and that I haven't missed anything.

Stu

[Kevin Downey]

so you are suggesting every time I edit a test file, I should add
:reload-all to the ns form while I am editing it, and remove it when I
am done and before I commit?

[Stuart Halloway]

Quite the opposite. I said :reload-all should be done only from the REPL.

The whole point of reload-all is that it is transitive. You don't have to add :reload-all to your test namespace. If you are at the REPL, pass :reload-all when you are reloading your test file. This will reload the test *and* any namespace it depends on.

Is this a toolchain issue? E.g. slime doesn't provide a way to reload-all, so you feel painted into a corner and have to put it into source code?

Stu

[Kevin Downey]

I suppose it could be a tool chain issue, but all slime does is load
the file, does clojure have some *reload-all* var that should be bound
so ns always does a :reload-all? are you suggesting that slime should
intersect namespace forms and slip in a :reload-all?

On Wed, Jun 30, 2010 at 7:21 PM, Stuart Halloway

[Kevin Downey]

if last var wins is kept, I think the correct behavior would be to
allow vars that have been overwritten to be overwritten multiple
times. but this is madness. much better to just throw an exception if
the name is already mapped to another var. last var wins needless
complicates the model with all the special casing.

[Stuart Halloway]

I am not a slime user, but my initial guess is that slime should provide two commands: one to load a namespace, and one to load it with reload-all.

Solving this problem by including :reload-all in test namespaces (as lein encourages) is deadly to perf. I have been on projects where the test suite got 15x faster by eliminating :remove-all. Imagine a C++ program that reruns the compiler between every test file and you will get an idea how bad this is.

Stu

[Laurent PETIT]

2010/7/1 Chas Emerick <ceme...@snowtide.com>:

> That is definitely out of scope for this change, which is fundamentally
> fairly simple.
>
> That said, I have to think that such decisions should be entirely made and
> implemented by tools, at least for now.  I don't think anyone knows what is
> best in this area yet.

I guessed so. Just wanted to make public this particular annoyance, so
that we can start thinking about it "in the background" of our heads.

[Phil Hagelberg]

On Wed, Jun 30, 2010 at 8:53 PM, Stuart Halloway
<stuart....@gmail.com> wrote:
> Solving this problem by including :reload-all in test namespaces (as lein encourages) is deadly to perf. I have been on projects where the test suite got 15x faster by eliminating :remove-all. Imagine a C++ program that reruns the compiler between every test file and you will get an idea how bad this is.

I certainly hope we can do better than C++. =)

In our project at work a full test run takes 68 seconds with
:reload-all and 65 with :reload.

In our case it doesn't seem worth having to worry about accidentally
missing changes on disk for a 4% speedup. Perhaps our project is an
outlier in that regard? I'd be interested in seeing more numbers for
others. I suppose it depends on how much I/O a particular project
needs.

-Phil

[Stuart Halloway]

It's not about accidentally missing changes on disk. It's about getting the right semantics. Having a :reload-all in source code to work around tooling issues is like having printlns in code to work around debugger issues. It's ok to do it locally, but you shouldn't commit it. (Would you add a call to reload! at the top of every test in a Rails app?)

Nor is it about I/O. The performance implications of :reload-all are based on the volume of code that gets reloaded. It is worse than linear as projects grow, and totally unnecessary.

This is important. Please change the leiningen default.

Stu

[Stuart Sierra]

On Jun 30, 9:51 pm, Stuart Halloway <stuart.hallo...@gmail.com> wrote:
> (3) Where is :reload-all appropriate? At the REPL, *not* in your test files. Leiningen has this wrong. Nobody has responded to any of my comments on this subject and I want to make sure people agree and that I haven't missed anything.

:reload-all should not even be valid inside 'ns'. The fact that it
does is probably a side-effect of implementation.

-S

[Phil Hagelberg]

On Thu, Jul 1, 2010 at 9:48 AM, Stuart Halloway
<stuart....@gmail.com> wrote:
> It's not about accidentally missing changes on disk. It's about getting the right semantics. Having a :reload-all in source code to work around tooling issues is like having printlns in code to work around debugger issues. It's ok to do it locally, but you shouldn't commit it. (Would you add a call to reload! at the top of every test in a Rails app?)

Ruby has a culture of always launching a fresh interpreter for every
test run. Clojure has a culture of keeping long-running instances
open. This is clearly a net win, but it makes it easy to "get slimed",
or have the current instance in memory fail to reflect the state of
the project on disk.

I will look at how much work it would be to add :reload-all semantics
to swank-clojure. If it's relatively simple then I will remove it from
the Leiningen project skeleton output, but that pushes the
responsibility out to other IDEs to handle it like swank.

-Phil

[Kevin Downey]

is there something var, etc, that slime should be setting so that
clojure knows we want to reload-all? you keep saying this is a tooling
issue, but does clojure have a hook for tools to specify what they
want? currently the only way I know of to specify this behavior is by
putting :reload-all into the ns form. you say this is bad, and that
slime should just have a reload-all mechanism, but how would you
suggest implementing it? what mechanisms in clojure should slime be
using?

On Thu, Jul 1, 2010 at 9:48 AM, Stuart Halloway
<stuart....@gmail.com> wrote:

--

[Phil Hagelberg]

On Thu, Jul 1, 2010 at 10:54 AM, Kevin Downey <red...@gmail.com> wrote:
> is there something var, etc, that slime should be setting so that
> clojure knows we want to reload-all? you keep saying this is a tooling
> issue, but does clojure have a hook for tools to specify what they
> want? currently the only way I know of to specify this behavior is by
> putting :reload-all into the ns form. you say this is bad, and that
> slime should just have a reload-all mechanism, but how would you
> suggest implementing it? what mechanisms in clojure should slime be
> using?

It looks like I could make swank-clojure use the require function
(which takes :reload-all in its flags) instead of load-file; the only
problem is I only have access to the filename at that point rather
than the ns name. It would be helpful if there were a pair of
functions exposed by Clojure that could translate between the two
(assuming a classpath-relative filename of course) since I've had to
reimplement this myself a couple times already.

Anyway, I think this is doable. Slime already has a distinction
between compiling a file and "loading" it (vestigal from its CL
roots); I could hijack that distinction and have the "load" version
imply reload-all.

-Phil

[Stuart Halloway]

> is there something var, etc, that slime should be setting so that
> clojure knows we want to reload-all?

Yes, slime should call use or require, passing :reload-all.

> currently the only way I know of to specify this behavior is by
> putting :reload-all into the ns form.

; example correct usage, not in an ns form
; this should be easy for tools to call
(require 'some-ns :reload-all)

I am really glad you raised this issue! I don't use slime, so I am ignorant of the pain you are feeling. But now that I know about it I want to make it go away.

Stu

[Meikel Brandmeyer]

Hi,

Am 01.07.2010 um 19:42 schrieb Phil Hagelberg:

> I will look at how much work it would be to add :reload-all semantics
> to swank-clojure. If it's relatively simple then I will remove it from
> the Leiningen project skeleton output, but that pushes the
> responsibility out to other IDEs to handle it like swank.

For the casefile: This should be a trivial change. VimClojure allows that since almost two years now. You can require the namespace of the current buffer with the :reload-all flag on a keystroke.

Sincerely
Meikel

[Kevin Downey]

regardless of how the reload-all is done, last-var-wins still does not
play nice with it

> --
> You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
> To post to this group, send email to cloju...@googlegroups.com.
> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.
>
>

--

[Stuart Halloway]

I would appreciate an example. In my testing last-var-wins works just fine as shown below.

; define a namespace example.bar
(ns example.bar)
(defn flatten [])

; require it multiple times:
user=> (require :reload-all 'example.bar)
WARNING: flatten already refers to: #'clojure.core/flatten in namespace: example.bar, being replaced by: #'example.bar/flatten
nil

user=> (require :reload-all 'example.bar)
WARNING: flatten already refers to: #'clojure.core/flatten in namespace: example.bar, being replaced by: #'example.bar/flatten
nil

Note this fix is on the development edge of master at github.com/clojure/clojure.

[Chas Emerick]

I wanted to bump this thread. Can we get a yay or nay w.r.t. #322
being in 1.2?

Insofar as the currently-available patch is roughly the only
implementation route I can see, applying it to master/HEAD would be
fine by me at this point. Anyone that does any kind of AOT
compilation can easily test the current proposed (and preferred, it
seems) change by pulling from here:

http://github.com/cemerick/clojure

That's the master branch (slightly behind on minor bugfixes, it looks
like), with the fix for #322 applied that changes the default
compilation behaviour to [*transitive-compile* false].

- Chas

[Stuart Halloway]

We are nay on all feature changes at this point barring exceptional circumstances. Have to stop somewhere. :-(

Stu

[Howard Lewis Ship]

I think this is pretty exceptional problem; I think you'll see a lot
of very confused error messages as new and old code (compiled
inadventently against 1.1 and 1.2 and thrown together unintentionally
via third-party libraries). I'm already hitting that now using lein
push from Clojars with 1.2.

--

[Chas Emerick]

Well, code AOT'd in 1.1 is incompatible with 1.2 in general due to
some Java-side signature changes.

But, I would argue:

(a) The proposed change is a fix, not a new feature -- defaulting AOT
compilation to be applied transitively was an oversight, but there was
almost surely no way to know the extent of the issue until people had
a chance to build large(r) systems from various AOT'd artifacts and
realize the problems caused downstream.

(b) As things stand, transitivity guarantees showstopper issues in
some circumstances. E.g. the processes that build NetBeans modules
and (mandatorily) analyze the classfiles contained therein (and their
declared module dependencies) simply fail when they detect that
classes will be loaded from modules that don't declare those classes
as being "exported" (but are nonetheless present because of the
transitivity of AOT compilation). The same issue appears to affect
OSGI bundling. The workarounds are unpleasant -- either ship source
(a no-go in some situations) or modify builds to scrub "improperly"-
included classfiles from the modules-to-be-built. This is the
situation that finally motivated me.

(c) For most (honestly, functionally all) users of AOT compilation,
this can and will be perceived as a change to their build tools –
almost assuredly either lein or clojure-maven-plugin – and not a
change in Clojure itself. Further, those build tools' authors can
determine what the right default behaviour should be for their users
and communities, which may be to ensure backwards-compatible behaviour
for some period of time (although AFAIK, both Phil and Mark have
indicated support for #322 and changing the transitivity default).

If those points are unpersuasive, then I would suggest a lesser course
of action: apply the version of the patch that leaves transitive
compilation as the default, but exposes the option to disable it.
AFAICT, this would have zero impact upon anyone in the (likely rare)
circumstance of not using lein or clojure-maven-plugin *and* depending
(probably unwittingly) upon transitive AOT compilation, but allowing
tools authors and others to readily/easily work around the issues
transitivity causes. Changing the transitivity default can then be
pushed back as a change in 1.2+.

- Chas

> --
> Howard M. Lewis Ship
>
> Creator of Apache Tapestry
>
> The source for Tapestry training, mentoring and support. Contact me to
> learn how I can get you up and productive in Tapestry fast!
>
> (971) 678-5210
> http://howardlewisship.com
>

[Phil Hagelberg]

On Tue, Jul 6, 2010 at 6:42 PM, Chas Emerick <ceme...@snowtide.com> wrote:
> If those points are unpersuasive, then I would suggest a lesser course of
> action: apply the version of the patch that leaves transitive compilation as
> the default, but exposes the option to disable it.  AFAICT, this would have
> zero impact upon anyone in the (likely rare) circumstance of not using lein
> or clojure-maven-plugin *and* depending (probably unwittingly) upon
> transitive AOT compilation, but allowing tools authors and others to
> readily/easily work around the issues transitivity causes.  Changing the
> transitivity default can then be pushed back as a change in 1.2+.

Strongly agree. This maintains the old behaviour for people who don't
care, but it lets tools get it right. Sounds like a win-win to me.

-Phil

[Laurent PETIT]

2010/7/7 Phil Hagelberg <ph...@hagelb.org>:

+1

[Stuart Halloway]

Persuasive. Will discuss with Rich this afternoon and get an answer today.

Stu

> Well, code AOT'd in 1.1 is incompatible with 1.2 in general due to some Java-side signature changes.
>
> But, I would argue:
>
> (a) The proposed change is a fix, not a new feature -- defaulting AOT compilation to be applied transitively was an oversight, but there was almost surely no way to know the extent of the issue until people had a chance to build large(r) systems from various AOT'd artifacts and realize the problems caused downstream.
>

> (b) As things stand, transitivity guarantees showstopper issues in some circumstances. E.g. the processes that build NetBeans modules and (mandatorily) analyze the classfiles contained therein (and their declared module dependencies) simply fail when they detect that classes will be loaded from modules that don't declare those classes as being "exported" (but are nonetheless present because of the transitivity of AOT compilation). The same issue appears to affect OSGI bundling. The workarounds are unpleasant -- either ship source (a no-go in some situations) or modify builds to scrub "improperly"-included classfiles from the modules-to-be-built. This is the situation that finally motivated me.

>> --
>> Howard M. Lewis Ship
>>
>> Creator of Apache Tapestry
>>
>> The source for Tapestry training, mentoring and support. Contact me to
>> learn how I can get you up and productive in Tapestry fast!
>>
>> (971) 678-5210
>> http://howardlewisship.com
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
>> To post to this group, send email to cloju...@googlegroups.com.
>> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.

[Laurent PETIT]

In fact, I'm not sure if I made this point sufficiently clear, but solving this issue will finally correctly enable correct management of compilation in IDEs for people with projects made of multiple subprojects.

Indeed currently, even for the own needs of Counterclockwise, I some times ended up with ccw.core needing to AOT compile those classes that the Eclipse framework must see. If I do so but any of the libraries ccw.core depends on was not itself AOT compiled, then the classes/ directory of ccw.core ended up with all the contents (in .class files formats) of its not-already-AOT-compiled dependencies !

It is even more subtle for the intra-project dependencies, 'cause if you did make changes to deps in the wrong order, you could have ccw.debug with the old classes from ccw.core, while ccw.core had new classes of it in its own classes/ folder ...

To try make this issue not as problematic as it is, I have made ccw AOT compile *everything* in each project. This helps with the intra-project dependencies issue.

When the patch is applied (which parts of this thread will finally make it into 1.2 ?), I can stop AOT compiling all my user's projects (at least for users switching to 1.2), and start doing the "right" approach anybody does: just AOT compile the strict minimum.

Regards,

--
Laurent

2010/7/7 Stuart Halloway <stuart....@gmail.com>

[Chas Emerick]

This is a fairly belated followup on the last-var-wins semantics, but
I thought I'd point out this sort of interaction:

1:159 bar=> (ns a)
nil
1:160 a=> (defn get [])
WARNING: get already refers to: #'clojure.core/get in namespace: a,
being replaced by: #'a/get
#'a/get
1:161 a=> (ns b)
nil
1:162 b=> (refer 'a)
WARNING: get already refers to: #'clojure.core/get in namespace: b,
being replaced by: #'a/get
nil
1:163 b=> (ns b)
1:164 b=> java.lang.IllegalStateException: get already refers to: #'a/
get in namespace: b

That error is very unfortunate IMO, and occurs upon every file reload
in a REPL where a referred ns defines vars that clash with any in
clojure.core. Looking at Namespace.warnOrFailOnReplace, it doesn't
appear that this can be addressed in any clean way given that the
relative order of a refer-clojure invocation via ns is not fixed (i.e.
if present, it is invoked in order that it is found, relative to
other :use and :require references). If it can be decided that :refer-
clojure invocations prompted by ns are always performed first, then it
seems that warnOrFailOnReplace could be changed to throw only if
neither the old or new var is from clojure.core.

Thoughts?

- Chas



On Jun 29, 10:00 pm, Stuart Halloway <stuart.hallo...@gmail.com>
wrote:
> Ok, I am seeing this issue now. will investigate further and get back to you.
>
> Thanks,
> Stu
>
> > I am running tests interactively, with slime and when I rewrite
> > something, or re-def something and want to reload the whole file to
> > make sure I pick up any changes *boom* it breaks
>
> > On Tue, Jun 29, 2010 at 6:31 PM, Stuart Halloway
> > <stuart.hallo...@gmail.com> wrote:
> >> I believe that the bigger problem here is misuse of :reload-all. Reloading is an interactive feature, to be driven from the repl. If I want to reload a test (and the code under test) then I can pass :reload-all at the repl. The test itself should *not* use :reload-all. If this is done across a test suite, it can cause the tests to run an order of magnitude or more slower as the entire app is recompiled unnecessarily between every test namespace.
>
> >> I have already fixed a variant of this problem in leiningen itself [1], and I think the generated test stub should be similarly fixed.
>
> >> Would this solve your problem, or are there cases for :reload-all in source files that I have missed?
>
> >> [1]http://github.com/technomancy/leiningen/commit/12b4bf10a32810b4478cee...
>
> >>> it might be late to mention this, butlastvarwins, in any

> >>> incarnation is horrible and I'd rather it didn't get released in any
> >>> form.
>
> >>> most tests (including the default stubs lein generates) load the
> >>> namespace they are testing with :reload-all, this breaks if you load
> >>> the namespace form for the test namespace again (say by hitting C-c
> >>> C-k for slime). I have tests where at the top of the test file there
> >>> are things like (ns-unmap (create-ns 'clojure.core) 'spit) to get the
> >>> tests to run.
>

> >>> On Tue, Jun 29, 2010 at 5:42 PM, Dimitry Gashinsky
> >>> <i+cloj...@gashinsky.com> wrote:
>
> >>>> On Jun 29, 2010, at 19:39 , Rich Hickey wrote:
>
> >>>>> On Jun 29, 2010, at 7:02 PM, Laurent PETIT wrote:
>
> >>>>>> Maybe I missed something, but it is not clear to me if all or some of
> >>>>>> the num/equiv ... branches will be part of 1.2 ?
>
> >>>>> No they will not. They will move into master shortly after we go to beta on 1.2, and will be part of the next release, after some more shake-out.
>
> >>>> What about prim branch? Is it hidden in the ...
>
> >>>>  _   __
> >>>> | \o/__
> >>>> |_/|\_|

>
> >>>> --
> >>>> You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
> >>>> To post to this group, send email to cloju...@googlegroups.com.
> >>>> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.

> >>>> For more options, visit this group athttp://groups.google.com/group/clojure-dev?hl=en.

>
> >>> --
> >>> And what is good, Phaedrus,
> >>> And what is not good—
> >>> Need we ask anyone to tell us these things?
>

> >>> --
> >>> You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
> >>> To post to this group, send email to cloju...@googlegroups.com.
> >>> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.

> >>> For more options, visit this group athttp://groups.google.com/group/clojure-dev?hl=en.

>
> >> --
> >> You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
> >> To post to this group, send email to cloju...@googlegroups.com.
> >> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.

> >> For more options, visit this group athttp://groups.google.com/group/clojure-dev?hl=en.

>
> > --
> > And what is good, Phaedrus,
> > And what is not good—
> > Need we ask anyone to tell us these things?
>