Re: Reloading file doesn't reload the multimethod dispatch function

[Chas Emerick]

On Jun 10, 2012, at 12:15 AM, Phil Hagelberg wrote:

> On Sat, Jun 9, 2012 at 9:04 PM, Orion Hickman
> <irespondprompt...@gmail.com> wrote:
>> This one threw me, hard. I do all my development with a long running
>> process (as, I'm sure, most people do). Is there some way we can tweak
>> the implementation to avoid this problem?
>
> Unfortunately defonce semantics on defmulti is intentional.
>
> Usually what I do as a workaround is clear the defmulti right before
> defining it:
>
> (def mymulti nil)
> (defmulti mymulti type)
>
> Ugly but preferable to the alternative.
>
> I believe the motivation behind the current behaviour is to preserve
> the method table upon reload; the unchanging dispatch function may
> just be collateral damage. I haven't looked at the implementation of
> defmulti, but it may be possible to have reloads change the dispatch
> function without clearing the method table.

I quote from a contemporary thread in the main Clojure ML, but this (the fact that the dispatch fn — and to a lesser extent, default dispatch value — is not affected when reloading a defmulti form) has been a long-standing issue.

I'm finding myself using multimethods more and more, and would like to see this addressed. It's easily the most common thorn I encounter in my REPL-ing.

I don't see an issue in JIRA for this, but thought I'd open a thread to establish what the concrete objectives should be first. (Do correct me if I somehow missed the relevant JIRA ticket.)

As I see it, defmulti's defonce semantics are a net positive, insofar as preserving the method table is desirable (esp. when methods are defined across multiple namespaces, or are expected to be provided by "third-party" libraries). The real problem is that the object defined in the multimethod's var — an instance of MultiFn — combines a number of different values, including:

* the dispatch function
* the default dispatch value
* the hierarchy used by the multimethod
* the multimethod's name (also used to name the var)
* the method table (the map of dispatch value => method that is updated via defmethod)
* the prefers table (the map of dispatch value => [dominated dispatch values] that is updated via prefer-method)
* perf implementation details

I personally expect def* forms, when loaded, to effect only the changes that they describe; in defmulti's case, that includes:

* the dispatch function
* the default dispatch value
* the hierarchy used by the multimethod

(The other bits of data that live in MultiFn are managed elsewhere, via prefer-method, defmethod, and so on, and so I wouldn't expect that reloading a defmulti would affect those things.)

While defonce semantics ensures that the most common dynamic use case is supported (i.e. defining or redefining new methods), it's too blunt an instrument in this case, since the MultiFn houses so many different (potentially) moving pieces.

I see two options:

(a) Put the dispatch fn and value into respective IRefs. This will allow defmulti to update these values (along with the hierarchy, which is already an IRef) when it is reloaded. The downside here is that this will add one (sometimes two) derefs to every multimethod invocation (and multimethods are already not the fastest dog in the pack), even in the common case where redefinition of the fn and dispatch value simply isn't going to happen.

(b) There is nothing AFAICT in MultiFn or other parts of the multimethod apparatus that requires that MultiFn be implemented as a Java class. The precise shape of a reimplementation of MultiFn in Clojure is TBD (though I have some ideas); however it shakes out, it shouldn't have any difficulty avoiding many of the tradeoffs between dynamicity and performance that the Java impl necessarily forces. i.e. it would be straightforward to (for example) provide knobs to choose which component values should be dynamically modifiable, or not (for all multimethods, or select few). A Clojure implementation may also make possible certain optimizations and a greater degree of introspection (e.g. easy visibility into the method cache) that are simply impossible now.

In the case that option (b) is desirable (or option (a) is considered too much of a burden on perf to accept), it may be that incubating a Clojure multimethod implementation as a library would be the prudent thing to do.

Thoughts?

Cheers,

- Chas