On Nov 13, 2:13 am, Mark Engelberg <mark.engelb...@gmail.com
> I'm still trying to get my head around the new features. Seeing more
> code examples will definitely help. In the meantime, here is some
> stream-of-consciousness thoughts and questions.
> I'm a little worried about the strong overlap between reify/proxy,
> deftype/defstruct, and defclass/gen-class. I can just imagine the
> questions a year from now when people join the Clojure community and
> want to understand how they differ. So I think that eventually, there
> needs to be a very clear "story" as to why you'd choose one over the
> other. Or better yet, maybe some of the older constructs can be
> phased out completely.
Yes, but there will be a transition period. I certainly tried to
explain the decision points on the wiki.
A big part of the design thinking behind these features went like
Clojure is built on a set of abstractions, and leverages/requires that
the host platform provide some sort of high-performance polymorphism
construct in order to make that viable. That said, Clojure was
bootstrapped on the host language and didn't really provide similar
constructs itself (multimethods are more powerful but slower), leaving
people that wanted to do things similar to what I did, in order to
write Clojure and its data structures, to either write Java or use
Clojure interop to, effectively, write Java in Clojure clothing.
So I took a step back and said, what part of Java did I *need* in
order to implement Clojure and its data structures, what could I do
without, and what semantics was I willing to support - for Clojure -
i.e. not in terms of interop. What I ended up with was - a high-
performance way to define and implement interfaces. What I explicitly
left out was - concrete derivation and implementation inheritance.
reify is Clojure semantics and proxy is Java/host semantics. Why
doesn't it replace proxy? Because proxy can derive from concrete
classes with constructors that take arguments. Supporting that
actually brings in a ton of semantics from Java, things I don't want
in Clojure's semantics. reify should be possible and portable in any
port of Clojure, proxy may not. Will the performance improvements of
reify make it into proxy? Probably at some point, not a priority now.
*** Prefer reify to proxy unless some interop API forces you to use
proxy. You shouldn't be creating things in Clojure that would require
you to use proxy. ***
defstruct is likely to be completely replaced by deftype, and at some
point could be deprecated/removed.
*** Prefer deftype to defstruct, unconditionally. ***
AOT deftype vs gen-class touches on the same Clojure semantics vs Java/
host semantics, with the objectives from before - support implementing
interfaces but not concrete derivation. So, no concrete base classes,
no super calls, self-ctor calls, statics, methods not implementing
interface methods etc. Will the performance improvements of deftype
make it into gen-class? Probably at some point, not a priority now.
Like proxy, gen-class will remain as an interop feature.
*** Prefer deftype to gen-class unless some interop API forces you to
use gen-class. ***
There will be a definterface similar to and probably replacing gen-
interface, with an API to match deftype.
So, with definterface, deftype, and reify you have a very clean way to
specify and implement a subset of the Java/C# polymorphism model, that
subset which I find clean and reasonable, with an expectation of
portability, and performance exactly equivalent to the same features
on the host.
I could have stopped there, and almost did. But there are three
aspects of that polymorphism model that aren't sufficient for Clojure:
- It is insufficiently dynamic. There is a static component - named
interfaces, that must be AOT compiled.
- Client code must use the interop style (.method x), and type hints,
in order to tap into the performance
- It is 'closed' polymorphism, i.e. the set of things a type can do
is fixed at the definition time of the type. This results in the
'expression problem', in this case the inability to extend types with
We've all experienced the expression problem - sometimes you simply
can't request/require that some type implement YourInterface in order
to play nicely with your design. You can see this in Clojure's
implementation as well - RT.count/seq/get etc all try to use Clojure's
abstraction interface first, but then have hand-written clauses for
types (e.g. String) that couldn't be retrofitted with the interface.
Multimethods, OTOH, don't suffer from this problem. But it is
difficult to get something as generic as Clojure's multimethods to
compete with interface dispatch in Java. Also, multimethods are kind
of atomic, often you need a set of them to completely specify an
abstraction. Finally, multimethods are a good story for the Clojure
side of an abstraction, but should you define a valuable abstraction
and useful code in Clojure and want to enable extension or
interoperation from Java or other JVM langs, what's the recipe?
Protocols take a subset of multimethod power, open extension, combine
it with a fixed, but extremely common, dispatch mechanism (single
dispatch on 'type' of first arg), allow a set of functions
constituting an abstraction to be named, specified, and implemented as
group, and provide a clear way to extend the protocol using ordinary
capabilities of the host (:on interface).
*** Prefer using protocols to specify your abstractions, vs
This will give you open extension and a dynamic system. You can always
make your protocol reach any type, and, you can always make your
protocol extensible through an interface using :on interface. In
particular note, calls to a protocol fn to an instance of the :on
interface go straight through, and are as fast as calls using (.method
#^AnInterface x), so there is no up-front performance compromise in
> While these datatype and protocol constructs are taking shape, maybe
> now is the time to discuss what kind of "privacy" settings are
> worthwhile in a language like Clojure. I think Java's system of
> private/public/protected is probably overkill for Clojure. But do
> people feel that some degree of data hiding is worthwhile?
> I don't understand whether there's any way to provide a partial
> implementation or default implementation of a given
> protocol/interface, and I believe this to be an important issue.
> For example, a protocol for < and > that provides a default
> implementation of > in terms of < and a default implementation of < in
> terms of >, so that you only need to implement one and you get the
> other for free.
> I'm also thinking about the relationship in Clojure's source between
> ISeq and ASeq. ASeq provides the partial, default implementation of
> more in terms of next, for example. How does this kind of thing look
> with the new protocol system?
This was an important consideration in the deftype/protocol design.
One reasonable argument for concrete implementation is abstract
superclasses, especially when used correctly. And Clojure's
implementation does use them, as you note. Some of the problems with
abstract classes are:
- they create a hierarchical type relationship (for no good reason).
- unless you are going to open that huge can of worms that is
multiple concrete inheritance, you only get a single inheritable
- they, too, are closed. If you are going to allow open extension,
but implementation reuse requires derivation, there is an open/closed
Protocols are designed to support hierarchy-free, open, multiple,
mechanical mixins. This is enabled by the fact that extend is an
ordinary function, and the mappings of names to implementation
functions are ordinary maps. One can create mixins by simply making
maps of names to functions. And one can use mixins in an ad hoc
manner, merging and replacing functions using ordinary map
(extend ::MyType AProtocol (assoc a-mixin-map :a-fn-to-replace a-
I think people will find this quite powerful and programmable.