Any ways to prevent protocol functions from being hardcoded in?

[Brian Marick]

I'm looking to do something like this:

(defprotocol Addable
(add-fields [this]))

(defrecord MyRecord [a b]
Addable
(add-fields [this] (+ a b)))

;;; Magic happens here

(defn indirect-adder [a b]
(add-fields (MyRecord. a b)))

(with-definition-of-add-fields-changed-to (fn [_] "hi mom")
(indirect-adder 1 2)
=> "hi mom" ; rather than 3

I expect there are no tricks like :dynamic true <http://blog.n01se.net/?p=134> that work, but I thought I'd check.

-----
Brian Marick, Artisanal Labrador
Contract programming in Ruby and Clojure
Occasional consulting on Agile
www.exampler.com, www.twitter.com/marick

[Stuart Sierra]

The protocol method `add-fields` becomes an ordinary Clojure Var.  You can temporarily change its root binding using `with-redefs` in 1.3.

-S

[Brian Marick]

On Jun 28, 2011, at 4:17 PM, Stuart Sierra wrote:

> The protocol method `add-fields` becomes an ordinary Clojure Var. You can temporarily change its root binding using `with-redefs` in 1.3.

`with-redefs` has no effect on an already-compiled function that uses a defrecord-defined function. I would need something that says "O compiler, being able to redef functions is actually more important to me right now than efficiency, so please emit code that notices redefs in, for example:

(defn indirect-adder [a b]
(add-fields (MyRecord. a b)))"

I have a kinda-kludgy workaround that does essentially that. It suffices.

[Stuart Sierra]

Yeah, inline method definitions in a deftype/defrecord are compiled right into the generated class; no Vars involved. That's why they're fast.

Recently the received wisdom has been: protocols are a low-level implementation detail. Actual APIs should be built with normal functions that call the protocol methods.

-Stuart Sierra
clojure.com

[Brian Marick]

On Jun 30, 2011, at 7:54 AM, Stuart Sierra wrote:

> Recently the received wisdom has been: protocols are a low-level implementation detail. Actual APIs should be built with normal functions that call the protocol methods.

Interesting. So I'm not encouraged to think "protocols" except when I care about efficiency or Java interop?

[Laurent PETIT]

2011/6/30 Brian Marick <mar...@exampler.com>:

>
> On Jun 30, 2011, at 7:54 AM, Stuart Sierra wrote:
>
>> Recently the received wisdom has been: protocols are a low-level implementation detail. Actual APIs should be built with normal functions that call the protocol methods.
>
>
> Interesting. So I'm not encouraged to think "protocols" except when I care about efficiency or Java interop?

I don't think so. I see it more that the design of your lib should not
be centered around expecting the user to extend and call protocol
functions all over the place.
But rather provide classic hof leveraging internally polymorphic parts
via protocols, and exposing those hof as APIs.

If required, the user may extend some of your librarie's protocols to
his own types or to types he doesn't own (at the risk of shadowing
your librarie's own extension).
But still, the functions of the protocol(s) the user will extend will
not be those which will be called in his code.

HTH,

--
Laurent

[Meikel Brandmeyer]

Hi,

Am 01.07.2011 um 00:16 schrieb Laurent PETIT:

>> On Jun 30, 2011, at 7:54 AM, Stuart Sierra wrote:
>>
>>> Recently the received wisdom has been: protocols are a low-level implementation detail. Actual APIs should be built with normal functions that call the protocol methods.
>

> But still, the functions of the protocol(s) the user will extend will
> not be those which will be called in his code.

I don't buy that. Why should protocol functions not be part of the public API?

(defprotocol Associative
(get [this k]))

(extend-protocol Associative
Vector
(get [this idx] ...)
Map
(get [this k] ...))

Looks perfectly fine to me. In fact: how do you want to hide the protocol here? I think the point is that the protocol does not have to comprise the whole API. It can be perfectly complemented by normal functions. Or it could be just "internal" in terms of the user side being only normal functions (the scenario you describe). But I think there are relatively simple use cases (see above), where it doesn't have to be that way.

Sincerely
Meikel

[Chas Emerick]

But, you'd never do that, right? Each implementation of Vector and Map would have their own idiosyncratic implementations of `get`, and you _want_ (need) that bolted into the deftyped class.

Brian was looking to be able to dynamically rebind a protocol fn, which is simply impossible for classes that have inline implementations. The solution in such cases seems to be that such protocol fns should be implemented via extend, which would then allow for any kind of rebinding or HOF action you want.

Unless I'm missing something, I'd revise the "received wisdom" to say "inlined protocol methods are a low-level implementation detail". If you want the perf, go for it, but know that you're giving up some of the benefits of vars for instances of classes that have direct impls. That sounds like a perfectly reasonable tradeoff.

- Chas

[Laurent PETIT]

2011/7/1 Chas Emerick <ceme...@snowtide.com>:

To be honest, i was just trying to "expand on my own current
understanding of the 'received wisdom'", 'cause I also (as Brian did)
found that this 'wisdom' is currently poorly explained, letting each
and every person "extend it to its own fully qualified demonstration".

By being explicit about what I have inferred from the condensed bits
that are generally served, my intention is to be proven wrong or not,
but by means of arguments and more details.

And I'm not sure that the 'received wisdom' is just about "inlining
protocol methods" vs other more flexible (but less performant)
techniques of extending protocols ...

[Meikel Brandmeyer]

Hi,

Am Freitag, 1. Juli 2011 02:34:01 UTC+2 schrieb Chas Emerick:

But, you'd never do that, right?  Each implementation of Vector and Map would have their own idiosyncratic implementations of `get`, and you _want_ (need) that bolted into the deftyped class.

Brian was looking to be able to dynamically rebind a protocol fn, which is simply impossible for classes that have inline implementations.  The solution in such cases seems to be that such protocol fns should be implemented via extend, which would then allow for any kind of rebinding or HOF action you want.

Unless I'm missing something, I'd revise the "received wisdom" to say "inlined protocol methods are a low-level implementation detail".  If you want the perf, go for it, but know that you're giving up some of the benefits of vars for instances of classes that have direct impls.  That sounds like a perfectly reasonable tradeoff.

That's not what I understood. I know about the trade-offs between extend and inline methods. I blogged some time ago about it and voted for extend being the default until speed matters. However Stuart wrote "protocols are a low-level implementation detail". And I don't think that this is reasonable. Take any of the collection functions: get, nth, peek, pop, seq, ... Each is a perfect candidate for a protocol function. Why (and how) would I hide this? I think this is a different question than using extend or inlining.

Re-reading Stuart's email, I may have actually misunderstood his second paragraph.

Sincerely
Meikel

[David McNeil]

On Jun 30, 7:54 am, Stuart Sierra <the.stuart.sie...@gmail.com> wrote:
> Recently the received wisdom has been: protocols are a low-level
> implementation detail. Actual APIs should be built with normal functions
> that call the protocol methods.

Stuart- I am a bit confused by this statement, and judging by this
thread others are as well. Any chance you could elaborate on the
received wisdom with respect to protocols and APIs?

Thank you.

-David

[Aaron Cohen]

I think a great example of this is clojure.java.io. The api is
"reader" and "writer", but the implementation is in the Coercions and
IOFactory protocols.

It's the largest "real world" usage of protocols I've seen so far, and
it's quite a nice, I think.

--Aaron

[Christophe Grand]

Hi

On Jul 1, 8:09 am, Meikel Brandmeyer <m...@kotka.de> wrote:
> That's not what I understood. I know about the trade-offs between extend and
> inline methods. I blogged some time ago about it and voted for extend being
> the default until speed matters. However Stuart wrote "protocols are a
> low-level implementation detail".

In my understanding of the protocols, low-level is not the exact word.
Protocols are designed with the implementer in mind, not the user.
Sometimes user-facing API and implementer-facing API overlap but it's
not a given. So, from the user point of view, protocols are an
implementation detail, they are somewhat "low-level" -- lower than the
user-level at least.

NB: here I draw a distinction between the user which uses your API
without providing his own implementation of your protocols, and the
"power user"/implementer which does. Two groups with different trade-
offs in API design

If you have given them overlapping APIs you won't be able to change
the user-facing API (eg adding fns or arities for convenience) without
impacting the implementer.

If you are the only implementer then your protocol is not pubic so you
can get away with letting the user directly call protocol fns.

> And I don't think that this is reasonable.
> Take any of the collection functions: get, nth, peek, pop, seq, ... Each is
> a perfect candidate for a protocol function. Why (and how) would I hide
> this? I think this is a different question than using extend or inlining.

conj, assoc, dissoc are counterexamples: their vararg forms can be
implemented in terms of their shorter form.
Let's forget that protocols fns can't be varargs. If you were
specifying that conj as a protocol should accept a indefinite number
of arguments, you would be putting the burden of providing this
convenient signature onto the implementor, making your protocol harder
to implement.
That's exactly why Java frameworks are littered with abstract
classes : interfaces are littered with helper methods that are
tiresome to implement, so there's the abstract companion class which
provide implementation of the helper methods, leaving only the core
methods abstract. And as soon as you are creating a class which must
implements two of such interfaces you are screwed because you won't be
able to inherit implementations from two different abstract classes.

Even nth and get are counterexamples too: given [coll index not-found]
one can easily implement [coll index].

my 2 cents,

Christophe

[Meikel Brandmeyer]

Salut Christophe,

Am 02.07.2011 um 22:07 schrieb Christophe Grand:

> my 2 cents,

Understatement!

Thank you for the clarifications. I wrote them up with an example (hopefully) illustrating the issue. http://bit.ly/nOHgle

Cordialement
Meikel

[David McNeil]

On Jul 2, 3:07 pm, Christophe Grand <christo...@cgrand.net> wrote:
> Protocols are designed with the implementer in mind, not the user.
> Sometimes user-facing API and implementer-facing API overlap but it's
> not a given. So, from the user point of view, protocols are an
> implementation detail, they are somewhat "low-level" -- lower than the
> user-level at least.

Christophe & Meikel - Thank you for elaborating on this point. I can
see the motivation for separating out the user level API from the
implementer API. However, it is not clear to me why we would say that
protocols are suitable for one of these APIs but not the other. Do you
have any thoughts on this?

Thanks.
-David

[Stuart Sierra]

"Recently the received wisdom has been: protocols are a low-level implementation detail. Actual APIs should be built with normal functions that call the protocol methods."

I misspoke: This is not to say that protocols cannot be API functions, but that protocols are not necessarily APIs, as interfaces would be in Java.

That said, it is often convenient to wrap protocol methods in normal functions to handle special cases, default arguments, varargs, etc.

-Stuart Sierra
clojure.com