Implementing a protocol with using a base implementation?

[tbatchelli]

Hi all,

I am writing a network protocol handler based on events (to wrap
netty, if you're curious). I created a protocol that defines the
functions needed to handle every possible (channel) event:

(defprotocol channel-handler-strategy
(message-received [this ctx evt])
(exception-caught [this ctx evt])
...
)

And I also provide a default event handler that will call the
appropriate handler function of the channel-handler-strategy protocol
depending on the type of event and some other data, for example:

(defrecord simple-channel-handler [strategy]
channel-handler
(handle-upstream [this ctx evt]
(if (= ctx "hello")
(message-received strategy ctx evt)
(exception-caught strategy ctx evt))))

Note: this is toy code, the real code hasn't been written yet ;)

Here is the problem I am trying to solve: the list of handler
functions that the protocol will define is loooong, and for most of
the cases, the developer will only want to perform some custom logic
in one or two of the handler functions, expecting that the rest of the
handler functions will perform some sort of default action (forward
the event in this case). If my understanding of the new type system in
1.2 is correct, there is no way create a record by extending another
record, nor a way to override functions of a record. Thus, with the
current tools, every channel-handler-strategy implementor would have
to provide a definition for *all* the functions defined in the
protocol.

I thought of some solutions, but I get the feeling I might just be
approaching the problem from the wrong angle.

1) The channel-handler-stategy implementor only implements the handler
functions he/she cares about, leaving the rest unimplemented. When the
event handler calls one of the unimplemented handler functions and
exception will be thrown, and so the event handler can catch them and
execute the default behavior.

2) If there was a way to build a record from a map e.g: {fn-name fn-
definition}, then I'd be able to create a default handler function map
which the channel-handler-strategy implementors would merge with their
own custom maps that contained only the redefined handler functions,
and then use this merged map to create the record. I could not find
any existing way to do this but I believe it should be possible to
write a macro to do create a record from such map.

3) Use structs, but that would be too slow for my use case.

So here is the question: Am I approaching this problem the wrong way?
Or this use case for types is an only edge case and I should go ahead
and build a custom solution based on any of the above proposed
solutions?

Thanks in advance for your thoughts,

Toni.

[Nicolas Oury]

Not very clean suggestion.

Split the protocol in one protocol per function.

Instance every one on Object, with the default protocol.

Instance each specific on the function for which it has a special instance.

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

[Nicolas Oury]

I read my mail and couldn't understand it.

Here is what I meant:

(defprotocol MessageReceived

   (message-received ...))

(defprotocol ExceptionCaught

 .....)

(extend Object MessageReceved {:message-received default-message-received-function}...

)

(deftype channel-handler

  ExceptionCaught ....)

(not the lack of message received implementation)

[Toni Batchelli]

Hi Nicolas,

I get the idea, but I don't see how this would help provide a default
implementation for the functions inside a protocol. It looks to me
like this would be the same as creating a record with only some of the
functions implemented. Or am I reading it wrong?

Thanks for your input.

Toni.

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

--
Antoni Batchelli
- twitter: @tbatchelli , @disclojure
--- email: tbatc...@gmail.com
----- web: tbatchelli.org , disclojure.org

[Nicolas Oury]

Hi Toni,

The key thing is that you can implement protocol on Object.

So if you have one protocol per function, (Let's call your function f1,..., fn and the protocols implementing them F1,....,Fn)

You can define a default value for each function:

(extend-type Object

   F1 (f1 ...) .... Fn (fn....))

and you have a type (let's call your type T1, ... Tm) T1, that only implements f1...fp (p <=n) (without loss of generality, up to renaming)

(deftype T1 

   F1

   (f1 .....)

   ... Fp (fp...))

Now, if you call fi on something of type T1. If i <=p , then the implemntation of F1 on T1 will be called. Else the default instance on Object will be called.

Nota: You can makes this more type safe, by creating an empty interface HasF1_Fn (definterface HasF1_Fn), and defining the default function

by extending this interface instead of Object.

You then have to had HasF1_Fn to your deftype.

Best regards,

Nicolas.

[Meikel Brandmeyer]

Hi,

one way to do that is using extend.

(def defaults
{fn1 (fn ...)
fn2 (fn ...)
fn3 (fn ...)})

(defrecord R1 [...])
(def R1-fns
{fn1 (fn ...)})

(defrecord R2 [...])
(def R2-fns
{fn2 (fn ...)
fn3 (fn ...)})

(extend YourProtocol
R1 (merge defaults R1-fns)
R2 (merge defaults R2-fns))

Another way is the following macro:

(defmacro defrecord-with-defaults
[record fields & protocol-default-fns]
(let [process-protocol
(fn [[protocol defaults fns]]
(let [defaults (when defaults
(->> defaults
resolve
var-get
(map (fn [[k f]]
(vector
k (cons (symbol (name k)) (next f)))))
(into {})))]
(list* protocol
(->> fns
(map #(array-map (-> % first name keyword) %))
(apply merge defaults)
(map second)))))
split-protocols
(fn split-protocols
[p-d-fs]
(lazy-seq
(when-let [[p d & fs] (seq p-d-fs)]
(let [[fs next-p-d-fs] (split-with (complement symbol?) fs)]
(cons [p d fs] (split-protocols next-p-d-fs))))))]
`(defrecord ~record ~fields
~@(mapcat process-protocol (split-protocols protocol-default-fns)))))

Usage:

(def foo-defaults
`{:bar (fn ([this# that#] nil))
:baz (fn [this# that#] nil)})

(defrecord-with-defaults R [a b]
Foo foo-defaults
(bar [this that] nil)
Frob nil
(frobnicator [this] nil))

Note: you have to syntax-quote the functions in the default map (as for definline). And you can have only one arity per function. Here the corresponding expansion:

(clojure.core/defrecord FooImpl1 [a b]
Foo
(bar [this that] nil)
(baz [this__711__auto__ that__712__auto__] nil)
Frob
(frobnicator [this] nil))

Sincerely
Meikel

[Toni Batchelli]

Hi Meikel,

This is awesome! You just did a big chunk of what I was about to try
to do :). Sorry for the late response, I've been off the grid for a
few days...

With your second proposed solution, the defrecord-with-defaults macro,
one can achieve very good performance while keeping some of the
features that implementation inheritance provides. I'll look into your
macro more closely and see if I can make it a seamless substitute for
defmacro.

Toni.

[Toni Batchelli]

So it looks I made a young programmer mistake (and I am not that young anymore!) I *assumed* that creating a defrecord would result in faster code than extending an existing one and therefore was looking to perform an early (and thus unnecessary) optimization. According to the code and results below, it would seem that I was wrong by an order of magnitude no less and that calling the method on an extended protocol is over 10x faster than calling the same method on a record . I have reviewed this test a few times, but I can't see any flaw.

Toni.

(defprotocol P

  (m1 [this p11])

  (m2 [this p21 p22]))

(defrecord simple-P-impl []

  P

  (m1 [this p11] (str "simple m1 called with p11=" p11) nil ) ;; returning nil on all methods so that I don't get a lot of output and end up measuring how fast is Emacs at scrolling the REPL.

  (m2 [this p21 p22] (str "simple m2 called with p21=" p21 "and p22=" p22) nil))

(defrecord empty-record [])

(extend empty-record

  P {:m1 (fn [this p11] (str "extended m1 called with p11=" p11) nil)

     :m2 (fn [this p21 p22] (str "extended m2 called with p21=" p21 "and p22=" p22) nil)})

(def my-empty-record (empty-record.))

(def my-simple-P (simple-P-impl.))

(dotimes [_ 10] (time (dotimes [_ 10000] (.m1 my-simple-P "hello")))) ; 

"Elapsed time: 131.973 msecs"

"Elapsed time: 142.72 msecs"

"Elapsed time: 95.51 msecs"

"Elapsed time: 95.724 msecs"

"Elapsed time: 83.646 msecs"

"Elapsed time: 97.44 msecs"

"Elapsed time: 79.077 msecs"

"Elapsed time: 94.275 msecs"

"Elapsed time: 78.541 msecs"

"Elapsed time: 95.75 msecs"

(dotimes [_ 10] (time (dotimes [_ 10000] (m1 my-empty-record "hello"))))

"Elapsed time: 6.257 msecs"

"Elapsed time: 35.18 msecs"

"Elapsed time: 5.674 msecs"

"Elapsed time: 2.615 msecs"

"Elapsed time: 3.372 msecs"

"Elapsed time: 5.169 msecs"

"Elapsed time: 3.651 msecs"

"Elapsed time: 3.429 msecs"

"Elapsed time: 4.706 msecs"

"Elapsed time: 2.878 msecs"

(note: similar results if calling m2)

[Nicolas Oury]

On Sat, Aug 21, 2010 at 6:33 AM, Toni Batchelli <tbatc...@gmail.com> wrote:
> P-impl.))
> (dotimes [_ 10] (time (dotimes [_ 10000] (.m1 my-simple-P "hello")))) ;
> "Elapsed time: 131.973 msecs"
> "Elapsed time: 142.72 msecs"
> "Elapsed time: 95.51 msecs"
> "Elapsed time: 95.724 msecs"
> "Elapsed time: 83.646 msecs"

Could you try by calling m and not .m1?

[Nicolas Oury]

m1 I meant.

Apologies.

[Meikel Brandmeyer]

Hi,

Am 21.08.2010 um 07:33 schrieb Toni Batchelli:

> (dotimes [_ 10] (time (dotimes [_ 10000] (.m1 my-simple-P "hello")))) ;
>
> "Elapsed time: 131.973 msecs"

I think you get caught by reflection here. As Nicholas said, you should call m1, not .m1.

Sincerely
Meikel

[tbatchelli]

That was it. I didn't think that calling the method directly on the
record would use reflection.

(dotimes [_ 10] (time (dotimes [_ 10000] (m1 my-simple-P "hello"))))

"Elapsed time: 14.765 msecs"
"Elapsed time: 5.347 msecs"
"Elapsed time: 21.427 msecs"
"Elapsed time: 7.267 msecs"
"Elapsed time: 2.88 msecs"
"Elapsed time: 2.675 msecs"
"Elapsed time: 6.643 msecs"
"Elapsed time: 2.548 msecs"
"Elapsed time: 2.773 msecs"
"Elapsed time: 2.833 msecs"

So from this it seems that using extend might be somewhat slower, but
not that much, and it is much more convenient for code reuse (the
numbers I am getting are not very reliable since there are other
processes going on in this desktop computer).

Thanks for your help,

Toni.