Datatype Usage Examples
SinDoc
I was wondering if someone could point me to recent usage examples of
deftype, defrecord, and reify. Reading [1] helped a lot but it wasn't
particularly easy to find it since it's not linked from the sidebar.
Specifically, what I'd like to know is:
- How to define and access member data fields -also mutable in case
of deftype- to my ADTs.
- Whether I can refer to type instances -an equivalent to the 'this'
keyword in Java.
- How to define constructors.
Kind regards,
SinDoc
[1] http://clojure.org/datatypes
Meikel Brandmeyer
On May 28, 9:52 am, SinDoc <s...@khakbaz.com> wrote:
> I was wondering if someone could point me to recent usage examples of
> deftype, defrecord, and reify. Reading [1] helped a lot but it wasn't
> particularly easy to find it since it's not linked from the sidebar.
>
> Specifically, what I'd like to know is:
>
> - How to define and access member data fields -also mutable in case
> of deftype- to my ADTs.
> - Whether I can refer to type instances -an equivalent to the 'this'
> keyword in Java.
> - How to define constructors.
http://kotka.de/blog/2010/03/memoize_done_right.html#protocols
Sincerely
Meikel
Krukow
>
> I was wondering if someone could point me to recent usage examples of
> deftype, defrecord, and reify. Reading [1] helped a lot but it wasn't
> particularly easy to find it since it's not linked from the sidebar.
"Circuit-breaker" pattern.
The circuit breaker is basically a state-machine, with transitions
that occur on various events.
To me, it was very natural to model the transition functions as a
protocol
(defprotocol CircuitBreakerTransitions
"Transition functions for circuit-breaker states"
(proceed [s] "true if breaker should proceed with call in this
state")
(on-success [s] "transition from s to this state after a successful
call")
(on-error [s] "transition from s to this state after an unsuccessful
call")
(on-before-call [s] "transition from s to this state before a
call"))
The states are then datatypes (records in this case) that are extended
to reach this protocol, e.g.,
(defrecord ClosedState [#^TransitionPolicy policy #^int fail-count])
...
(extend ClosedState CircuitBreakerTransitions ...)
Blog: http://blog.higher-order.net/2010/05/05/circuitbreaker-clojure-1-2/
Code: http://github.com/krukow/clojure-circuit-breaker
> Specifically, what I'd like to know is:
>
> - How to define and access member data fields -also mutable in case
> of deftype- to my ADTs.
> - Whether I can refer to type instances -an equivalent to the 'this'
> keyword in Java.
keyword in Java, e.g.,
(extend ClosedState CircuitBreakerTransitions
...
:on-success
(fn [{f :fail-count p :policy, :as s}] ;; note we can destructure
the 'this' argument (s)
(if (zero? f) s (ClosedState. p 0)))
...)
> - How to define constructors.
two params:
(ClosedState. policy fail-count)
If you need more flexibility, I believe you need gen-class, but I am
unsure.
Hope that helps. Kind Regards,
- Karl
Sina K. Heshmati
Meikel Brandmeyer wrote:
Thank you for the link, Meikel. It's very well-written.
Kind regards,
SinDoc
Sina K. Heshmati
Krukow wrote:
> SinDoc wrote:
>> I was wondering if someone could point me to recent usage examples of
>> deftype, defrecord, and reify. Reading [1] helped a lot but it wasn't
>> particularly easy to find it since it's not linked from the sidebar.
>
> Blog: http://blog.higher-order.net/2010/05/05/circuitbreaker-clojure-1-2/
>
> Code: http://github.com/krukow/clojure-circuit-breaker
This is a good example. I like the way merge is used to avoid redundancy while dispatching methods. Deftype is not being used in the example but if I want the map, I should go for defrecord anyway.
The only member data _I'm_ able find are the ones that are passed to the default constructor, namely at the time that the abstraction is reified. What if I'd have to give create a member field that is not necessarily known by the caller or instantiator. That is, it's the abstraction itself that has to create and maintain the value of that field.
One of the abstraction that I was hoping to implement in Clojure is a Scheme-like pair in order to demonstrate various memory management techniques. Once we do (cons a b), an abstract pair should be made that only contains a pointer to its location in memory: (tag address). Here's the pair implementation [2] in Scheme R5RS.
I'm thinking optional parameters could be used. Since callers don't have to provide values for them, I can use them to maintain the state for each instance. I might be missing the obvious here since these are data types and there should be an explicit way to say that this _thing_ is my data.
>> Specifically, what I'd like to know is:
>>
>> - How to define and access member data fields -also mutable in case
>> of deftype- to my ADTs.
>
> The example is there for immutable fields. Why do you need mutability?
I don't need it. Just wanted to see some examples. I'd rather go immutable anyway ;)
>> - Whether I can refer to type instances -an equivalent to the 'this'
>> keyword in Java.
>
> The first argument to a protocol function corresponds to the "this"
> keyword in Java, e.g.,
>
> (extend ClosedState CircuitBreakerTransitions
> ...
> :on-success
> (fn [{f :fail-count p :policy, :as s}] ;; note we can destructure
> the 'this' argument (s)
> (if (zero? f) s (ClosedState. p 0)))
> ...)
Nice!
>> - How to define constructors.
>
> A single constructor is automatically defined for you. In my case with
> two params:
>
> (ClosedState. policy fail-count)
>
> If you need more flexibility, I believe you need gen-class, but I am
> unsure.
I can live with one constructor for now.
>> [1]http://clojure.org/datatypes
[2] http://is.gd/ctoSg
> Hope that helps. Kind Regards,
Thank you very much, Krukow. It sure did help.
Kind regards,
SinDoc
Krukow
On May 28, 9:59 pm, "Sina K. Heshmati" <s...@khakbaz.com> wrote:
> Hi Krukow,
[snip]
as a constructor argument, but computed as a function of the other
fields, and which is not necessarily accessible from callers. If I
understand correctly what it is you want, I think you are moving away
from Clojure, trying to somehow encapsulate an "object". See RH's note
on encapsulation: "Encapsulation of information is folly. Fields are
public. Use protocols/interfaces to avoid dependencies" [1].
Again, I believe that you can use gen-class if you really need to do
this.
> One of the abstraction that I was hoping to implement in Clojure is a Scheme-like pair in order to demonstrate various memory management techniques. Once we do (cons a b), an abstract pair should be made that only contains a pointer to its location in memory: (tag address). Here's the pair implementation [2] in Scheme R5RS.
>
arbitrarily? This is definately not the Clojure way :) In Clojure,
you'd either use a pair of atoms or refs, giving you managed
mutability, or you'd simply write a functional pair.
[snip]
>
> Kind regards,
> SinDoc
Sina K. Heshmati
> Sina K. Heshmati wrote:
> [snip]
>> The only member data _I'm_ able find are the ones that are passed to the default
>> constructor, namely at the time that the abstraction is reified. What if I'd have
>> to give create a member field that is not necessarily known by the caller or
>> instantiator. That is, it's the abstraction itself that has to create and
>> maintain the value of that field.
>
> You mean something like a private field in Java that is not supplied
> as a constructor argument, but computed as a function of the other
> fields, and which is not necessarily accessible from callers. If I
> understand correctly what it is you want, I think you are moving away
> from Clojure, trying to somehow encapsulate an "object". See RH's note
> on encapsulation: "Encapsulation of information is folly. Fields are
> public. Use protocols/interfaces to avoid dependencies" [1].
I chose Clojure to experiment with new things rather than applying my old tricks so I'm definitely not looking for encapsulation per se but I should be able to map the concepts from one language to the other in order to be able to effectively code. That said, I'd rather make sure that my low-level data structures are being operated on by only one implementation.
The fact that a certain field is immutable reassures me that other programs can't alter it and this is good enough. If I want my abstraction to be able to alter its own state, then I'd use Refs, Atoms, and the like. I hope this is Clojure idiomatic.
My implementation of the pair abstraction in Clojure is here:
http://github.com/sindoc/algorithms/blob/master/src/main/clojure/memory-management/manual/pairs/pair.clj
Right now, external programs can access my low-level data structures e.g. car-mem and alter them since they're 'refs'. This is not good to my eyes, unless someone says: no matter what you do, external programs can still find a way to access the private fields of my implementation e.g. using reflection.
Macros could also be used to generate most of the helper functions for car-mem, cdr-mem and next-free.
Please feel free to report any bad practices or improvements.
>> One of the abstraction that I was hoping to implement in Clojure is a Scheme-like
>> pair in order to demonstrate various memory management techniques. Once we do
>> (cons a b), an abstract pair should be made that only contains a pointer to its
>> location in memory: (tag address). Here's the pair implementation [2] in Scheme
>> R5RS.
>>
>
> So you want a mutable pair (fst, snd) where fst and snd can be mutated
> arbitrarily? This is definately not the Clojure way :) In Clojure,
> you'd either use a pair of atoms or refs, giving you managed
> mutability, or you'd simply write a functional pair.
Obviously, this is not the Clojure way but I have to simulate the Scheme pair system.
Thanks for the follow-up, Krukow.
SinDoc
Adrian Cuthbertson
You could use closures to encapsulate the refs/atoms ...
(let [car-mem (ref nil)]
(defn set-car-mem [new-car-mem]
(dosync (ref-set car-mem new-car-mem)))
(defn update-car-mem [new-car-mem]
(dosync (set-car-mem new-car-mem)))
(defn get-car-mem [] @car-mem))
user=> (set-car-mem 0)
user=> (get-car-mem)
0
user=> @car-mem
java.lang.Exception: Unable to resolve symbol: car-mem in this context
(NO_SOURCE_FILE:0)
(note that you need a do-sync around ref-set - your code didn't have one.)
-Rgds, Adrian
Sina K. Heshmati
>> That said, I'd rather make sure that my low-level data structures are being
>> operated on by only one implementation.
>
> You could use closures to encapsulate the refs/atoms ...
>
> (let [car-mem (ref nil)]
> (defn set-car-mem [new-car-mem]
> (dosync (ref-set car-mem new-car-mem)))
> (defn update-car-mem [new-car-mem]
> (dosync (set-car-mem new-car-mem)))
> (defn get-car-mem [] @car-mem))
>
> user=> (set-car-mem 0)
> user=> (get-car-mem)
> 0
> user=> @car-mem
> java.lang.Exception: Unable to resolve symbol: car-mem in this context
> (NO_SOURCE_FILE:0)
This is how I'd implemented state-hiding in Scheme [1]. The problem though, is exporting a public interface. In [1], (export x y) is just a macro for (define x null) and (define y null). So I define my public bindings and re-assign them to their actual values while I'm inside the closure.
I can't do the same in Clojure, since I'm not allowed to re-assign a definition --i.e. its root.
A dispatch function should do the trick [2].
> (note that you need a do-sync around ref-set - your code didn't have one.)
'set-car-mem' and the like are always called from within a dosync. I should probably remove those set-* functions since (1) they don't abstract too much and (2) they confuse readers.
Thank you Adrian, for your help.
Kind regards,
SinDoc
Meikel Brandmeyer
Am 30.05.2010 um 16:39 schrieb Sina K. Heshmati:
I'm almost sure, that this code does not what you expect. Nested def's, and in particular defn's, are almost surely wrong. You want:
(defn make-module
[]
(let [state (atom 0)
say-hello (fn [] (println "Hello"))
public-op (fn [x y] (+ @state x y))]
(fn [op]
(case op
:hello say-hello
:public public-op))))
And I'm not sure with what you mean by "you are not allowed to change the root of a Var". Of course you can do that.
(declare x)
(defn fn-using-x ...)
(def x 5)
The declare is also (more or less) equivalent to (def x nil).
But I'm not sure I misunderstand something here.
Sincerely
Meikel
Sina K. Heshmati
Meikel Brandmeyer wrote:
> Am 30.05.2010 um 16:39 schrieb Sina K. Heshmati:
>
>> [2]
>> http://github.com/sindoc/algorithms/blob/master/src/test/clojure/whiteboard/y2010/hide-adt-state-using-closure.clj
>
> I'm almost sure, that this code does not what you expect. Nested def's, and in
> particular defn's, are almost surely wrong. You want:
>
> (defn make-module
> []
> (let [state (atom 0)
> say-hello (fn [] (println "Hello"))
> public-op (fn [x y] (+ @state x y))]
> (fn [op]
> (case op
> :hello say-hello
> :public public-op))))
Could you elaborate a bit please? Is there a reason why nested defns are wrong?
> And I'm not sure with what you mean by "you are not allowed to change the root of
> a Var". Of course you can do that.
I referred to the fact that we can't do as follows:
> (def x 10)
> (set! x (* x x))
It should be possible to achieve the exact same effect using some other means but not as a result of evaluating the two expressions above.
> (declare x)
> (defn fn-using-x ...)
> (def x 5)
>
> The declare is also (more or less) equivalent to (def x nil).
(declare symbol) seems to work [3]. I'll later try it with deftypes to see if I can also export datatypes since my method as in [2] failed to export datatypes.
Thank you Meikel.
Kind regards,
SinDoc
Sina K. Heshmati
> Meikel Brandmeyer wrote:
>> Am 30.05.2010 um 16:39 schrieb Sina K. Heshmati:
>>
The atomic 'state' doesn't seem to be visible to the datatype methods. The question is why?
(defprotocol prot-a
(op-a [self x y]))
(let [state (atom 10)]
(deftype t-a [member]
prot-a
(op-a [self x y]
(+ (.member self) x y @state))))
(def t-a-instance (t-a. 5))
This example is in:
http://github.com/sindoc/algorithms/blob/master/src/test/clojure/whiteboard/y2010/hide-adt-state/datatype-in-closure-01.clj
Thank you in advance,
SinDoc
Meikel Brandmeyer
On May 31, 9:37 am, "Sina K. Heshmati" <s...@khakbaz.com> wrote:
> The atomic 'state' doesn't seem to be visible to the datatype methods. The question is why?
>
> (defprotocol prot-a
> (op-a [self x y]))
>
> (let [state (atom 10)]
> (deftype t-a [member]
> prot-a
> (op-a [self x y]
> (+ (.member self) x y @state))))
>
> (def t-a-instance (t-a. 5))
methods are not closures. (And even if they were, your state would be
the same for all instances.) You have to pass the state as an argument
to the constructor.
(defprotocol prot-a ...)
(deftype t-a [state member] ...)
(defn make-t-a [member] (t-a. (atom 0) member))
(def t-a-instance (make-t-a 5))
If you actually want the same state for instances, use a private var.
(defprotocol prot-a ...)
(def ^{:private true} state (atom 0))
(deftype t-a [member] ...)
(def t-a-instance (t-a. 5))
Sincerely
Meikel
Sincerely
Meikel
Sina K. Heshmati
> On May 31, 9:37 am, "Sina K. Heshmati" <s...@khakbaz.com> wrote:
>
>> The atomic 'state' doesn't seem to be visible to the datatype methods. The
>> question is why?
>>
>> (defprotocol prot-a
>> (op-a [self x y]))
>>
>> (let [state (atom 10)]
>> (deftype t-a [member]
>> prot-a
>> (op-a [self x y]
>> (+ (.member self) x y @state))))
>>
>> (def t-a-instance (t-a. 5))
>
> Of course not. deftype defines a type and not an environment. It's
> methods are not closures. (And even if they were, your state would be
> the same for all instances.) You have to pass the state as an argument
> to the constructor.
Now it makes more sense. Thanks.
> (defprotocol prot-a ...)
> (deftype t-a [state member] ...)
>
> (defn make-t-a [member] (t-a. (atom 0) member))
>
> (def t-a-instance (make-t-a 5))
I'll pass.
> If you actually want the same state for instances, use a private var.
>
> (defprotocol prot-a ...)
> (def ^{:private true} state (atom 0))
> (deftype t-a [member] ...)
>
> (def t-a-instance (t-a. 5))
This is a pretty solution but I tested it and it doesn't work the way it should. That is, when I load [1] in a separate REPL, I can still see and (reset!) the allegedly private 'state'.
user=> (load-file "path/tp/datatype-01.clj")
#'foo.datatype-01/t-a-instance
user=> (ns PREFIX.datatype-01)
nil
foo.datatype-01 => (op-a t-a-instance 1 2)
18
foo.datatype-01 => state
#<Atom@302e67: 10>
foo.datatype-01 => @state
10
foo.datatype-01 => (reset! state 13)
13
foo.datatype-01 => @state
13
Kind regards,
SinDoc
Meikel Brandmeyer
On May 31, 10:58 am, "Sina K. Heshmati" <s...@khakbaz.com> wrote:
> foo.datatype-01 => (reset! state 13)
Again: of course you can! You are in the same namespace! In Clojure
the "unit" is a namespace and not a type. If you want this level
privateness you have to use one namespace per type. However this is
not really the Clojure Way.
Sincerely
Meikel
Sina K. Heshmati
Here's my concern:
- My program (A) is running.
- B is running on the same VM.
- B accesses the state of A.
- B alters the state of A in an inconsistent way e.g. whenever the internal state x changes, the internal state y also has to change accordingly, but B only changes x.
- A is screwed.
The author of Clojure calls encapsulation a folly but I don't understand why (?) How can we avoid situations like the one explained above?
I came up with the following, which does what I want by relying on the lexical scope of functions. This is OK but the only problem is the fact that I have to hardcode function calls for each type method.
(defprotocol prot-a
(op-a [self x y]))
(let [state (atom 10)]
(defn op-a
[self x y]
(+ x y (.member self) @state)))
(deftype t-a [member]
prot-a
(op-a [self x y]
(op-a self x y)))
(def t-a-1 (t-a. 5))
Is there a way to pass the function itself rather than making a new function that calls the exact same function with the same arguments?
Kind regards,
SinDoc
Meikel Brandmeyer
On May 31, 1:46 pm, "Sina K. Heshmati" <s...@khakbaz.com> wrote:
> Here's my concern:
>
> - My program (A) is running.
> - B is running on the same VM.
> - B accesses the state of A.
> - B alters the state of A in an inconsistent way
> e.g. whenever the internal state x changes, the
> internal state y also has to change accordingly,
> but B only changes x.
> - A is screwed.
^{:private true} state (atom 0)). B is namespace my.ns.b. B cannot
change state in A.
user=> (ns my.ns.a)
nil
my.ns.a=> (def ^{:private true} state (atom 0))
#'my.ns.a/state
my.ns.a=> (ns my.ns.b)
nil
my.ns.b=> (reset! my.ns.a/state :messed-up-your-state-harharhar)
java.lang.IllegalStateException: var: my.ns.a/state is not public
(NO_SOURCE_FILE:4)
From a philosophical point of view:
Either you develop an application and
you are in charge of B. => You are in control: don't do it.
you are not in charge of B. => B is obviously a crappy library which
you should not use in project.
You develop a library: A. Why bother? Since your library is well
documented it clearly defines the contract. Why care for people who
are too lazy to read?
> Is there a way to pass the function itself rather
> than making a new function that calls the exact
> same function with the same arguments?
(extend t-a
prot-a
{:op-a (fn [this x y] (+ (.member this) x y @state))}))
Sincerely
Meikel
Sina K. Heshmati
True but my main concern is security of a running application. It could very well be that B is just a bunch of interactions, in which case B can enter A's namespace.
I still haven't tried to use reflection to see if I can still access the state. In case a program can use metaprogramming to access private fields of an abstraction, no matter what the programmer does, then my concerns are not relevant.
>> Is there a way to pass the function itself rather
>> than making a new function that calls the exact
>> same function with the same arguments?
>
> (let [state (atom 0)]
> (extend t-a
> prot-a
> {:op-a (fn [this x y] (+ (.member this) x y @state))}))
Thanks a bunch, Meikel.
Problem solved. Please see:
http://github.com/sindoc/algorithms/blob/master/src/main/clojure/whiteboard/y2010/hide-adt-state/datatype-and-closure-01.clj
Kind regards,
SinDoc
Meikel Brandmeyer
On May 31, 3:15 pm, "Sina K. Heshmati" <s...@khakbaz.com> wrote:
> True but my main concern is security of a running application.
> It could very well be that B is just a bunch of interactions,
> in which case B can enter A's namespace.
of A? If you mean you have a Repl and someone arbitrary can
enter commands there as she likes... You can always mess up
everything.
> I still haven't tried to use reflection to see if I can still
> access the state. In case a program can use metaprogramming
> to access private fields of an abstraction, no matter what the
> programmer does, then my concerns are not relevant.
fuzzy "the user might do something". Do you have a particular
scenario in mind?
> Problem solved.
Just some minor remarks: you can move the extend outside the
let and the declare is not necessary. defprotocol will take
care of that for you.
Sincerely
Meikel
Sina K. Heshmati
> On May 31, 3:15 pm, "Sina K. Heshmati" <s...@khakbaz.com> wrote:
>
>> True but my main concern is security of a running application.
>> It could very well be that B is just a bunch of interactions,
>> in which case B can enter A's namespace.
>
> ??? I'm not sure I understand. How can B "enter" the namespace
> of A? If you mean you have a Repl and someone arbitrary can
> enter commands there as she likes... You can always mess up
> everything.
>
>> I still haven't tried to use reflection to see if I can still
>> access the state. In case a program can use metaprogramming
>> to access private fields of an abstraction, no matter what the
>> programmer does, then my concerns are not relevant.
>
> To be honest: I don't see the problem. You talk about a very
> fuzzy "the user might do something". Do you have a particular
> scenario in mind?
You're right. It's rather a contract problem as you said than a security concern because no matter what we do, people can use reflection to mess things up anyway.
I'm likely to drop the whole idea of encapsulating the state in a closure and rely on private annotations.
Thanks Meikel for bearing with me :)
>> Problem solved.
>
> Just some minor remarks: you can move the extend outside the
> let and the declare is not necessary. defprotocol will take
> care of that for you.
Nice. That's much better.
I'd like to thank Meikel, Krukow, and Adrian for participating in this discussion.
Kind regards,
SinDoc
Meikel Brandmeyer
On Mon, May 31, 2010 at 07:34:24AM +0200, Sina K. Heshmati wrote:
> > (defn make-module
> > []
> > (let [state (atom 0)
> > say-hello (fn [] (println "Hello"))
> > public-op (fn [x y] (+ @state x y))]
> > (fn [op]
> > (case op
> > :hello say-hello
> > :public public-op))))
>
> Could you elaborate a bit please? Is there a reason why nested defns are wrong?
I'm sorry. I missed this question.
def always introduces a global binding. It is unlike schemes define.
So invoking make-module twice basically overwrites the first defs
with the second ones.
Sincerely
Meikel
Sina K. Heshmati
Meikel Brandmeyer wrote:
> On Mon, May 31, 2010 at 07:34:24AM +0200, Sina K. Heshmati wrote:
>
>> > (defn make-module
>> > []
>> > (let [state (atom 0)
>> > say-hello (fn [] (println "Hello"))
>> > public-op (fn [x y] (+ @state x y))]
>> > (fn [op]
>> > (case op
>> > :hello say-hello
>> > :public public-op))))
>>
>> Could you elaborate a bit please? Is there a reason why nested defns are wrong?
>
> I'm sorry. I missed this question.
Thanks for the follow-up.
> def always introduces a global binding. It is unlike schemes define.
> So invoking make-module twice basically overwrites the first defs
> with the second ones.
I see now. I used to think of them as defines in Scheme but I started to notice this when I saw them survive closures.
Kind regards,
SinDoc