Struggling with encapsulation

[Colin Yates]

(newbie, but trying hard!)

I am designing a Woobly.  A Woobly is non-trivial and has a number of internal data structures (queues, worker threads etc.).  You can 'add' and 'remove' jobs from a Woobly.

In OO land I might have a Woobly interface with the various methods which provides a public API.  I might also have a factory or more likely builder somewhere which creates Wooblies.  

The part I am struggling with is how to create a Woobly without exposing its internals.  Let's imagine that Woobly uses an internal LinkedBlockingQueue of a certain size.

Option 1 - a big bag of data.

I could create a map of config/state/data that comprises the implementation and have the creator function return that.  Consumers can then call other methods passing back that bag of config/state, but what stops them simply diving into that bag themselves?  For example:

[code]

(defn create-woobly

 ([] (create-woobly 100)

 ([queue-size] {:queue (java.util.concurrent.LinkedBlockingQueue queue-size)}))

(defn add-job [woobly job] ....)

;; nothing stopping me diving into that state...

(.clear (:queue (create-wobbly)))

[/code]

Option 2 - protocol and deftype

Alternatively I could implement an IWoobly protocol and create a single deftype which is instantiated and returned from the 'create-woobly' function?  I am not sure I like this as it is really back to OO isn't it?  

I want to retain the notion of create returning a handle which is the first argument in the other public functions, but the first way just feels far too dangerous. 

Am I overthinking this - what do you all do to address this?

Thanks a bunch.

Col

[Gary Trakhman]

If the interface provides everything that's needed, then there will be no need to dive in?  

I find attempts to hide that stuff frustrating when I'm a consumer of the code, if I need it, and I acknowledge that implementation details are subject-to-change when I take that risk, so only having something clearly marked off by documentation of intent would be what I would do as a producer of the code.

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Jonathan Fischer Friberg]

I agree with Gary, there's normally not really any need to obfuscate the implementation,
and using the underlying structure can sometimes be useful.

That said, if you really want to, you can create a "woobly" protocol and
implement it using reify, this will make the underlying implementation
completely hidden.

(defprotocol Woobly
  (add-job [woobly job]))

(defn create-woobly

 ([] (create-woobly 100)

 ([queue-size]
  (let [queue (java.util.concurrent.LinkedBlockingQueue. queue-size)]
    (reify Woobly
      (add-job [woobly job]
        ...use queue...)))))

Jonathan

[Dave Ray]

I agree that you probably don't need to go overboard with hiding
stuff. For option 2 though there's no need for deftype. Just implement
the protocol with reifiy within the create function and use the
closure for state.

(defn create-woobly
[...]
(let [... put your queues and stuff here ...]
(reify Woobly
... implement protocol here ...)))

Dave

[James Reeves]

On 9 May 2013 17:07, Colin Yates <colin...@gmail.com> wrote:

The part I am struggling with is how to create a Woobly without exposing its internals.

To what end? What's the benefit?

If you take a look at some internal data structures Clojure uses, like zippers or protocols, you'll notice that they're just maps. In general there's no need to try and obfuscate data to stop people from diving into the internals; just don't provide a public API for the internal parts and people will get the hint.

- James 

[Colin Yates]

Thanks for all the helpful responses.

One reason I want to hide the internals is that I don't want people to add jobs directly to the queue.  (add-job) will put a map containing the provided function onto the queue.  Not really relevant, but this is so that I can track queue timings that I can later on use to determine how much capacity the system can handle.

I am nervous as well about "expose internals but trust people to do the right thing" because in this case, if I was a consumer and saw that queue, particularly given the emphasis about data being the contract etc. then why would I think *not* to use it.  

I do appreciate the point about not needlessly obfuscating information - this is a slightly different case.

Sounds like in this case, either reify is the way to go or maybe return a bad of data but have this stuff in an 'internal' map (i.e. {:internal {:queue...}})

Thanks a bunch - really helpful.

--

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/D2OBBPTxGfY/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.

[James Reeves]

On 9 May 2013 18:03, Colin Yates <colin...@gmail.com> wrote:

I am nervous as well about "expose internals but trust people to do the right thing" because in this case, if I was a consumer and saw that queue, particularly given the emphasis about data being the contract etc. then why would I think *not* to use it.

If this were a problem in general I think we'd find more people poking at the internals of protocols, but I've never seen that happen.

You could use namespaced keywords, like :woobly.internal/queue, to give people more of a hint that the data shouldn't be used without the API, but I really think you're borrowing trouble with this.

Encapsulation is like inheritance, in that it's one of those ideas that's nice on paper, but ultimately not very useful in practise.

- James

[Colin Yates]

Hi James - thanks for your response.  I am still trying to wrap my head around this as I cannot agree.  

To re-frame, woobly allows you to add a job.  The internal implementation of this is that it is decorates that job and puts the *decorated job* onto an internal LBQ.  The implementation also depends upon being the only user of that LBQ as well.  It is critical that jobs are added using the 'add-job' function.

I try and write software that is 'easy to do the right thing, hard to do the wrong thing', and exposing the LBQ makes it trivial to do the wrong thing.  Indeed - if I was using the woobly library (I could just say 'scheduler' right? :)) why wouldn't I think 'hey, I've got the LBQ, that is probably how I enter the pipeline' and 'great, now I can dequeue my jobs by calling .clear' etc.  Documentation - sure, but documentation will never win a war when it is fighting the 'I know you can do this in the code but don't' fight.  The 'bad thing' is that I have exposed too much.

And please don't think I am making the 'code should stop bad programmers doing the wrong thing' argument, I'm not (been there, lost).  I just know that if I released a scheduler library and the main construct was a map containing a LBQ (or in fact any sequence) then I would be inundated with a bunch of 'I did X to the queue and it broke'....

Is it really 'good Clojure' to expose internals which allow a user to easily and trivially break things?

Please don't think I am nit-picking or splitting hairs, I genuinely want to improve my Clojure intuition.

--

[Korny Sietsma]

I would generally handle this sort of encapsulation at the namespace level.

Put (create-woobly) and (add-job) and all the other woobly-related functions into a woobly namespace.  Also add any functions that access info from a woobly bag-o-state, or mutate a woobly to make a woobly-with-extras.

Functions that might dangerously expose internals of a woobly can be made private, or possibly you can just document them in a way to warn folks away from "bad" behaviour.

While external users of (woobly/create-woobly) can in theory dig into the internals of the woobly "object", but it should be relatively obvious that this isn't a good idea.

I'd defer making protocols until you actually need polymorphism.

- Korny

You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Kornelis Sietsma  korny at my surname dot com http://korny.info

.fnord { display: none !important; }

[Colin Yates]

Thanks Korny.

Ok, the over-ridding theme seems to be: expose state, even if it is dangerous, but expect consumers to 'do the right thing' and read the documentation.

I can see that.  I guess I have worked (and to be honest, been guilty myself) with too many people who don't read the documentation, use auto-complete to find something that looks like it might work and then move on to the next chunk of wiring things up in XML :).

I think I also got hung up on the 'data as a contract'.  The point here is that I am not returning data, rather I am defining a service.  A subtle difference but an important one I think.

Keep the comments coming!

[John D. Hume]

I agree with the advice you've gotten, but since no one has mentioned it, I wanted to point out that you can have encapsulation w/o protocols with something like this.

Assume a queue is your only state and `add` and `clear` are your private fns that take a queue as first argument.

(defn new-scheduler []
  (let [queue (...)]
    {:add (partial add queue)
     :clear (partial clear queue)}))

There are several disadvantages to this, however. The biggest in my book is that it achieves your goal, and you're limited in the same way your users are. You can't add behavior to an already created scheduler (unless it's built on adding and clearing). Furthermore, if you dynamically recompile `add` or `clear`, it won't change the behavior of an already created scheduler, since partial has the fns, not the symbols or vars that point at them. (These same disadvantages apply to a reified protocol.)

As others have recommended, just return a map. Keep in mind that the documentation is just a `(doc new-scheduler)` away and that auto-completion will tend to send people back into your ns's fns rather than into the internals of a data structure.

[Colin Yates]

Thanks John.  To be explicit - the add method shouldn't be private - it is the only way users should add to the queue.  I think this is what you meant but you wrote "..and `add` and `clear` are your private fns..".

Again, this paradigm shift of 'trust your users' is unfortunately alien to me based on my experience with most of the Java devs I have come across :).  I say that not be snarky, but to highlight how much it really does change things.  The open-closed principle now becomes much simpler to realise for example.

Thanks again.

[John D. Hume]

The add method that you partially apply in new-scheduler should be private, because a user can't supply the first argument it expects. You might do something like this.

(defn- add* [queue item] (...))

(defn add [scheduler item]
  ((scheduler :add) item))

(defn new-scheduler []
  (let [queue (...)]

    {:add (partial add* queue)}))

But again, I am not recommending this. I just wanted to point out that a closure gives you encapsulation w/o extra language features.

[James Reeves]

On 10 May 2013 11:32, Colin Yates <colin...@gmail.com> wrote:

And please don't think I am making the 'code should stop bad programmers doing the wrong thing' argument, I'm not (been there, lost).  I just know that if I released a scheduler library and the main construct was a map containing a LBQ (or in fact any sequence) then I would be inundated with a bunch of 'I did X to the queue and it broke'....

Have you tried it? :)

I've authored about 40 Clojure libraries over 5 years, some with data structures with internal components. The number of times someone has said "I did X to this internal data and it broke" is exactly zero. It's never ever happened.

So I think you're probably borrowing trouble, and I'd certainly be very surprised if you have any support requests like the one you describe.

- James

[Colin Yates]

On 10 May 2013 14:10, James Reeves <ja...@booleanknot.com> wrote:

Have you tried it? :)

I've authored about 40 Clojure libraries over 5 years, some with data structures with internal components. The number of times someone has said "I did X to this internal data and it broke" is exactly zero. It's never ever happened.

So I think you're probably borrowing trouble, and I'd certainly be very surprised if you have any support requests like the one you describe.

 

This is all about changing my mindset from 15-odd years of Java dev by learning from others, so let's give it a go.  Years of enterprise Java dev have gotten their cynical, 'data is precious and should be hidden away', 'other devs will do the wrong thing' etc. claws into me though, so it is with trepidation I set out on this gloriously liberating new path :).

[Jim - FooBar();]

On 10/05/13 14:20, Colin Yates wrote:

This is all about changing my mindset from 15-odd years of Java dev by learning from others, so let's give it a go.  Years of enterprise Java dev have gotten their cynical, 'data is precious and should be hidden away', 'other devs will do the wrong thing' etc. claws into me though, so it is with trepidation I set out on this gloriously liberating new path :).

the mindset you're describing is a direct consequence of unrestrained mutability ...nothing bad can happen to your immutable clojure data :)... very liberating indeed!

Jim

[Colin Yates]

I get your point, but Java's LinkedBlockingQueue is mutable.  Of course, I should replace it with a plain sequence or Clojure's persistent queue but I really want the blocking semantics.  That's tomorrow's job :).

--

[Timo Mihaljov]

On 10.05.2013 14:44, John D. Hume wrote:
> Assume a queue is your only state and `add` and `clear` are your private
> fns that take a queue as first argument.
>
> (defn new-scheduler []
> (let [queue (...)]
> {:add (partial add queue)
> :clear (partial clear queue)}))
>
> There are several disadvantages to this, however. The biggest in my book
> is that it achieves your goal, and you're limited in the same way your
> users are. You can't add behavior to an already created scheduler
> (unless it's built on adding and clearing). Furthermore, if you
> dynamically recompile `add` or `clear`, it won't change the behavior of
> an already created scheduler, since partial has the fns, not the symbols
> or vars that point at them. (These same disadvantages apply to a reified
> protocol.)

You can make dynamic recompilation work by referring to the vars and not
the functions:

{:add (partial #'add queue)
:clear (partial #'clear queue)}

This works because vars implement `IFn` by delegating to the function
they contain.

--
Timo