Cells in Clojure

288 views
Skip to first unread message

Stuart Sierra

unread,
Sep 20, 2008, 1:46:54 AM9/20/08
to clo...@googlegroups.com
On Sep 19, 7:56 pm, MikeM <michael.messini...@invista.com> wrote:
> Someone's been working on cells for clojure:http://paste.lisp.org/display/66688

I'm not sure what that code is trying to do, but it seems to rely on
the Qt libraries.

I've attached a trivial implementation of Cells in Clojure. It's not
as fancy as the original CL version, but I think it gets the idea
across.

-Stuart Sierra

cells.clj

kyle smith

unread,
Sep 20, 2008, 7:54:26 PM9/20/08
to Clojure
Maybe you could attach a cell to a JComponent and call setEnabled as
necessary.

Stuart Sierra

unread,
Sep 21, 2008, 12:06:07 AM9/21/08
to clo...@googlegroups.com
Exactly. Here's some demo code, attached, that uses a cell in a
JPanel. This is a simple example, but maybe it shows how this could
be useful when you have many objects with complex dependencies.
-Stuart
square_demo.clj

Albert Cardona

unread,
Sep 21, 2008, 8:08:52 AM9/21/08
to clo...@googlegroups.com
Stuart,


> I've attached a trivial implementation of Cells in Clojure. It's not
> as fancy as the original CL version, but I think it gets the idea
> across.
>


clojure.lang.Compiler$CompilerException: cells.clj:35: Unable to resolve
symbol: sequential? in this context


What am I missing? I have latest svn clojure.

Albert

--
Albert Cardona
http://www.mcdb.ucla.edu/Research/Hartenstein/acardona

Stuart Sierra

unread,
Sep 21, 2008, 8:32:22 AM9/21/08
to Clojure
On Sep 21, 8:08 am, Albert Cardona <sapri...@gmail.com> wrote:
> clojure.lang.Compiler$CompilerException: cells.clj:35: Unable to resolve
> symbol: sequential? in this context
>
> What am I missing? I have latest svn clojure.

Hi Albert,
I'm not sure. I can't reproduce this on Clojure SVN rev. 1035.
"sequential?" has been in boot.clj since rev. 1016.
-Stuart

MikeM

unread,
Sep 21, 2008, 7:29:50 PM9/21/08
to Clojure
Thanks for doing this. I've been curious about cells for a while, and
wondered how it could be done in Clojure.

Your comment (in the other thread) about using agents inspired me to
see how this would work. I've uploaded acells.clj to the group, which
is my attempt at converting your cells implementation to use agents in
place of refs. Your examples work, but I haven't done anything with it
beyond this. I'd like to hear your comments.

Stuart Sierra

unread,
Sep 21, 2008, 11:45:56 PM9/21/08
to Clojure
Hi Mike,
This is interesting, thanks for writing it. I understand Refs better
than Agents, but even so, I'm thinking I was wrong when I first
suggested using Agents for Cells.

Pondering aloud here: The critical property of a Cell is that it is
completely finished updating before any of its dependent ("child")
cells are updated. Does this property still hold when using Agents
instead of Refs? I'm not sure it does, since an action on an Agent
occurs at an unspecified time in the future.

In the first examples I wrote, it probably doesn't matter. Here's a
more stressful example:

(def first-cell (cell 1))
(trace-cell first-cell)

(def bunch-of-cells
(doall
(take 5000
(iterate (fn [previous]
(cell (inc (cv previous))))
first-cell))))

(def last-cell (cell (inc (cv (last bunch-of-cells)))))
(trace-cell last-cell)

This works using cells.clj:
user=> (alter-cell first-cell 2)
first-cell changed from 1 to 2
last-cell changed from 5001 to 5002

But using acells.clj, you get an "Exception: Agent has errors" on (def
bunch-of-cells ...). The stack trace ends in a NullPointerException,
I think because new Cells are trying to dereference Cells that have
not yet been initialized. This is not to say definitively that Agents
won't work for Cells -- just that they may require more protections.

Incidentally, even with Refs, bunch-of-cells can't be much longer than
5000 without causing a StackOverflow. But I'm guessing a 5000+ chain
of serial dependencies is pretty rare.

-Stuart

Stuart Sierra

unread,
Sep 22, 2008, 10:31:37 AM9/22/08
to Clojure
On Sep 21, 11:45 pm, Stuart Sierra <the.stuart.sie...@gmail.com>
wrote:
> Pondering aloud here: The critical property of a Cell is that it is
> completely finished updating before any of its dependent ("child")
> cells are updated. Does this property still hold when using Agents
> instead of Refs? I'm not sure it does, since an action on an Agent
> occurs at an unspecified time in the future.

On second thought, maybe Agents will work, if one can enforce the
ordering of updates. It might be useful to have Cells that update
asynchronously. I don't know, and I don't have time to play with it
just now, but it's worth thinking about.

-Stuart

MikeM

unread,
Sep 22, 2008, 12:07:31 PM9/22/08
to Clojure

> On second thought, maybe Agents will work, if one can enforce the
> ordering of updates.  

"Actions dispatched to an agent from another single agent or thread
will occur in the order they were sent" (from clojure.org/agents) -
seems like this may be helpful, but since updates can occur from
multiple threads (the agent thread pool), it seems that the agent
cells implementation must explicitly enforce update ordering to match
the dependency relationship.

Rich Hickey

unread,
Sep 22, 2008, 12:56:53 PM9/22/08
to Clojure


On Sep 22, 12:07 pm, MikeM <michael.messini...@invista.com> wrote:
> > On second thought, maybe Agents will work, if one can enforce the
> > ordering of updates.
>
> "Actions dispatched to an agent from another single agent or thread
> will occur in the order they were sent" (from clojure.org/agents) -
> seems like this may be helpful,

It also says:

"If during the function execution any other dispatches are made
(directly or indirectly), they will be held until after the state of
the Agent has been changed."

That means that if, as a result of an update, some agent sends
messages to others, they will definitely arrive after the original
agent's state has been changed.

> but since updates can occur from
> multiple threads (the agent thread pool), it seems that the agent
> cells implementation must explicitly enforce update ordering to match
> the dependency relationship.

I don't know anything about Cells, but if it has global time ordering
it might not be a good fit for multithreading, and not for agents.

Agents provide the useful multithreaded guarantees above, if you can
architect with the concurrency multithreading implies.

The difference is, if agent A notifies B and C, and B notifies C, then
a change to A will send actions to B and C, and B will subsequently
send to C, but A's message to C and B's message to C can arrive in any
order. Once you are ok with that, you can do multithreaded reactive
programming.

I've often thought of providing change notification queues on the
agents themselves, letting Clojure do reactive programming right out
of the box.

Rich

MikeM

unread,
Sep 22, 2008, 8:57:25 PM9/22/08
to Clojure

> "If during the function execution any other dispatches are made
> (directly or indirectly), they will be held until after the state of
> the Agent has been changed."
>
> That means that if, as a result of an update, some agent sends
> messages to others, they will definitely arrive after the original
> agent's state has been changed.

This should help with proper ordering of updates to cells, since the
actions sent by an agent will, when executed, see the updated value of
the sending agent.

>
> The difference is, if agent A notifies B and C, and B notifies C, then
> a change to A will send actions to B and C, and B will subsequently
> send to C, but A's message to C and B's message to C can arrive in any
> order. Once you are ok with that, you can do multithreaded reactive
> programming.

I think this is consistent with the cells approach because the order
of notifications shouldn't matter. Each time C gets a notification it
will query A and B for their values. This is true even for Stuart's
original refs implementation (please correct me if I'm wrong about
this).

> I've often thought of providing change notification queues on the
> agents themselves, letting Clojure do reactive programming right out
> of the box.
>
This sounds like a powerful capability to have, if only to be able to
trace agent changes easily for debugging and monitoring. Would this
also be a way to attach side-effect-producing code to agents without
concern about retries?

Rich Hickey

unread,
Sep 22, 2008, 9:07:50 PM9/22/08
to Clojure
> > I've often thought of providing change notification queues on the
> > agents themselves, letting Clojure do reactive programming right out
> > of the box.
>
> This sounds like a powerful capability to have, if only to be able to
> trace agent changes easily for debugging and monitoring. Would this
> also be a way to attach side-effect-producing code to agents without
> concern about retries?

Side effects are ok in agent actions - there are no retries with
agents.

Rich

Stuart Sierra

unread,
Sep 22, 2008, 10:55:21 PM9/22/08
to clo...@googlegroups.com
Ok, here's another version of Cells, attached. This one uses Agents,
like MikeM wrote. It works even on a 10000-link chain of dependencies
-- see examples in the comment at the end. Cells get updated
asynchronously via actions on Agents. I may be missing some (dosync
...) wrappers in there.

I discovered, however, that asynchronous updates may not be what you
want! For example, my Swing demo doesn't work correctly now -- the
red square leaves behind a trail of visual artifacts, probably because
the repaint triggers are getting called in random order from multiple
threads. So maybe Refs were better, or maybe I'm using Agents
incorrectly.

-Stuart

acells.clj

mac

unread,
Sep 23, 2008, 6:42:18 AM9/23/08
to Clojure
I just started looking at this today, I think it looks very cool.
Can't say I completely understand the code yet because I just read
through it rather quickly.
But I have a question about the alter-cell function in the last
version posted by Stuart:
The last two lines of alter-cell are sends to the same agent (the cell
being operated on). In the first, a function that is to be run on
update is stored in the state of the cell ((send c# assoc :fn (fn []
~expr))) and in the second the cell is told to update itself.
Does this not mean that if several alter-cell statements execute in
parallell on the same cell then it is possible that some of the
functions set in the first send are discarded?
It could get ordered something like this in the queue of functions to
run on the agent:
(assoc ...)
(assoc ...) <-- "destroys" the :fn set by the first assoc
(update...)
(update...)
If this was not intended, a way to fix it is to make the update-cell
into a single call, perhaps taking the update function as an argument
rather than reading it from the cell state.

But like I said I've not completely understood the code so please
don't be mean to me if this was a retarded thing to point out :)

/mac
>  acells.clj
> 7KVisaHämta

Rich Hickey

unread,
Sep 23, 2008, 9:50:49 AM9/23/08
to Clojure


On Sep 22, 10:55 pm, "Stuart Sierra" <the.stuart.sie...@gmail.com>
wrote:
I've added some support for agent watches. You should consider it
experimental, but it will allow the creation of cell-like and other
reactive systems.

It takes the form of 2 functions, add-watch and remove-watch.

add-watch [#^clojure.lang.Agent a watcher callback]

Adds a watcher to an agent. Whenever the agent runs an action, any
registered watchers will have their callback function called. The
callback fn will be passed 3 args, the watcher, the agent and a
boolean
which will be true if the agent's state was (potentially) changed by
the action. The callback fn is run synchronously with the action,
and thus derefs of the agent in the callback will see the value set
during that action. Because it is run on the action thread, the
callback should not block, but can send messages.

This should make constructing things like cells much easier and more
robust.

In particular, it will work with the native agent interface, no extra
layer needed.

Try it out - feedback welcome!

Rich

Stuart Sierra

unread,
Sep 23, 2008, 10:16:11 AM9/23/08
to Clojure
Hi mac,

On Sep 23, 6:42 am, mac <markus.gustavs...@gmail.com> wrote:
> Does this not mean that if several alter-cell statements execute in
> parallell on the same cell then it is possible that some of the
> functions set in the first send are discarded?

I *think* this shouldn't matter, because only the *last* call to alter-
cell is important. I'm assuming that in normal usage, alter-cell is
only called on Cells that hold constant values, not expressions.
Cells defined with an expression shouldn't change that often.

The code doesn't make this distinction clear. It should probably
treat "value cells", which have no dependencies and don't need a fn at
all, differently from "expression cells".

-Stuart

Stuart Sierra

unread,
Sep 23, 2008, 10:47:24 AM9/23/08
to Clojure
On Sep 23, 9:50 am, Rich Hickey <richhic...@gmail.com> wrote:
> I've added some support for agent watches. You should consider it
> experimental, but it will allow the creation of cell-like and other
> reactive systems.

Cool, thanks Rich! This will be fun to play with. One question -- is
it possible to get the previous value of the agent (before the action)
from within the callback fn?

I noticed this -- if a callback throws an exception, the agent stops
accepting messages, but agent-errors remains empty:

user=> (def a1 (agent 1))
user=> @a1
1
user=> (add-watch a1 :dumb (fn [w a c] (eval 'missing-var)))
user=> (send a1 (fn [a] 2))
user=> @a1
2
user=> (send a1 (fn [a] 3))
user=> @a1
2 ;; The last action didn't run
user=> (agent-errors a1)
nil

-Stuart
Message has been deleted

Rich Hickey

unread,
Sep 23, 2008, 11:11:49 AM9/23/08
to Clojure


On Sep 23, 10:47 am, Stuart Sierra <the.stuart.sie...@gmail.com>
wrote:
> On Sep 23, 9:50 am, Rich Hickey <richhic...@gmail.com> wrote:
>
> > I've added some support for agent watches. You should consider it
> > experimental, but it will allow the creation of cell-like and other
> > reactive systems.
>
> Cool, thanks Rich! This will be fun to play with. One question -- is
> it possible to get the previous value of the agent (before the action)
> from within the callback fn?
>

It would be possible, but I'd like to hear more about why it is a good
idea. It seems to me to be a potential violation of encapsulation to
transmit state as part of a fundamental API. Many agents will/should
have a set of functions one should use to access their state, for
instance. If the main use is to see if something has changed, that's
covered by the boolean.

> I noticed this -- if a callback throws an exception, the agent stops
> accepting messages, but agent-errors remains empty:
>

I'll look into how best to handle that. I think logically it's not a
problem of the agent itself, but it shouldn't get lost either.

I've been thinking about adding error streams for agents, so an
application could have agent-related exceptions put on one or more
queues for centralized handling...

Rich

Rich Hickey

unread,
Sep 23, 2008, 11:38:42 AM9/23/08
to Clojure
I've added exceptions thrown by watchers to agent's errors, for now
(svn 1043)

Rich

Stuart Sierra

unread,
Sep 23, 2008, 12:22:58 PM9/23/08
to Clojure
On Sep 23, 11:11 am, Rich Hickey <richhic...@gmail.com> wrote:
> On Sep 23, 10:47 am, Stuart Sierra <the.stuart.sie...@gmail.com>
> wrote:
> > Cool, thanks Rich!  This will be fun to play with.  One question -- is
> > it possible to get the previous value of the agent (before the action)
> > from within the callback fn?
>
> It would be possible, but I'd like to hear more about why it is a good
> idea.

Only use I can think of is calculating deltas, e.g. "How much did this
value change? If it changed by more than X, do ..." I guess the
action itself could do that, instead.

-Stuart
Reply all
Reply to author
Forward
0 new messages