Potential issue with clojure 1.3 and build of 1.4 master from 13/12 or perhaps just my lack of understanding !

[Graham Berks]

Hi, learning clojure so this may just be a lack of understanding on my
part. Been looking at a post about swing and clojure and it used an
agent to implement a flipper with code below.

When I run the code in clojure 1.2.1 via
(def flipper (new-flipper))
(send flipper start)

all is fine and the counts increase etc.

Under 1.3 or latest 1.4 it runs out of heap, shortly after I attempt
to defref flipper and see the state I get the exception below.

Any ideas ? My lack of understanding or a bug ??

Thanks
Graham

-----------------------

AgentTest.core=> @flipper
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2427)
at java.lang.Class.privateGetPublicMethods(Class.java:2547)
at java.lang.Class.getMethods(Class.java:1410)
at clojure.lang.Reflector.getMethods(Reflector.java:375)
at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:224)
at clojure.main$repl_caught.invoke(main.clj:172)
at clojure.main$repl$fn__6020.invoke(main.clj:267)
at clojure.main$repl.doInvoke(main.clj:265)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.main$repl_opt.invoke(main.clj:331)
at clojure.main$main.doInvoke(main.clj:427)
at clojure.lang.RestFn.invoke(RestFn.java:397)
at clojure.lang.Var.invoke(Var.java:397)
at clojure.lang.AFn.applyToHelper(AFn.java:159)
at clojure.lang.Var.applyTo(Var.java:518)
at clojure.main.main(main.java:37)

------------- code ----------------

(ns AgentTest.core)

(defn new-flipper []
(agent {:total 0, :heads 0,
:running false,
:random (java.util.Random.)}))

(defn calculate [state]
(if (:running state)
(do (send *agent* calculate)
(assoc state
:total (inc (:total state))
:heads (if (.nextBoolean (:random state))
(inc (:heads state))
(:heads state))))
state))

(defn start [state]
(send *agent* calculate)
(assoc state :running true))

(defn stop [state]
(assoc state :running false))

[Stuart Sierra]

This is a problem with a feature introduced in 1.3: binding-conveyance for Agent sends. Starting in 1.3, agent actions are called with the same thread-local Var bindings that were in effect when they were called. In some situations, this can build up a stack of thread-local Var binding frames, one for every `send`.

I've seen this problem before, but this is a particularly nice demonstration of it. Don't have a solution yet.

-S

[Tassilo Horn]

Stuart Sierra <the.stua...@gmail.com> writes:

Hi Stuart,

> This is a problem with a feature introduced in 1.3: binding-conveyance
> for Agent sends. Starting in 1.3, agent actions are called with the
> same thread-local Var bindings that were in effect when they were
> called. In some situations, this can build up a stack of thread-local
> Var binding frames, one for every `send`.

But shouldn't at least the thread-local Var binding frames be freed
after the actions have been performed?

I've ran the OP's code using 1.3.0, waited till the JVM used up about
1.7GiB, then called (send flipper stop), but the allocated memory stays
constantly at the 1.7GiB (and even increased to 1.8 after running
(System/gc)). So it seems like the bindings are still referenced from
somewhere...

Bye,
Tassilo

[Stuart Sierra]

Tassilo wrote: "But shouldn't at least the thread-local Var binding frames be freed after the actions have been performed?"

Yes. It's a bug. Just created CLJ-898: http://dev.clojure.org/jira/browse/CLJ-898

-S

[Tassilo Horn]

Stuart Sierra <the.stua...@gmail.com> writes:

Hi Stuart,

> Tassilo wrote: "But shouldn't at least the thread-local Var binding

I've attached a patch to the issue which at least works for me and
doesn't make any existing tests fail. That's my first encounter with
clojure internals, so I'm not completely sure if it's the right thing to
do. What you cannot do anymore in an agent action is popping back to
any previous thread-binding, but I guess you shouldn't do so anyway...

Bye,
Tassilo