Lazyness and dynamic binding

[David Powell]

Hi,

The interactions between laziness and dynamic-binding are a bit
unpleasant.

It effectively means that if you write some function that returns a
(nested) data-structure, created using higher-order functions, such as
for and map, then you probably shouldn't use dynamic bindings anywhere
in that function, as the bindings will have gone out of scope by the
time the lazy sequence gets realised when the client iterates it.

There is a good description of the problem here:

http://kotka.de/blog/2009/11/Taming_the_Bound_Seq.html

The solution is interesting, but I don't think it will work in
situations with nested lazy-seqs.

Is this situation ever likely to be improved? Or should people just
avoid using dynamic bindings? It is hard to avoid laziness as it is
built into things like map/filter/for.

--
Dave

[Stuart Sierra]

Look at scopes, a potential solution to this problem:

http://dev.clojure.org/display/design/Resource+Scopes

[David Powell]

Stuart Sierra wrote:

> Look at scopes, a potential solution to this problem:
> http://dev.clojure.org/display/design/Resource+Scopes

I was wondering whether scopes might help the situation, but I didn't
see any specific mention of dynamic binding in the rationale, and I
don't know enough about them to understand whether they will help with
it.

If scopes are a solution to this problem, then can the rationale be
updated to include it - or if they aren't should we start a new dev
page for this problem?

Btw - as an experiment, I just tried hacking LazySeq to use the new
binding frame stuff, and to record the binding frame when the LazySeq
is constructed, and to temporarily install the frame when the
underlying data is realised.

(I guess that this adds some overhead. Though perhaps the overhead
could be minimised if there was some way to detect when the underlying
seq was chunked, and to only reinstall the bindings when when new
chunks were realised, tho I don't quite understand how lazyness and
chunking work together...)

--
Dave

[David Powell]

> Btw - as an experiment, I just tried hacking LazySeq to use the new
> binding frame stuff, and to record the binding frame when the LazySeq
> is constructed, and to temporarily install the frame when the
> underlying data is realised.

Well, I'm not sure whether scopes will help with this problem or not,
but I thought I'd post my experiment anyway: Attached.

Is installing the lazy-seq's bindings during the generation of the
sequence items better? Is it more intuitive? Does it have any sharp
edges? Is it too slow?

Is it safe to use Var.resetThreadBindingFrame in this way?

Interestingly, when the underlying sequence is chunked, the binding
only has to be installed once per chunk. And presumably the new
binding frames are pretty cheap to install.

--
Dave