[groovy-user] break in closure

27 views
Skip to first unread message

Francesco Pasqualini

unread,
Aug 4, 2008, 4:23:48 AM8/4/08
to us...@groovy.codehaus.org
What is the status of this proposal ?

http://docs.codehaus.org/display/GroovyJSR/break+in+closure

thanks

On Tue, Jul 8, 2008 at 10:20 PM, phil swenson <phil.s...@gmail.com> wrote:
I need a function that returns if a directory has any files in it (directories don't count).  Here's my take:

def noFilesInDirectory(dir){
def file = new File(dir)
def i = 0
file.eachFileRecurse(){
    if (it.isFile()){
        i++
    }
}
return i == 0
}

I think there is a better way... what is it?

Guillaume Laforge

unread,
Aug 4, 2008, 4:35:50 AM8/4/08
to us...@groovy.codehaus.org
Currently stalled.
Which doesn't mean we won't revisit it at some point.

--
Guillaume Laforge
Groovy Project Manager
G2One, Inc. Vice-President Technology
http://www.g2one.com

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


Francesco Pasqualini

unread,
Aug 4, 2008, 4:46:00 AM8/4/08
to us...@groovy.codehaus.org
i think it's very important due to the heavy use in Groovy of clodure in loops.
Groovy SQL, FILE, and many other libraries... it seems

thanks

Mingfai

unread,
Aug 4, 2008, 11:41:23 AM8/4/08
to us...@groovy.codehaus.org
i second this. supporting break for closure should be in high priority. which JIRA issue refer to it?

Jochen Theodorou

unread,
Aug 4, 2008, 12:31:08 PM8/4/08
to us...@groovy.codehaus.org
Mingfai schrieb:

> i second this. supporting break for closure should be in high priority.
> which JIRA issue refer to it?

I think there are several... haven't looked at it. There were several
huge discussions about how to implement break/continue in closures. We
didn't implement it till today, because we where not satisfied with the
solutions we found. Some solutions simply didn't work in a multi
threading environment (like using fields), others are slow (exceptions,
even those preallocated ones). We also wanted to wait a bit and see how
the closure proposal for Java develops.

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

Charles Oliver Nutter

unread,
Aug 4, 2008, 7:27:50 PM8/4/08
to us...@groovy.codehaus.org
Jochen Theodorou wrote:
> Mingfai schrieb:
>> i second this. supporting break for closure should be in high
>> priority. which JIRA issue refer to it?
>
> I think there are several... haven't looked at it. There were several
> huge discussions about how to implement break/continue in closures. We
> didn't implement it till today, because we where not satisfied with the
> solutions we found. Some solutions simply didn't work in a multi
> threading environment (like using fields), others are slow (exceptions,
> even those preallocated ones). We also wanted to wait a bit and see how
> the closure proposal for Java develops.

Exception-based flow control is only slow if you use it. And the keep to
keeping it from being "too slow" is to shrink stack size down a lot, so
that there's at least a change for the thrower and catcher to be inlined
together. That may be impossible to do with Groovy at present since
reflection adds so many stack frames. But I doubt a moderately slow
non-local flow option would be worse than none at all.

- Charlie

Randall R Schulz

unread,
Aug 4, 2008, 7:43:04 PM8/4/08
to us...@groovy.codehaus.org
On Monday 04 August 2008 16:27, Charles Oliver Nutter wrote:
> ...

>
> Exception-based flow control is only slow if you use it. And the keep
> to keeping it from being "too slow" is to shrink stack size down a
> lot, so that there's at least a change for the thrower and catcher to
> be inlined together.

Don't forget that the biggest cost in exception instance creation is
{@link Throwable#fillInStackTrace}. There's a well-known (and fairly
obvious) trick to avoiding this overhead when the stack trace is of no
interest and / or the cost of creating it is excessive: Define a
Throwable derivative that overrides this method with a no-op method.


> ...
>
> - Charlie


Randall Schulz

Jochen Theodorou

unread,
Aug 4, 2008, 8:39:13 PM8/4/08
to us...@groovy.codehaus.org
Randall R Schulz schrieb:

> On Monday 04 August 2008 16:27, Charles Oliver Nutter wrote:
>> ...
>>
>> Exception-based flow control is only slow if you use it. And the keep
>> to keeping it from being "too slow" is to shrink stack size down a
>> lot, so that there's at least a change for the thrower and catcher to
>> be inlined together.
>
> Don't forget that the biggest cost in exception instance creation is
> {@link Throwable#fillInStackTrace}. There's a well-known (and fairly
> obvious) trick to avoiding this overhead when the stack trace is of no
> interest and / or the cost of creating it is excessive: Define a
> Throwable derivative that overrides this method with a no-op method.

Charles is very well aware of that. What we (Charles and myself) are
talking about are not the obvious performance tricks you can achieve by
reusing exceptions and avoid the stack trace being filled.What Charles
is talking about is, is about inlining. Because things only become
really fast when JIT is able to inline. Exception in Java can be inlined
on certain circumstances. But for this the number of stack frames must
be as low as possible.

bye blackdrag


--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------

Charles Oliver Nutter

unread,
Aug 4, 2008, 11:22:24 PM8/4/08
to us...@groovy.codehaus.org
Jochen Theodorou wrote:
> Charles is very well aware of that. What we (Charles and myself) are
> talking about are not the obvious performance tricks you can achieve by
> reusing exceptions and avoid the stack trace being filled.What Charles
> is talking about is, is about inlining. Because things only become
> really fast when JIT is able to inline. Exception in Java can be inlined
> on certain circumstances. But for this the number of stack frames must
> be as low as possible.

FWIW, the magic number is 9 on HotSpot. Reflection tends to take up at
least half of that.

- Charlie

Jochen Theodorou

unread,
Aug 5, 2008, 6:52:28 AM8/5/08
to us...@groovy.codehaus.org
Charles Oliver Nutter schrieb:

> Jochen Theodorou wrote:
>> Charles is very well aware of that. What we (Charles and myself) are
>> talking about are not the obvious performance tricks you can achieve
>> by reusing exceptions and avoid the stack trace being filled.What
>> Charles is talking about is, is about inlining. Because things only
>> become really fast when JIT is able to inline. Exception in Java can
>> be inlined on certain circumstances. But for this the number of stack
>> frames must be as low as possible.
>
> FWIW, the magic number is 9 on HotSpot. Reflection tends to take up at
> least half of that.

well, in case of "each" we can do it without reflection now. That leaves
maybe 2 layers in the closure itself, another 3 (?) in the call
site... well that should work in 1.6. In 1.5 it won't, because the call
would contain at last 2 reflective calls, which means the stack is
already breaking the length of 9

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------

Alexander Veit

unread,
Aug 5, 2008, 9:42:15 AM8/5/08
to us...@groovy.codehaus.org
> Charles Oliver Nutter wrote:
> Exception-based flow control is only slow if you use it. And
> the keep to keeping it from being "too slow" is to shrink
> stack size down a lot, so that there's at least a change
> for the thrower and catcher to be inlined together. That
> may be impossible to do with Groovy at present since
> reflection adds so many stack frames. But I doubt a moderately slow
> non-local flow option would be worse than none at all.

Isn't it inappropriate to use Exceptions and the break(+) keyword to
terminate an iteration?

The only advantage that I could see at the moment is that, if break in
closures would throw a BreakNotSupportedException to indicate the end of
iteration, existing code that's not aware of breaks in closures would fail
with a dedicated exception.

However, personally I would prefer a solution like

public final class ITERATION {
public static final Object END = new ITERATION();
private ITERATION() {}
}

that could be used within closures

{
// ...
return ITERATION.END
// ...
}

to stop the iteration.

Regards,
Alex

(+) what about switch-case in closures?

Jochen Theodorou

unread,
Aug 5, 2008, 9:51:33 AM8/5/08
to us...@groovy.codehaus.org
Alexander Veit schrieb:

>> Charles Oliver Nutter wrote:
>> Exception-based flow control is only slow if you use it. And
>> the keep to keeping it from being "too slow" is to shrink
>> stack size down a lot, so that there's at least a change
>> for the thrower and catcher to be inlined together. That
>> may be impossible to do with Groovy at present since
>> reflection adds so many stack frames. But I doubt a moderately slow
>> non-local flow option would be worse than none at all.
>
> Isn't it inappropriate to use Exceptions and the break(+) keyword to
> terminate an iteration?

if you have a better idea, let's hear it. The problem is that we need to
make a jump, a jump across stack frames... this is not supported by the JVM

> The only advantage that I could see at the moment is that, if break in
> closures would throw a BreakNotSupportedException to indicate the end of
> iteration, existing code that's not aware of breaks in closures would fail
> with a dedicated exception.
>
> However, personally I would prefer a solution like
>
> public final class ITERATION {
> public static final Object END = new ITERATION();
> private ITERATION() {}
> }
>
> that could be used within closures
>
> {
> // ...
> return ITERATION.END
> // ...
> }
>
> to stop the iteration.

the problem is, that this is not thread safe. if the same closure is
used by multiple threads, then it is possible that one Thread will
overwrite the value in a way, which is invalid for another Thread.

> (+) what about switch-case in closures?

a break inside a switch-case inside a closure behaves like if it is not
inside the switch-case. You can also use loops inside the closure, which
do use break/continue without problem Only when you call methods like
each, find, findAll, grep and others we have a problem.

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------

Francesco Pasqualini

unread,
Aug 5, 2008, 9:59:18 AM8/5/08
to us...@groovy.codehaus.org
what about use some special value passed back from the closure to the caller loop ?
something like
   it.eof =true 
   return

or it.next = false

Alexander Veit

unread,
Aug 5, 2008, 10:21:05 AM8/5/08
to us...@groovy.codehaus.org
> Jochen Theodorou wrote

> > public final class ITERATION {
> > public static final Object END = new ITERATION();
> > private ITERATION() {}
> > }
> >
> > that could be used within closures
> >
> > {
> > // ...
> > return ITERATION.END
> > // ...
> > }
> >
> > to stop the iteration.
>
> the problem is, that this is not thread safe. if the same closure is
> used by multiple threads, then it is possible that one Thread will
> overwrite the value in a way, which is invalid for another Thread.

Hmm, I don't understand that. ITERATION.END is a singleton instance that's
being returned by the closure.

If Closure itself is stateful (how?), then an iteration method that supports
'break' should probably clone the closure before calling it.

-Alex

Jochen Theodorou

unread,
Aug 5, 2008, 10:31:07 AM8/5/08
to us...@groovy.codehaus.org
Francesco Pasqualini schrieb:

> what about use some special value passed back from the closure to the
> caller loop ?
> something like
> it.eof =true
> return
>
> or it.next = false

that would work for each, but not for example for findAll or grep

Jochen Theodorou

unread,
Aug 5, 2008, 10:56:18 AM8/5/08
to us...@groovy.codehaus.org
Alexander Veit schrieb:

>> Jochen Theodorou wrote
>>> public final class ITERATION {
>>> public static final Object END = new ITERATION();
>>> private ITERATION() {}
>>> }
>>>
>>> that could be used within closures
>>>
>>> {
>>> // ...
>>> return ITERATION.END
>>> // ...
>>> }
>>>
>>> to stop the iteration.
>> the problem is, that this is not thread safe. if the same closure is
>> used by multiple threads, then it is possible that one Thread will
>> overwrite the value in a way, which is invalid for another Thread.
>
> Hmm, I don't understand that. ITERATION.END is a singleton instance that's
> being returned by the closure.

yes, sorry I did misread it... But still, how would you use that in findAll?

> If Closure itself is stateful (how?), then an iteration method that supports
> 'break' should probably clone the closure before calling it.

I think there are two kinds of state... ones that belong to all usages
of this closure, like the resolving strategy and others like delegate.
Only... do we want to "always" clone a closure? Isn't that as much of a
problem (performance wise) as using an exception?Well a test shows, that
it makes no big difference.

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------

Alexander Veit

unread,
Aug 5, 2008, 2:41:09 PM8/5/08
to us...@groovy.codehaus.org
> Jochen Theodorou wrote:
> [...]

> yes, sorry I did misread it... But still, how would you use
> that in findAll?

e.g.

public static Collection findAll(Collection self, Closure closure) {
Collection answer = createSimilarCollection(self);
for (Iterator iter = self.iterator(); iter.hasNext();) {
final Object value = iter.next();
final Object ret = closure.call(value);
if (ITERATION.END == ret) {
break;
}
if (DefaultTypeTransformation.castToBoolean(ret)) {
answer.add(value);
}
}
return answer;
}

> > If Closure itself is stateful (how?), then an iteration
> > method that supports 'break' should probably clone
> > the closure before calling it.
>
> I think there are two kinds of state... ones that belong to
> all usages of this closure, like the resolving strategy
> and others like delegate.
> Only... do we want to "always" clone a closure? Isn't that as
> much of a problem (performance wise) as using an exception?
> Well a test shows, that it makes no big difference.

OK, as long as code does not change these (resolving strategy, delegate,
properties), it does not need to create a clone of the closure. If custom
code breaks this rule, it has to deal with the consequences to it's closures
by itself.

-Alex

Charles Oliver Nutter

unread,
Aug 5, 2008, 2:47:05 PM8/5/08
to us...@groovy.codehaus.org
Jochen Theodorou wrote:
> Charles Oliver Nutter schrieb:
>> Jochen Theodorou wrote:
>>> Charles is very well aware of that. What we (Charles and myself) are
>>> talking about are not the obvious performance tricks you can achieve
>>> by reusing exceptions and avoid the stack trace being filled.What
>>> Charles is talking about is, is about inlining. Because things only
>>> become really fast when JIT is able to inline. Exception in Java can
>>> be inlined on certain circumstances. But for this the number of stack
>>> frames must be as low as possible.
>>
>> FWIW, the magic number is 9 on HotSpot. Reflection tends to take up at
>> least half of that.
>
> well, in case of "each" we can do it without reflection now. That leaves
> maybe 2 layers in the closure itself, another 3 (?) in the call site...
> well that should work in 1.6. In 1.5 it won't, because the call would
> contain at last 2 reflective calls, which means the stack is already
> breaking the length of 9

The call to each is reflected though, isn't it? So this would only be
useful for flow control that remains inside a Java-based method and the
closure it invokes I guess.

Note another case I think is more important than break: non-local return.

- Charlie

Charles Oliver Nutter

unread,
Aug 5, 2008, 2:52:36 PM8/5/08
to us...@groovy.codehaus.org
Alexander Veit wrote:
> Isn't it inappropriate to use Exceptions and the break(+) keyword to
> terminate an iteration?
>
> The only advantage that I could see at the moment is that, if break in
> closures would throw a BreakNotSupportedException to indicate the end of
> iteration, existing code that's not aware of breaks in closures would fail
> with a dedicated exception.

A non-local break needs to unroll all the way back to some appropriate
target. The easiest way to do that across many calls is with an exception.

> However, personally I would prefer a solution like
>
> public final class ITERATION {
> public static final Object END = new ITERATION();
> private ITERATION() {}
> }
>
> that could be used within closures
>
> {
> // ...
> return ITERATION.END
> // ...
> }
>
> to stop the iteration.

If you had two nested iterations, how would you break the outer one?
Also, this adds the overhead of having to check the return value from
every iteration *always* because at any time someone might return the
special return value. Exception-handling would only have to wrap the
entire iteration, catching the break exception on the way out and
terminating iteration then.

A special return value would work for some limited cases, but it would
always add overhead to every iteration and it isn't easy to target it to
a specific closure or iteration.

- Charlie

Jochen Theodorou

unread,
Aug 5, 2008, 3:05:37 PM8/5/08
to us...@groovy.codehaus.org
Charles Oliver Nutter schrieb:
> Jochen Theodorou wrote:
>> Charles Oliver Nutter schrieb:
>>> Jochen Theodorou wrote:
>>>> Charles is very well aware of that. What we (Charles and myself) are
>>>> talking about are not the obvious performance tricks you can achieve
>>>> by reusing exceptions and avoid the stack trace being filled.What
>>>> Charles is talking about is, is about inlining. Because things only
>>>> become really fast when JIT is able to inline. Exception in Java can
>>>> be inlined on certain circumstances. But for this the number of
>>>> stack frames must be as low as possible.
>>>
>>> FWIW, the magic number is 9 on HotSpot. Reflection tends to take up
>>> at least half of that.
>>
>> well, in case of "each" we can do it without reflection now. That
>> leaves maybe 2 layers in the closure itself, another 3 (?) in the
>> call site... well that should work in 1.6. In 1.5 it won't, because
>> the call would contain at last 2 reflective calls, which means the
>> stack is already breaking the length of 9
>
> The call to each is reflected though, isn't it?

methods from DefaultGroovyMethods don't need to be reflective. They are
transformed directly in MetaMethods, which means only one additional
layer is required (the call from the MetaMethod to the real method,
which is not done by reflection, but by generated code). For example
this script:

[1].each {(new Exception("each!!")).}

gives this trace:

> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[...]
> at test$_run_closure1.doCall(test.groovy:141)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
> at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:234)
> at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:271)
> at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:845)
> at groovy.lang.Closure.call(Closure.java:292)
> at groovy.lang.Closure.call(Closure.java:305)
> at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1080)
> at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1057)
> at org.codehaus.groovy.runtime.dgm$84.invoke(Unknown Source)
> [...]

this shows there is dgm$84, which represents the each method wrapper,
then there is the each method itself and about 12 frames later the
doCall method of the closure... That is of course still too long, but if
we get the path shortened a little, then it should be possible. The call
to the dgm method itself looks like this:

> at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
> at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:51)
> at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:41)
> at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
> at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
> at test.run(test.groovy:141)

which is of course too long too. Reflection takes 4 frames, our call
sites 5 frames... this should be reduced to 2 frames in best case.

> So this would only be
> useful for flow control that remains inside a Java-based method and the
> closure it invokes I guess.
>
> Note another case I think is more important than break: non-local return.

well, yes the return is a class of its own. I guess a simple return
value won't do it here ;)

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------

Alexander Veit

unread,
Aug 5, 2008, 4:02:01 PM8/5/08
to us...@groovy.codehaus.org
> Charles Oliver Nutter wrote:
> If you had two nested iterations, how would you break the outer one?

- How could the compiler decide how many iteration levels are to be unwinded
without an explicit label (e.g. a catch block) in at least one of the outer
iteration loops?

- What is the return value from nested iterations that are breaked by an
exception? I think it would be rather unclear and probably dependant on
implementation details of the iteration methods.

- If many iterations are to be unwinded what about the ~9-stackframe
problem? ;)

I think it would be the most natural way to only break the innermost
iteration level.

> Also, this adds the overhead of having to check the return value from
> every iteration *always* because at any time someone might return the

> special return value. [...]

That's true, but is an identity comparison really more overhead the a
try-catch-block? I think it would work with comparable performance on any
JVM.

-Alex

Alex Tkachman

unread,
Aug 5, 2008, 4:06:58 PM8/5/08
to us...@groovy.codehaus.org
5 frames on 1st invokaction of call site but only 1 unneeded on every
next (may be 2 but as far as I remember only 1)

Jochen Theodorou

unread,
Aug 5, 2008, 4:16:20 PM8/5/08
to us...@groovy.codehaus.org
Alex Tkachman schrieb:

> 5 frames on 1st invokaction of call site but only 1 unneeded on every
> next (may be 2 but as far as I remember only 1)

ah sorry ;)

Jochen Theodorou

unread,
Aug 5, 2008, 4:26:22 PM8/5/08
to us...@groovy.codehaus.org
Alexander Veit schrieb:

>> Charles Oliver Nutter wrote:
>> If you had two nested iterations, how would you break the outer one?
>
> - How could the compiler decide how many iteration levels are to be unwinded
> without an explicit label (e.g. a catch block) in at least one of the outer
> iteration loops?

you could store and count the levels in a ThreadLocal ;) No, for break
you can let the compiler decorate the information you need. Continue is
a bit more difficult of course

> - What is the return value from nested iterations that are breaked by an
> exception? I think it would be rather unclear and probably dependant on
> implementation details of the iteration methods.

you have the same problem with a normal iteration, or not? Just think of
findAll, which usually returns something.

> - If many iterations are to be unwinded what about the ~9-stackframe
> problem? ;)

it becomes even bigger ;)

> I think it would be the most natural way to only break the innermost
> iteration level.

you say it so easily... imagine a list of list and made an iteration method:

def myIter(listOfList,closure) {
listOfList.each { it.each(closure) }
}

myIter(list) {
if (it==5) break
[...]
}

now, what is this break addressing? From the looks of it, it should
break out of the iteration defined by myIter, but myIter is actually two
iterations, which is invisible for the user. So breaking the inner most
is not as natural as you may think, since using such blocks/closures
allows you to move the inner part. It is not as lexical as in Java. And
I really would like this break instruction to break out of the iteration
defined by myIter, not only the it.each in the implementation

>> Also, this adds the overhead of having to check the return value from
>> every iteration *always* because at any time someone might return the
>> special return value. [...]
>
> That's true, but is an identity comparison really more overhead the a
> try-catch-block? I think it would work with comparable performance on any
> JVM.

depends... a try-catch block can be optimized away completely... I am
almost sure the same could be done for the if, but the states are
changing for each iteration

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------

Francesco Pasqualini

unread,
Aug 5, 2008, 6:27:57 PM8/5/08
to us...@groovy.codehaus.org
What about a syntax to specify 'while' 'until' and 'skip' closures by named parameters ?
 
file.eachFileRecurse( while: {it.isFile  }, skip: {it.size < 10}, until: { it.executable }    ){
  println it.name
}
 
the example is just to illustrate the syntax.

Francesco Pasqualini

unread,
Aug 5, 2008, 6:57:18 PM8/5/08
to us...@groovy.codehaus.org
the 'until' closure is tested after the body closure
 
 
file.eachFileRecurse( while: {it.isFile  }, skip: {it.size < 10}, until: { it.found }    ){
   println it.name;
   if(it.name='test'){
     it.found = true;
     return
   }
   [.....]

Charles Oliver Nutter

unread,
Aug 5, 2008, 7:03:58 PM8/5/08
to us...@groovy.codehaus.org
Jochen Theodorou wrote:
> methods from DefaultGroovyMethods don't need to be reflective. They are
> transformed directly in MetaMethods, which means only one additional
> layer is required (the call from the MetaMethod to the real method,
> which is not done by reflection, but by generated code). For example
> this script:
>
> [1].each {(new Exception("each!!")).}
>
> gives this trace:

Here's a similar trace in JRuby, calling through each and a closure to a
Thread.dumpStack (I've removed the frames relating to the Java
call...more on that in a moment):

➔ jruby -rjava -e "[1,2,3].each { java.lang.Thread.dumpStack }"
...
at ruby.__dash_e__.block_0$RUBY$__block__(-e:1)
at ruby.__dash_e__BlockCallback$block_0$RUBY$__block__xx1.call(Unknown
Source)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:100)
at org.jruby.runtime.Block.yield(Block.java:100)
at org.jruby.RubyArray.each(RubyArray.java:1410)
at
org.jruby.RubyArray$i_method_0_0$RUBYFRAMEDINVOKER$each.call(org/jruby/RubyArray$i_method_0_0$RUBYFRAMEDINVOKER$each.gen)
at
org.jruby.runtime.CallSite$InlineCachingCallSite.cacheAndCall(CallSite.java:135)
at
org.jruby.runtime.CallSite$InlineCachingCallSite.callIter(CallSite.java:364)
at ruby.__dash_e__.__file__(-e:1)

Inside the __block__ method is where a non-local return exception would
be fired, since it's instantiated directly from bytecode there. The
interpreted trace is longer, but not substantially so in most cases
(only one extra level deep for each nesting in the AST).

Note that from __file__ which represents the body of the script there's
only three intermediate frames to get to the actual each logic. Not
quite as short as the one or two frames in Groovy, but it's important to
note there's no custom logic or hand-written code like your
DefaultGroovyMethods.each involved here; every Ruby method has about the
same frame load. The closure invocation path is the same...only three
frames from each to the actual closure body. Again, this will vary, but
it's pretty tight.

Of course the Java integration layer is somewhat different. Here's the
missing bits from the Thread.dumpStack call:

java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Thread.java:1176)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.jruby.javasupport.JavaMethod.invokeWithExceptionHandling(JavaMethod.java:292)
at org.jruby.javasupport.JavaMethod.invoke_static(JavaMethod.java:272)
at
org.jruby.javasupport.methods.StaticMethodInvoker.call(StaticMethodInvoker.java:38)
at org.jruby.runtime.CallSite$InlineCachingCallSite.call(CallSite.java:333)
at ruby.__dash_e__.block_0$RUBY$__block__(-e:1)

(This is a call path I've recently improved a lot...it used to be a lot
deeper)

Because we also use reflection for invoking Java methods, we have a
similar stack depth issue to Groovy when calling from Ruby to Java.
Here, from the __block__ method to the dumpStack method we have 8
intermediate frames; 4 of those are reflection, 1 is the inline cache,
and 3 are plumbing for JRuby. We could probably trim that down to 1
layer of plumbing, but reflection pretty much kills us here.
invokedynamic's method handles or my own handle generator should be a
big help.

Luckily, most people using non-local flow control have not been doing it
across Java calls, so frequently we do get the "fast" version of
exception-based flow. But when I say that it's slower in some cases,
we're still not talking that slow...here's a benchmark of a non-local
return in JRuby versus just letting the return value "fall out":

Non-Local Return:
➔ jruby --server -rbenchmark -e "def foo; 1.times { return }; end;
5.times { puts Benchmark.measure { 1_000_000.times { foo } } }"
1.840000 0.000000 1.840000 ( 1.840454)
1.691000 0.000000 1.691000 ( 1.691069)
1.609000 0.000000 1.609000 ( 1.609168)
1.605000 0.000000 1.605000 ( 1.604125)
1.609000 0.000000 1.609000 ( 1.609253)
"Fall Out" Return:
➔ jruby --server -rbenchmark -e "def foo; 1.times { }; end; 5.times {
puts Benchmark.measure { 1_000_000.times { foo } } }"
0.792000 0.000000 0.792000 ( 0.791247)
0.633000 0.000000 0.633000 ( 0.632990)
0.581000 0.000000 0.581000 ( 0.581429)
0.578000 0.000000 0.578000 ( 0.578351)
0.581000 0.000000 0.581000 ( 0.580972)

Divided by the million invocations happening here, that's an additional
performance hit of something like 1 microsecond per iteration. Seems
like it's probably not such a big cost.

- Charlie

Charles Oliver Nutter

unread,
Aug 5, 2008, 7:09:30 PM8/5/08
to us...@groovy.codehaus.org
Alexander Veit wrote:
> - How could the compiler decide how many iteration levels are to be unwinded
> without an explicit label (e.g. a catch block) in at least one of the outer
> iteration loops?

A simple strategy is to construct the break exception object when
calling a method that takes a closure that contains a break; then you
just catch that break exception and make sure it's yours.

> - What is the return value from nested iterations that are breaked by an
> exception? I think it would be rather unclear and probably dependant on
> implementation details of the iteration methods.

In Ruby, it depends on the method, but usually a break causes the method
under iteration to return that value.

[1,2,3].each {|x| break 'foo' if x == 2} # => returns 'foo'

> - If many iterations are to be unwinded what about the ~9-stackframe
> problem? ;)

That wouldn't be improved by a special return value, and you'd have more
conditional logic in the mix to slow things down further.

> That's true, but is an identity comparison really more overhead the a
> try-catch-block? I think it would work with comparable performance on any
> JVM.

If you're iterating over a million elements...yes. Maybe even if you're
iterating over a thousand or a hundred elements, especially if you're
not using a non-local jump. But checking the return value each time
penalizes even iterations that don't do non-local jumps, slowing down
every iteration.

- Charlie

Charles Oliver Nutter

unread,
Aug 5, 2008, 7:10:26 PM8/5/08
to us...@groovy.codehaus.org
Charles Oliver Nutter wrote:
> ➔ jruby -rjava -e "[1,2,3].each { java.lang.Thread.dumpStack }"
> ...
> at ruby.__dash_e__.block_0$RUBY$__block__(-e:1)
> at
> ruby.__dash_e__BlockCallback$block_0$RUBY$__block__xx1.call(Unknown Source)
> at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:100)
> at org.jruby.runtime.Block.yield(Block.java:100)
> at org.jruby.RubyArray.each(RubyArray.java:1410)
> at
> org.jruby.RubyArray$i_method_0_0$RUBYFRAMEDINVOKER$each.call(org/jruby/RubyArray$i_method_0_0$RUBYFRAMEDINVOKER$each.gen)
>
> at
> org.jruby.runtime.CallSite$InlineCachingCallSite.cacheAndCall(CallSite.java:135)
>
> at
> org.jruby.runtime.CallSite$InlineCachingCallSite.callIter(CallSite.java:364)
>
> at ruby.__dash_e__.__file__(-e:1)

It occurred to me after Alex's email that I included the cache frame
here. So typically it would go from 2 frames for inline cache to 1
frame, plus the method handle.

Charles Oliver Nutter

unread,
Aug 5, 2008, 7:14:52 PM8/5/08
to us...@groovy.codehaus.org
Jochen Theodorou wrote:
> Alexander Veit schrieb:

>> - How could the compiler decide how many iteration levels are to be
>> unwinded
>> without an explicit label (e.g. a catch block) in at least one of the
>> outer
>> iteration loops?
>
> you could store and count the levels in a ThreadLocal ;) No, for break
> you can let the compiler decorate the information you need. Continue is
> a bit more difficult of course

"next" in Ruby is just a local return from the closure in almost all cases.

>> - What is the return value from nested iterations that are breaked by an
>> exception? I think it would be rather unclear and probably dependant on
>> implementation details of the iteration methods.
>
> you have the same problem with a normal iteration, or not? Just think of
> findAll, which usually returns something.

It also eliminates future possibilities of specifying more specific
return types on Groovy code since it would always have to be able to
return this custom return object.

>> That's true, but is an identity comparison really more overhead the a
>> try-catch-block? I think it would work with comparable performance on any
>> JVM.
>
> depends... a try-catch block can be optimized away completely... I am
> almost sure the same could be done for the if, but the states are
> changing for each iteration

The try/catch can almost be optimized away; there's still framing and
flow guarantees it makes that can impact performance. But in this case
we're talking about one try/catch wrapping the entire iteration. It
would be little more than noise.

The identity check could be optimized away only if it were never used.
The minute someone passes a special return value through that check,
hotspot (for example) would have to deoptimize and start running the checks.

- Charlie

Jochen Theodorou

unread,
Aug 5, 2008, 7:29:07 PM8/5/08
to us...@groovy.codehaus.org
Charles Oliver Nutter schrieb:

> Alexander Veit wrote:
>> - How could the compiler decide how many iteration levels are to be
>> unwinded
>> without an explicit label (e.g. a catch block) in at least one of the
>> outer iteration loops?
>
> A simple strategy is to construct the break exception object when
> calling a method that takes a closure that contains a break; then you
> just catch that break exception and make sure it's yours.

which is not so nice in our case, since you do want to define your own
iteration methods, but not handle the implementation detail exception

>> - What is the return value from nested iterations that are breaked by an
>> exception? I think it would be rather unclear and probably dependant on
>> implementation details of the iteration methods.
>
> In Ruby, it depends on the method, but usually a break causes the method
> under iteration to return that value.
>
> [1,2,3].each {|x| break 'foo' if x == 2} # => returns 'foo'

it is a break with a "return value". This was suggested for Groovy too...

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------

Charles Oliver Nutter

unread,
Aug 5, 2008, 8:28:36 PM8/5/08
to us...@groovy.codehaus.org
Jochen Theodorou wrote:
> which is not so nice in our case, since you do want to define your own
> iteration methods, but not handle the implementation detail exception

In Ruby this is mitigated somewhat because most non-local flow control
appears syntactically in the same block of code as the place it should
jump to:

def foo
ary.each do |x|
return x
end
end

It's easy when compiling the body of "foo" to construct a return
exception the eventual non-local return will use; you then provide this
to the closure you've constructed or on the closure's set of heap
variables (perhaps as a "secret" variable) and the eventual return
throws it back to its creator.

We don't use this technique in JRuby at present, because there are some
other weird cases where you can pass in a closure to use for each, and
break is expected to work as though it lived in that method. There's
some confusion about those more complex cases in Ruby at the moment, but
for now we essentially use a marker on our heap-allocated frames to
indicate where a jump is supposed to go.

I believe we could eliminate most of that complication/confusion with a
combination of better static analysis of contained closures and more
runtime flow analysis to show whether such jumps are actually happening
or not.

- Charlie

Alexander Veit

unread,
Aug 6, 2008, 4:41:14 PM8/6/08
to us...@groovy.codehaus.org
Hi,

Sorry for my late reply.

> Jochen Theodorou wrote:
> Alexander Veit schrieb:
> >> Charles Oliver Nutter wrote:
> >> If you had two nested iterations, how would you break the
> outer one?
> >
> > - How could the compiler decide how many iteration levels
> > are to be unwinded without an explicit label (e.g.
> > a catch block) in at least one of the outer iteration
> > loops?
>
> you could store and count the levels in a ThreadLocal ;) No,
> for break you can let the compiler decorate the information
> you need. Continue is a bit more difficult of course

1. Yes, the compiler could create decorations. But how can it decide where
to put it into the bytecode?

- Can it place the exception handler into the closure's body? No.
- Can it place the exception handler into the code that calls the iteration
function (e.g. findAll)? No, since it would only work for iteration
functions that have no return values. Return values would fall victim to
stack unwinding.

So the exception handler has to be placed inside the body of an iteration
function. Since more than one level of iterations is to be supported, it
cannot merely try to decorate/modify existing bytecode, but has to somehow
copy it, infer it's semantics, decorate it, and inline the decorated code. I
wonder if this could be done for methods that are not static.

2. Will it work across compilation units?
Let's assume the iteration call is defined in compilation unit A.groovy and
a closure that uses 'break' in compilation unit B.groovy. I cannot see how a
compiler for a non-static typed language would deal with this situation. And
further, how should the compiler decide that it does not have to add
decorations in compilation unit C instead of A or B?

> > - What is the return value from nested iterations that are
> > breaked by an exception? I think it would be rather
> > unclear and probably dependant on implementation details
> > of the iteration methods.
>
> you have the same problem with a normal iteration, or not?
> Just think of findAll, which usually returns something.

Yes, but it would work if Groovy would only break the innermost iteration
level and if the relevant iteration methods would implement support for
break.


I hope that I'm not completely wrong with my arguments ;)
Alex

Jochen Theodorou

unread,
Aug 7, 2008, 6:58:56 AM8/7/08
to us...@groovy.codehaus.org
Alexander Veit schrieb:

> Hi,
>
> Sorry for my late reply.
>
>> Jochen Theodorou wrote:
>> Alexander Veit schrieb:
>>>> Charles Oliver Nutter wrote:
>>>> If you had two nested iterations, how would you break the
>> outer one?
>>> - How could the compiler decide how many iteration levels
>>> are to be unwinded without an explicit label (e.g.
>>> a catch block) in at least one of the outer iteration
>>> loops?
>> you could store and count the levels in a ThreadLocal ;) No,
>> for break you can let the compiler decorate the information
>> you need. Continue is a bit more difficult of course
>
> 1. Yes, the compiler could create decorations. But how can it decide where
> to put it into the bytecode?
>
> - Can it place the exception handler into the closure's body? No.
> - Can it place the exception handler into the code that calls the iteration
> function (e.g. findAll)? No, since it would only work for iteration
> functions that have no return values. Return values would fall victim to
> stack unwinding.

I guess I should outline the code a bit:

given the following groovy code:

list.each {break}

we wouldcreate the this code (in pseudo java):

Marker marker = MarkerFactory.getMarker()
Closure cl = new Closure(){
Object doCall(Object x) {
throw new BreakExpeption(marker);
}
}
makeMethodCall("each",list,cl, marker)

and each would look like:

Object each(List l, Closure cl, Marker marker) {
for (Object o: l) {
try {
cl.call(o);
} catch (ContinueException ce) {
if (ce.marker == marker) continue;
throw ce;
} catch (BreakException be) {
if (be.marker == marker) break;
throw be;
}
}
}

using a threalocal, you could remove the parameter... if you have code
like this:

outer: outer.each { inner ->
inner.each {
break outer
}
}

then the code would look like this:

Marker marker = MarkerFactory.getMarker()
Closure cl = new Closure(){
Object doCall(Object inner) {
Closure cl2 = new Closure(){
Object doCall(Object inner) {
throw new BreakExpeption(marker);
}
}
makeMethodCall("each",inner,cl2, null)
}
}
makeMethodCall("each",outer,cl, marker)


The problem I have with this is, that I don't want to let the user have
to do all this stuff in the methods. It is too complicated. The other
problem is, that this technique works only for appended blocks. If the
Closure is defined somewhere else, then the coupling between the
iteration method and the exception marker is lost.

> So the exception handler has to be placed inside the body of an iteration
> function. Since more than one level of iterations is to be supported, it
> cannot merely try to decorate/modify existing bytecode, but has to somehow
> copy it, infer it's semantics, decorate it, and inline the decorated code. I
> wonder if this could be done for methods that are not static.

automatic transformation of bytecode at runtime you mean? That is a
chapter I didn't dare to open in this discussion yet.

> 2. Will it work across compilation units?
> Let's assume the iteration call is defined in compilation unit A.groovy and
> a closure that uses 'break' in compilation unit B.groovy. I cannot see how a
> compiler for a non-static typed language would deal with this situation. And
> further, how should the compiler decide that it does not have to add
> decorations in compilation unit C instead of A or B?

see above

>>> - What is the return value from nested iterations that are
>>> breaked by an exception? I think it would be rather
>>> unclear and probably dependant on implementation details
>>> of the iteration methods.
>> you have the same problem with a normal iteration, or not?
>> Just think of findAll, which usually returns something.
>
> Yes, but it would work if Groovy would only break the innermost iteration
> level and if the relevant iteration methods would implement support for
> break.

yex, but as I said... innermost isn't as clear as it seems at first.

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------

Alexander Veit

unread,
Aug 7, 2008, 8:43:21 AM8/7/08
to us...@groovy.codehaus.org

OK, here you have introduced explicit labels, that are to be inserted by the
user ;)

This probably make things easier. Also nested labels are conceivable.

But don't use ThreadLocals - they're really ugly due too their lack of
locality in respect to method bodies. Let the user forget a finally block
and he/she may have a problem that's origin is not easily to locate ;)

> then the code would look like this:
>
> Marker marker = MarkerFactory.getMarker()
> Closure cl = new Closure(){
> Object doCall(Object inner) {
> Closure cl2 = new Closure(){
> Object doCall(Object inner) {
> throw new BreakExpeption(marker);
> }
> }
> makeMethodCall("each",inner,cl2, null)
> }
> }
> makeMethodCall("each",outer,cl, marker)
>
>
> The problem I have with this is, that I don't want to let the
> user have to do all this stuff in the methods. It is too complicated.

TANSTAAFL. And modifying existing (byte)code is not really an option, IMHO
(see below).

> The other problem is, that this technique works only for appended
> blocks. If the Closure is defined somewhere else, then the coupling
> between the iteration method and the exception marker is lost.

There's yet another problem if iteration methods are involved that return
values and if the break skips more than one iteration level. Think e.g. of a
tree data structure and a closure (predicate) that operates on the tree to
return all nodes in tree order up to a node that has certain properties. You
probably wouldn't get the expected results with this labelled-break
implementation.

> > So the exception handler has to be placed inside the body
> > of an iteration function. Since more than one level of
> > iterations is to be supported, it cannot merely try to
> > decorate/modify existing bytecode, but has to somehow
> > copy it, infer it's semantics, decorate it, and inline
> > the decorated code. I wonder if this could be done for
> > methods that are not static.
>
> automatic transformation of bytecode at runtime you mean? That is a
> chapter I didn't dare to open in this discussion yet.

Don't take this too seriously. I would never do such things. Consider it to
be a joke ;)


Cheers,
Alex

Jochen Theodorou

unread,
Aug 7, 2008, 9:04:20 AM8/7/08
to us...@groovy.codehaus.org
Alexander Veit schrieb:
[...]

>> outer: outer.each { inner ->
>> inner.each {
>> break outer
>> }
>> }
>
> OK, here you have introduced explicit labels, that are to be inserted by the
> user ;)

yes... it is the same as with Java, or not? Also it is valid code already

> This probably make things easier. Also nested labels are conceivable.

yepp ;)

> But don't use ThreadLocals - they're really ugly due too their lack of
> locality in respect to method bodies. Let the user forget a finally block
> and he/she may have a problem that's origin is not easily to locate ;)

we must find a way to hide that all... I really don't like the idea of
having the user to handle this..

>> then the code would look like this:
>>
>> Marker marker = MarkerFactory.getMarker()
>> Closure cl = new Closure(){
>> Object doCall(Object inner) {
>> Closure cl2 = new Closure(){
>> Object doCall(Object inner) {
>> throw new BreakExpeption(marker);
>> }
>> }
>> makeMethodCall("each",inner,cl2, null)
>> }
>> }
>> makeMethodCall("each",outer,cl, marker)
>>
>>
>> The problem I have with this is, that I don't want to let the
>> user have to do all this stuff in the methods. It is too complicated.
>
> TANSTAAFL. And modifying existing (byte)code is not really an option, IMHO
> (see below).

TANSTAAFL? What's that? Oh: "There Ain't No Such Thing As A Free Lunch"

hehe, ok, but there is a difference in how much you should pay... For me
the costs here seem to be a bit too high. The Java Closure proposal has
more or less the same problem I guess... well no, that's not true... In
the Java proposal for closure you can jump only to a label and default
labels are given only to normal loops. This way the iterator methods do
not need to handle the exceptions, but of course then you can not break
an iteration given by an iteration method too. The examples I showed
above are then for example invalid.

>> The other problem is, that this technique works only for appended
>> blocks. If the Closure is defined somewhere else, then the coupling
>> between the iteration method and the exception marker is lost.
>
> There's yet another problem if iteration methods are involved that return
> values and if the break skips more than one iteration level. Think e.g. of a
> tree data structure and a closure (predicate) that operates on the tree to
> return all nodes in tree order up to a node that has certain properties. You

> probably wouldn't get the expected results with this labeled-break
> implementation.

again I should probably mention the Gafter closure proposal... when you
look closely, then you will see that all his "closure" based iteration
methods are void. And of course exactly because of the problem with the
return value. On the other hand it should be quite clear, that in case
of break nothing is returned... which means the return value would be
null I would say... just as if you would have called a void method

>>> So the exception handler has to be placed inside the body
>>> of an iteration function. Since more than one level of
>>> iterations is to be supported, it cannot merely try to
>>> decorate/modify existing bytecode, but has to somehow
>>> copy it, infer it's semantics, decorate it, and inline
>>> the decorated code. I wonder if this could be done for
>>> methods that are not static.
>> automatic transformation of bytecode at runtime you mean? That is a
>> chapter I didn't dare to open in this discussion yet.
>
> Don't take this too seriously. I would never do such things. Consider it to
> be a joke ;)

hehe, ok

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------

Charles Oliver Nutter

unread,
Aug 7, 2008, 3:15:44 PM8/7/08
to us...@groovy.codehaus.org
Jochen Theodorou wrote:
> given the following groovy code:
>
> list.each {break}
>
> we wouldcreate the this code (in pseudo java):
>
> Marker marker = MarkerFactory.getMarker()
> Closure cl = new Closure(){
> Object doCall(Object x) {
> throw new BreakExpeption(marker);
> }
> }
> makeMethodCall("each",list,cl, marker)

If you're instantiating the closure object for each call anyway, why
wouldn't you just use the closure object as the marker? This is
analogous to how we do it in JRuby.

I also need to point out again that non-local flow control is generally
expected to be used for *literal* closures in Ruby, not reified
closures. You almost never see people do this:

p = proc { break }
[1,2,3].each &p

But this is very common:

[1,2,3].each { break }

This is because in the first case, the target of the break would need to
be the "proc" method. Since the proc method has already returned, that
"activation" is no longer available on the stack. So the break is no
longer valid. In the latter case, the break breaks to each, which is
still active at the time the closure is invoked.

- Charlie

Jochen Theodorou

unread,
Aug 7, 2008, 4:22:00 PM8/7/08
to us...@groovy.codehaus.org
Charles Oliver Nutter schrieb:

> Jochen Theodorou wrote:
>> given the following groovy code:
>>
>> list.each {break}
>>
>> we wouldcreate the this code (in pseudo java):
>>
>> Marker marker = MarkerFactory.getMarker()
>> Closure cl = new Closure(){
>> Object doCall(Object x) {
>> throw new BreakExpeption(marker);
>> }
>> }
>> makeMethodCall("each",list,cl, marker)
>
> If you're instantiating the closure object for each call anyway, why
> wouldn't you just use the closure object as the marker? This is
> analogous to how we do it in JRuby.

well.. it is just a prototype ;)

> I also need to point out again that non-local flow control is generally
> expected to be used for *literal* closures in Ruby, not reified
> closures. You almost never see people do this:
>
> p = proc { break }
> [1,2,3].each &p
>
> But this is very common:
>
> [1,2,3].each { break }
>
> This is because in the first case, the target of the break would need to
> be the "proc" method. Since the proc method has already returned, that
> "activation" is no longer available on the stack. So the break is no
> longer valid. In the latter case, the break breaks to each, which is
> still active at the time the closure is invoked.

which means [1,2,3].each &p will cause an error? well... that's what I
would expect at last. I called it appended block instead of literal
closure, but we mean the same I guess and we came to the same
conclusion. But unlike Ruby we have no runtime representation that makes
a difference between a block and whatever proc produces. Well, maybe we
should investigate this for Ruby a bit more.. how do you write in Ruby
such an each-method, that is able to react to a break? What would the
code look like?

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------

Charles Oliver Nutter

unread,
Aug 7, 2008, 10:41:37 PM8/7/08
to us...@groovy.codehaus.org
Jochen Theodorou wrote:
> which means [1,2,3].each &p will cause an error? well... that's what I
> would expect at last. I called it appended block instead of literal
> closure, but we mean the same I guess and we came to the same
> conclusion. But unlike Ruby we have no runtime representation that makes
> a difference between a block and whatever proc produces. Well, maybe we
> should investigate this for Ruby a bit more.. how do you write in Ruby
> such an each-method, that is able to react to a break? What would the
> code look like?

Because break is a syntactic construct, there's no way to create code
that responds to it directly. The logic basically goes like this:

* a return in a closure returns from the method frame in which the
closure is directly constructed in. If a closure is reified into a Proc
object, the return produces an error. If the closure is reified into a
lambda (more method-like, more similar to Groovy closures) the return
returns from the lambda.

* a break in a closure returns from the method receiving the literal
closure, as in .each { break }. If the closure is reified into a Proc
object, the break produces an error. If the closure is reified into a
lambda, the break returns from the lambda.

* a next in a closure returns from the closure in all cases.

* a redo in a closure jumps to the top of the closure body and continues
executing.

And there's a really weird one I convinced Matz to remove from future
versions of Ruby. JRuby does not support it:

* a retry in a closure jumps back to the original call that received the
closure, reevaluates the receiver and arguments, and starts the call
over again.

I won't go over the cases where procs can become lambdas and what
happens when either of them are uwrapped back into blocks or used as the
body of method definitions. It gets a little involved.

But in each case, there's a specific point in the execution flow/stack
where a flow-control keyword in a given closure is supposed to jump to,
and there's no way to directly influence that from within Ruby code.

- Charlie

Francesco Pasqualini

unread,
Aug 16, 2008, 10:02:59 AM8/16/08
to us...@groovy.codehaus.org
The BGGA java closures proposal (short for Bracha, Gafter, Gosling, Ahe, the authors' last names)  include an interresting solution to support  break in closures.
Perhaps it's a possible solution for groovy too.

Take a look (the latest paraph at the end of the page) the section "Loop abstraction" :

http://www.javac.info/closures-v05.html

Jochen Theodorou

unread,
Aug 16, 2008, 6:59:23 PM8/16/08
to us...@groovy.codehaus.org
Francesco Pasqualini schrieb:
> The BGGA java closures proposal (short for *B*racha, *G*after,
> *G*osling, *A*he, the authors' last names) include an interresting
> solution to support break in closures.
> Perhaps it's a possible solution for groovy too.
>
> Take a look (the latest paraph at the end of the page) the section "Loop
> abstraction" :
>
> http://www.javac.info/closures-v05.html

I know that proposal... it extends the normal for loop to take a method
and a block, it requires the method to be marked and when you read
carefully on the blog it might even require a new bytecode or at last
flag. An if you look further you will see that this for loop is no
expression, but a statement, which means it has no return value.

That means you can express "each", but not for example "find" and "grep"
or many other methods.

A "break" is simple in this case, because the "for" marks the spot of
usage, so we have something to place an exception handler to catch the
exception. We also need the marked method, because the exception handler
for "continue" must be inside the iterator method and coupled with the
loop there... would be interesting to know what happens if more than one
loop is used... maybe continue will count for the most outer loop.
Anyway, the method parameter list will need to be extended to be able to
get the information through which "continue" is determined to continue
the current loop and not another one. Hmm.... when I think about it I am
not sure how continue really works there... the actual loop could be
somewhere else... in a method called by the method used in the
for-loop... we have code like that in DGM... maybe they intend to use
the first loop in the last method marked with "for"

Among BGGA, CICE and FCM only BGGA is trying to solve break/continue.

If we had full continuations built into the JVM, then I would have nice
implementation ideas, easy ones even.

Anyway... why do I not like the BGGA proposal for Groovy?
* it requires strange method signatures that reveal implementation
details if you don't support the user with some compiler based code
generation (no problem inside groovy code, a big problem in java code)
* it requires to mark the loop as such.. which isn't actually bad, I it
is just not nice
* it does not solve the handling of return values in case of a "break".

there would be also the point of having to declare so many types, but
that is not needed in Groovy anyway.

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------

Jochen Theodorou

unread,
Oct 1, 2008, 6:56:56 PM10/1/08
to us...@groovy.codehaus.org
sorry... this thread got buried somehow...

Charles Oliver Nutter schrieb:


> Jochen Theodorou wrote:
>> which means [1,2,3].each &p will cause an error? well... that's what I
>> would expect at last. I called it appended block instead of literal
>> closure, but we mean the same I guess and we came to the same
>> conclusion. But unlike Ruby we have no runtime representation that
>> makes a difference between a block and whatever proc produces. Well,
>> maybe we should investigate this for Ruby a bit more.. how do you
>> write in Ruby such an each-method, that is able to react to a break?
>> What would the code look like?
>
> Because break is a syntactic construct, there's no way to create code
> that responds to it directly. The logic basically goes like this:
>
> * a return in a closure returns from the method frame in which the
> closure is directly constructed in. If a closure is reified into a Proc
> object, the return produces an error. If the closure is reified into a
> lambda (more method-like, more similar to Groovy closures) the return
> returns from the lambda.

in Groovy the Closure/lambda has its own method frame, so return returns
from the closure, but not from the iterating function. So yes, your
lambdas are fitting our Groovy Closures more I guess...

> * a break in a closure returns from the method receiving the literal
> closure, as in .each { break }. If the closure is reified into a Proc
> object, the break produces an error. If the closure is reified into a
> lambda, the break returns from the lambda.

so break==return for a lambda, true?

> * a next in a closure returns from the closure in all cases.

so break==return==next for lambda? Or invalid in a lambda?

> * a redo in a closure jumps to the top of the closure body and continues
> executing.

well, Java continue is more like next I think... redo is something that
we could easily implement.... but I am not sure we really need that.

> And there's a really weird one I convinced Matz to remove from future
> versions of Ruby. JRuby does not support it:
>
> * a retry in a closure jumps back to the original call that received the
> closure, reevaluates the receiver and arguments, and starts the call
> over again.

oh... evil.... ;)

> I won't go over the cases where procs can become lambdas and what
> happens when either of them are uwrapped back into blocks or used as the
> body of method definitions. It gets a little involved.
>
> But in each case, there's a specific point in the execution flow/stack
> where a flow-control keyword in a given closure is supposed to jump to,
> and there's no way to directly influence that from within Ruby code.

I, see thanks.

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------

Reply all
Reply to author
Forward
0 new messages